Instead of Module setting up the root_scope with the root source file,
instead, Module relies on the package table graph being set up properly,
and inside `update()`, it does the equivalent of `_ = @import("std");`.
This, in term, imports start.zig, which has the logic to call main (or
not). `Module` no longer has `root_scope` - the root source file is no
longer special, it's just in the package table mapped to "root".
I also went ahead and implemented proper detection of updated files.
mtime, inode, size, and source hash are kept in `Scope.File`.
During an update, iterate over `import_table` and stat each file to find
out which ones are updated.
The source hash is redundant with the source hash used by the struct
decl that corresponds to the file, so it should be removed in a future
commit before merging the branch.
* AstGen: add "previously declared here" notes for variables shadowing
decls.
* Parse imports as structs. Module now calls `AstGen.structDeclInner`,
which is called by `AstGen.containerDecl`.
- `importFile` is a bit kludgy with how it handles the top level Decl
that kinda gets merged into the struct decl at the end of the
function. Be on the look out for bugs related to that as well as
possibly cleaner ways to implement this.
* Module: factor out lookupDeclName into lookupIdentifier and lookupNa
* Rename `Scope.Container` to `Scope.Namespace`.
* Delete some dead code.
This branch won't work until `usingnamespace` is implemented because it
relies on `@import("builtin").OutputMode` and `OutputMode` comes from a
`usingnamespace`.
This branch adds "builtin" and "std" to the import table when using the
self-hosted backend.
"builtin" gains one additional item:
```
pub const zig_is_stage2 = true; // false when using stage1 backend
```
This allows the std lib to do conditional compilation based on detecting
which backend is being used. This will be removed from builtin as soon
as self-hosted catches up to feature parity with stage1.
Keep a sharp eye out - people are going to be tempted to abuse this.
The general rule of thumb is do not use `builtin.zig_is_stage2`. However
this commit breaks the rule so that we can gain limited start.zig support
as we incrementally improve the self-hosted compiler.
This commit also implements `fullyQualifiedNameHash` and related
functionality, which effectively puts all Decls in their proper
namespaces. `fullyQualifiedName` is not yet implemented.
Stop printing "todo" log messages for test decls unless we are in test
mode.
Add "previous definition here" error notes for Decl name collisions.
This commit does not bring us yet to a newly passing test case.
Here's what I'm working towards:
```zig
const std = @import("std");
export fn main() c_int {
const a = std.fs.base64_alphabet[0];
return a - 'A';
}
```
Current output:
```
$ ./zig-cache/bin/zig build-exe test.zig
test.zig:3:1: error: TODO implement more analyze elemptr
zig-cache/lib/zig/std/start.zig:38:46: error: TODO implement structInitExpr ty
```
So the next steps are clear:
* Sema: improve elemptr
* AstGen: implement structInitExpr
Before, incremental compilation would crash when trying to emit compile
errors for the update after introducing a parse error.
Parse errors are handled by not invalidating any existing semantic
analysis. However, only the parse error must be reported, with all the
other errors suppressed. Once the parse error is fixed, the new file can
be treated as an update to the previously-succeeded update.
* `analyzeContainer` now has an `outdated_decls` set as well as
`deleted_decls`. Instead of queuing up outdated Decls for re-analysis
right away, they are added to this new set. When processing the
`deleted_decls` set, we remove deleted Decls from the
`outdated_decls` set, to avoid deleted Decl pointers from being in
the work_queue. Only after processing the deleted decls do we add
analyze_decl work items to the queue.
* Module.deletion_set is now an `AutoArrayHashMap` rather than `ArrayList`.
`declareDeclDependency` will now remove a Decl from it as appropriate.
When processing the `deletion_set` in `Compilation.performAllTheWork`,
it now assumes all Decl in the set are to be deleted.
* Fix crash when handling parse errors. Currently we unload the
`ast.Tree` if any parse errors occur. Previously the code emitted a
LazySrcLoc pointing to a token index, but then when we try to resolve
the token index to a byte offset to create a compile error message,
the ast.Tree` would be unloaded. Now we use
`LazySrcLoc.byte_abs` instead of `token_abs` so the error message can
be created even with the `ast.Tree` unloaded.
Together, these changes solve a crash that happened with incremental
compilation when Decls were added and removed in some combinations.
We are now passing this test:
```zig
export fn _start() noreturn {}
```
```
test.zig:1:30: error: expected noreturn, found void
```
I ran into an issue where we get an integer overflow trying to compute
node index offsets from the containing Decl. The problem is that the
parser adds the Decl node after adding the child nodes. For some things,
it is easy to reserve the node index and then set it later, however, for
this case, it is not a trivial code change, because depending on tokens
after parsing the decl determines whether we want to add a new node or
not.
Possible strategies here:
1. Rework the parser code to make sure that Decl nodes are before
children nodes in the AST node array.
2. Use signed integers for Decl node offsets.
3. Just flip the order of subtraction and addition. Expect Decl Node
index to be greater than children Node indexes.
I opted for (3) because it seems like the simplest thing to do. We'll
want to unify the logic for computing the offsets though because if the
logic gets repeated, it will probably get repeated wrong.
The memory layout for ZIR instructions is completely reworked. See
zir.zig for those changes. Some new types:
* `zir.Code`: a "finished" set of ZIR instructions. Instead of allocating
each instruction independently, there is now a Tag and 8 bytes of
data available for all ZIR instructions. Small instructions fit
within these 8 bytes; larger ones use 4 bytes for an index into
`extra`. There is also `string_bytes` so that we can have 4 byte
references to strings. `zir.Inst.Tag` describes how to interpret
those 8 bytes of data.
- This is shared by all `Block` scopes.
* `Module.WipZirCode`: represents an in-progress `zir.Code`. In this
structure, the arrays are mutable, and get resized as we add/delete
things. There is extra state to keep track of things. This struct is
stored on the stack. Once it is finished, it produces an immutable
`zir.Code`, which will remain on the heap for the duration of a
function's existence.
- This is shared by all `GenZir` scopes.
* `Sema`: represents in-progress semantic analysis of a `zir.Code`.
This data is stored on the stack and is shared among all `Block`
scopes. It is now the main "self" argument to everything in the file
that was previously named `zir_sema.zig`.
Additionally, I moved some logic that was in `Module` into here.
`Module.Fn` now stores its parameter names inside the `zir.Code`,
instead of inside ZIR instructions. When the TZIR memory layout
reworking time comes, codegen will be able to reference this data
directly instead of duplicating it.
astgen.zig is (so far) almost entirely untouched, but nearly all of it
will need to be reworked to adhere to this new memory layout structure.
I have no benchmarks to report yet, as I am still working through
compile errors and fixing various things that I broke in this branch.
Overhaul of Source Locations:
Previously we used `usize` everywhere to mean byte offset, but sometimes
also mean other stuff. This was error prone and also made us do
unnecessary work, and store unnecessary bytes in memory.
Now there are more types involved into source locations, and more ways
to describe a source location.
* AllErrors.Message: embrace the assumption that files always have less
than 2 << 32 bytes.
* SrcLoc gets more complicated, to model more complicated source
locations.
* Introduce LazySrcLoc, which can model interesting source locations
with very little stored state. Useful for avoiding doing unnecessary
work when no compile errors occur.
Also, previously, we had `src: usize` on every ZIR instruction. This is
no longer the case. Each instruction now determines whether it even cares
about source location, and if so, how that source location is stored.
This requires more careful work inside `Sema`, but it results in fewer
bytes stored on the heap, without compromising accuracy and power of
compile error messages.
Miscellaneous:
* std.zig: string literals have more helpful result values for
reporting errors. There is now a lower level API and a higher level
API.
- side note: I noticed that the string literal logic needs some love.
There is some unnecessarily hacky code there.
* cut & pasted some TZIR logic that was in zir.zig to ir.zig. This
probably broke stuff and needs to get fixed.
* Removed type/Enum.zig, type/Union.zig, and type/Struct.zig. I don't
think this quite how this code will be organized. Need some more
careful planning about how to implement structs, unions, enums. They
need to be independent Decls, just like a top level function.
LLVM 12 included a patch that changed the way availability annotations
are specified. We now have to define the _LIBCPP_DISABLE_VISIBILITY_ANNOTATIONS
flag to make sure that we tell the c++ headers that we don't use
visibility annotations.
Related LLVM patch: D90843
This commit does not reach any particular milestone, it is
work-in-progress towards getting things to build.
There's a `@panic("TODO")` in translate-c that should be removed when
working on translate-c stuff.
The CLI gains -flto and -fno-lto options to override the default.
However, the cool thing about this is that the defaults are great! In
general when you use build-exe in release mode, Zig will enable LTO if
it would work and it would help.
zig cc supports detecting and honoring the -flto and -fno-lto flags as
well. The linkWithLld functions are improved to all be the same with
regards to copying the artifact instead of trying to pass single objects
through LLD with -r. There is possibly a future improvement here as
well; see the respective TODOs.
stage1 is updated to support outputting LLVM bitcode instead of machine
code when lto is enabled. This allows LLVM to optimize across the Zig and
C/C++ code boundary.
closes#2845
string literals and error set types were allocating the ty/val fields of
the anonymous Decl into the owner Decl's arena, rather than the
new anonymous Decl's arena as intended. This caused use of undefined
value later on in the pipeline.
Previously you had to recompile if you wanted to change the log scopes
that get printed. Now, log scopes can be set at runtime, and -Dlog
controls whether all logging is available at runtime.
Purpose here is a nicer development experience. Most likely stage2
developers will always want -Dlog enabled and then pass --debug-log
scopes when debugging particular issues.
* Implement error notes
- note: other symbol exported here
- note: previous else prong is here
- note: previous '_' prong is here
* Add Compilation.CObject.ErrorMsg. This object properly converts to
AllErrors.Message when the time comes.
* Add Compilation.CObject.failure_retryable. Properly handles
out-of-memory and other transient failures.
* Introduce Module.SrcLoc which has not only a byte offset but also
references the file which the byte offset applies to.
* Scope.Block now contains both a pointer to the "owner" Decl and the
"source" Decl. As an example, during inline function call, the
"owner" will be the Decl of the caller and the "source" will be the
Decl of the callee.
* Module.ErrorMsg now sports a `file_scope` field so that notes can
refer to source locations in a file other than the parent error
message.
* Some instances where a `*Scope` was stored, now store a
`*Scope.Container`.
* Some methods in the `Scope` namespace were moved to the more specific
type, since there was only an implementation for one particular tag.
- `removeDecl` moved to `Scope.Container`
- `destroy` moved to `Scope.File`
* Two kinds of Scope deleted:
- zir_module
- decl
* astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly
changed to be a reference; this commit fixes it. Fewer ZIR
instructions processed as a result.
- declval_in_module is renamed to declval
- previous declval ZIR instruction is deleted; it was only for .zir
files.
* Test harness: friendlier diagnostics when an unexpected set of errors
is encountered.
* zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst
on the last zir instruction in the block.
Compile log implementation:
* Write to a buffer rather than directly to stderr.
* Only keep track of 1 callsite per Decl.
* No longer mutate the ZIR Inst struct data.
* "Compile log statement found" errors are only emitted when there are
no other compile errors.
-femit-zir and support for .zir source files is regressed. If we wanted
to support this again, outputting .zir would need to be done as yet
another backend rather than in the haphazard way it was previously
implemented.
For parsing .zir, it was implemented previously in a way that was not
helpful for debugging. We need tighter integration with the test harness
for it to be useful; so clearly a rewrite is needed. Given that a
rewrite is needed, and it was getting in the way of progress and
organization of the rest of stage2, I regressed the feature.
* move concurrency primitives that always operate on kernel threads to
the std.Thread namespace
* remove std.SpinLock. Nobody should use this in a non-freestanding
environment; the other primitives are always preferable. In
freestanding, it will be necessary to put custom spin logic in there,
so there are no use cases for a std lib version.
* move some std lib files to the top level fields convention
* add std.Thread.spinLoopHint
* add std.Thread.Condition
* add std.Thread.Semaphore
* new implementation of std.Thread.Mutex for Windows and non-pthreads Linux
* add std.Thread.RwLock
Implementations provided by @kprotty
* CLI: change to -mred-zone and -mno-red-zone to match gcc/clang.
* build.zig: remove the double negative and make it an optional bool.
This follows precedent from other flags, allowing the compiler CLI to
be the decider of what is default instead of duplicating the default
value into the build system code.
* Compilation: make it an optional `want_red_zone` instead of a
`no_red_zone` bool. The default is decided by a call to
`target_util.hasRedZone`.
* When creating a Clang command line, put -mred-zone on the command
line if we are forcing it to be enabled.
* Update update_clang_options.zig with respect to the recent {s}/{} format changes.
* `zig cc` integration with red zone preference.
This will enable code to perform version checks and make it easier to
support multiple versions of Zig.
Within the SemVer implementation, an intermediate value needed to be
coerced to a slice to workaround a comptime bug.
Closes#6466
Zig's format system is flexible enough to add custom formatters. This PR removes the new z/Z format specifiers that were added for printing Zig identifiers and replaces them with custom formatters.
