37a9a4e0f1
This commit makes some big changes to how we track state for Zig source files. In particular, it changes: * How `File` tracks its path on-disk * How AstGen discovers files * How file-level errors are tracked * How `builtin.zig` files and modules are created The original motivation here was to address incremental compilation bugs with the handling of files, such as #22696. To fix this, a few changes are necessary. Just like declarations may become unreferenced on an incremental update, meaning we suppress analysis errors associated with them, it is also possible for all imports of a file to be removed on an incremental update, in which case file-level errors for that file should be suppressed. As such, after AstGen, the compiler must traverse files (starting from analysis roots) and discover the set of "live files" for this update. Additionally, the compiler's previous handling of retryable file errors was not very good; the source location the error was reported as was based only on the first discovered import of that file. This source location also disappeared on future incremental updates. So, as a part of the file traversal above, we also need to figure out the source locations of imports which errors should be reported against. Another observation I made is that the "file exists in multiple modules" error was not implemented in a particularly good way (I get to say that because I wrote it!). It was subject to races, where the order in which different imports of a file were discovered affects both how errors are printed, and which module the file is arbitrarily assigned, with the latter in turn affecting which other files are considered for import. The thing I realised here is that while the AstGen worker pool is running, we cannot know for sure which module(s) a file is in; we could always discover an import later which changes the answer. So, here's how the AstGen workers have changed. We initially ensure that `zcu.import_table` contains the root files for all modules in this Zcu, even if we don't know any imports for them yet. Then, the AstGen workers do not need to be aware of modules. Instead, they simply ignore module imports, and only spin off more workers when they see a by-path import. During AstGen, we can't use module-root-relative paths, since we don't know which modules files are in; but we don't want to unnecessarily use absolute files either, because those are non-portable and can make `error.NameTooLong` more likely. As such, I have introduced a new abstraction, `Compilation.Path`. This type is a way of representing a filesystem path which has a *canonical form*. The path is represented relative to one of a few special directories: the lib directory, the global cache directory, or the local cache directory. As a fallback, we use absolute (or cwd-relative on WASI) paths. This is kind of similar to `std.Build.Cache.Path` with a pre-defined list of possible `std.Build.Cache.Directory`, but has stricter canonicalization rules based on path resolution to make sure deduplicating files works properly. A `Compilation.Path` can be trivially converted to a `std.Build.Cache.Path` from a `Compilation`, but is smaller, has a canonical form, and has a digest which will be consistent across different compiler processes with the same lib and cache directories (important when we serialize incremental compilation state in the future). `Zcu.File` and `Zcu.EmbedFile` both contain a `Compilation.Path`, which is used to access the file on-disk; module-relative sub paths are used quite rarely (`EmbedFile` doesn't even have one now for simplicity). After the AstGen workers all complete, we know that any file which might be imported is definitely in `import_table` and up-to-date. So, we perform a single-threaded graph traversal; similar to what `resolveReferences` plays for `AnalUnit`s, but for files instead. We figure out which files are alive, and which module each file is in. If a file turns out to be in multiple modules, we set a field on `Zcu` to indicate this error. If a file is in a different module to a prior update, we set a flag instructing `updateZirRefs` to invalidate all dependencies on the file. This traversal also discovers "import errors"; these are errors associated with a specific `@import`. With Zig's current design, there is only one possible error here: "import outside of module root". This must be identified during this traversal instead of during AstGen, because it depends on which module the file is in. I tried also representing "module not found" errors in this same way, but it turns out to be much more useful to report those in Sema, because of use cases like optional dependencies where a module import is behind a comptime-known build option. For simplicity, `failed_files` now just maps to `?[]u8`, since the source location is always the whole file. In fact, this allows removing `LazySrcLoc.Offset.entire_file` completely, slightly simplifying some error reporting logic. File-level errors are now directly built in the `std.zig.ErrorBundle.Wip`. If the payload is not `null`, it is the message for a retryable error (i.e. an error loading the source file), and will be reported with a "file imported here" note pointing to the import site discovered during the single-threaded file traversal. The last piece of fallout here is how `Builtin` works. Rather than constructing "builtin" modules when creating `Package.Module`s, they are now constructed on-the-fly by `Zcu`. The map `Zcu.builtin_modules` maps from digests to `*Package.Module`s. These digests are abstract hashes of the `Builtin` value; i.e. all of the options which are placed into "builtin.zig". During the file traversal, we populate `builtin_modules` as needed, so that when we see this imports in Sema, we just grab the relevant entry from this map. This eliminates a bunch of awkward state tracking during construction of the module graph. It's also now clearer exactly what options the builtin module has, since previously it inherited some options arbitrarily from the first-created module with that "builtin" module! The user-visible effects of this commit are: * retryable file errors are now consistently reported against the whole file, with a note pointing to a live import of that file * some theoretical bugs where imports are wrongly considered distinct (when the import path moves out of the cwd and then back in) are fixed * some consistency issues with how file-level errors are reported are fixed; these errors will now always be printed in the same order regardless of how the AstGen pass assigns file indices * incremental updates do not print retryable file errors differently between updates or depending on file structure/contents * incremental updates support files changing modules * incremental updates support files becoming unreferenced Resolves: #22696
A general-purpose programming language and toolchain for maintaining robust, optimal, and reusable software.
Documentation
If you are looking at this README file in a source tree, please refer to the Release Notes, Language Reference, or Standard Library Documentation corresponding to the version of Zig that you are using by following the appropriate link on the download page.
Otherwise, you're looking at a release of Zig, so you can find the language
reference at doc/langref.html, and the standard library documentation by
running zig std, which will open a browser tab.
Installation
A Zig installation is composed of two things:
- The Zig executable
- The lib/ directory
At runtime, the executable searches up the file system for the lib/ directory, relative to itself:
- lib/
- lib/zig/
- ../lib/
- ../lib/zig/
- (and so on)
In other words, you can unpack a release of Zig anywhere, and then begin
using it immediately. There is no need to install it globally, although this
mechanism supports that use case too (i.e. /usr/bin/zig and /usr/lib/zig/).
Building from Source
Ensure you have the required dependencies:
- CMake >= 3.15
- System C/C++ Toolchain
- LLVM, Clang, LLD development libraries == 20.x
Then it is the standard CMake build process:
mkdir build
cd build
cmake ..
make install
For more options, tips, and troubleshooting, please see the Building Zig From Source page on the wiki.
Building from Source without LLVM
In this case, the only system dependency is a C compiler.
cc -o bootstrap bootstrap.c
./bootstrap
This produces a zig2 executable in the current working directory. This is a
"stage2" build of the compiler,
without LLVM extensions, and is
therefore lacking these features:
- Release mode optimizations
- aarch64 machine code backend
- @cImport
- zig translate-c
- Ability to compile assembly files
- Some ELF linking features
- Most COFF/PE linking features
- Some WebAssembly linking features
- Ability to create import libs from def files
- Ability to create static archives from object files
- Ability to compile C, C++, Objective-C, and Objective-C++ files
However, a compiler built this way does provide a C backend, which may be useful for creating system packages of Zig projects using the system C toolchain. In this case, LLVM is not needed!
Furthermore, a compiler built this way provides an LLVM backend that produces bitcode files, which may be compiled into object files via a system Clang package. This can be used to produce system packages of Zig applications without the Zig package dependency on LLVM.
Contributing
Zig is Free and Open Source Software. We welcome bug reports and patches from everyone. However, keep in mind that Zig governance is BDFN (Benevolent Dictator For Now) which means that Andrew Kelley has final say on the design and implementation of everything.
One of the best ways you can contribute to Zig is to start using it for an open-source personal project.
This leads to discovering bugs and helps flesh out use cases, which lead to further design iterations of Zig. Importantly, each issue found this way comes with real world motivations, making it straightforward to explain the reasoning behind proposals and feature requests.
You will be taken much more seriously on the issue tracker if you have a personal project that uses Zig.
The issue label Contributor Friendly exists to help you find issues that are limited in scope and/or knowledge of Zig internals.
Please note that issues labeled Proposal but do not also have the Accepted label are still under consideration, and efforts to implement such a proposal have a high risk of being wasted. If you are interested in a proposal which is still under consideration, please express your interest in the issue tracker, providing extra insights and considerations that others have not yet expressed. The most highly regarded argument in such a discussion is a real world use case.
For more tips, please see the Contributing page on the wiki.
Community
The Zig community is decentralized. Anyone is free to start and maintain their own space for Zig users to gather. There is no concept of "official" or "unofficial". Each gathering place has its own moderators and rules. Users are encouraged to be aware of the social structures of the spaces they inhabit, and work purposefully to facilitate spaces that align with their values.
Please see the Community wiki page for a public listing of social spaces.