* Indices of referenced captures
* Line and column of `@src()`
The second point aligns with a reversal of the "incremental compilation"
section of https://github.com/ziglang/zig/issues/2029#issuecomment-645793168.
This reversal was already done as #17688 (46a6d50), with the idea to
push incremental compilation down the line. My proposal is to keep it as
comptime-known, and simply re-analyze uses of `@src()` whenever their
line/column change.
I think this decision is reasonable for a few reasons:
* The Zig compiler is quite fast. Occasionally re-analyzing a few
functions containing `@src()` calls is perfectly acceptable and won't
noticably impact update times.
* The system described by Andrew in #2029 is currently vaporware.
* The system described by Andrew in #2029 is non-trivial to implement.
In particular, it requires some way to have backends update a single
global in certain cases, without re-doing semantic analysis. There is
no other part of incremental compilation which requires this.
* Having `@src().line` be comptime-known is useful. For instance, #17688
was justified by broken Tracy integration because the source line
couldn't be comptime-known.
* std.c.darwin: add missing CPUFAMILY fields
* std.zig.system.detectNativeCpuAndFeatures: add missing darwin fields
* add comment so the prong isnt lost and easily discoverable during next llvm upgrade
A compilation build step for which the binary is not required could not
be compiled previously. There were 2 issues that caused this:
- The compiler communicated only the results of the emitted binary and
did not properly communicate the result if the binary was not emitted.
This is fixed by communicating the final hash of the artifact path (the
hash of the corresponding /o/<hash> directory) and communicating this
instead of the entire path. This changes the zig build --listen protocol
to communicate hashes instead of paths, and emit_bin_path is accordingly
renamed to emit_digest.
- There was an error related to the default llvm object path when
CacheUse.Whole was selected. I'm not really sure why this didn't manifest
when the binary is also emitted.
This was fixed by improving the path handling related to flush() and
emitLlvmObject().
In general, this commit also improves some of the path handling throughout
the compiler and standard library.
This replaces the constant `Zir.Inst.Ref` tags (and the analagous tags
in `Air.Inst.Ref`, `InternPool.Index`) referring to types in
`std.builtin` with a ZIR instruction `extended(builtin_type(...))` which
instructs Sema to fetch such a type, effectively as if it were a
shorthand for the ZIR for `@import("std").builtin.xyz`.
Previously, this was achieved through constant tags in `Ref`. The
analagous `InternPool` indices began as `simple_type` values, and were
later rewritten to the correct type information. This system was kind of
brittle, and more importantly, isn't compatible with incremental
compilation of std, since incremental compilation relies on the ability
to recreate types at different indices when they change. Replacing the
old system with this instruction slightly increases the size of ZIR, but
it simplifies logic and allows incremental compilation to work correctly
on the standard library.
This shouldn't have a significant impact on ZIR size or compiler
performance, but I will take measurements in the PR to confirm this.
The old logic here had bitrotted, largely because there were some
incorrect `else` cases. This is now implemented correctly for all
current ZIR instructions. This prevents instructions being lost in
incremental updates, which is important for updates to be minimal.
* new .zig-cache subdirectory: 'v'
- stores coverage information with filename of hash of PCs that want
coverage. This hash is a hex encoding of the 64-bit coverage ID.
* build runner
* fixed bug in file system inputs when a compile step has an
overridden zig_lib_dir field set.
* set some std lib options optimized for the build runner
- no side channel mitigations
- no Transport Layer Security
- no crypto fork safety
* add a --port CLI arg for choosing the port the fuzzing web interface
listens on. it defaults to choosing a random open port.
* introduce a web server, and serve a basic single page application
- shares wasm code with autodocs
- assets are created live on request, for convenient development
experience. main.wasm is properly cached if nothing changes.
- sources.tar comes from file system inputs (introduced with the
`--watch` feature)
* receives coverage ID from test runner and sends it on a thread-safe
queue to the WebServer.
* test runner
- takes a zig cache directory argument now, for where to put coverage
information.
- sends coverage ID to parent process
* fuzzer
- puts its logs (in debug mode) in .zig-cache/tmp/libfuzzer.log
- computes coverage_id and makes it available with
`fuzzer_coverage_id` exported function.
- the memory-mapped coverage file is now namespaced by the coverage id
in hex encoding, in `.zig-cache/v`
* tokenizer
- add a fuzz test to check that several properties are upheld
This target triple was weird on multiple levels:
* The `ilp32` ABI is the soft float ABI. This is not the main ABI we want to
support on RISC-V; rather, we want `ilp32d`.
* `gnuilp32` is a bespoke tag that was introduced in Zig. The rest of the world
just uses `gnu` for RISC-V target triples.
* `gnu_ilp32` is already the name of an ILP32 ABI used on AArch64. `gnuilp32` is
too easy to confuse with this.
* We don't use this convention for `riscv64-linux-gnu`.
* Supporting all RISC-V ABIs with this convention will result in combinatorial
explosion; see #20690.
I pointed a fuzzer at the tokenizer and it crashed immediately. Upon
inspection, I was dissatisfied with the implementation. This commit
removes several mechanisms:
* Removes the "invalid byte" compile error note.
* Dramatically simplifies tokenizer recovery by making recovery always
occur at newlines, and never otherwise.
* Removes UTF-8 validation.
* Moves some character validation logic to `std.zig.parseCharLiteral`.
Removing UTF-8 validation is a regression of #663, however, the existing
implementation was already buggy. When adding this functionality back,
it must be fuzz-tested while checking the property that it matches an
independent Unicode validation implementation on the same file. While
we're at it, fuzzing should check the other properties of that proposal,
such as no ASCII control characters existing inside the source code.
Other changes included in this commit:
* Deprecate `std.unicode.utf8Decode` and its WTF-8 counterpart. This
function has an awkward API that is too easy to misuse.
* Make `utf8Decode2` and friends use arrays as parameters, eliminating a
runtime assertion in favor of using the type system.
After this commit, the crash found by fuzzing, which was
"\x07\xd5\x80\xc3=o\xda|a\xfc{\x9a\xec\x91\xdf\x0f\\\x1a^\xbe;\x8c\xbf\xee\xea"
no longer causes a crash. However, I did not feel the need to add this
test case because the simplified logic eradicates most crashes of this
nature.
This is a misfeature that we inherited from LLVM:
* https://reviews.llvm.org/D61259
* https://reviews.llvm.org/D61939
(`aarch64_32` and `arm64_32` are equivalent.)
I truly have no idea why this triple passed review in LLVM. It is, to date, the
*only* tag in the architecture component that is not, in fact, an architecture.
In reality, it is just an ILP32 ABI for AArch64 (*not* AArch32).
The triples that use `aarch64_32` look like `aarch64_32-apple-watchos`. Yes,
that triple is exactly what you think; it has no ABI component. They really,
seriously did this.
Since only Apple could come up with silliness like this, it should come as no
surprise that no one else uses `aarch64_32`. Later on, a GNU ILP32 ABI for
AArch64 was developed, and support was added to LLVM:
* https://reviews.llvm.org/D94143
* https://reviews.llvm.org/D104931
Here, sanity seems to have prevailed, and a triple using this ABI looks like
`aarch64-linux-gnu_ilp32` as you would expect.
As can be seen from the diffs in this commit, there was plenty of confusion
throughout the Zig codebase about what exactly `aarch64_32` was. So let's just
remove it. In its place, we'll use `aarch64-watchos-ilp32`,
`aarch64-linux-gnuilp32`, and so on. We'll then translate these appropriately
when talking to LLVM. Hence, this commit adds the `ilp32` ABI tag (we already
have `gnuilp32`).
This was used for LoongArch64, where:
* `gnuf64` -> `ilp32d` / `lp64d` (full hard float)
* `gnuf32` -> `ilp32f` / `lp64f` (hard float for `f32` only)
* `gnusf` -> `ilp32` / `lp64` (soft float)
But Loongson eventually settled on just `gnu` for the first case since that's
what most people will actually be targeting outside embedded scenarios. The
`gnuf32` and `gnusf` specifiers remain in use.
Changes the `make` function signature to take an options struct, which
additionally includes `watch: bool`. I intentionally am not exposing
this information to configure phase logic.
Also adds global zig cache to the compiler cache prefixes.
Closes#20600
Updates the build runner to unconditionally require a zig lib directory
parameter. This parameter is needed in order to correctly understand
file system inputs from zig compiler subprocesses, since they will refer
to "the zig lib directory", and the build runner needs to place file
system watches on directories in there.
The build runner's fanotify file watching implementation now accounts
for when two or more Cache.Path instances compare unequal but ultimately
refer to the same directory in the file system.
Breaking change: std.Build no longer has a zig_lib_dir field. Instead,
there is the Graph zig_lib_directory field, and individual Compile steps
can still have their zig lib directories overridden. I think this is
unlikely to break anyone's build in practice.
The compiler now sends a "file_system_inputs" message to the build
runner which shares the full set of files that were added to the cache
system with the build system, so that the build runner can watch
properly and redo the Compile step. This is implemented for whole cache
mode but not yet for incremental cache mode.
Since we track `reify` instructions across incremental updates, it is
acceptable to treat it as the baseline for a relative source location.
This turns out to be a good idea, since it makes it easy to define the
source location for a reified type.
🦀 src_decl is gone 🦀
This commit eliminates the `src_decl` field from `Sema.Block`. This
change goes further to eliminating unnecessary responsibilities of
`Decl` in preparation for its major upcoming refactor.
The two main remaining reponsibilities had to do with namespace types:
`src_decl` was used to determine their line number and their name. The
former use case is solved by storing the line number alongside type
declarations (and reifications) in ZIR; this is actually more correct,
since previously the line number assigned to the type was really the
line number of the source declaration it was syntactically contained
within, which does not necessarily line up. Consequently, this change
makes debug info for namespace types more correct, although I am not
sure how debuggers actually utilize this line number, if at all. Naming
types was solved by a new field on `Block`, called `type_name_ctx`. In a
sense, it represents the "namespace" we are currently within, including
comptime function calls etc. We might want to revisit this in future,
since the type naming rules seem to be a bit hand-wavey right now.
As far as I can tell, there isn't any more preliminary work needed for
me to start work on the behemoth task of splitting `Zcu.Decl` into the
new `Nav` (Named Addressable Value) and `Cau` (Comptime Analysis Unit)
types. This will be a sweeping change, impacting essentially every part
of the pipeline after `AstGen`.
This is in preparation for some upcoming changes to how we represent
source locations in the compiler. The bulk of the change here is dealing
with the removal of `src()` methods from `Zir` types.
