The compiler actually doesn't need any functional changes for this: Sema
does reification based on the tag indices of `std.builtin.Type` already!
So, no zig1.wasm update is necessary.
This change is necessary to disallow name clashes between fields and
decls on a type, which is a prerequisite of #9938.
Implements the accepted proposal to introduce `@branchHint`. This
builtin is permitted as the first statement of a block if that block is
the direct body of any of the following:
* a function (*not* a `test`)
* either branch of an `if`
* the RHS of a `catch` or `orelse`
* a `switch` prong
* an `or` or `and` expression
It lowers to the ZIR instruction `extended(branch_hint(...))`. When Sema
encounters this instruction, it sets `sema.branch_hint` appropriately,
and `zirCondBr` etc are expected to reset this value as necessary. The
state is on `Sema` rather than `Block` to make it automatically
propagate up non-conditional blocks without special handling. If
`@panic` is reached, the branch hint is set to `.cold` if none was
already set; similarly, error branches get a hint of `.unlikely` if no
hint is explicitly provided. If a condition is comptime-known, `cold`
hints from the taken branch are allowed to propagate up, but other hints
are discarded. This is because a `likely`/`unlikely` hint just indicates
the direction this branch is likely to go, which is redundant
information when the branch is known at comptime; but `cold` hints
indicate that control flow is unlikely to ever reach this branch,
meaning if the branch is always taken from its parent, then the parent
is also unlikely to ever be reached.
This branch information is stored in AIR `cond_br` and `switch_br`. In
addition, `try` and `try_ptr` instructions have variants `try_cold` and
`try_ptr_cold` which indicate that the error case is cold (rather than
just unlikely); this is reachable through e.g. `errdefer unreachable` or
`errdefer @panic("")`.
A new API `unwrapSwitch` is introduced to `Air` to make it more
convenient to access `switch_br` instructions. In time, I plan to update
all AIR instructions to be accessed via an `unwrap` method which returns
a convenient tagged union a la `InternPool.indexToKey`.
The LLVM backend lowers branch hints for conditional branches and
switches as follows:
* If any branch is marked `unpredictable`, the instruction is marked
`!unpredictable`.
* Any branch which is marked as `cold` gets a
`llvm.assume(i1 true) [ "cold"() ]` call to mark the code path cold.
* If any branch is marked `likely` or `unlikely`, branch weight metadata
is attached with `!prof`. Likely branches get a weight of 2000, and
unlikely branches a weight of 1. In `switch` statements, un-annotated
branches get a weight of 1000 as a "middle ground" hint, since there
could be likely *and* unlikely *and* un-annotated branches.
For functions, a `cold` hint corresponds to the `cold` function
attribute, and other hints are currently ignored -- as far as I can tell
LLVM doesn't really have a way to lower them. (Ideally, we would want
the branch hint given in the function to propagate to call sites.)
The compiler and standard library do not yet use this new builtin.
Resolves: #21148
My main gripes with this design were that it was incorrectly namespaced, the naming was inconsistent and a bit wrong (`fooAlign` vs `fooAlignment`).
This commit moves all the logic from `PerThread.zig` to use the zcu + tid system that the previous couple commits introduce.
I've organized and merged the functions to be a bit more specific to their own purpose.
- `fieldAlignment` takes a struct or union type, an index, and a Zcu (or the Sema version which takes a Pt), and gives you the alignment of the field at the index.
- `structFieldAlignment` takes the field type itself, and provides the logic to handle special cases, such as externs.
A design goal I had in mind was to avoid using the word 'struct' in the function name, when it worked for things that aren't structs, such as unions.
This commit makes more progress towards incremental compilation, fixing
some crashes in the frontend. Notably, it fixes the regressions introduced
by #20964. It also cleans up the "outdated file root" mechanism, by
virtue of deleting it: we now detect outdated file roots just after
updating ZIR refs, and re-scan their namespaces.
The type `Zcu.Decl` in the compiler is problematic: over time it has
gained many responsibilities. Every source declaration, container type,
generic instantiation, and `@extern` has a `Decl`. The functions of
these `Decl`s are in some cases entirely disjoint.
After careful analysis, I determined that the two main responsibilities
of `Decl` are as follows:
* A `Decl` acts as the "subject" of semantic analysis at comptime. A
single unit of analysis is either a runtime function body, or a
`Decl`. It registers incremental dependencies, tracks analysis errors,
etc.
* A `Decl` acts as a "global variable": a pointer to it is consistent,
and it may be lowered to a specific symbol by the codegen backend.
This commit eliminates `Decl` and introduces new types to model these
responsibilities: `Cau` (Comptime Analysis Unit) and `Nav` (Named
Addressable Value).
Every source declaration, and every container type requiring resolution
(so *not* including `opaque`), has a `Cau`. For a source declaration,
this `Cau` performs the resolution of its value. (When #131 is
implemented, it is unsolved whether type and value resolution will share
a `Cau` or have two distinct `Cau`s.) For a type, this `Cau` is the
context in which type resolution occurs.
Every non-`comptime` source declaration, every generic instantiation,
and every distinct `extern` has a `Nav`. These are sent to codegen/link:
the backends by definition do not care about `Cau`s.
This commit has some minor technically-breaking changes surrounding
`usingnamespace`. I don't think they'll impact anyone, since the changes
are fixes around semantics which were previously inconsistent (the
behavior changed depending on hashmap iteration order!).
Aside from that, this changeset has no significant user-facing changes.
Instead, it is an internal refactor which makes it easier to correctly
model the responsibilities of different objects, particularly regarding
incremental compilation. The performance impact should be negligible,
but I will take measurements before merging this work into `master`.
Co-authored-by: Jacob Young <jacobly0@users.noreply.github.com>
Co-authored-by: Jakub Konka <kubkon@jakubkonka.com>
This allows the mutate mutex to only be locked during actual grows,
which are rare. For the lists that didn't previously have a mutex, this
change has little effect since grows are rare and there is zero
contention on a mutex that is only ever locked by one thread. This
change allows `extra` to be mutated without racing with a grow.
Primarily, this commit removes 2 fields from File, relying on the data
being stored in the `files` field, with the key as the path digest, and
the value as the struct decl corresponding to the File. This table is
serialized into the compiler state that survives between incremental
updates.
Meanwhile, the File struct remains ephemeral data that can be
reconstructed the first time it is needed by the compiler process, as
well as operated on by independent worker threads.
A key outcome of this commit is that there is now a stable index that
can be used to refer to a File. This will be needed when serializing
error messages to survive incremental compilation updates.
This change modifies `Zcu.ErrorMsg` to store a `Zcu.LazySrcLoc` rather
than a `Zcu.SrcLoc`. Everything else is dominoes.
The reason for this change is incremental compilation. If a failed
`AnalUnit` is up-to-date on an update, we want to re-use the old error
messages. However, the file containing the error location may have been
modified, and `SrcLoc` cannot survive such a modification. `LazySrcLoc`
is designed to be correct across incremental updates. Therefore, we
defer source location resolution until `Compilation` gathers the compile
errors into the `ErrorBundle`.
This commit reworks our representation of exported Decls and values in
Zcu to be memory-optimized and trivially serialized.
All exports are now stored in the `all_exports` array on `Zcu`. An
`AnalUnit` which performs an export (either through an `export`
annotation or by containing an analyzed `@export`) gains an entry into
`single_exports` if it performs only one export, or `multi_exports` if
it performs multiple.
We no longer store a persistent mapping from a `Decl`/value to all
exports of that entity; this state is not necessary for the majority of
the pipeline. Instead, we construct it in `Zcu.processExports`, just
before flush. This does not affect the algorithmic complexity of
`processExports`, since this function already iterates all exports in
the `Zcu`.
The elimination of `decl_exports` and `value_exports` led to a few
non-trivial backend changes. The LLVM backend has been wrangled into a
more reasonable state in general regarding exports and externs. The C
backend is currently disabled in this commit, because its support for
`export` was quite broken, and that was exposed by this work -- I'm
hoping @jacobly0 will be able to pick this up!
This patch is a pure rename plus only changing the file path in
`@import` sites, so it is expected to not create version control
conflicts, even when rebasing.
`LazySrcLoc` now stores a reference to the "base AST node" to which it
is relative. The previous tagged union is `LazySrcLoc.Offset`. To make
working with this structure convenient, `Sema.Block` contains a
convenience `src` method which takes an `Offset` and returns a
`LazySrcLoc`.
The "base node" of a source location is no longer given by a `Decl`, but
rather a `TrackedInst` representing either a `declaration`,
`struct_decl`, `union_decl`, `enum_decl`, or `opaque_decl`. This is a
more appropriate model, and removes an unnecessary responsibility from
`Decl` in preparation for the upcoming refactor which will split it into
`Nav` and `Cau`.
As a part of these `Decl` reworks, the `src_node` field is eliminated.
This change aids incremental compilation, and simplifies `Decl`. In some
cases -- particularly in backends -- the source location of a
declaration is desired. This was previously `Decl.srcLoc` and worked for
any `Decl`. Now, it is `Decl.navSrcLoc` in reference to the upcoming
refactor, since the set of `Decl`s this works for precisely corresponds
to what will in future become a `Nav` -- that is, source-level
declarations and generic function instantiations, but *not* type owner
Decls.
This commit introduces more tags to `LazySrcLoc.Offset` so as to
eliminate the concept of `error.NeededSourceLocation`. Now, `.unneeded`
should only be used to assert that an error path is unreachable. In the
future, uses of `.unneeded` can probably be replaced with `undefined`.
The `src_decl` field of `Sema.Block` no longer has a role in type
resolution. Its main remaining purpose is to handle namespacing of type
names. It will be eliminated entirely in a future commit to remove
another undue responsibility from `Decl`.
It is worth noting that in future, the `Zcu.SrcLoc` type should probably
be eliminated entirely in favour of storing `Zcu.LazySrcLoc` values.
This is because `Zcu.SrcLoc` is not valid across incremental updates,
and we want to be able to reuse error messages from previous updates
even if the source file in question changed. The error reporting logic
should instead simply resolve the location from the `LazySrcLoc` on the
fly.
This is in preparation for some upcoming changes to how we represent
source locations in the compiler. The bulk of the change here is dealing
with the removal of `src()` methods from `Zir` types.
This reverts commit a7de02e05216db9a04e438703ddf1b6b12f3fbef.
This did not implement the accepted proposal, and I did not sign off
on the changes. I would like a chance to review this, please.
As of Clang 18, calling memcpy() with a misaligned pointer trips UBSAN,
even if the length is zero. This unfortunately includes any call to
`@memcpy` when source or destination are undefined and the length is
zero.
This patch makes the C backend avoid calling memcpy when the length is
zero, thereby avoiding undefined behavior.
A zig1.wasm update will be needed in the llvm18 branch to activate this
code.
this patch renames ComptimeStringMap to StaticStringMap, makes it
accept only a single type parameter, and return a known struct type
instead of an anonymous struct. initial motivation for these changes
was to reduce the 'very long type names' issue described here
https://github.com/ziglang/zig/pull/19682.
this breaks the previous API. users will now need to write:
`const map = std.StaticStringMap(T).initComptime(kvs_list);`
* move `kvs_list` param from type param to an `initComptime()` param
* new public methods
* `keys()`, `values()` helpers
* `init(allocator)`, `deinit(allocator)` for runtime data
* `getLongestPrefix(str)`, `getLongestPrefixIndex(str)` - i'm not sure
these belong but have left in for now incase they are deemed useful
* performance notes:
* i posted some benchmarking results here:
https://github.com/travisstaloch/comptime-string-map-revised/issues/1
* i noticed a speedup reducing the size of the struct from 48 to 32
bytes and thus use u32s instead of usize for all length fields
* i noticed speedup storing KVs as a struct of arrays
* latest benchmark shows these wall_time improvements for
debug/safe/small/fast builds: -6.6% / -10.2% / -19.1% / -8.9%. full
output in link above.
We've got a big one here! This commit reworks how we represent pointers
in the InternPool, and rewrites the logic for loading and storing from
them at comptime.
Firstly, the pointer representation. Previously, pointers were
represented in a highly structured manner: pointers to fields, array
elements, etc, were explicitly represented. This works well for simple
cases, but is quite difficult to handle in the cases of unusual
reinterpretations, pointer casts, offsets, etc. Therefore, pointers are
now represented in a more "flat" manner. For types without well-defined
layouts -- such as comptime-only types, automatic-layout aggregates, and
so on -- we still use this "hierarchical" structure. However, for types
with well-defined layouts, we use a byte offset associated with the
pointer. This allows the comptime pointer access logic to deal with
reinterpreted pointers far more gracefully, because the "base address"
of a pointer -- for instance a `field` -- is a single value which
pointer accesses cannot exceed since the parent has undefined layout.
This strategy is also more useful to most backends -- see the updated
logic in `codegen.zig` and `codegen/llvm.zig`. For backends which do
prefer a chain of field and elements accesses for lowering pointer
values, such as SPIR-V, there is a helpful function in `Value` which
creates a strategy to derive a pointer value using ideally only field
and element accesses. This is actually more correct than the previous
logic, since it correctly handles pointer casts which, after the dust
has settled, end up referring exactly to an aggregate field or array
element.
In terms of the pointer access code, it has been rewritten from the
ground up. The old logic had become rather a mess of special cases being
added whenever bugs were hit, and was still riddled with bugs. The new
logic was written to handle the "difficult" cases correctly, the most
notable of which is restructuring of a comptime-only array (for
instance, converting a `[3][2]comptime_int` to a `[2][3]comptime_int`.
Currently, the logic for loading and storing work somewhat differently,
but a future change will likely improve the loading logic to bring it
more in line with the store strategy. As far as I can tell, the rewrite
has fixed all bugs exposed by #19414.
As a part of this, the comptime bitcast logic has also been rewritten.
Previously, bitcasts simply worked by serializing the entire value into
an in-memory buffer, then deserializing it. This strategy has two key
weaknesses: pointers, and undefined values. Representations of these
values at comptime cannot be easily serialized/deserialized whilst
preserving data, which means many bitcasts would become runtime-known if
pointers were involved, or would turn `undefined` values into `0xAA`.
The new logic works by "flattening" the datastructure to be cast into a
sequence of bit-packed atomic values, and then "unflattening" it; using
serialization when necessary, but with special handling for `undefined`
values and for pointers which align in virtual memory. The resulting
code is definitely slower -- more on this later -- but it is correct.
The pointer access and bitcast logic required some helper functions and
types which are not generally useful elsewhere, so I opted to split them
into separate files `Sema/comptime_ptr_access.zig` and
`Sema/bitcast.zig`, with simple re-exports in `Sema.zig` for their small
public APIs.
Whilst working on this branch, I caught various unrelated bugs with
transitive Sema errors, and with the handling of `undefined` values.
These bugs have been fixed, and corresponding behavior test added.
In terms of performance, I do anticipate that this commit will regress
performance somewhat, because the new pointer access and bitcast logic
is necessarily more complex. I have not yet taken performance
measurements, but will do shortly, and post the results in this PR. If
the performance regression is severe, I will do work to to optimize the
new logic before merge.
Resolves: #19452Resolves: #19460
Legacy anon decls now have three uses:
* Type owner decls
* Function owner decls
* `@export` and `@extern`
Therefore, there are no longer any cases where we wish to explicitly
omit legacy anon decls from the binary. This means we can remove the
concept of an "alive" vs "dead" `Decl`, which also allows us to remove
the separate `anon_work_queue` in `Compilation`.
Good riddance!
Most of these changes are trivial. There's a fix for a minor bug this
exposed in `Value.readFromPackedMemory`, but aside from that, it's all
just things like changing `intern` calls to `toIntern`.
`Decl` can no longer store un-interned values, so this field is now
unnecessary. The type can instead be fetched with the new `typeOf`
helper method, which just gets the type of the Decl's `Value`.
This commit changes how we represent comptime-mutable memory
(`comptime var`) in the compiler in order to implement the intended
behavior that references to such memory can only exist at comptime.
It does *not* clean up the representation of mutable values, improve the
representation of comptime-known pointers, or fix the many bugs in the
comptime pointer access code. These will be future enhancements.
Comptime memory lives for the duration of a single Sema, and is not
permitted to escape that one analysis, either by becoming runtime-known
or by becoming comptime-known to other analyses. These restrictions mean
that we can represent comptime allocations not via Decl, but with state
local to Sema - specifically, the new `Sema.comptime_allocs` field. All
comptime-mutable allocations, as well as any comptime-known const allocs
containing references to such memory, live in here. This allows for
relatively fast checking of whether a value references any
comptime-mtuable memory, since we need only traverse values up to
pointers: pointers to Decls can never reference comptime-mutable memory,
and pointers into `Sema.comptime_allocs` always do.
This change exposed some faulty pointer access logic in `Value.zig`.
I've fixed the important cases, but there are some TODOs I've put in
which are definitely possible to hit with sufficiently esoteric code. I
plan to resolve these by auditing all direct accesses to pointers (most
of them ought to use Sema to perform the pointer access!), but for now
this is sufficient for all realistic code and to get tests passing.
This change eliminates `Zcu.tmp_hack_arena`, instead using the Sema
arena for comptime memory mutations, which is possible since comptime
memory is now local to the current Sema.
This change should allow `Decl` to store only an `InternPool.Index`
rather than a full-blown `ty: Type, val: Value`. This commit does not
perform this refactor.
This fixes an issue with boostrapping the compiler using MSVC. There is a CircularBuffer with
an array of length 65536 initialized to undefined, and because the undefined path of `renderValue`
was using `StringLiteral` to render this, the resulting zig2.c would fail to compile using MSVC.
The solution was to move the already-existing array initializer path (used in the non-undefined path)
into StringLiteral, and make StringLiteral aware of the total length so it could decide between which
style of initialization to use. We prefer to use string literals if we can, as this results in the least
amount of emitted C source.
A pointer type already has an alignment, so this information does not
need to be duplicated on the function type. This already has precedence
with addrspace which is already disallowed on function types for this
reason. Also fixes `@TypeOf(&func)` to have the correct addrspace and
alignment.
Namespace types (`struct`, `enum`, `union`, `opaque`) do not use
structural equality - equivalence is based on their Decl index (and soon
will change to AST node + captures). However, we previously stored all
other information in the corresponding `InternPool.Key` anyway. For
logical consistency, it makes sense to have the key only be the true key
(that is, the Decl index) and to load all other data through another
function. This introduces those functions, by the name of
`loadStructType` etc. It's a big diff, but most of it is no-brainer
changes.
In future, it might be nice to eliminate a bunch of the loaded state in
favour of accessor functions on the `LoadedXyzType` types (like how we
have `LoadedUnionType.size()`), but that can be explored at a later
date.
This commit eliminates the `dbg_block_{begin,end}` instructions from
both ZIR and AIR. Instead, lexical scoping of `dbg_var_{ptr,val}`
instructions is decided based on the AIR block they exist within. This
is a much more robust system, and also results in a huge drop in ZIR
bytes - around 7% for Sema.zig.
This required some enhancements to Sema to prevent elision of blocks
when they are required for debug variable scoping. This can be observed
by looking at the AIR for the following simple test program with and
without `-fstrip`:
```zig
export fn f() void {
{
var a: u32 = 0;
_ = &a;
}
{
var a: u32 = 0;
_ = &a;
}
}
```
When `-fstrip` is passed, no AIR blocks are generated. When `-fno-strip`
is passed, the ZIR blocks are lowered to true AIR blocks to give correct
lexical scoping to the debug vars.
The changes here incidentally reolve #19060. A corresponding behavior
test has been added.
Resolves: #19060
