KangarooTwelve is a family of two fast and secure extendable-output
functions (XOFs): KT128 and KT256. These functions generalize
traditional hash functions by allowing arbitrary output lengths.
KangarooTwelve was designed by SHA-3 authors. It aims to deliver
higher performance than the SHA-3 and SHAKE functions defined in
FIPS 202, while preserving their flexibility and core security
principles.
On high-end platforms, it can take advantage of parallelism,
whether through multiple CPU cores or SIMD instructions.
As modern SHA-3 constructions, KT128 and KT256 can serve as
general-purpose hash functions and can be used, for example, in
key-derivation, and with arbitrarily large inputs.
RFC9861: https://datatracker.ietf.org/doc/rfc9861/
It seems to me this was simply forgotten.
Or there is some reason I don't know why this code doesn't work for `comptime_float`.
For a more comprehensive fix, https://github.com/ziglang/zig/pull/24057 is the place to look.
This method is called on an identifier token, so let's rename the parameter to make this clear.
This is also how it's named on most of the caller's sides.
This also unifies the rename implementations, since previously `posix.renameW` used `MoveFileEx` while `posix.renameatW` used `NtOpenFile`/`NtSetInformationFile`. This, in turn, allows the `MoveFileEx` bindings to be deleted as `posix.renameW` was the only usage.
This functionality -- if it's actually needed -- can be reintroduced through
some other mechanism. An ABI is clearly not the right way to represent it.
closes#25918
This reverts commit b34a13da38f60737477e62dd0acd1a7a6b6be0a0.
This is not the right fix; rather, the C backend needs to translate all of ccr,
icc, and xcc to just icc.
https://github.com/ziglang/zig/issues/25471
This is not the only test that aborts like this, nor does it happen only on
FreeBSD, but it happens to be disproportionally disruptive on FreeBSD in
particular.
The new builtins are:
* `@EnumLiteral`
* `@Int`
* `@Fn`
* `@Pointer`
* `@Tuple`
* `@Enum`
* `@Union`
* `@Struct`
Their usage is documented in the language reference.
There is no `@Array` because arrays can be created like this:
if (sentinel) |s| [n:s]T else [n]T
There is also no `@Float`. Instead, `std.meta.Float` can serve this use
case if necessary.
There is no `@ErrorSet` and intentionally no way to achieve this.
Likewise, there is intentionally no way to reify tuples with comptime
fields, or function types with comptime parameters. These decisions
simplify the Zig language specification, and moreover make Zig code more
readable by discouraging overly complex metaprogramming.
Co-authored-by: Ali Cheraghi <alichraghi@proton.me>
Resolves: #10710
This reverts commit d54fbc01234688b37a48b29fee499529a500ccf5.
Since all incremental tests are flaky on Windows, this is reinstated and
all test-incremental tests will be skipped on Windows until the
flakiness is resolved.
Closes#26003
If a Reader implementation implements `stream` by ignoring the Writer, writing directly to its internal buffer, and returning 0, then `defaultDiscard` would not update `seek` and also return 0, which is incorrect and can cause `discardShort` to violate the contract of `VTable.discard` by calling into `vtable.discard` with a non-empty buffer.
This commit fixes the problem by advancing seek up to the limit after the stream call. This logic could likely be somewhat simplified in the future depending on how #25170 is resolved.
This commit flips usage of PathType.isSep from requiring the caller to convert to native to assuming the input is LE encoded, which is a breaking change. This makes usage a bit nicer, though, and moves the endian conversion work from runtime to comptime.
while still preserving the guarantee about async() being assigned a unit
of concurrency (or immediately running the task), this change:
* retains the error from calling getCpuCount()
* spawns all threads in detached mode, using WaitGroup to join them
* treats all workers the same regardless of whether they are processing
concurrent or async tasks. one thread pool does all the work, while
respecting async and concurrent limits.
This is a reimplementation of Io.Threaded that fixes the issues
highlighted in the recent Zulip discussion. It's poorly tested but it
does successfully run to completion the litmust test example that I
offered in the discussion.
This implementation has the following key design decisions:
- `t.cpu_count` is used as the threadpool size.
- `t.concurrency_limit` is used as the maximum number of
"burst, one-shot" threads that can be spawned by `io.concurrent` past
`t.cpu_count`.
- `t.available_thread_count` is the number of threads in the pool that
is not currently busy with work (the bookkeeping happens in the worker
function).
- `t.one_shot_thread_count` is the number of active threads that were
spawned by `io.concurrent` past `t.cpu_count`.
In this implementation:
- `io.async` first tries to decrement `t.available_thread_count`. If
there are no threads available, it tries to spawn a new one if possible,
otherwise it runs the task immediately.
- `io.concurrent` first tries to use a thread in the pool same as
`io.async`, but on failure (no available threads and pool size limit
reached) it tries to spawn a new one-shot thread. One shot threads
run a different main function that just executes one task, decrements
the number of active one shot threads, and then exits.
A relevant future improvement is to have one-shot threads stay on for a
few seconds (and potentially pick up a new task) to amortize spawning
costs.
I would like a chance to review this before it lands, please. Feel free
to submit the work again without changes and I will make review
comments.
In the meantime, these reverts avoid intermittent CI failures, and
remove bad patterns from occurring in the standard library that other
users might copy.
Revert "std.crypto: improve KT documentation, use key_length for B3 key length (#25807)"
This reverts commit 4b593a6c24797484e68a668818736b0f6a8d81a2.
Revert "crypto - threaded K12: separate context computation from thread spawning (#25793)"
This reverts commit ee4df4ad3edad160fb737a1935cd86bc2f9cfbbe.
Revert "crypto.kt128: when using incremental hashing, use SIMD when possible (#25783)"
This reverts commit bf9082518c32ce7d53d011777bf8d8056472cbf9.
Revert "Add std.crypto.hash.sha3.{KT128,KT256} - RFC 9861. (#25593)"
This reverts commit 95c76b1b4aa7302966281c6b9b7f6cadea3cf7a6.
When calling QueryObjectName, NT namespaced paths can be returned. This
change appropriately strips the prefix to turn it into an absolute path.
(The above behaviour was observed at least in Wine so far)
Co-authored-by: Ryan Liptak <squeek502@hotmail.com>
Previously, fs.path handled a few of the Windows path types, but not all of them, and only a few of them correctly/consistently. This commit aims to make `std.fs.path` correct and consistent in handling all possible Win32 path types.
This commit also slightly nudges the codebase towards a separation of Win32 paths and NT paths, as NT paths are not actually distinguishable from Win32 paths from looking at their contents alone (i.e. `\Device\Foo` could be an NT path or a Win32 rooted path, no way to tell without external context). This commit formalizes `std.fs.path` being fully concerned with Win32 paths, and having no special detection/handling of NT paths.
Resources on Windows path types, and Win32 vs NT paths:
- https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html
- https://chrisdenton.github.io/omnipath/Overview.html
- https://learn.microsoft.com/en-us/windows/win32/fileio/naming-a-file
API additions/changes/deprecations
- `std.os.windows.getWin32PathType` was added (it is analogous to `RtlDetermineDosPathNameType_U`), while `std.os.windows.getNamespacePrefix` and `std.os.windows.getUnprefixedPathType` were deleted. `getWin32PathType` forms the basis on which the updated `std.fs.path` functions operate.
- `std.fs.path.parsePath`, `std.fs.path.parsePathPosix`, and `std.fs.path.parsePathWindows` were added, while `std.fs.path.windowsParsePath` was deprecated. The new `parsePath` functions provide the "root" and the "kind" of a path, which is platform-specific. The now-deprecated `windowsParsePath` did not handle all possible path types, while the new `parsePathWindows` does.
- `std.fs.path.diskDesignator` has been deprecated in favor of `std.fs.path.parsePath`, and same deal with `diskDesignatorWindows` -> `parsePathWindows`
- `relativeWindows` is now a compile error when *not* targeting Windows, while `relativePosix` is now a compile error when targeting Windows. This is because those functions read/use the CWD path which will behave improperly when used from a system with different path semantics (e.g. calling `relativePosix` from a Windows system with a CWD like `C:\foo\bar` will give you a bogus result since that'd be treated as a single relative component when using POSIX semantics). This also allows `relativeWindows` to use Windows-specific APIs for getting the CWD and environment variables to cut down on allocations.
- `componentIterator`/`ComponentIterator.init` have been made infallible. These functions used to be able to error on UNC paths with an empty server component, and on paths that were assumed to be NT paths, but now:
+ We follow the lead of `RtlDetermineDosPathNameType_U`/`RtlGetFullPathName_U` in how it treats a UNC path with an empty server name (e.g. `\\\share`) and allow it, even if it'll be invalid at the time of usage
+ Now that `std.fs.path` assumes paths are Win32 paths and not NT paths, we don't have to worry about NT paths
Behavior changes
- `std.fs.path` generally: any combinations of mixed path separators for UNC paths are universally supported, e.g. `\/server/share`, `/\server\share`, `/\server/\\//share` are all seen as equivalent UNC paths
- `resolveWindows` handles all path types more appropriately/consistently.
+ `//` and `//foo` used to be treated as a relative path, but are now seen as UNC paths
+ If a rooted/drive-relative path cannot be resolved against anything more definite, the result will remain a rooted/drive-relative path.
+ I've created [a script to generate the results of a huge number of permutations of different path types](https://gist.github.com/squeek502/9eba7f19cad0d0d970ccafbc30f463bf) (the result of running the script is also included for anyone that'd like to vet the behavior).
- `dirnameWindows` now treats the drive-relative root as the dirname of a drive-relative path with a component, e.g. `dirname("C:foo")` is now `C:`, whereas before it would return null. `dirnameWindows` also handles local device paths appropriately now.
- `basenameWindows` now handles all path types more appropriately. The most notable change here is `//a` being treated as a partial UNC path now and therefore `basename` will return `""` for it, whereas before it would return `"a"`
- `relativeWindows` will now do its best to resolve against the most appropriate CWD for each path, e.g. relative for `D:foo` will look at the CWD to check if the drive letter matches, and if not, look at the special environment variable `=D:` to get the shell-defined CWD for that drive, and if that doesn't exist, then it'll resolve against `D:\`.
Implementation details
- `resolveWindows` previously looped through the paths twice to build up the relevant info before doing the actual resolution. Now, `resolveWindows` iterates backwards once and keeps track of which paths are actually relevant using a bit set, which also allows it to break from the loop when it's no longer possible for earlier paths to matter.
- A standalone test was added to test parts of `relativeWindows` since the CWD resolution logic depends on CWD information from the PEB and environment variables
Edge cases worth noting
- A strange piece of trivia that I found out while working on this is that it's technically possible to have a drive letter that it outside the intended A-Z range, or even outside the ASCII range entirely. Since we deal with both WTF-8 and WTF-16 paths, `path[0]`/`path[1]`/`path[2]` will not always refer to the same bits of information, so to get consistent behavior, some decision about how to deal with this edge case had to be made. I've made the choice to conform with how `RtlDetermineDosPathNameType_U` works, i.e. treat the first WTF-16 code unit as the drive letter. This means that when working with WTF-8, checking for drive-relative/drive-absolute paths is a bit more complicated. For more details, see the lengthy comment in `std.os.windows.getWin32PathType`
- `relativeWindows` will now almost always be able to return either a fully-qualified absolute path or a relative path, but there's one scenario where it may return a rooted path: when the CWD gotten from the PEB is not a drive-absolute or UNC path (if that's actually feasible/possible?). An alternative approach to this scenario might be to resolve against the `HOMEDRIVE` env var if available, and/or default to `C:\` as a last resort in order to guarantee the result of `relative` is never a rooted path.
- Partial UNC paths (e.g. `\\server` instead of `\\server\share`) are a bit awkward to handle, generally. Not entirely sure how best to handle them, so there may need to be another pass in the future to iron out any issues that arise. As of now the behavior is:
+ For `relative`, any part of a UNC disk designator is treated as the "root" and therefore isn't applicable for relative paths, e.g. calling `relative` with `\\server` and `\\server\share` will result in `\\server\share` rather than just `share` and if `relative` is called with `\\server\foo` and `\\server\bar` the result will be `\\server\bar` rather than `..\bar`
+ For `resolve`, any part of a UNC disk designator is also treated as the "root", but relative and rooted paths are still elligable for filling in missing portions of the disk designator, e.g. `resolve` with `\\server` and `foo` or `\foo` will result in `\\server\foo`
Fixes#25703Closes#25702
This is relevant to PIEs, which are notably enabled by default on macOS.
The build system needs to only see virtual addresses, that is, those
which do not have the slide applied; but the fuzzer itself naturally
sees relocated addresses (i.e. with the slide applied). We just need to
subtract the slide when we communicate addresses to the build system.
Like ELF, we now have `std.debug.MachOFile` for the host-independent
parts, and `std.debug.SelfInfo.MachO` for logic requiring the file to
correspond to the running program.
When reporting a compile error, we would load the new file, but assume
we could apply old AST/token indices (etc) to it, potentially causing
crashes. Instead, if the file stat has changed since it was loaded, just
emit an error that the file was modified mid-update.
This has no business being here. Tests for our compiler-rt routines should be in
compiler-rt, and tests for our C ABI compliance should be in `test-c-abi`.
Little-endian is what `std.zig.Server` expects, but the old logic just
send the raw bytes of the struct, so sent in native endian (causing a
crash on big-endian targets).
The big-endian logic here was simply incorrect. Luckily, it was also
overcomplicated; after calling `Value.writeToPackedMemory`, there is a
method on `std.math.big.int.Mutable` which just does the correct
endianness load for us.
Integers with padding bits on big-endian targets cannot quite be bitcast
with a trivial memcpy, because the padding bits (which are zext or sext)
are the most-significant, so are at the *lowest* addresses. So to
bitcast to something which doesn't have padding bits, we need to offset
past the padding.
The logic I've added here definitely doesn't handle all possibilities
correctly; I think that would actually be quite complicated. However, it
handles a common case, and so prevents the Zig compiler itself from
being miscompiled on big-endian targets (hence fixing a bootstrapping
problem on big-endian).
- Affects the following functions:
+ `std.fs.Dir.readLinkW`
+ `std.os.windows.ReadLink`
+ `std.os.windows.ntToWin32Namespace`
+ `std.posix.readlinkW`
+ `std.posix.readlinkatW`
Each of these functions (except `ntToWin32Namespace`) took WTF-16 as input and would output WTF-8, which makes optimal buffer re-use difficult at callsites and could force unnecessary WTF-16 <-> WTF-8 conversion during an intermediate step.
The functions have been updated to output WTF-16, and also allow for the path and the output to re-use the same buffer (i.e. in-place modification), which can reduce the stack usage at callsites. For example, all of `std.fs.Dir.readLink`/`readLinkZ`/`std.posix.readlink`/`readlinkZ`/`readlinkat`/`readlinkatZ` have had their stack usage reduced by one PathSpace struct (64 KiB) when targeting Windows.
The new `ntToWin32Namespace` takes an output buffer and returns a slice from that instead of returning a PathSpace, which is necessary to make the above possible.
The reasoning in the comment deleted by this commit no longer applies, since that same benefit can be obtained by using OpenFile with `.filter = .any`.
Also removes a stray debug.print
We were already using this for `stage1/zig.h`, but `stage1/zig1.wasm`
was being modified directly by the `wasm-opt` command. That's a bad idea
because it forces the build system to assume that `wasm-opt` has side
effects, so it is re-run every time you run `zig build update-zig1`,
i.e. it does not interact with the cache system correctly. It is much
better to create non-side-effecting `Run` steps (using `addOutput*Arg`)
where possible so that the build system has a more correct understanding
of the step graph.
A new `Legalize.Feature` tag is introduced for each float bit width
(16/32/64/80/128). When e.g. `soft_f16` is enabled, all arithmetic and
comparison operations on `f16` are converted to calls to the appropriate
compiler_rt function using the new AIR tag `.legalize_compiler_rt_call`.
This includes casts where the source *or* target type is `f16`, or
integer<=>float conversions to or from `f16`. Occasionally, operations
are legalized to blocks because there is extra code required; for
instance, legalizing `@floatFromInt` where the integer type is larger
than 64 bits requires calling an arbitrary-width integer conversion
function which accepts a pointer to the integer, so we need to use
`alloc` to create such a pointer, and store the integer there (after
possibly zero-extending or sign-extending it).
No backend currently uses these new legalizations (and as such, no
backend currently needs to implement `.legalize_compiler_rt_call`).
However, for testing purposes, I tried modifying the self-hosted x86_64
backend to enable all of the soft-float features (and implement the AIR
instruction). This modified backend was able to pass all of the behavior
tests (except for one `@mod` test where the LLVM backend has a bug
resulting in incorrect compiler-rt behavior!), including the tests
specific to the self-hosted x86_64 backend.
`f16` and `f80` legalizations are likely of particular interest to
backend developers, because most architectures do not have instructions
to operate on these types. However, enabling *all* of these legalization
passes can be useful when developing a new backend to hit the ground
running and pass a good amount of tests more easily.
Simplifies the logic, clarifies the comment, and fixes a minor bug,
which is that we exported the Windows ABI name *instead* of the standard
compiler-rt name, but it's meant to be exported *in addition* to the
standard name (this is LLVM's behavior and it is more useful).
The build runner was previously forcing child processes to have their
stderr colorization match the build runner by setting `CLICOLOR_FORCE`
or `NO_COLOR`. This is a nice idea in some cases---for instance a simple
`Run` step which we just expect to exit with code 0 and whose stderr is
not being programmatically inspected---but is a bad idea in others, for
instance if there is a check on stderr or if stderr is captured, in
which case forcing color on the child could cause checks to fail.
Instead, this commit adds a field to `std.Build.Step.Run` which
specifies a behavior for the build runner to employ in terms of
assigning the `CLICOLOR_FORCE` and `NO_COLOR` environment variables. The
default behavior is to set `CLICOLOR_FORCE` if the build runner's output
is colorized and the step's stderr is not captured, and to set
`NO_COLOR` otherwise. Alternatively, colors can be always enabled,
always disabled, always match the build runner, or the environment
variables can be left untouched so they can be manually controlled
through `env_map`.
Notably, this fixes a failure when running `zig build test-cli` in a
TTY (or with colors explicitly enabled). GitHub CI hadn't caught this
because it does not request color, but Codeberg CI now does, and we were
seeing a failure in the `zig init` test because the actual output had
color escape codes in it due to 6d280dc.
Apple's own headers and tbd files prefer to think of Mac Catalyst as a distinct
OS target. Earlier, when DriverKit support was added to LLVM, it was represented
a distinct OS. So why Apple decided to only represent Mac Catalyst as an ABI in
the target triple is beyond me. But this isn't the first time they've ignored
established target triple norms (see: armv7k and aarch64_32) and it probably
won't be the last.
While doing this, I also audited all Darwin OS prongs throughout the codebase
and made sure they cover all the tags.
There is approximately zero chance of the Zig team ever spending any effort on
supporting Cygwin; the MSVC and MinGW-w64 ABIs are superior in every way that
matters, and not least because they lead to binaries that just run natively on
Windows without needing a POSIX emulation environment installed.
It's easy to do FP unwinding from a CPU context: you just report the
captured ip/pc value first, and then unwind from the captured fp value.
All this really needed was a couple of new functions on the
`std.debug.cpu_context` implementations so that we don't need to rely on
`std.debug.Dwarf` to access the captured registers.
Resolves: #25576
The changes to `codegen.c` are blatant hacks, but the problem they work
around isn't a regression: it's an existing miscompilation. This branch
happened to *expose* that miscompilation in more cases by changing how
an incorrect result is *used*.
It turns out we did use these in the C backend. However, it's really
just as easy, if not easier, to replicate the logic directly in C.
Synchronizes stage1/zig.h to make sure the bootstrap doesn't depend on
these functions either. The actual zig1 tarball is unmodified because
regenerating it is unnecessary in this instance.
I had tried unrolling the loops to avoid requiring the
`vector_store_elem` instruction, but it's arguably a problem to generate
O(N) code for an operation on `@Vector(N, T)`. In addition, that
lowering emitted a lot of `.aggregate_init` instructions, which is
itself a quite difficult operation to codegen.
This requires reintroducing runtime vector indexing internally. However,
I've put it in a couple of instructions which are intended only for use
by `Air.Legalize`, named `legalize_vec_elem_val` (like `array_elem_val`,
but for indexing a vector with a runtime-known index) and
`legalize_vec_store_elem` (like the old `vector_store_elem`
instruction). These are explicitly documented as *not* being emitted by
Sema, so need only be implemented by backends if they actually use an
`Air.Legalize.Feature` which emits them (otherwise they can be marked as
`unreachable`).
`__addosi4`, `__addodi4`, `__addoti4`, `__subosi4`, `__subodi4`, and
`__suboti4` were all functions which we invented for no apparent reason.
Neither LLVM, nor GCC, nor the Zig compiler use these functions. It
appears the functions were created in a kind of misunderstanding of an
old language proposal; see https://github.com/ziglang/zig/pull/10824.
There is no benefit to these functions existing; if a Zig compiler
backend needs this operation, it is trivial to implement, and *far*
simpler than calling a compiler-rt routine. Therefore, this commit
deletes them. A small amount of that code was used by other parts of
compiler-rt; the logic is trivial so has just been inlined where needed.
I also chose to quickly implement `__addvdi3` (a standard function)
because it is trivial and we already implement the `sub` parallel.
I started this diff trying to remove a little dead code from the C
backend, but ended up finding a bunch of dead code sprinkled all over
the place:
* `packed` handling in the C backend which was made dead by `Legalize`
* Representation of pointers to runtime-known vector indices
* Handling for the `vector_store_elem` AIR instruction (now removed)
* Old tuple handling from when they used the InternPool repr of structs
* Straightforward unused functions
* TODOs in the LLVM backend for features which Zig just does not support
The main goal of this change was to avoid emitting the
`vector_store_elem` AIR tag, because this represents an operation which
Zig no longer supports (and hence Sema no longer emits) as of 010d9a6
(because runtime vector indices are now forbidden). Backends should not
need to lower this operation, so I rewrote the legalizations which
emitted it (scalarizations of vector operations) to instead unroll the
loop and hence emit comptime-known vector indices.
In doing this, I actually reworked those legalizations to use a
different strategy; instead of using an `alloc` and storing to
individual vector elements, the vector is constructed by-val, for
instance by performing the scalar operation on all elements and passing
them to an `aggregate_init`. This is vastly simpler to implement in
Legalize, conceptually simpler, and doesn't severely pessimise memory
usage, because a non-optimizing backend will store the full vector on
the stack either way.
Given the above rationale, I also ended up reworking several other
legalizations to use simpler lowerings. The legalizations in question
were bitcast scalarization, `struct_field_val` of `packed struct`s
(where we just bitcast to an integer and perform the appropriate
shift/trunc sequence), and `aggregate_init` of a `packed struct` (also
implemented in terms of integer bitwise operations with bitcasts to and
from the actual types). This hugely simplified some parts of `Legalize`.
So, `Legalize` is now much simpler, and the `vector_store_elem`
instruction is no longer emitted by any part of the compiler so can be
removed in a future commit.
This fixes package fetching on Windows.
Previously, `Async/GroupClosure` allocations were only aligned for the
closure struct type, which resulted in panics when `context_alignment`
(or `result_alignment` for that matter) had a greater alignment.
`Clock.real` being defined to return timestamps relative to an
implementation-specific epoch means that there's currently no way for
the user to translate returned timestamps to actual calendar dates
without digging into implementation details of any particular `Io`
implementation. Redefining it to return timestamps relative to
1970-01-01T00:00:00Z fixes this problem.
There are other ways to solve this, such as adding a new vtable function
for returning the implementation-specific epoch, but in terms of
complexity this redefinition is by far the simplest solution and only
amounts to a simple 96-bit integer addition's worth of overhead on OSes
like Windows that use non-POSIX/Unix epochs.
ML-DSA is a post-quantum signature scheme that was recently
standardized by NIST.
Keys and signatures are pretty large, not making it a drop-in
replacement for classical signature schemes.
But if you are shipping keys that may still be used in 10 years
or whenever large quantum computers able to break ECC arrive,
it that ever happens, and you don't have the ability to replace
these keys, ML-DSA is for you.
Performance is great, verification is faster than Ed25519 / ECDSA.
I tried manual vectorization, but it wasn't worth it, the compiler
does at good job at auto-vectorization already.
This configuration hasn't had much work put into it yet, so is all but
guaranteed to miscompile or crash. Since users are starting to try out
`-fincremental`, and LLVM is still the default backend in many cases,
it's worth having this warning to avoid bug reports like
https://github.com/ziglang/zig/issues/25873.
To match the new default implementation. In fact, I implemented this by
simply dispatching *to* the default implementation after the debug log
guard; no need to complicate things!
This change removes the ref_start_index from the possible enum values of
Index and OptionalIndex. It is not really a index, but a constant that
tells the offset of static Refs, so lets move it where such constant
belongs i.e. to the Ref.
Since the child process is spawned with the tmp directory as its CWD, the child process opens it without DELETE access. On error, the child process would still be alive while the tmp directory is attempting to be deleted, so it would fail with `.SHARING_VIOLATION => return error.FileBusy`.
Fixes arguably the least important part of #22510, since it's only the directory itself that would fail to get deleted, all the files inside would get deleted just fine.
It was not obvious that the KT128/KT256 customization string can be
used to set a key, or what it was designed to be used for at all.
Also properly use key_length and not digest_length for the BLAKE3
key length (no practical changes as they are both 32, but that was
confusing).
Remove unneeded simd_degree copies by the way, and that doesn't need
to be in the public interface.
This seems to work around a very puzzling miscompilation first
present in LLVM 21.x. We already unconditionally add these
clobbers to inline assembly that came from the source, the
valgrind requests should also contain them.
If we use `undefined`, then `netReceive` can `@intCast` the
control slice len to msghdr controllen, which is sometimes `u32`,
even on 64-bit platforms.
`init` just avoids this entirely by setting `control` to an empty
slice rather than undefined.
68d2f68ed introduced special handling for StructInit fields
containing multiline strings to prevent inserting whitespace after =.
However, this logic didn't handle cases without a trailing comma,
which resulted in unwanted trailing whitespace.
The subsystem detection was flaky and often incorrect and was not
actually needed by the compiler or standard library. The actual
subsystem won't be known until at link time, so it doesn't make
sense to try to determine it at compile time.
* threaded K12: separate context computation from thread spawning
Compute all contexts and store them in a pre-allocated array,
then spawn threads using the pre-computed contexts.
This ensures each context is fully materialized in memory with the
correct values before any thread tries to access it.
* kt128: unroll the permutation rounds only twice
This appears to deliver the best performance thanks to improved cache
utilization, and it’s consistent with what we already do for SHA3.
KT128 and KT256 are fast, secure cryptographic hash functions based on Keccak (SHA-3).
They can be seen as the modern version of SHA-3, and evolution of SHAKE, with better performance.
After the SHA-3 competition, the Keccak team proposed these variants in 2016, and the constructions underwent 8 years of public scrutiny before being standardized in October 2025 as RFC 9861.
They uses a tree-hashing mode on top of TurboSHAKE, providing both high security and excellent performance, especially on large inputs.
They support arbitrary-length output and optional customization strings.
Hashing of very large inputs can be done using multiple threads, for high throughput.
KT128 provides 128-bit security strength, equivalent to AES-128 and SHAKE128, which is sufficient for virtually all applications.
KT256 provides 256-bit security strength, equivalent to SHA-512. For virtually all applications, KT128 is enough (equivalent to SHA-256 or BLAKE3).
For small inputs, TurboSHAKE128 and TurboSHAKE256 (which KT128 and KT256 are based on) can be used instead as they have less overhead.
Also remove the example implementation from the file doc comment; it's
better to just link to `defaultLog` as an example, since this avoids
writing the example implementation twice and prevents the example from
bitrotting.
`std.Io.tty.Config.detect` may be an expensive check (e.g. involving
syscalls), and doing it every time we need to print isn't really
necessary; under normal usage, we can compute the value once and cache
it for the whole program's execution. Since anyone outputting to stderr
may reasonably want this information (in fact they are very likely to),
it makes sense to cache it and return it from `lockStderrWriter`. Call
sites who do not need it will experience no significant overhead, and
can just ignore the TTY config with a `const w, _` destructure.
As with Solaris (dba1bf935390ddb0184a4dc72245454de6c06fd2), we have no way to
actually audit contributions for these OSs. IBM also makes it even harder than
Oracle to actually obtain these OSs.
closes#23695closes#23694closes#3655closes#23693
restores code closer to master branch in hopes of avoiding a regression
that was introduced when this was based on openSelfExe rather than
GetModuleFileNameExW.
The compiler crashed when we tried to call a function pointer for which
the type signature does not match any function body or function import
in the entire wasm executable, because there is no way to create a
reference to a function without it being in the function table or import
table. Solution is to make this instruction lower to unreachable.
let's handle this in a follow-up change. implementation needs to use
ConvertInterfaceNameToLuidW and the additional dependency on
Iphlpapi.dll poses some challenges.
Now std.Io.Threaded can return error.ConcurrencyUnavailable rather than
asserting. This is handy for logic that wants to try a concurrent
implementation but then fall back to a synchronous one.
Microsoft documentation says "The if_nametoindex function is implemented
for portability of applications with Unix environments, but the
ConvertInterface functions are preferred."
This was also the only dependency on iphlpapi.
The previous implementation would eagerly attempt TCP connection upon
receiving a DNS reply, but it would still wait for all the DNS results
before returning from the function.
This implementation returns immediately upon first successful TCP
connection, canceling not only in-flight TCP connection attempts but
also unfinished DNS queries.
Unfortunately this can't be implemented "above the vtable" because
various operating systems don't provide low level DNS resolution
primitives such as just putting the list of nameservers in a file.
Without libc on Linux it works great though!
Anyway this also changes the API to be based on Io.Queue. By using a
large enough buffer, reusable code can be written that does not require
concurrent, yet takes advantage of responding to DNS queries as they
come in. I sketched out a new implementation of `HostName.connect` to
demonstrate this, but it will require an additional API (`Io.Select`) to
be implemented in a future commit.
This commit also introduces "uncancelable" variants for mutex locking,
waiting on a condition, and putting items into a queue.
This only shows in release mode, the compiler tries to preserve some
value in rdi, but that gets replaced inside the fiber. This would not
happen in the C calling convention, but in these normal Zig functions,
it can happen.
In the future, it might be nice to introduce a type for file system path
names. This would be a way to avoid having InvalidFileName in the error
set, since construction of such type could validate it above the
interface.
Calling recvmsg first means no poll syscall needed when messages are
already in the operating system queue. Empirically, this happens when
repeating a DNS query that has been already been made recently. In such
case, poll() is never called!
glibc and linux kernel use size_t for some field lengths while POSIX and
musl use int. This bug would have caused breakage the first time someone
tried to call sendmsg on a 64-bit big endian system when linking musl
libc.
my opinion:
* msghdr.iovlen: kernel and glibc have it right. This field should
definitely be size_t. With int, the padding bytes are wasted for no
reason.
* msghdr.controllen: POSIX and musl have it right. 4 bytes is plenty for
the length, and it saves 4 bytes next to flags.
* cmsghdr.len: POSIX and musl have it right. 4 bytes is plenty for the
length, and it saves 4 bytes since the other fields are also 32-bits
each.
The one about INT_MAX is self-evident from the type system.
The one about kernel having bad types doesn't seem accurate as I checked
the source code and it uses size_t for all the appropriate types,
matching the libc struct definition for msghdr and msghdr_const.
extract pure functional logic into pure functions and then layer the
scope crap on top properly
the formatting code incorrectly didn't do the reverse operation
(if_indextoname). fix that with some TODO panics
1. a fiber can't put itself on a queue that allows it to be rescheduled
2. allow the idle fiber to unlock a mutex held by another fiber by
ignoring reschedule requests originating from the idle fiber
When the previous fiber did not request to be registered as an awaiter,
it may not have actually been a full blown `Fiber`, so only create the
`Fiber` pointer when needed.
The logic for computing reference traces was unintentionally finding the
*longest* possible trace (approximately). I think I already tried to fix
this before, but misunderstood how my own code works. Here, we fix it
properly: by slightly reworking the logic to use one ArrayHashMap for
both the result and the traversal queue, we trivially get a proper
breadth-first traversal so that we can find the shortest possible
reference trace for every referenced unit.
There is no straightforward way for the Zig team to access the Solaris system
headers; to do this, one has to create an Oracle account, accept their EULA to
download the installer ISO, and finally install it on a machine or VM. We do not
have to jump through hoops like this for any other OS that we support, and no
one on the team has expressed willingness to do it.
As a result, we cannot audit any Solaris contributions to std.c or other
similarly sensitive parts of the standard library. The best we would be able to
do is assume that Solaris and illumos are 100% compatible with no way to verify
that assumption. But at that point, the solaris and illumos OS tags would be
functionally identical anyway.
For Solaris especially, any contributions that involve APIs introduced after the
OS was made closed-source would also be inherently more risky than equivalent
contributions for other proprietary OSs due to the case of Google LLC v. Oracle
America, Inc., wherein Oracle clearly demonstrated its willingness to pursue
legal action against entities that merely copy API declarations.
Finally, Oracle laid off most of the Solaris team in 2017; the OS has been in
maintenance mode since, presumably to be retired completely sometime in the 2030s.
For these reasons, this commit removes all Oracle Solaris support.
Anyone who still wishes to use Zig on Solaris can try their luck by simply using
illumos instead of solaris in target triples - chances are it'll work. But there
will be no effort from the Zig team to support this use case; we recommend that
people move to illumos instead.
Renames arePointersLogical to shouldBlockPointerOps for clarity
adds capability check to allow pointer ops on .storage_buffer when
variable_pointers capability is enabled.
Fixes#25638
Zig uses "rN" for MIPS register clobbers which are more
ergonomic and easier to write (.rN vs. .@"$N").
However, GCC and Clang uses "$N".
Bug: #25613
Signed-off-by: Bingwu Zhang <xtex@xtexx.eu.org>
MIPS I has load hazards so we need to insert nops in a few places. This is not a
problem for MIPS II and later.
While doing this, I also touched up all the inline asm to use ABI register
aliases and a consistent formatting convention. Also fixed a few places that
didn't properly check if the syscall return value should be negated.
This allows us to rule out support for certain address spaces based on the OS.
This commit is just a refactor, however, and doesn't actually make use of that
opportunity yet.
build system: unit test enhancements
Contributes towards https://github.com/ziglang/zig/issues/19821, but does not close it, since the timeout currently cannot be modified per unit test.
The last commit passed CI, so this final bump is just to allow for
deviation caused by different loads on the runner machines. With this
change, I don't expect any current unit test to ever time out, even when
CI is under extreme load.
Unfortunately, Windows' scheduler means that test timeouts get hit very
easily, because it seems the system can refuse to schedule a waiting
process for *upwards of 10 minutes*. We should look for a better
solution for this problem going forwards, but for now, just give Windows
a very high test timeout.
The 30 minute timeout set here is around the duration of a *full CI run*
on Windows, so it should be impossible to hit normally, but it means
that if a test gets stuck we'll at least get told (eventually).
The Wycheproof test suite is extensive, but takes a long time to
complete on CI.
Keep only the most relevant ones and take it as an opportunity to describe
what they are.
The remaining ones are still available for manual testing when required.
This test called `yield` 80,000 times, which is nothing on a system with
little load, but murder on a CI system. macOS' scheduler in particular
doesn't seem to deal with this very well. The `yield` calls also weren't
even necessarily doing what they were meant to: if the optimizer could
figure out that it doesn't clobber some memory, then it could happily
reorder around the `yield`s anyway!
The test has been simplified and made to work better, and the number of
yields have been reduced. The number of overall iterations has also been
reduced, because with the `yield` calls making races very likely, we
don't really need to run too many iterations to be confident that the
implementation is race-free.
For instance, when running a Zig test using the self-hosted aarch64
backend, this logic was previously expecting `std.zig.Server` to be
used, but the default test runner intentionally does not do this because
the backend is too immature to handle it. On 'master', this is causing
sporadic failures; on this branch, they became consistent failures.
The new `--error-style` option decides how build failures are printed.
The default mode "verbose" prints all context including the step graph
fragment and the failed command (if any). The alternative mode "minimal"
prints only the failed step itself, and does not print the failed
command. There are also "verbose_clear" and "minimal_clear" modes, which
have the distinction that the output is cleared (through ANSI escape
codes) between updates, preventing different updates from being confused
in the output. If `--error-style` is not specified, the environment
variable `ZIG_BUILD_ERROR_STYLE` is checked before falling back to the
default of "verbose"; this means the value can effectively be chosen
system-wide since it is generally a personal preference.
Also introduced is a `--multiline-errors` option which decides how to
print errors which span multiple lines. By default, non-initial lines
are indented to align with the first. Alternatively, a leading newline
can be printed to align everyting on the first column, or no special
treatment can be applied, resulting in misaligned output. Again, there
is an environment variable (`ZIG_BUILD_MULTILINE_ERRORS`) to specify a
preferred default if the option is not explicitly provided.
Resolves: #23472
This is a major refactor to `Step.Run` which adds new functionality,
primarily to the execution of Zig tests.
* All tests are run, even if a test crashes. This happens through the
same mechanism as timeouts where the test processes is repeatedly
respawned as needed.
* The build status output is more precise. For each unit test, it
differentiates pass, skip, fail, crash, and timeout. Memory leaks are
reported separately, as they do not indicate a test's "status", but
are rather an additional property (a test with leaks may still pass!).
* The number of memory leaks is tracked and reported, both per-test and
for a whole `Run` step.
* Reporting is made clearer when a step is failed solely due to error
logs (`std.log.err`) where every unit test passed.
For now, there is a flag to `zig build` called `--test-timeout-ms` which
accepts a value in milliseconds. If the execution time of any individual
unit test exceeds that number of milliseconds, the test is terminated
and marked as timed out.
In the future, we may want to increase the granularity of this feature
by allowing timeouts to be specified per-step or even per-test. However,
a global option is actually very useful. In particular, it can be used
in CI scripts to ensure that no individual unit test exceeds some
reasonable limit (e.g. 60 seconds) without having to assign limits to
every individual test step in the build script.
Also, individual unit test durations are now shown in the time report
web interface -- this was fairly trivial to add since we're timing tests
(to check for timeouts) anyway.
This commit makes progress on #19821, but does not close it, because
that proposal includes a more sophisticated mechanism for setting
timeouts.
Co-Authored-By: David Rubin <david@vortan.dev>
The ABIs do not define a frame pointer register, nor do they define a guaranteed
and fixed area on the stack where one might find saved registers such as a frame
pointer or return address.
The compile-time check against the minimum version here wasn't appropriate, since it still makes sense to try using FILE_RENAME_INFORMATION_EX even if the minimum version is something like `xp`, since that doesn't rule out the possibility of the compiled code running on Windows 10/11. This compile-time check was doubly bad since the default minimum windows version (`.win10`) was below the `.win10_rs5` that was checked for, so when providing a target like `x86_64-windows-gnu` it'd always rule out using this syscall.
After this commit, we always try using FILE_RENAME_INFORMATION_EX and then let the operating system tell us when some aspect of it is not supported. This allows us to get the benefits of these new syscalls/flags whenever it's actually possible.
The possible error returns were validated experimentally:
- INVALID_PARAMETER is returned when the underlying filesystem is FAT32
- INVALID_INFO_CLASS is returned on Windows 7 when trying to use FileRenameInformationEx/FileDispositionInformationEx
- NOT_SUPPORTED is returned on Windows 10 >= .win10_rs5 when setting a bogus flag value (I used `0x1000`)
This is very likely full of wrong stuff. It's effectively just a copy of the
x86_64 file - needed because the former stopped using usize/isize. To be clear,
this is no more broken than the old situation was; this just makes the
brokenness explicit.
This is very likely full of wrong stuff. It's effectively just a copy of the
mips64 file - needed because the former stopped using usize/isize. To be clear,
this is no more broken than the old situation was; this just makes the
brokenness explicit.
This reverts commit fcfdf99122b17a928c144c3d70418b35e6b1620e.
This added too much load to the x86_64-linux machines, resulting in timeouts
pretty much across the board.
This is a rewrite of the BLAKE3 implementation, with vectorization.
On Apple Silicon, the new implementation is about twice as fast as the previous one.
With AVX2, it is more than 4 times faster.
With AVX512, it is more than 7.5x faster than the previous implementation (from 678 MB/s to 5086 MB/s).
The return address points to the call instruction on SPARC, so the actual return
address is 8 bytes after. This means that we shouldn't do the return address
adjustment that we normally do.
The FP would point to the register save area for the previous frame, while the
SP points to the register save area for the current frame. So use the latter.
I have no idea if this is a QEMU bug or real kernel behavior. Either way, the
register save area specifically exists for asynchronous spilling of incoming and
local registers, so there should be no harm in doing this.
* std.crypto: add AES-CCM and CBC-MAC
Add AES-CCM (Counter with CBC-MAC) authenticated encryption and
CBC-MAC message authentication code implementations to the standard
library.
AES-CCM combines CTR mode encryption with CBC-MAC authentication as
specified in NIST SP 800-38C and RFC 3610. It provides authenticated
encryption with support for additional authenticated data (AAD).
CBC-MAC is a simple MAC construction used internally by CCM, specified
in FIPS 113 and ISO/IEC 9797-1.
Includes comprehensive test vectors from RFC 3610 and NIST SP 800-38C.
* std.crypto: add CCM* (encryption-only) support to AES-CCM
Implements CCM* mode per IEEE 802.15.4 specification, extending
AES-CCM to support encryption-only mode when tag_len=0. This is
required by protocols like ZigBee, Thread, and WirelessHART.
Changes:
- Allow tag_len=0 for encryption-only mode (no authentication)
- Skip CBC-MAC computation when tag_len=0 in encrypt/decrypt
- Correctly encode M'=0 in B0 block for CCM* mode
- Add Aes128Ccm0 and Aes256Ccm0 convenience instances
- Add IEEE 802.15.4 test vectors and CCM* tests
* std.crypto: add doc comments for AES-CCM variants
If these ever get allocated, it's most likely going to be for things that don't
matter to us anyway, so completely abandoning DWARF unwinding just because we
see these doesn't seem justified. We will still do so if we're actually asked to
read from such a register, which is the only actually problematic case; see
c23a5ccd19 for more details.
This is to help diagnose #25498. We can't use `unexpectedErrno` here,
because `std.posix.munmap` is infallible. So, when the flag is set to
report unexpected errnos, we just call `std.debug.panic` to provide
details instead of doing `unreachable`.
Pushing straight to master after running checks locally; there's no
point waiting for CI on the PR just for this.
This path being relative is unconventional and causes issues for us
if the output artifact is ever used from a different cwd than the one it
was built from. The behavior implemented by this commit of always
emitting these paths as absolute was actually the behavior in 0.14.x,
but it regressed in 0.15.1 due to internal reworks to path handling
which led to relative paths being more common in the compiler internals.
Resolves: #25433
This type is useful for two things:
* Doing non-local control flow with ucontext.h functions.
* Inspecting machine state in a signal handler.
The first use case is not one we support; we no longer expose bindings to those
functions in the standard library. They're also deprecated in POSIX and, as a
result, not available in musl.
The second use case is valid, but is very poorly served by the standard library.
As evidenced by my changes to std.debug.cpu_context.signal_context_t, users will
be better served rolling their own ucontext_t and especially mcontext_t types
which fit their specific situation. Further, these types tend to evolve
frequently as architectures evolve, and the standard library has not done a good
job keeping up, or even providing them for all supported targets.
I made a couple of decisions for this based on the fact that we don't expose the
signal_ucontext_t type outside of the file:
* Adding all the floating point and vector state to every ucontext_t and
mcontext_t variant was way, way too much work, especially when we don't even
use the stuff. So I deleted all that and kept only the bare minimum needed to
reach into general-purpose registers.
* There is no particularly compelling reason to stick to the naming and struct
nesting used in the system headers. So we can actually unify the access
patterns for almost all of these variants by taking some liberties here; as a
result, fromPosixSignalContext() is now much nicer to read and extend.
I broke this when porting this logic for the `std.debug` rework in
https://github.com/ziglang/zig/pull/25227. The offset that I copied was
actually being treated as relative to the address of the *saved* base
pointer. I think it makes more sense to do what I did and just treat all
offsets as relative to this frame's base.
- Revive some of the removed cache integration logic in `cmdTranslateC` now that `translate-c` can return error bundles
- Fixup inconsistent path separators (on Windows) when building the aro include path
- Move some error bundle logic from resinator into aro.Diagnostics
- Add `ErrorBundle.addRootErrorMessageWithNotes` (extracted from resinator)
Now it's based on calling fillMore rather than an illegal aliased stream
into the Reader buffer.
This commit also includes a disambiguation block inspired by #25162. If
`StreamTooLong` was added to `RebaseError` then this logic could be
replaced by removing the exit condition from the while loop. That error
code would represent when `buffer` capacity is too small for an
operation, replacing the current use of asserts.
Fix `takeDelimiter` and `takeDelimiterExclusive` tossing too many bytes
(#25132)
Also add/improve test coverage for all delimiter and sentinel methods,
update usages of `takeDelimiterExclusive` to not rely on the fixed bug,
tweak a handful of doc comments, and slightly simplify some logic.
I have not fixed#24950 in this commit because I am a little less
certain about the appropriate solution there.
Resolves: #25132
Co-authored-by: Andrew Kelley <andrew@ziglang.org>
* File.Writer.seekBy passed wrong offset to setPosAdjustingBuffer.
* File.Writer.sendFile incorrectly used non-logical position.
Related to 1d764c1fdf04829cec5974d82cec901825a80e49
Test case provided by:
Co-authored-by: Kendall Condon <goon.pri.low@gmail.com>
Previously, the logic in peekDelimiterInclusive (when the delimiter was not found in the existing buffer) used the `n` returned from `r.vtable.stream` as the length of the slice to check, but it's valid for `vtable.stream` implementations to return 0 if they wrote to the buffer instead of `w`. In that scenario, the `indexOfScalarPos` would be given a 0-length slice so it would never be able to find the delimiter.
This commit changes the logic to assume that `r.vtable.stream` can both:
- return 0, and
- modify seek/end (i.e. it's also valid for a `vtable.stream` implementation to rebase)
Also introduces `std.testing.ReaderIndirect` which helps in being able to test against Reader implementations that return 0 from `stream`/`readVec`
Fixes#25428
This reverts commit 27aba2d776caf59bb6569934626af587fdba9c75.
I'd like to review this contribution more carefully, particularly with
the alternate implementation that is also open as a pull request
(#25109).
Reopens#25093
`findScalarPos` might do repetitive work, even if using simd. For
example, when searching the string `/abcde/fghijk/lm` for the character
`/`, a 16-byte wide search would yield `1000001000000100` but would only
count the first `1` and re-search the remaining of the string.
When testing locally, the difference was quite significative:
```
count scalar
5737 iterations 522.83us per iterations
0 bytes per iteration
worst: 2370us median: 512us stddev: 107.64us
count v2
38333 iterations 78.03us per iterations
0 bytes per iteration
worst: 713us median: 76us stddev: 10.62us
count scalar v2
99565 iterations 29.80us per iterations
0 bytes per iteration
worst: 41us median: 29us stddev: 1.04us
```
Note that `count v2` is a simpler string search, similar to the
remaining version of the simd approach:
```
pub fn countV2(comptime T: type, haystack: []const T, needle: T) usize {
const n = haystack.len;
if (n < 1) return 0;
var count: usize = 0;
for (haystack[0..n]) |item| {
count += @intFromBool(item == needle);
}
return count;
}
```
Which implies the compiler yields some optimized code for a simpler loop
that is more performant than the `findScalarPos`-based approach, hence
the usage of iterative approach for the remaining of the haystack.
Co-authored-by: StAlKeR7779 <stalkek7779@yandex.ru>
For unwinding purposes, we don't care about unsupported registers. Yet because
we added these rules to the cache entry, we'd later try to evaluate them and
thus fail the unwind attempt for no good reason. They'd also take up cache rule
slots that would be better spent on actually relevant registers.
Note that any attempt to read unsupported registers during unwinding will still
fail the unwind attempt as expected.
The previous version (ported from musl) used bit-by-bit calculations and was slow, but the current version (also ported from musl) uses lookup tables combined with Goldschmidt iterations to significantly improve the speed.
* ELF v1 on powerpc64 is only barely kept on life support in a couple of Linux
distros. I don't anticipate that this will last much longer.
* Most of the Linux world has moved to powerpc64le which requires ELF v2.
* Some Linux distros have even started supporting powerpc64 with ELF v2.
* The BSD world has long since moved to ELF v2.
* We have no actual linking support for ELF v1.
* ELF v1 had confused DWARF register mappings which is becoming a problem in
our DWARF code in std.debug.
It's clear that ELF v1 is on its way out, and we never fully supported it
anyway. So let's not waste any time or energy on it going forward.
closes#5927
FreeBSD doesn't support the same number of platforms as Linux, and even then,
only has usermode emulation for a subset of its supported platforms.
NetBSD's usermode emulation support is apparently just broken at the moment.
The amount of cross compilation required for these tests was too time-consuming
for how much value they added. test-stack-traces now cover these well enough,
especially as we add more exotic machines to the CI fleet to run native tests.
For the supported COFF machine types of X64 (x86_64), I386 (x86), ARMNT (thumb), and ARM64 (aarch64), this new Zig implementation results in byte-for-byte identical .lib files when compared to the previous LLVM-backed implementation.
Previously, `setAlignment` would set the value to 1 fewer than it should, so if you were intending to set alignment to 8 bytes, it would actually set it to 4 bytes, etc.
This enables depth-related use cases without any dependency on the Walker's internal stack which doesn't always pertain to the actual depth of the current entry (i.e. recursing into a directory immediately affects the stack).
Some decision-making might depend on the level of the traversal, so
it makes sense to expose depth here since it's stable, and not in the
automatic walker where it's not.
This matches all other platforms. Even if this field is defined as 'int'
in the C definition, the expectation is that the full 32-bit unsigned
integer range can be used. In particular this Sigaction initializer in
the new std.debug code was causing a build failure:
```zig
.flags = (posix.SA.SIGINFO | posix.SA.RESTART | posix.SA.RESETHAND)
```
This is a little different from how C/C++ compilers do this, but I think it's
justified because it's what users actually *mean* when the use frame pointer
options.
This is another one of those LLVM "CPU" features that have nothing to do with
CPU at all and should really be a TargetMachine option or something. One day
we'll figure out a better way of dealing with these...
contributing is in the readme already, and code of conduct should go on
the website. this is a code repository; it doesn't dictate social norms.
the reason for these documents being in .github/ was to satisfy GitHub
demands so that the UI would look more favorably upon ziglang/zig but
that is no longer a concern.
Before, this had a subtle ordering bug where duplicate
deps that are specified as both lazy and eager in different
parts of the dependency tree end up not getting fetched
depending on the ordering. I modified it to resubmit lazy
deps that were promoted to eager for fetching so that it will
be around for the builds that expect it to be eager downstream
of this.
Implements deflate compression from scratch. A history window is kept in
the writer's buffer for matching and a chained hash table is used to
find matches. Tokens are accumulated until a threshold is reached and
then outputted as a block. Flush is used to indicate end of stream.
Additionally, two other deflate writers are provided:
* `Raw` writes only in store blocks (the uncompressed bytes). It
utilizes data vectors to efficiently send block headers and data.
* `Huffman` only performs Huffman compression on data and no matching.
The above are also able to take advantage of writer semantics since they
do not need to keep a history.
Literal and distance code parameters in `token` have also been reworked.
Their parameters are now derived mathematically, however the more
expensive ones are still obtained through a lookup table (expect on
ReleaseSmall).
Decompression bit reading has been greatly simplified, taking advantage
of the ability to peek on the underlying reader. Additionally, a few
bugs with limit handling have been fixed.
There were only a few dozen lines of common logic, and they frankly
introduced more complexity than they eliminated. Instead, let's accept
that the implementations of `SelfInfo` are all pretty different and want
to track different state. This probably fixes some synchronization and
memory bugs by simplifying a bunch of stuff. It also improves the DWARF
unwind cache, making it around twice as fast in a debug build with the
self-hosted x86_64 backend, because we no longer have to redundantly go
through the hashmap lookup logic to find the module. Unwinding on
Windows will also see a slight performance boost from this change,
because `RtlVirtualUnwind` does not need to know the module whatsoever,
so the old `SelfInfo` implementation was doing redundant work. Lastly,
this makes it even easier to implement `SelfInfo` on freestanding
targets; there is no longer a need to emulate a real module system,
since the user controls the whole implementation!
There are various other small refactors here in the `SelfInfo`
implementations as well as in the DWARF unwinding logic. This change
turned out to make a lot of stuff simpler!
Apparently the `__eh_frame` in Mach-O binaries doesn't include the
terminator entry, but in all other respects it acts like `.eh_frame`
rather than `.debug_frame`. I have no idea.
By my estimation, these changes speed up DWARF unwinding when using the
self-hosted x86_64 backend by around 7x. There are two very significant
enhancements: we no longer iterate frames which don't fit in the stack
trace buffer, and we cache register rules (in a fixed buffer) to avoid
re-parsing and evaluating CFI instructions in most cases. Alongside this
are a bunch of smaller enhancements, such as pre-caching the result of
evaluating the CIE's initial instructions, avoiding re-parsing of CIEs,
and big simplifications to the `Dwarf.Unwind.VirtualMachine` logic.
This was causing a zig2 miscomp, which emitted slightly broken debug
information, which caused extremely slow stack unwinding. We're working
on fixing or reporting this upstream, but we can use this workaround for
now, because GCC guarantees arithmetic signed shift.
This logic was causing some occasional infinite looping on ARM, where
the `.debug_frame` section is often incomplete since the `.exidx`
section is used for unwind information. But the information we're
getting from the compiler is totally *valid*: it's leaving the rule as
the default, which is (as with most architectures) equivalent to
`.undefined`!
This has been a TODO for ages, but in the past it didn't really matter
because stack traces are typically printed to stderr for which a mutex
is held so in practice there was a mutex guarding usage of `SelfInfo`.
However, now that `SelfInfo` is also used for simply capturing traces,
thread safety is needed. Instead of just a single mutex, though, there
are a couple of different mutexes involved; this helps make critical
sections smaller, particularly when unwinding the stack as `unwindFrame`
doesn't typically need to hold any lock at all.
Calling `current` here causes compilation failures as the C backend
currently does not emit valid MSVC inline assembly. This change means
that when building for MSVC with the self-hosted C backend, only FP
unwinding can be used.
Processes should reasonably be able to expect their children to abort
with typical exit codes, rather than a debugger breakpoint signal. This
flag in the PEB is what would be checked by `IsDebuggerPresent` in
kernel32, which is the function you would typically use for this
purpose.
This fixes `test-stack-trace` failures on Windows, as these tests were
expecting exit code 3 to indicate abort.
...and just deal with signal handlers by adding 1 to create a fake
"return address". The system I tried out where the addresses returned by
`StackIterator` were pre-subtracted didn't play nicely with error
traces, which in hindsight, makes perfect sense. This definition also
removes some ugly off-by-one issues in matching `first_address`, so I do
think this is a better approach.
This crash exists on master, and seems to have existed since 2019; I
think it's just very rare and depends on the exact binary generated. In
theory, a stream block should always be a "data" block rather than a FPM
block; the FPMs use blocks `1, 4097, 8193, ...` and `2, 4097, 8194, ...`
respectively. However, I have observed LLVM emitting an otherwise valid
PDB which maps FPM blocks into streams. This is not a bug in
`std.debug.Pdb`, because `llvm-pdbutil` agrees with our stream indices.
I think this is arguably an LLVM bug; however, we don't really lose
anything from just weakening this check. To be fair, MSF doesn't have an
explicit specification, and LLVM's documentation (which is the closest
thing we have) does not explicitly state that FPM blocks cannot be
mapped into streams, so perhaps this is actually valid.
In the rare case that LLVM emits this, previously, stack traces would
have been completely useless; now, stack traces will work okay.
Mostly on macOS, since Loris showed me a not-great stack trace, and I
spent 8 hours trying to make it better. The dyld shared cache is
designed in a way which makes this really hard to do right, and
documentation is non-existent, but this *seems* to work pretty well.
I'll leave the ruling on whether I did a good job to CI and our users.
Our usage of `ucontext_t` in the standard library was kind of
problematic. We unnecessarily mimiced libc-specific structures, and our
`getcontext` implementation was overkill for our use case of stack
tracing.
This commit introduces a new namespace, `std.debug.cpu_context`, which
contains "context" types for various architectures (currently x86,
x86_64, ARM, and AARCH64) containing the general-purpose CPU registers;
the ones needed in practice for stack unwinding. Each implementation has
a function `current` which populates the structure using inline
assembly. The structure is user-overrideable, though that should only be
necessary if the standard library does not have an implementation for
the *architecture*: that is to say, none of this is OS-dependent.
Of course, in POSIX signal handlers, we get a `ucontext_t` from the
kernel. The function `std.debug.cpu_context.fromPosixSignalContext`
converts this to a `std.debug.cpu_context.Native` with a big ol' target
switch.
This functionality is not exposed from `std.c` or `std.posix`, and
neither are `ucontext_t`, `mcontext_t`, or `getcontext`. The rationale
is that these types and functions do not conform to a specific ABI, and
in fact tend to get updated over time based on CPU features and
extensions; in addition, different libcs use different structures which
are "partially compatible" with the kernel structure. Overall, it's a
mess, but all we need is the kernel context, so we can just define a
kernel-compatible structure as long as we don't claim C compatibility by
putting it in `std.c` or `std.posix`.
This change resulted in a few nice `std.debug` simplifications, but
nothing too noteworthy. However, the main benefit of this change is that
DWARF unwinding---sometimes necessary for collecting stack traces
reliably---now requires far less target-specific integration.
Also fix a bug I noticed in `PageAllocator` (I found this due to a bug
in my distro's QEMU distribution; thanks, broken QEMU patch!) and I
think a couple of minor bugs in `std.debug`.
Resolves: #23801Resolves: #23802
This only matters if `callMain` is called by a user, since `std.start`
will never itself call `callMain` when `target.os.tag == .other`.
However, it *is* a valid use case for a user to call
`std.start.callMain` in their own startup logic, so this makes sense.
The input path could be cwd-relative, in which case it must be modified
before it is written into the batch script.
Also, remove usage of deprecated `GeneralPurposeAllocator` alias, rename
`allocator` to `gpa`, use unmanaged `ArrayList`.
Turns out that RtlCaptureStackBackTrace is actually just doing FP (ebp)
unwinding under the hood, making this logic completely redundant with
our own FP-walking implementation; see added comment for details.
Previously, the `test-stack-traces` step was essentially just testing
error traces, and even there we didn't have much coverage. This commit
solves that by splitting the "stack trace" tests into two separate
harnesses: the "stack trace" tests are for actual stack traces (i.e.
involving stack unwinding), while the "error trace" tests are
specifically for error return traces.
The "stack trace" tests will test different configurations of:
* `-lc`
* `-fPIE`
* `-fomit-frame-pointer`
* `-fllvm`
* unwind tables (currently disabled)
* strip debug info (currently disabled)
The main goal there is to test *stack unwinding* under different
conditions. Meanwhile, the "error trace" tests will test different
configurations of `-O` and `-fllvm`; the main goal here, aside from
checking that error traces themselves do not miscompile, is to check
whether debug info is still working even in optimized builds. Of course,
aggressive optimizations *can* thwart debug info no matter what, so as
before, there is a way to disable cases for specific targets / optimize
modes.
The program which converts stack traces into a more validatable format
by removing things like addresses (previously `check-stack-trace.zig`,
now `convert-stack-trace.zig`) has been rewritten and simplified. Also,
thanks to various fixes in this branch, several workarounds have become
unnecessary: for instance, we don't need to ignore the function name
printed in stack traces in release modes, because `std.debug.Dwarf` now
uses the correct DIE for inlined functions!
Neither `test-stack-traces` nor `test-error-traces` does general foreign
architecture testing, because it seems that (at least for now) external
executors often aren't particularly good at handling stack tracing
correctly (looking at you, Wine). Generally, they just test the native
target (this matches the old behavior of `test-stack-traces`). However,
there is one exception: when on an x86_64 or aarch64 host, we will also
test the 32-bit version (x86 or arm) if the OS supports it, because such
executables can be trivially tested without an external executor.
Oh, also, I wrote a bunch of stack trace tests. Previously there was,
erm, *one* test in `test-stack-traces` which wasn't for error traces.
Now there are a good few!
It was possible for `arg6` to be passed as an operand relative to esp.
In that case, the `push` at the top clobbered esp and hence made the
reference to arg6 invalid. This was manifesting in this branch as broken
stack traces on x86-linux due to an `mmap2` syscall accidentally passing
the page offset as non-zero!
This commit fixes a bug introduced in cb0e6d8aa.
We mustn't emit the DT_PLTGOT entry in `.dynamic` in a statically-linked
PIE, because there's no dl to relocate it (and `std.pie.relocate`, or
the PIE relocator in libc, won't touch it). In that case, there cannot
be any PLT entries, so there's no point emitting the `.got.plt` section
at all. If we just don't create that section, `link.Elf` already knows
not to add the DT_PLTGOT entry to `.dynamic`.
Co-authored-by: Jacob Young <jacobly0@users.noreply.github.com>
This abstraction isn't really tied to DWARF at all! Really, we're just
loading some information from an ELF file which is useful for debugging.
That *includes* DWARF, but it also includes other information. For
instance, the other change here:
Now, if DWARF information is missing, `debug.SelfInfo.ElfModule` will
name symbols by finding a matching symtab entry. We actually already do
this on Mach-O, so it makes obvious sense to do the same on ELF! This
change is what motivated the restructuring to begin with.
The symtab work is derived from #22077.
Co-authored-by: geemili <opensource@geemili.xyz>
If it's not given, we should set `first_address` to the return address
of `dumpCurrentStackTrace` to avoid the call to `writeCurrentStackTrace`
appearing in the trace. However, we must only do that if no `context` is
given; if there's a context then we're starting the stack unwind
elsewhere.
At least, when there's not a ZigObject. The old behavior was incorrect
in the presence of a ZigObject, and this doesn't really mix nicely with
incremental compilation anyway; but when the objects are all external,
we may as well build the search table.
Far simpler, because everything which `crash_report.zig` did is now
handled pretty well by `std.debug` anyway. All we want is to print some
context around panics and segfaults. Using the new ability to override
the default segfault handler while still having std handle the
target-specific bits for us, that's really simple.
The downside of this commit is that more precise errors are no longer
propagated up. However, these errors were pretty useless in isolation
due to them having no context; and regardless, we intentionally swallow
most of them in `std.debug` anyway. Therefore, this is better in
practice, because it allows `std.debug` to give slightly more useful
warnings when handling errors. This commit does that for unwind errors,
for instance, which differentiate between the unwind info being corrupt
vs missing vs inaccessible vs unsupported.
A better solution would be to also include more detailed information via
the diagnostics pattern, but this commit is an incremental improvement.
turns out this isn't technically specific to that target at all; other
targets just don't emit mid-function 'ret' instructions as much so
certain CFI instruction patterns were only seen on aarch64.
thanks to jacob for finding the bug <3
Because -fno-llvm is now the default on x86_64-linux, this target was
exactly equivalent to one specified earlier in the matrix. This was
probably just missed when doing the work to enable the self-hosted
backend by default for x86_64.
The memory operand might use one of the extended GPRs R8 through R15 and
hence require a REX prefix, but having a REX prefix makes the high-byte
register AH unencodeable as the src operand. This latent bug was exposed
by this branch, presumably because `select` now happens to be putting
something in an extended GPR instead of a legacy GPR.
In theory this could be fixed with minimal cost by introducing a way to
communicate to `select` that neither the destination memory nor the
other temporary can be in an extended GPR. However, I just went for the
simple solution which comes at a cost of one trivial instruction: copy
the remainder from AH to AL, and *then* copy AL to the destination.
`rep movsb` isn't usually a great idea here. This commit makes the logic
which tentatively existed in `genInlineMemcpy` apply in more cases, and
in particular applies it to the "new" backend logic. Put simply, all
copies of 128 bytes or fewer will now attempt this path first,
where---provided there is an SSE register and/or a general-purpose
register available---we will lower the operation using a sequence of 32,
16, 8, 4, 2, and 1 byte copy operations.
The feedback I got on this diff was "Push it to master and if it
miscomps I'll revert it" so don't blame me when it explodes
* Add missing functions like ISDIR() or ISREG(). This is required to
build the zig compiler
* Use octal notation for the S_ constants. This is how it is done for
".freebsd" and it is also the notation used by DragonFly in
"sys/stat.h"
* Reorder S_ constants in the same order as ".freebsd" does. Again, this
follows the ordering within "sys/stat.h"
Before https://github.com/ziglang/zig/pull/18160, error tracing defaulted to true in ReleaseSafe, but that is no longer the case. These option descriptions were never updating accordingly.
--debug-rt previously would make rt libs match the root module. Now they
are always debug when --debug-rt is passed. This includes compiler-rt,
fuzzer lib, and others.
Moving towards our function naming convention of having one word per
concept and constructing function names out of concatenated concepts.
In `std.mem` the concepts are:
* "find" - return index of substring
* "pos" - starting index parameter
* "last" - search from the end
* "linear" - simple for loop rather than fancy algo
* "scalar" - substring is a single element
This is f5fb720a5399ee98e45f36337b2f68a4d23a783c plus ehaas's nonnull
attribute pull request currently at 4b26cb3ac610a0a070fc43e43da8b4cdf0e9101b
with zig patches intact.
Adds the limit option to `--fuzz=[limit]`. the limit expresses a number
of iterations that *each fuzz test* will perform at maximum before
exiting. The limit argument supports also 'K', 'M', and 'G' suffixeds
(e.g. '10K').
Does not imply `--web-ui` (like unlimited fuzzing does) and prints a
fuzzing report at the end.
Closes#22900 but does not implement the time based limit, as after
internal discussions we concluded to be problematic to both implement
and use correctly.
Clang fails to compile the CBE translation of this code ("non-ASM
statement in naked function"). Similar to the implementations of
`restore_rt` on x86 and ARM, when the CBE is in use, this commit employs
alternative inline assembly that avoids using non-immediate input
operands.
In a library, the two `builtin.link_libc` and `builtin.output_mode ==
.Exe` checks could both be false. Thus, you would get a compile error
even if you specified an `env_map` at runtime. This change turns the
compile error into a runtime panic and updates the documentation to
reflect the runtime requirement.
Fixes#25209.
On PowerPC, some registers are both inputs to syscalls and clobbered by
them. An example is r0, which initially contains the syscall number, but
may be overwritten during execution of the syscall.
musl and glibc use a `+` (read-write) constraint to indicate this, which
isn't supported in Zig. The current implementation of PowerPC syscalls
in the Zig standard library instead lists these registers as both inputs
and clobbers, but this results in the C backend generating code that is
invalid for at least some C compilers, like GCC, which doesn't support
the specifying the same register as both an input and a clobber.
This PR changes the PowerPC syscall functions to list such registers as
inputs and outputs rather than inputs and clobbers. Thanks to jacobly0
who pointed out that it's possible to have multiple outputs; I had
gotten the wrong idea from the documentation.
If the compiler happens to pick `ret = r0`, then this will assemble to
`ag r0, 0` which is obviously not what we want. Using `a` instead of `r` will
ensure that we get an appropriate address register, i.e. `r1` through `r15`.
Re-enable pie_linux for s390x-linux which was disabled in
ed7ff0b693037078f451a7c6c1124611060f4892.
There are two reasons for this:
1. Apple is about to drop support for this target. Zig will keep support
but move it to a lower tier - one that does not require continuous CI
testing. Support for this target will be maintained by the enthusiasm
of contributors but will not block other bug fixes and enhancements.
2. This is our only non-self-hosted action runner. We are migrating away
from GitHub soon at which point this runner will no longer be
available.
Disabled due to no active maintainer (feel free to fix the failures and
then re-enable at any time). The failures occur due to backend
miscompilation of different AIR from the frontend.
Disabled due to no active maintainer (feel free to fix the failures and
then re-enable at any time). The failures occur due to changing AIR from
the frontend, and backend being incomplete.
C translation is in the process of switching to be aro-based
(see #24497)
That codebase will need to gain some kind of helper for translating C
code that uses runtime vector indexing.
with field_ptr_load and field_ptr_named_load.
These avoid doing by-val load operations for structs that are
runtime-known while keeping the previous semantics for comptime-known
values.
If `r.end` is updated in the `stream` implementation, then it's possible that `r.end += ...` will behave unexpectedly. What seems to happen is that it reverts back to its value before the function call and then the increment happens. Here's a reproduction:
```zig
test "fill when stream modifies `end` and returns 0" {
var buf: [3]u8 = undefined;
var zero_reader = infiniteZeroes(&buf);
_ = try zero_reader.fill(1);
try std.testing.expectEqual(buf.len, zero_reader.end);
}
pub fn infiniteZeroes(buf: []u8) std.Io.Reader {
return .{
.vtable = &.{
.stream = stream,
},
.buffer = buf,
.end = 0,
.seek = 0,
};
}
fn stream(r: *std.Io.Reader, _: *std.Io.Writer, _: std.Io.Limit) std.Io.Reader.StreamError!usize {
@memset(r.buffer[r.seek..], 0);
r.end = r.buffer.len;
return 0;
}
```
When `fill` is called, it will call into `vtable.readVec` which in this case is `defaultReadVec`. In `defaultReadVec`:
- Before the `r.end += r.vtable.stream` line, `r.end` will be 0
- In `r.vtable.stream`, `r.end` is modified to 3 and it returns 0
- After the `r.end += r.vtable.stream` line, `r.end` will be 0 instead of the expected 3
Separating the `r.end += stream();` into two lines fixes the problem (and this separation is done elsewhere in `Reader` so it seems possible that this class of bug has been encountered before).
Potentially related issues:
- https://github.com/ziglang/zig/issues/4021
- https://github.com/ziglang/zig/issues/12064
This clarifies that it is legal to return an invalid pointer from a
function, provided that such pointer is not dereferenced.
This matches current status quo of the language. Any change to this
should be a proposal that argues for different semantics.
It is also legal in C to return a pointer to a local. The C backend
lowers such thing directly, so the corresponding warning in C must be
disabled (`-Wno-return-stack-address`).
This test works by assuming that std.ArrayList will grow with a specific
capacity increasing pattern, which is an invalid assumption. Delete the
offending test.
I measured this against master branch and found no statistical
difference. Since this code is simpler and logically superior due to
always leaving sufficient unused capacity when growing, it is preferred
over status quo.
Adds `addFileContentArg` and `addPrefixedFileContentArg` to pass the content
of a file with a lazy path as an argument to a `std.Build.Step.Run`.
This enables replicating shell `$()` / cmake `execute_process` with `OUTPUT_VARIABLE`
as an input to another `execute_process` in conjuction with `captureStdOut`/`captureStdErr`.
To also be able to replicate `$()` automatically trimming trailing newlines and cmake
`OUTPUT_STRIP_TRAILING_WHITESPACE`, this patch adds an `options` arg to those functions
which allows specifying the desired handling of surrounding whitespace.
The `options` arg also allows to specify a custom `basename` for the output. e.g.
to add a file extension (concrete use case: Zig `@import()` requires files to have a
`.zig`/`.zon` extension to recognize them as valid source files).
The data structure was originally added in
41e1cd185b82a518c58c92544c45f0348c03ef74 and then removed in
50a336fff899ebd8a687c453ec6beb18a5a9baf9, but brought back in
711bf55eaa643c3d05640bebbf3e4315477b8ed8 for Decl in the compiler
frontend, and then the last reference to it was eliminated in
548a087fafeda5b07d2237d5137906b8d07da699 which removed Decl in favor of
Nav and Cau.
When building on macOS Tahoe, binaries were getting duplicate LC_RPATH
load commands which caused dyld to refuse to run them with a
"duplicate LC_RPATH" error that has become a hard error.
The duplicates occurred when library directories were being added
to rpath_list twice:
- from lib_directories
- from native system paths detection which includes the same dirs
This can be re-evaluated at a later time, but at the moment the
performance and stability concerns hold it back. Additionally, it
promotes a non-smithing approach to fuzz tests.
This PR significantly improves the capabilities of the fuzzer.
The changes made to the fuzzer to accomplish this feat mostly include
tracking memory reads from .rodata to determine fresh inputs, new
mutations (especially the ones that insert const values from .rodata
reads and __sanitizer_conv_const_cmp), and minimizing found inputs.
Additionally, the runs per second has greatly been increased due to
generating smaller inputs and avoiding clearing the 8-bit pc counters.
An additional feature added is that the length of the input file is now
stored and the old input file is rerun upon start.
Other changes made to the fuzzer include more logical initialization,
using one shared file `in` for inputs, creating corpus files with
proper sizes, and using hexadecimal-numbered corpus files for
simplicity.
Furthermore, I added several new fuzz tests to gauge the fuzzer's
efficiency. I also tried to add a test for zstandard decompression,
which it crashed within 60,000 runs (less than a second.)
Bug fixes include:
* Fixed a race conditions when multiple fuzzer processes needed to use
the same coverage file.
* Web interface stats now update even when unique runs is not changing.
* Fixed tokenizer.testPropertiesUpheld to allow stray carriage returns
since they are valid whitespace.
I’ve been typing `zig fmt **/.zig` for a long time, until I discovered
that the argument can actually be a directory.
Mention this feature explicitly in the help message.
* Remove the generic model; we already have generic_la32 and generic_la64 and
pick appropriately based on bitness.
* Remove the loongarch64 model. We used this as our baseline for 64-bit, but it's
actually pretty misleading and useless; it doesn't represent any real CPU and
has less features than generic_la64.
* Add la64v1_0 and la64v1_1 models.
* Change our baseline CPU model for 64-bit to be la64v1_0, thus adding LSX to
the baseline feature set.
Ascon is the family of cryptographic constructions standardized by NIST
for lightweight cryptography.
The Zig standard library already included the Ascon permutation itself,
but higher-level constructions built on top of it were intentionally
postponed until NIST released the final specification.
That specification has now been published as NIST SP 800-232:
https://csrc.nist.gov/pubs/sp/800/232/final
With this publication, we can now confidently include these constructions
in the standard library.
* std.sort.pdq: fix out-of-bounds access in partialInsertionSort
When sorting a sub-range that doesn't start at index 0, the
partialInsertionSort function could access indices below the range
start. The loop condition `while (j >= 1)` didn't respect the
arbitrary range boundaries [a, b).
This changes the condition to `while (j > a)` to ensure indices
never go below the range start, fixing the issue where pdqContext
would access out-of-bounds indices.
Fixes#25250
In ed25519.zig, we checked if a test succeeds, in which case we
returned an error. This was confusing, and Andrew pointed out that
Zig weights branches against errors by default.
* test: remove test-compare-output and test-asm-link tests
These were low value and unfocused tests. We already have coverage of the
important aspects of these tests elsewhere. Additionally, there was really no
need for these to have their own test harness.
* test: rename issue_8550 standalone test to compile_asm
* test: rename backend=stage2 to backend=selfhosted, and add backend=auto
backend=auto (now the default if backend is omitted) means to let the compiler
pick whatever backend it wants as the default. This is important for platforms
where we don't yet have a self-hosted backend, such as loongarch64.
Also purge a bunch of redundant target=native.
* test: delete old stage1 compile_errors tests
generic_function_returning_opaque_type.zig was salvaged as it's still worth
having.
* test: pull tests in test/cases/llvm/ up to test/cases/
There is nothing inherently LLVM-specific about any of these.
* test: remove @cImport usage in interdependent_static_c_libs
* test: move glibc_compat from link to standalone tests
This is not really testing the linker.
* build: -Dskip-translate-c now implies -Dskip-run-translated-c
* build: skip test-cimport when -Dskip-translate-c is given
backend=auto (now the default if backend is omitted) means to let the compiler
pick whatever backend it wants as the default. This is important for platforms
where we don't yet have a self-hosted backend, such as loongarch64.
Also purge a bunch of redundant target=native.
These were low value and unfocused tests. We already have coverage of the
important aspects of these tests elsewhere. Additionally, there was really no
need for these to have their own test harness.
The Zig standard library lacked schemes that resist nonce reuse.
AES-SIV and AES-GCM-SIV are the standard options for this.
AES-GCM-SIV can be very useful when Zig is used to target embedded
systems, and AES-SIV is especially useful for key wrapping.
Also take it as an opportunity to add a bunch of test vectors to
modes.ctr and make sure it works with block ciphers whose size is
not 16.
This bug was manifesting for user as a nasty link error because they
were calling their application's main entry point as a coerced function,
which essentially broke reference tracking for the entire ZCU, causing
exported symbols to silently not get exported.
I've been a little unsure about how coerced functions should interact
with the unit graph before, but the solution is actually really obvious
now: they shouldn't! `Sema` is now responsible for unwrapping
possibly-coerced functions *before* queuing analysis or marking unit
references. This makes the reference graph optimal (there are no
redundant edges representing coerced versions of the same function) and
simplifies logic elsewhere at the expense of just a few lines in Sema.
The switch from @bitCast() to @intCast() here safety-checks
Linux's assertion that these 3 calls never return errors (negative
values as pid_t). getppid() can legally return 0 if the parent is
in a different pid namespace, but this is not an error.
When not linking libc on 64-bit Linux and calling posix.setsid(),
we get a type error at compile time inside of posix.errno(). This
is because posix.errno()'s non-libc branch expects a usize-sized
value, which is what all the error-returning os.linux syscalls
return, and linux.setsid() instead returned a pid_t, which is only
32 bits wide.
This and the other 3 pid-related calls just below it (getpid(),
getppid(), and gettid()) are the only Linux syscall examples here
that are casting their return values to pid_t. For the other 3
this makes sense: those calls are documented to have no possible
errors and always return a valid pid_t value.
However, setsid() actually can return the error EPERM, and
therefore needs to return the raw value from syscall0 for
posix.errno() to process like normal.
Additionally, posix.setsid() needs an @intCast(rc) for the success
case as a result, like most other such cases.
We need std.os.linux and std.c to agree on the types here, or else
we'd have to pointlessly cast across the difference up in the
std.posix wrapper. I ran into this as a type error the first time
I tried to compile my code that calls posix.socketpair() on Linux
without libc.
All of our existing socket calls with these kinds of arguments in
std (including the existing c.socketpair as well as
os.linux.socket in this same file) use unsigned for all of these
parameters, and so this brings linux.socketpair() into alignment
with everything else.
FreeBSD normally provides this symbol in libc, but it's in the
FBSDprivate_1.0 namespace, so it doesn't get included in our abilists file.
Fortunately, the implementation is identical for Linux and FreeBSD, so we can
just provide it in compiler-rt.
It's interesting to note that the same is not true for NetBSD where the
implementation is more complex to support older Arm versions. But we do include
the symbol in our abilists file for NetBSD libc, so that's fine.
closes#25215
* Make cat in test/standalone/simple working again
- Fixes:
zig/0.15.1/lib/zig/std/Io/Writer.zig:939:11: 0x1049aef63 in sendFileAll (nclip)
assert(w.buffer.len > 0);
- because we are no using non zero buffers for stdout - "do not forget to flush"
* replace std.fs with fs because we are already importing it
This instruction actually has fairly useless semantics, and even the
cases that were semantically correct could save 1 cycle of latency by
using a different sequnce involving the avx version instead.
Closes#25174
Re-enable the test. Will trigger #24380 as-is, but follow-on change moes
this code over to test/standalone.
Make the test a bit easier to debug by stashing the "seen" signal number
in the shared `seen_sig` (instead of just incrementing a counter for each
hit). And only doing so if the `seen_sig` is zero.
These tests aren't (directly) using Posix APIs, so they don't need to be
in posix/test.zig. Put them over with the code and tests in Thread.zig.
Since the spawn/join test in the posix code was redundant, just dropped
that one.
Because these lists are very long in several cases and quite
varied, I opted to place them in the existing c/foo.zig files.
There are many other sets of network-related constants like this
to add over time across all the OSes. For now I picked these
because I needed a few constants from each of these namespaces for
my own project, so I tried to flesh out these namespaces
completely as best I could, at least for basic sockopt purposes.
Note windows has some of these already defined in ws2_32 as
individual constants rather than contained in a namespacing
struct. I'm not sure what to do with that in the long run (break
it and namespace them?), but this doesn't change the status quo
for windows in any case.
in_pktinfo is only used on a few targets for the IP_PKTINFO
sockopt, as many BSDs use an alternate mechanism (IP_RECVDSTADDR)
that doesn't require a special struct. in6_pktinfo is more
universal.
This is the struct type used as set/getsockopt() option data with
SO.ACCEPTFILTER, which is also only declared on this same limited
set of BSD-ish targets.
In theory this could be aliased over to std.posix as well, but I
think for a corner case like this, it's not unreasonable for a
user that is avoiding uneccessary std.c references to access it as
"posix.system.accept_filter_arg" (which would still work fine if,
in the future, FreeBSD escapes its libc dep and defines it in
std.os.freebsd).
missing `extern` on a struct.
but also all these instances that call pwriteAll with a `@ptrCast` are
endianness bugs.
this should be changed to use File.Writer and call writeSliceEndian
instead.
this commit fixes one immediate problem but does not fix everything.
Add verifyStrict() functions for cofactorless verification.
Also:
- Support messages < 64 characters in the test vectors
- Allow mulDoubleBasePublic to return the identity as a regular
value. There are valid use cases for this.
I noticed this by stress testing my tls server implementation. From time to time curl (and other tools: ab, vegeta) will report invalid signature. I trace the problem to the way how std lib is encoding raw signature into der format. Using raw signature I got in some cases different encoding using std and openssl. Std is not producing minimal der when signature `r` or `s` integers has leading zero(es).
Here is an example to illustrate difference. Notice leading 00 in `s`
integer which is removed in openssl encoding but not in std encoding.
```Zig
const std = @import("std");
test "ecdsa signature to der" {
// raw signature r and s bytes
const raw = hexToBytes(
\\ 49 63 0c 94 95 2e ff 4b 02 bf 35 c4 97 9e a7 24
\\ 20 dc 94 de aa 1b 17 ff e1 49 25 3e 34 ef e8 d0
\\ c4 43 aa 7b a9 f3 9c b9 f8 72 7d d7 0c 9a 13 1e
\\
\\ 00 56 85 43 d3 d4 05 62 a1 1d d8 a1 45 44 b5 dd
\\ 62 9f d1 e0 ab f1 cd 4a 85 d0 1f 5d 11 d9 f8 89
\\ 89 d4 59 0c b0 6e ea 3c 19 6a f7 0b 1a 4a ce f1
);
// encoded by openssl
const expected = hexToBytes(
\\ 30 63 02 30
\\ 49 63 0c 94 95 2e ff 4b 02 bf 35 c4 97 9e a7 24
\\ 20 dc 94 de aa 1b 17 ff e1 49 25 3e 34 ef e8 d0
\\ c4 43 aa 7b a9 f3 9c b9 f8 72 7d d7 0c 9a 13 1e
\\
\\ 02 2f
\\ 56 85 43 d3 d4 05 62 a1 1d d8 a1 45 44 b5 dd
\\ 62 9f d1 e0 ab f1 cd 4a 85 d0 1f 5d 11 d9 f8 89
\\ 89 d4 59 0c b0 6e ea 3c 19 6a f7 0b 1a 4a ce f1
);
// encoded by std
const actual = hexToBytes(
\\ 30 64 02 30
\\ 49 63 0c 94 95 2e ff 4b 02 bf 35 c4 97 9e a7 24
\\ 20 dc 94 de aa 1b 17 ff e1 49 25 3e 34 ef e8 d0
\\ c4 43 aa 7b a9 f3 9c b9 f8 72 7d d7 0c 9a 13 1e
\\
\\ 02 30
\\ 00 56 85 43 d3 d4 05 62 a1 1d d8 a1 45 44 b5 dd
\\ 62 9f d1 e0 ab f1 cd 4a 85 d0 1f 5d 11 d9 f8 89
\\ 89 d4 59 0c b0 6e ea 3c 19 6a f7 0b 1a 4a ce f1
);
_ = actual;
const Ecdsa = std.crypto.sign.ecdsa.EcdsaP384Sha384;
const sig = Ecdsa.Signature.fromBytes(raw);
var buf: [Ecdsa.Signature.der_encoded_length_max]u8 = undefined;
const encoded = sig.toDer(&buf);
try std.testing.expectEqualSlices(u8, &expected, encoded);
}
pub fn hexToBytes(comptime hex: []const u8) [removeNonHex(hex).len / 2]u8 {
@setEvalBranchQuota(1000 * 100);
const hex2 = comptime removeNonHex(hex);
comptime var res: [hex2.len / 2]u8 = undefined;
_ = comptime std.fmt.hexToBytes(&res, hex2) catch unreachable;
return res;
}
fn removeNonHex(comptime hex: []const u8) []const u8 {
@setEvalBranchQuota(1000 * 100);
var res: [hex.len]u8 = undefined;
var i: usize = 0;
for (hex) |c| {
if (std.ascii.isHex(c)) {
res[i] = c;
i += 1;
}
}
return res[0..i];
}
```
Trimming leading zeroes from signature integers fixes encoding.
Also, added EPIPE to recvfrom() error set (it's a documented error
for unix and tcp sockets, at least), which recvmsg() largely
shares. Windows has an odd, callback-based form of recvmsg() that
doesn't fit the normal interface here.
socketpair is something like a pipe2() for sockets, and generally
only works for AF_UNIX sockets for most platforms. Winsock2
explicitly does not support this call, even though it does have
AF_UNIX sockets.
* Document std.mem.* functions
Functions in std.mem are essential for virtually all applications,
yet many of them lacked documentation.
Co-authored-by: Andrew Kelley <andrew@ziglang.org>
This reverts commit ac42eaaadd0650ffc281f9a1ed1a642fde8984b7, reversing
changes made to 9fa2394f8c00d060931d69fb6f342f7f2e3d826e.
I would like a chance to review this, please. I already spotted some
issues.
- introduce seekToUnbuffered which asserts no buffered data and does not
have WriteFailed in the error set
- remove WriteFailed from SeekError
- make seekTo based on calling flush and then seekToUnbuffered
- revert the change to reset seek_err since the error sets are
compatible again
Call start/endBlock before/after `parseBlockInfoBlock` in order to not
use the current block context, which is wrong and leads to e.g. incorrect
abbrevlen being used.
Previously we had a single definition of std.c.cmsghdr for all
libc-linking platforms which aliased from the Solaris definition,
a superfluous matching one in std.os.dragonfly, and no others.
The existing definition from std.c didn't actually work for Linux,
as Linux's "len" field is usize in the kernel's definition.
Emscripten follows the Linux model of course (but uses the
binary-compatible musl definition, which has an endian-sensitive
padding scheme to make the len type "socklen_t" even though the
kernel uses a usize, which is fair).
This unifies and documents all the known *nix-ish cases (I'm not
sure if wasi or windows really has cmsghdr support? Could be added
later, void for now), such that c.cmsghdr and posix.system.cmsghdr
should work correctly for all the known cases here, libc or
otherwise.
This reverts commit f4ed35f800396f12c7cd6aa1f70cf2555ddf7c84.
This should no longer be needed now that stack traces are disabled on loongarch.
closes#24405
lzma2 Decoder already checks if decoding is finished or not inside the
process function, `range_decoder`finish does not mean the decoder has
finished, also need to check `ld.rep[0] == 0xFFFF_FFFF`, which was
already done inside the proccess function. This fix delete the redundant
`isFinish()` check for `range_decoder`.
Before this commit, -Mfoo=bar=baz would be incorrectly split into mod_name: `foo` and root_src_orig: `bar`
After this commit, -Mfoo=bar=baz will be correctly split into mod_name: `foo` and root_src_orig: `bar=baz`
Closes#25059
* update the MSG struct with the correct values for openbsd
* add comment with link to sys/sys/socket.h
---------
Co-authored-by: Brandon Mercer <bmercer@eutonian.com>
LLVM 21 has started recognizing strlen-like idioms and optimizing them to strlen
calls, so we need this function provided in compiler-rt for libc-less
compilations.
Before this commit, calling appendRemaining with an ArrayList where list.items.len != list.capacity could result in illegal behavior if the Writer.Allocating resized the list during the appendRemaining call.
Fixes#25057
We can't call `@frameAddress()` and then immediately `return`! That
invalidates the frame. This *usually* isn't a problem, because the stack
walk `next` call will *probably* have a stack frame and it will
*probably* be at the exact same address, but neither of those is a
guarantee. On powerpc, presumably some unfortunate inlining was going
on, so this frame was indeed invalidated when we started walking frames.
We need to explicitly pass `@frameAddress` into any function which will
return before we actually walk the stack. Pretty simple patch.
Resolves: #24970
The TLS 1.2 implementation was incorrectly hardcoded to always send the
secp256r1 public key in the client key exchange message, regardless of
which elliptic curve the server actually negotiated.
This caused TLS handshake failures with servers that preferred other curves
like X25519.
This fix:
- Tracks the negotiated named group from the server key exchange message
- Dynamically selects the correct public key (X25519, secp256r1, or
secp384r1) based on what the server negotiated
- Properly constructs the client key exchange message with the
appropriate key size for each curve type
Fixes TLS 1.2 connections to servers like ziglang.freetls.fastly.net
that prefer X25519 over secp256r1.
Note the previous "28" here for openbsd was some kind of copy
error long ago. That's the value of KERN.SOMAXCONN, which is an
entirely different thing.
missing these things:
- implementation of finish()
- detect packed bytes read for check and block padding
- implementation of discard()
- implementation of block stream checksum
Fixes#23993
Previously, if multiple build processes tried to create the same args file, there was a race condition with the use of the non-atomic `writeFile` function which could cause a spawned compiler to read an empty or incomplete args file. This commit avoids the race condition by first writing to a temporary file with a random path and renaming it to the desired path.
* add macos handling for totalSystemMemory
* fix return type cast for .freebsd in totalSystemMemory
* add handling for the whole Darwin family in totalSystemMemory
This make `fs.Dir.access` has compatibility like the zig version before.
With this change the `zig build --search-prefix` command would work again like
the zig 0.14 version when used on Ubuntu22.04, kernel version 5.4.
This is my penance for baiting andrew into deleting the existing generic
queue data structures with my talk of "too many ring buffers".
The new Reader and Writer interfaces are excellent ring buffers for many
use cases, but a generic queue container type is now missing.
This new double-ended queue, known more succinctly as a deque, is
implemented from scratch based on the API design lessons learned from
ArrayList over the years.
The API is not yet as featureful as ArrayList, but the core
functionality is in place and I will be using this in my personal
projects shortly. I think it makes sense to add further functions as
needed based on real-world use-cases.
* Adds "flat" alternatives to zon.parse.from* that don't support pointers
* Fixes documentation
* Removes flat postfix from non allocating functions, adds alloc to others
* Stops using alloc variant in tests where not needed
It is important we copy the left-overs in the message *before* we XOR
it into the ciphertext, because if we're encrypting in-place (i.e., m ==
c), we will manipulate the message that will be used for tag generation.
This will generate faulty tags when message length doesn't conform with
16 byte blocks.
The big endian RISC-V effort is mostly driven by MIPS (the company) which is
pivoting to RISC-V, and presumably needs a big endian variant to fill the niche
that big endian MIPS (the ISA) did.
GCC already supports these targets, but LLVM support will only appear in 22;
this commit just adds the necessary target knowledge and checks on our end.
Without this change, the docs are formatted s.t. the text "Edge case rules ordered by precedence:" is appended onto the prior line of text "Underflow: Absolute value of result smaller than 1", instead of getting its own line.
This API is based around the unsound idea that a process can perform
checked virtual memory loads to prevent crashing. This depends on
OS-specific APIs that may be unavailable, disabled, or impossible due to
virtualization.
It also makes collecting stack traces ridiculously slow, which is a
problem for users of DebugAllocator - in other words, everybody, all the
time. It also makes strace go from being superbly clean to being awful.
AtomicFile.finish() calls flush() which renders any previous updateTimes() calls
useless. Regression introduced in f2a3ac7c0534a74ee544fdf6ef9d2176a8d62389.
Closes#24927.
Linux already gained the relevant syscalls and consts in #24473
The basic mlock() and munlock() are fairly universal across the
*nix world with a consistent interface, but are missing on wasi
and windows.
The mlockall() and munlockall() calls are not as widely supported
as the basic ones. Notable non-implementers include darwin,
haiku, and serenity (and of course wasi and windows again).
mlock2() is Linux-only, as are its MLOCK flags.
This mainly just moves stuff around.
Justifications for other changes:
* `KEVENT.FLAGS` is backed by `c_uint` because that's what the `kevent64` flags param takes (according to the 'latest' manpage from 2008)
* `MACH_RCV_NOTIFY` is a legacy name and `MACH_RCV_OVERWRITE` is deprecated (xnu/osfmk/mach/message.h), so I removed them. They were 0 anyway and thus couldn't be represented
as a packed struct field.
* `MACH.RCV` and `MACH.SEND` are technically the same 'type' because they can both be supplied at the same time to `mach_msg`. I decided to still keep them separate because
naming works out better that way and all flags except for `MACH_MSG_STRICT_REPLY` aren't shared anyway. Both are part of a packed union `mach_msg_option_t` which supplies a
helper function to combine the two types.
* `PT` is backed by `c_int` because that's what `ptrace` takes as a request arg (according to the latest manpage from 2015)
* extend std.Io.Reader.peekDelimiterExclusive test to repeat successful end-of-stream path (fails)
* fix std.Io.Reader.peekDelimiterExclusive to not advance seek position in successful end-of-stream path
When an error response was encountered, such as 404 not found, the body
wasn't discarded, leading to the string "404 not found" being
incorrectly interpreted as the next request's response.
closes#24732
It doesn't really make sense for `target_util.canBuildLibCompilerRt`
(and its ubsan-rt friend) to take in `use_llvm`, because the caller
doesn't control that: they're just going to queue a sub-compilation for
the runtime. The only exception to that is the ZCU strategy, where we
effectively embed `_ = @import("compiler_rt")` into the Zig compilation:
there, the question does matter. Rather than trying to do multiple weird
calls to model this, just have `canBuildLibCompilerRt` return not just a
boolean, but also differentiate the self-hosted backend being capable of
building the library vs only LLVM being capable. Logic in `Compilation`
uses that difference to decide whether to use the ZCU strategy, and also
to disable the library if the compiler does not support LLVM and it is
required.
Also, remove a redundant check later on, when actually queuing jobs.
We've already checked that we can build `compiler_rt`, and
`compiler_rt_strat` is set accordingly. I'm guessing this was there to
work around a bug I saw in the old strategy assignment, where support
was ignored in some cases.
Resolves: #24623
they seem to be always `null` even when accessed through extern key so we have no way to tell whether they have natural alignment or not to decorate. And the reason we don't always decorate them is because some environments might be too dumb and crash for this.
This is theoretically a bugfix as well, since it enforces the correct
limit on the first write after writing the header. This theoretical bug
hasn't been hit in practice though as far as I know.
Writer.sendFileAll() asserts non-zero buffer capacity in the case that
the fallback is hit. It also requires the caller to flush. The buffer
may be bypassed as an optimization but this is not a guarantee.
Also improve the Writer documentation and add an earlier assert on
buffer capacity in sendFileAll().
On macOS, when using the LLVM backend, the output binary retains a
reference to this object file's debug info (as opposed to self-hosted
backends which instead emit a dSYM bundle). As such, we need to retain
this object file in such cases. This object does unfortunately "leak",
in that it won't be reused and will just sit in the cache forever (or
until GC'd in the future). But that's no worse than the cache behavior
prior to the rework that caused this, and it will become less of a
problem over time as the self-hosted backend gains usability for debug
builds and eventually becomes the default.
Resolves: #24369
* std.Io.Reader: fix confused semantics of rebase. Before it was
ambiguous whether it was supposed to be based on end or seek. Now it
is clearly based on seek, with an added assertion for clarity.
* std.crypto.tls.Client: fix panic due to not enough buffer size
available. Also, avoid unnecessary rebasing.
* std.http.Reader: introduce max_head_len to limit HTTP header length.
This prevents crash in underlying reader which may require a minimum
buffer length.
* std.http.Client: choose better buffer sizes for streams and TLS
client. Crucially, the buffer shared by HTTP reader and TLS client
needs to be big enough for all http headers *and* the max TLS record
size. Bump HTTP header size default from 4K to 8K.
fixes#24872
I have noticed however that there are still fetch problems
Previously, index out-of-bounds could occur when copying match_length bytes while decoding whatever sequence happened to overflow `dest`. Now, each sequence checks that there is enough room for the full sequence_length (literal_length + match_length) before doing any copying.
Fixes the failing inputs found here: https://github.com/ziglang/zig/issues/24817#issuecomment-3192927715
As well as the exact byte count, include a human-readable value so it's
clearer what the error is actually telling you. The exact byte count
might not be worth keeping, but I decided I would in case it's useful in
any scenario.
In the best case, this is redundant work, because we aren't actually
going to emit a working binary this update. In the worst case, it causes
bugs because the linker may not have *seen* the thing being exported due
to the compile errors.
Resolves: #24417
* std.Io.Reader: appendRemaining no longer supports alignment and has
different rules about how exceeding limit. Fixed bug where it would
return success instead of error.StreamTooLong like it was supposed to.
* std.Io.Reader: simplify appendRemaining and appendRemainingUnlimited
to be implemented based on std.Io.Writer.Allocating
* std.Io.Writer: introduce unreachableRebase
* std.Io.Writer: remove minimum_unused_capacity from Allocating. maybe
that flexibility could have been handy, but let's see if anyone
actually needs it. The field is redundant with the superlinear growth
of ArrayList capacity.
* std.Io.Writer: growingRebase also ensures total capacity on the
preserve parameter, making it no longer necessary to do
ensureTotalCapacity at the usage site of decompression streams.
* std.compress.flate.Decompress: fix rebase not taking into account seek
* std.compress.zstd.Decompress: split into "direct" and "indirect" usage
patterns depending on whether a buffer is provided to init, matching
how flate works. Remove some overzealous asserts that prevented buffer
expansion from within rebase implementation.
* std.zig: fix readSourceFileToAlloc returning an overaligned slice
which was difficult to free correctly.
fixes#24608
The previous code assumed that `initFrame` during the `new_frame` state would always result in the `in_frame` state, but that's not always the case. `initFrame` can also result in the `skippable_frame` state, which would lead to access of union field 'in_frame' while field 'skipping_frame' is active.
Now, the switch is re-entered with the updated state so either case is handled appropriately.
Fixes the crashes from https://github.com/ziglang/zig/issues/24817
Previously, the "allow EndOfStream" part of this logic was too permissive. If there are a few dangling bytes at the end of the stream, that should be treated as a bad magic number. The only case where EndOfStream is allowed is when the stream is truly at the end, with exactly zero bytes available.
Validate wildcard certificates as specified in RFC 6125.
In particular, `*.example.com` should match `foo.example.com` but
NOT `bar.foo.example.com` as it previously did.
The Lua headers are needed because, yes, NetBSD has a kernel module for Lua
support. soundcard.h is technically a system header but is installed by
libossaudio and so was missed previously.
This also removes some riscv headers that shouldn't have been added because
NetBSD does not yet officially support the riscv32/riscv64 ports.
Closes#24737.
Newer 32-bit Linux targets like 32-bit RISC-V only use the 64-bit
time ABI, with these syscalls having `time64` as their suffix.
This is a stopgap solution in favor of a full audit of `std.os.linux` to
prepare for #4726.
See also #21440 for prior art.
The generic syscall table has different names for syscalls that take a
timespec64 on 32-bit targets, in that it adds the `_time64` suffix.
Similarly, the `_time32` suffix has been removed.
I'm not sure if the existing logic for determining the proper timespec
struct to use was subtly broken, but it should be a good chance to
finish #4726 - we only have 12 years after all...
As for the changes since 6.11..6.16:
6.11:
- x86_64 gets `uretprobe`, a syscall to speed up returning BPF probes.
- Hexagon gets `clone3`, but don't be fooled: it just returns ENOSYS.
6.13:
- The `*xattr` family of syscalls have been enhanced with new `*xattrat`
versions, similar to the other file-based `at` calls.
6.15:
- Atomically create a detached mount tree and set mount options on it.
Finally, this commit also adds the syscall numbers for OpenRISC and maps
it to the `or1k` cpu.
Changes by Arnd Bergmann have migrated all supported architectures to
use a table for their syscall lists. This removes the need to use the
C pre-processor and simplifies the logic considerably.
All currently supported architectures have been added, with the ones Zig
doesn't support being commented out. Speaking of; OpenRisc has been
enabled for generation.
A little clunky -- maybe the frontend should give an answer here -- but
this patch makes sense with the surrounding logic just to fix the crash.
Resolves: #24265
`limit` in chunkedSendFile applies only to the file, not the entire
chunk. `limit` in sendFileHeader does not include the header.
Additionally adds a comment to clarify what `limit` applies to in
sendFileHeader and fixed a small bug in it (`drain` is able to return
less then `header.len`).
The LLVM backend lowers unions where all fields are zero-bit as
equivalent to their backing enum, and expects them to have the same
by-ref-ness in at least one place in the backend, probably more.
Resolves: #23577
This "get" is useless noise and was copied from FixedBufferWriter.
Since this API has not yet landed in a release, now is a good time
to make the breaking change to fix this.
`Aegis256XGeneric` behaves differently than `Aegis128XGeneric` in that
it currently encrypts associated data instead of just absorbing it. Even
though the end result is the same, there's no point in encrypting and
copying the ad into a buffer that gets overwritten anyway. This fix
makes `Aegis256XGeneric` behave the same as `Aegis128XGeneric`.
According to https://apilevels.com, 88.5% of Android users are on 29+. Older API
levels require libc as of https://github.com/ziglang/zig/pull/24629, which has
confused some users. Seems reasonable to bump the default so most people won't
be confused by this.
This commit expands on the foundations laid by https://github.com/ziglang/zig/pull/23177
and moves even more `Sema`-only functionality from `Value`
to `Sema.arith`. Specifically all shift and bitwise operations,
`@truncate`, `@bitReverse` and `@byteSwap` have been moved and
adapted to the new rules around `undefined`.
Especially the comptime shift operations have been basically
rewritten, fixing many open issues in the process.
New rules applied to operators:
* `<<`, `@shlExact`, `@shlWithOverflow`, `>>`, `@shrExact`: compile error if any operand is undef
* `<<|`, `~`, `^`, `@truncate`, `@bitReverse`, `@byteSwap`: return undef if any operand is undef
* `&`, `|`: Return undef if both operands are undef, turn undef into actual `0xAA` bytes otherwise
Additionally this commit canonicalizes the representation of
aggregates with all-undefined members in the `InternPool` by
disallowing them and enforcing the usage of a single typed
`undef` value instead. This reduces the amount of edge cases
and fixes a bunch of bugs related to partially undefined vecs.
List of operations directly affected by this patch:
* `<<`, `<<|`, `@shlExact`, `@shlWithOverflow`
* `>>`, `@shrExact`
* `&`, `|`, `~`, `^` and their atomic rmw + reduce pendants
* `@truncate`, `@bitReverse`, `@byteSwap`
This algorithm is non-trivial and makes sense for any data structure
that acts as an array list, so I thought it would make sense as a
method.
I have a real world case for this in a music player application
(deleting queue items).
Adds the method to:
* ArrayList
* ArrayHashMap
* MultiArrayList
This experimental target was never fully completed. The operating system
is not that interesting or popular anyway, and the maintainer is no
longer around.
Not worth the maintenance burden. This code can be resurrected later if
it is worth it. In such case it will be subject to greater scrutiny.
This is one way of partially addressing https://github.com/ziglang/zig/issues/24767
- These functions are unused
- These functions are untested
- These functions are broken
+ The same dangling pointer bug from 6219c015d8e8c958d96e5caa5ef0dbab9c414996 exists in `writePreserve`
+ The order of the bytes preserved in relation to the `bytes` being written can differ depending on unused buffer capacity at the time of the call and the drain implementation.
If there ends up being a need for these functions, they can be fixed and added back.
This commit re-enables the --webui functionality on windows, with the caveat that rebuild functionality is still disabled (due to deadlocks caused by reading to / writing from the same non-overlapped socket on multiple threads). I updated the UI to be aware of this, and hide the `Rebuild` button.
http.Server: Remove incorrect advance() call. This was causing browsers to disconnect the websocket, as we were sending undefined bytes.
build.WebServer: Re-enable on windows, but disable functionality that requires receiving messages from the client
build-web: Show total times in tables
The "completed" count in the "Semantic Analysis" progress node had
regressed since 0.14.0: the number got crazy big very fast, even on
simple cases. For instance, an empty `pub fn main` got to ~59,000 where
on 0.14 it only reached ~4,000. This was happening because I was
unintentionally introducing a node every time type resolution was
*requested*, even if (as is usually the case) it turned out to already
be done. The fix is simply to start the progress node a little later,
once we know we are actually doing semantic analysis. This brings the
number for that empty test case down to ~5,000, which makes perfect
sense. It won't exactly match 0.14, because the standard library has
changed, and also because the compiler's progress output does have some
*intentional* changes.
The functions `Compilation.create` and `Compilation.update` previously
returned inferred error sets, which had built up a lot of crap over
time. This meant that certain error conditions -- particularly certain
filesystem errors -- were not being reported properly (at best the CLI
would just print the error name). This was also a problem in
sub-compilations, where at times only the error name -- which might just
be something like `LinkFailed` -- would be visible.
This commit makes the error handling here more disciplined by
introducing concrete error sets to these functions (and a few more as a
consequence). These error sets are small: errors in `update` are almost
all reported via compile errors, and errors in `create` are reported
through a new `Compilation.CreateDiagnostic` type, a tagged union of
possible error cases. This allows for better error reporting.
Sub-compilations also report errors more correctly in several cases,
leading to more informative errors in the case of compiler bugs.
Also fixes some race conditions in library building by replacing calls
to `setMiscFailure` with calls to `lockAndSetMiscFailure`. Compilation
of libraries such as libc happens on the thread pool, so the logic must
synchronize its access to shared `Compilation` state.
While underlying writer is Allocating writer buffer can grow in
vtable.drain call. We should not hold pointer to the buffer before that
call and use it after.
This remembers positions instead of holding reference.
Running tar.pipeToFileSystem compressed_mingw_includes.tar file from #24732
finishes in infinite loop calling defaultReadVec with:
r.seek = 1024
r.end = 1024
r.buffer.len = 1024
first.len = 512
that combination calls vtable.stream with 0 capacity writer and loops
forever.
Comment is to use whichever has larger capacity, and this fix reflects that.
This way, if the ci-riscv64-linux label was added to a PR previously, removing
it will cause the concurrency group of the workflow to cancel the runs triggered
by the label being added.
It's a bit counter-intuitive, but there are two streams here: the
implementation here, and the connected output stream.
When we say "unflushed" we mean don't flush the connected output stream
because that's managed externally. But an "end" operation should always
flush the implementation stream.
Previously, when extracting a ZIP file, isBadFilename(), which is
designed to reject ../ patterns to prevent directory traversal, was
called before normalizing backslashes to forward slashes.
This allowed path traversal sequences like ..\\..\\..\\etc\\passwd
which pass validation but are then converted to ../../../etc/passwd
for file extraction.
Don't see why byte returned from specialPeek needs to be shifted by
remaining_needed_bits.
I believe that decision in specialPeek should be done on the number of
the remaining bits not of the content of that bits.
Some test result are changed, but they are now consistent with the
original state as found in:
5f790464b0/lib/std/compress/flate/Decompress.zig
Changing Bits from usize to u32 or u64 now returns same results.
* flate: simplify peekBitsEnding
`peekBits` returns at most asked number of bits. Fails with EndOfStream
when there are no available bits. If there are less bits available than
asked still returns that available bits.
Hopefully this change better reflects intention. On first input stream
peek error we break the loop.
If both are used, 'else' handles named members and '_' handles
unnamed members. In this case the 'else' prong will be unrolled
to an explicit case containing all remaining named values.
Mainly affects ZIR representation of switch_block[_ref]
and special prong (detection) logic for switch.
Adds a new SpecialProng tag 'absorbing_under' that allows
specifying additional explicit tags in a '_' prong which
are respected when checking that every value is handled
during semantic analysis but are not transformed into AIR
and instead 'absorbed' by the '_' branch.
In trying to reproduce the race in #24380, my system tripped over the stat
"blocks" field changing in this test. The value was almost always 8
(effectively 4k) or very infrequently 0 (I saw the 0 from both `fstat` and
`fstatat`). I believe the underlying filesystem is free to asynchronously
change this value. For example, if it migrates a file between some
"inline" or maybe journal storage, and actual on-disk blocks. So it seems
plausible that its allowed to change between stat calls.
Breaking up the struct comparison this way means we also don't compare any
of the padding or "reserved" fields, too. And we can narrow down the
s390x-linux work-around.
The `atime()`, etc wrappers here expect to create a `std.linux.timespec`
(defined in `linux.zig` to have `isize` fields), so the u32 causes errors:
error: expected type 'isize', found 'u32'
.nsec = self.atim_nsec,
Make the nsec fields signed for consistency with all the other structs,
with and with `std.linux.timespec`.
Also looks like the comment on `__pad1` was copied from `__pad0`, but it
only applies to `__pad0`.
If an error occured which prevented a prelink task from being queued,
then `pending_prelink_tasks` would never be decremented, which could
cause deadlocks in some cases. So, instead of calculating ahead of time
the number of prelink tasks to expect, we use a simpler strategy which
is much like a wait group: we add 1 to a value when we spawn a worker,
and in the worker function, `defer` decrementing the value. The initial
value is 1, and there's a decrement after all of the workers are
spawned, so once it hits 0, prelink is done (be it with a failure or a
success).
This reverts commit b461d07a5464aec86c533434dab0b58edfffb331.
After some discussion in the team, we've decided that this is too disruptive,
especially because the linker errors are less than helpful. That's a fixable
problem, so we might reconsider this in the future, but revert it for now.
This use case is handled by ArrayListUnmanaged via the "...Bounded"
method variants, and it's more optimal to share machine code, versus
generating multiple versions of each function for differing array
lengths.
This was a regression in #24588.
I have verified that this patch works by confirming that with the
downstream patches SerenityOS apply to the Zig source tree (sans the one
working around this regression), I can build the build runner for
SerenityOS.
Resolves: #24682
This option is similar to `--debug-target` in letting us override
details of the build runner target when debugging the build system.
While `--debug-target` lets us override the target query, this option
lets us override the libc installation. This option is only usable in a
compiler built with debug extensions.
I am using this to (try to) test the build runner targeting SerenityOS.
This reverts commit fa445d86a110f1171b75824fe5ec139089fa4733.
Narrator: It did, in fact, make a difference.
For whatever reason, building LLVM against spacemit_x60 or baseline makes no
noticeable difference in terms of performance, but building the Zig compiler
against spacemit_x60 does. Also, the miscompilation that was causing
riscv64-linux-debug to fail was in the LLVM libraries, not in the Zig compiler,
so we may as well take the win here.
GitHub is apparently very bad at arithmetic and so will cancel jobs that pass
the 5 hours mark, even if they're nowhere near the 6 hours timeout. So add an
hour to assist GitHub in this very difficult task.
The support was already there but somebody forgot to allow to use the
calling conventions spirv_fragment and spirv_vertex when having opengl
as os tag.
Previously, this only applied when using `-fincremental --watch`, but
`--webui` makes the build runner stay alive just like `--watch` does, so
the same logic applies here. Without this, attempting to perform
incremental updates with `--webui` performs full rebuilds. (I did test
that before merging the PR, but at that time I was passing `--watch`
too -- which has since been disallowed -- so I missed that it doesn't
work as expected without that option!)
This commit replaces the "fuzzer" UI, previously accessed with the
`--fuzz` and `--port` flags, with a more interesting web UI which allows
more interactions with the Zig build system. Most notably, it allows
accessing the data emitted by a new "time report" system, which allows
users to see which parts of Zig programs take the longest to compile.
The option to expose the web UI is `--webui`. By default, it will listen
on `[::1]` on a random port, but any IPv6 or IPv4 address can be
specified with e.g. `--webui=[::1]:8000` or `--webui=127.0.0.1:8000`.
The options `--fuzz` and `--time-report` both imply `--webui` if not
given. Currently, `--webui` is incompatible with `--watch`; specifying
both will cause `zig build` to exit with a fatal error.
When the web UI is enabled, the build runner spawns the web server as
soon as the configure phase completes. The frontend code consists of one
HTML file, one JavaScript file, two CSS files, and a few Zig source
files which are built into a WASM blob on-demand -- this is all very
similar to the old fuzzer UI. Also inherited from the fuzzer UI is that
the build system communicates with web clients over a WebSocket
connection.
When the build finishes, if `--webui` was passed (i.e. if the web server
is running), the build runner does not terminate; it continues running
to serve web requests, allowing interactive control of the build system.
In the web interface is an overall "status" indicating whether a build
is currently running, and also a list of all steps in this build. There
are visual indicators (colors and spinners) for in-progress, succeeded,
and failed steps. There is a "Rebuild" button which will cause the build
system to reset the state of every step (note that this does not affect
caching) and evaluate the step graph again.
If `--time-report` is passed to `zig build`, a new section of the
interface becomes visible, which associates every build step with a
"time report". For most steps, this is just a simple "time taken" value.
However, for `Compile` steps, the compiler communicates with the build
system to provide it with much more interesting information: time taken
for various pipeline phases, with a per-declaration and per-file
breakdown, sorted by slowest declarations/files first. This feature is
still in its early stages: the data can be a little tricky to
understand, and there is no way to, for instance, sort by different
properties, or filter to certain files. However, it has already given us
some interesting statistics, and can be useful for spotting, for
instance, particularly complex and slow compile-time logic.
Additionally, if a compilation uses LLVM, its time report includes the
"LLVM pass timing" information, which was previously accessible with the
(now removed) `-ftime-report` compiler flag.
To make time reports more useful, ZIR and compilation caches are ignored
by the Zig compiler when they are enabled -- in other words, `Compile`
steps *always* run, even if their result should be cached. This means
that the flag can be used to analyze a project's compile time without
having to repeatedly clear cache directory, for instance. However, when
using `-fincremental`, updates other than the first will only show you
the statistics for what changed on that particular update. Notably, this
gives us a fairly nice way to see exactly which declarations were
re-analyzed by an incremental update.
If `--fuzz` is passed to `zig build`, another section of the web
interface becomes visible, this time exposing the fuzzer. This is quite
similar to the fuzzer UI this commit replaces, with only a few cosmetic
tweaks. The interface is closer than before to supporting multiple fuzz
steps at a time (in line with the overall strategy for this build UI,
the goal will be for all of the fuzz steps to be accessible in the same
interface), but still doesn't actually support it. The fuzzer UI looks
quite different under the hood: as a result, various bugs are fixed,
although other bugs remain. For instance, viewing the source code of any
file other than the root of the main module is completely broken (as on
master) due to some bogus file-to-module assignment logic in the fuzzer
UI.
Implementation notes:
* The `lib/build-web/` directory holds the client side of the web UI.
* The general server logic is in `std.Build.WebServer`.
* Fuzzing-specific logic is in `std.Build.Fuzz`.
* `std.Build.abi` is the new home of `std.Build.Fuzz.abi`, since it now
relates to the build system web UI in general.
* The build runner now has an **actual** general-purpose allocator,
because thanks to `--watch` and `--webui`, the process can be
arbitrarily long-lived. The gpa is `std.heap.DebugAllocator`, but the
arena remains backed by `std.heap.page_allocator` for efficiency. I
fixed several crashes caused by conflation of `gpa` and `arena` in the
build runner and `std.Build`, but there may still be some I have
missed.
* The I/O logic in `std.Build.WebServer` is pretty gnarly; there are a
*lot* of threads involved. I anticipate this situation improving
significantly once the `std.Io` interface (with concurrency support)
is introduced.
if I remove the last input byte from "don't read past deflate stream's
end" (on master branch), the test fails with error.EndOfStream. what,
then, is it supposed to be testing?
It's quite silly to have this override which nonetheless makes
assumptions about the input type. Encode the actual complexity of the
sort.
Also, simplify the sorting logic, and fix a bug (grab min and max
*after* the sort, not *before*!)
fsync blocks until the contents have been actually written to disk,
which would be useful if we didn't want to report success until having
achieved durability. But the OS will ensure coherency; i.e. if one
process writes stuff without calling fsync, then another process reads
that stuff, the writes will be seen even if they didn't get flushed to
disk yet.
Since this code deals with ephemeral cache data, it's not worth trying
to achieve this kind of durability guarantee. This is consistent with
all the other tooling on the system.
Certainly, if we wanted to change our stance on this, it would not be
something that affects only the git fetching logic.
Unlike all other platforms where RDONLY is 0 it does not work as a
default for the O flags on serenity - various syscalls other than
'open', e.g. 'pipe', return EINVAL if unexpected bits are set in the
flags.
when a Run step that captures stderr fails, no output from it is visible
by the user and, since the step failed, any downstream step that would
process the captured stream will not run, making it impossible for the
user to see the stderr output from the failed process invocation, which
makes for a frustrating puzzle when this happens in CI.
* Sema: remove redundant comptime-known initializer tracking
This logic predates certain Sema enhancements whose behavior it
essentially tries to emulate in one specific case in a problematic way.
In particular, this logic handled initializing comptime-known `const`s
through RLS, which was reworked a few years back in 644041b to not rely
on this logic, and catching runtime fields in comptime-only
initializers, which has since been *correctly* fixed with better checks
in `Sema.storePtr2`. That made the highly complex logic in
`validateStructInit`, `validateUnionInit`, and `zirValidatePtrArrayInit`
entirely redundant. Worse, it was also causing some tracked bugs, as
well as a bug which I have identified and fixed in this PR (a
corresponding behavior test is added).
This commit simplifies union initialization by bringing the runtime
logic more in line with the comptime logic: the tag is now always
populated by `Sema.unionFieldPtr` based on `initializing`, where this
previously happened only in the comptime case (with `validateUnionInit`
instead handling it in the runtime case). Notably, this means that
backends are now able to consider getting a pointer to an inactive union
field as Illegal Behavior, because the `set_union_tag` instruction now
appears *before* the `struct_field_ptr` instruction as you would
probably expect it to.
Resolves: #24520Resolves: #24595
* Sema: fix comptime-known union initialization with OPV field
The previous commit uncovered this existing OPV bug by triggering this
logic more frequently.
* Sema: remove dead logic
This is redundant because `storePtr2` will coerce to the return type
which (in `Sema.coerceInMemoryAllowedErrorSets`) will add errors to the
current function's IES if necessary.
* Sema: don't rely on Liveness
We're currently experimenting with backends which effectively do their
own liveness analysis, so this old trick of mine isn't necessarily valid
anymore. However, we can fix that trivially: just make the "nop"
instruction we jam into here have the right type. That way, the leftover
field/element pointer instructions are perfectly valid, but still
unused.
Wow, *lots* of backends were reliant on Sema doing the heavy lifting for
them. CBE, Wasm, and SPIR-V have all regressed in places now that they
actually need to, like, initialize unions and such.
We're currently experimenting with backends which effectively do their
own liveness analysis, so this old trick of mine isn't necessarily valid
anymore. However, we can fix that trivially: just make the "nop"
instruction we jam into here have the right type. That way, the leftover
field/element pointer instructions are perfectly valid, but still
unused.
This is redundant because `storePtr2` will coerce to the return type
which (in `Sema.coerceInMemoryAllowedErrorSets`) will add errors to the
current function's IES if necessary.
This logic predates certain Sema enhancements whose behavior it
essentially tries to emulate in one specific case in a problematic way.
In particular, this logic handled initializing comptime-known `const`s
through RLS, which was reworked a few years back in 644041b to not rely
on this logic, and catching runtime fields in comptime-only
initializers, which has since been *correctly* fixed with better checks
in `Sema.storePtr2`. That made the highly complex logic in
`validateStructInit`, `validateUnionInit`, and `zirValidatePtrArrayInit`
entirely redundant. Worse, it was also causing some tracked bugs, as
well as a bug which I have identified and fixed in this PR (a
corresponding behavior test is added).
This commit simplifies union initialization by bringing the runtime
logic more in line with the comptime logic: the tag is now always
populated by `Sema.unionFieldPtr` based on `initializing`, where this
previously happened only in the comptime case (with `validateUnionInit`
instead handling it in the runtime case). Notably, this means that
backends are now able to consider getting a pointer to an inactive union
field as Illegal Behavior, because the `set_union_tag` instruction now
appears *before* the `struct_field_ptr` instruction as you would
probably expect it to.
Resolves: #24520Resolves: #24595
Changes fmtId to return the FormatId type directly, and renames the
FormatId.render function to FormatId.format, so it can be used in a
format expression directly.
Why? Since `render` is private, you can't create functions that wrap
`fmtId` or `fmtIdFlags`, since you can't name the return type of those
functions outside of std itself.
The current setup _might_ be intentional? In which case I can live with
it, but I figured I'd make a small contrib to upstream zig :)
This eliminates a footgun and special case handling with fixed buffers,
as well as allowing decompression streams to keep a window in the output
buffer.
Not only are `Step.Compile` methods like `linkLibC()` redundant because
`Module` exposes the same APIs, it also might not be immediately obvious
to users that these methods modify the underlying root module, which can
be a footgun and lead to unintended results if the module is exported to
package consumers or shared by multiple compile steps.
Using `compile.root_module.link_libc = true` makes it more clear to
users which of the compile step and the module owns which options.
Also check that FileNotFound is consistently returned when the path is missing.
The new `run_relative` step will test spawning paths like:
child_path: ../84385e7e669db0967d7a42765011dbe0/child
missing_child_path: ../84385e7e669db0967d7a42765011dbe0/child_intentionally_missing
besides simply being redundant work, the now removed normalize call would cause
spawn to errantly fail (BadPath) when passing a relative path which traversed
'above' the current working directory. This case is already handled by leaving
normalization to the windows.wToPrefixedFileW call in
windowsCreateProcessPathExt
This passes tests but it doesn't provide as big a window size as is
required to decompress larger streams.
The next commit in this branch will work towards that, without
introducing an additional buffer.
- factor out `loadReg`
- support all general system control registers in inline asm
- fix asserts after iterating field offsets
- fix typo in `slice_elem_val`
- fix translation of argument locations
This option never worked properly (it emitted wrongly-formatted code),
and it doesn't seem particularly *useful* -- someone who's proficient
enough with `std.Build` to not need explanations probably just wants to
write their own thing. Meanwhile, the use case of writing your own
`build.zig` was extremely poorly served, because `build.zig.zon` *needs*
to be generated programmatically for a correct `fingerprint`, but the
only ways to do that were to a) do it wrong and get an error, or b) get
the full init template and delete the vast majority of it. Both of these
were pretty clunky, and `-s` didn't really help.
So, replace this flag with a new one, `--minimal`/`-m`, which uses a
different template. This template is trivial enough that I opted to just
hardcode it into the compiler for simplicity. The main job of
`zig init -m` is to generate a correct `build.zig.zon` (if it is unable
to do this, it exits with a fatal error). In addition, it will *attempt*
to generate a tiny stub `build.zig`, with only an `std` import and an
empty `pub fn build`. However, if `build.zig` already exists, it will
avoid overwriting it, and doesn't even complain. This serves the use
case of writing `build.zig` manually and *then* running `zig init -m`
to generate an appropriate `build.zig.zon`.
https://github.com/ziglang/zig/issues/23635
I also added tests for `@rem()` with `denominator < 0` cause there were none before
I hope I added them in the correct place, if not I can change it ofc
Its design keeps evolving. See
https://github.com/Nicoshev/rapidhash/releases
It's great to see the design improving, but over time, this will lead to
code rot; versions that aren't widely used but would still have to live
in the standard library forever and be maintained.
Better to be maintained as an external dependency that applications can
opt into. Then, in a few years, if a version proves to be stable and
widely adopted, it could be considered for inclusion in the standard
library.
The rejection of #6025 indicates that if stackless coroutines return to
Zig, they will look quite different; see #23446 for the working draft
proposal for their return (though it will definitely be tweaked before
being accepted). Some of this test coverage was deleted in 40d11cc, but
because stackless coroutines will take on a new form if re-introduced, I
anticipate that essentially *none* of this coverage will be relevant. Of
course, if it for some reason is, we can always grab it from the Git
history.
* std.os.uefi.protocol.file: use @alignCast in getInfo() method to fix#24480
* std.os.uefi.protocol.file: pass alignment responsabilities to caller by redefining the buffer type instead of blindly calling @alignCast
ensure that it issues a stream call that includes the buffer to detect
the end when needed, but otherwise does not offer Reader buffer to
append directly to the list.
LLVM always assumes these are on. Zig backends do not observe them.
If Zig backends want to start using them, they can be introduced, one
arch at a time, with proper documentation.
Soft float is a very rare use case for riscv*-linux. No point wasting CI
resources on these targets, especially since our arm and mips soft float
coverage is already likely to catch most soft float bugs.
these are taking too long. let's use a different workflow for now until
these runs are not holding up the pipeline, then they can be
reintroduced on master branch
Without this change, by default you get a failure when trying to cross
compile for these targets.
freebsd was error: undefined symbol: __libc_start1
netbsd was warning: invalid target NetBSD libc version: 9.4.0
error: unable to build NetBSD libc shared objects: InvalidTargetLibCVersion
now they work by default
don't forget to save the list. this allows a
`testing.checkAllAllocationFailures()` test to pass in one of my
projects which newly failed since #24329 was merged.
Rather than having the endian-suffixed functions be the preferred ones
the unsuffixed ones are the preferred ones and the tricky functions get
a special suffix.
Makes packed structs read and written the same as integers.
closes#12960
The idea is to have 2 runners per machine, since a lot of time is spent building
stage3 and stage4, both of which are largely single-core affairs. This will make
the test steps take longer, however, so the timeouts have been bumped a bit, and
max RSS for the test step has been lowered from 64G to 32G to prevent OOM.
Finally, we now only run a single ReleaseSafe job on PRs; Debug and Release jobs
are limited to pushes.
Add an additional check before emitting `.loop_switch_br` instead
of `.switch_br` in a tagged switch statement for whether any of the
continues referencing its tag are actually runtime reachable.
This fixes triggering an assertion in Liveness caused by the invalid
assumption that every tagged switch must be a loop if its tag is
referenced in any way even if this reference is not runtime reachable.
The rule: `pub fn main` owns file descriptors 0, 1, and 2. If you didn't
write `pub fn main()` it is, in general, not your business to print to
stderr.
As written, I think langref's example is actually a poor reason to use
`inline`.
If you have
if (foo(1200, 34) != 1234) {
@compileError("bad");
}
and you want to make sure that the call is executed at compile time, the
right way to fix it is to add comptime
if (comptime foo(1200, 34) != 1234) {
@compileError("bad");
}
and not to make the function `inline`. I _think_ that inlining functions
just to avoid `comptime` at a call-site is an anti-pattern. When the
reader sees `foo(123)` at the call-site, they _expect_ this to be a
runtime call, as that's the normal rule in Zig.
Inline is still necessary when you can't make the _whole_ call
`comptime`, because it has some runtime effects, but you still want
comptime-known result.
A good example here is
inline fn findImportPkgHashOrFatal(b: *Build, comptime asking_build_zig: type, comptime dep_name: []const u8) []const u8 {
from Build.zig, where the `b` argument is runtime, and is used for
side-effects, but where the result is comptime.
I don't know of a good small example to demonstrate the subtelty here,
so I went ahead with just adding a runtime print to `foo`. Hopefully
it'll be enough for motivated reader to appreciate the subtelty!
* std.os.uefi.tables: ziggify boot and runtime services
* avoid T{} syntax
Co-authored-by: linusg <mail@linusgroh.de>
* misc fixes
* work
* self-review quickfixes
* dont make MemoryMapSlice generic
* more review fixes, work
* more work
* more work
* review fixes
* update boot/runtime services references throughout codebase
* self-review fixes
* couple of fixes i forgot to commit earlier
* fixes from integrating in my own project
* fixes from refAllDeclsRecursive
* Apply suggestions from code review
Co-authored-by: truemedian <truemedian@gmail.com>
* more fixes from review
* fixes from project integration
* make natural alignment of Guid align-8
* EventRegistration is a new opaque type
* fix getNextHighMonotonicCount
* fix locateProtocol
* fix exit
* partly revert 7372d65
* oops exit data_len is num of bytes
* fixes from project integration
* MapInfo consistency, MemoryType update per review
* turn EventRegistration back into a pointer
* forgot to finish updating MemoryType methods
* fix IntFittingRange calls
* set uefi.Page nat alignment
* Back out "set uefi.Page nat alignment"
This backs out commit cdd9bd6f7f5fb763f994b8fbe3e1a1c2996a2393.
* get rid of some error.NotFound-s
* fix .exit call in panic
* review comments, add format method
* fix resetSystem data alignment
* oops, didnt do a final refAllDeclsRecursive i guess
* review comments
* writergate update MemoryType.format
* fix rename
---------
Co-authored-by: linusg <mail@linusgroh.de>
Co-authored-by: truemedian <truemedian@gmail.com>
This silences the excessive default stderr logging from Wine. The user can still
override this by setting WINEDEBUG in the environment; this just provides a more
sensible default.
Closes#24139.
Basically everything that has a direct replacement or no uses left.
Notable omissions:
- std.ArrayHashMap: Too much fallout, needs a separate cleanup.
- std.debug.runtime_safety: Too much fallout.
- std.heap.GeneralPurposeAllocator: Lots of references to it remain, not
a simple find and replace as "debug allocator" is not equivalent to
"general purpose allocator".
- std.io.Reader: Is being reworked at the moment.
- std.unicode.utf8Decode(): No replacement, needs a new API first.
- Manifest backwards compat options: Removal would break test data used
by TestFetchBuilder.
- panic handler needs to be a namespace: Many tests still rely on it
being a function, needs a separate cleanup.
Apparently raw LLVM IR Bitcode files ("Bitstreams") may appear in
archives with LTO enabled. I observed this in the wild on
Chimera Linux.
I'm not yet sure if it's in scope for Zig to support these special
archives, but we should at least give a correct error message.
Deprecates all existing std.io readers and writers in favor of the newly
provided std.io.Reader and std.io.Writer which are non-generic and have the
buffer above the vtable - in other words the buffer is in the interface, not
the implementation. This means that although Reader and Writer are no longer
generic, they are still transparent to optimization; all of the interface
functions have a concrete hot path operating on the buffer, and only make
vtable calls when the buffer is full.
Alignment and fill options only apply to numbers.
Rework the implementation to mainly branch on the format string rather
than the type information. This is more straightforward to maintain and
more straightforward for comptime evaluation.
Enums support being printed as decimal, hexadecimal, octal, and binary.
`formatInteger` is another possible format method that is
unconditionally called when the value type is struct and one of the
integer-printing format specifiers are used.
for structs, enums, and unions.
auto untagged unions are no longer printed as pointers; instead they are
printed as "{ ... }".
extern and packed untagged unions have each field printed, similar to
what gdb does.
also fix bugs in delimiter based reading
added adapter to AnyWriter and GenericWriter to help bridge the gap
between old and new API
make std.testing.expectFmt work at compile-time
std.fmt no longer has a dependency on std.unicode. Formatted printing
was never properly unicode-aware. Now it no longer pretends to be.
Breakage/deprecations:
* std.fs.File.reader -> std.fs.File.deprecatedReader
* std.fs.File.writer -> std.fs.File.deprecatedWriter
* std.io.GenericReader -> std.io.Reader
* std.io.GenericWriter -> std.io.Writer
* std.io.AnyReader -> std.io.Reader
* std.io.AnyWriter -> std.io.Writer
* std.fmt.format -> std.fmt.deprecatedFormat
* std.fmt.fmtSliceEscapeLower -> std.ascii.hexEscape
* std.fmt.fmtSliceEscapeUpper -> std.ascii.hexEscape
* std.fmt.fmtSliceHexLower -> {x}
* std.fmt.fmtSliceHexUpper -> {X}
* std.fmt.fmtIntSizeDec -> {B}
* std.fmt.fmtIntSizeBin -> {Bi}
* std.fmt.fmtDuration -> {D}
* std.fmt.fmtDurationSigned -> {D}
* {} -> {f} when there is a format method
* format method signature
- anytype -> *std.io.Writer
- inferred error set -> error{WriteFailed}
- options -> (deleted)
* std.fmt.Formatted
- now takes context type explicitly
- no fmt string
preparing to rearrange std.io namespace into an interface
how to upgrade:
std.io.getStdIn() -> std.fs.File.stdin()
std.io.getStdOut() -> std.fs.File.stdout()
std.io.getStdErr() -> std.fs.File.stderr()
Macos uses the BSD definition of msghdr
All linux architectures share a single msghdr definition. Many
architectures had manually inserted padding fields that were endian
specific and some had fields with different integers. This unifies all
architectures to use a single correct msghdr definition.
necessary because to pass `zig fmt --check` we need to use the canonical
identifier syntax, which means changing `.@"async"` to `.async` which
previous zig1 is unable to parse.
Also remove `@frameSize`, closing #3654.
While the other machinery might remain depending on #23446, it is
settled that there will not be `async`/ `await` keywords in the
language.
This matches what we do for small helper libraries like this in MinGW-w64. It
simplifies the compiler a bit, and also means the build system doesn't have to
treat these library names specially.
Closes#24325.
It's kind of unclear what `*-windows-none` actually means, but as far as LLVM is
concerned, it's equivalent to `*-windows-msvc`. For clarity, only test
`*-windows-msvc` and `*-windows-gnu`. #20690 will clean this situation up
eventually.
Also ensure coverage of `link_libc = true` and `link_libc = false` for both.
The overflow check for safe signed subtraction was using the formula (rhs < 0) == (lhs > result). This logic is flawed and incorrectly reports an overflow when the right-hand side is zero.
For the expression 42 - 0, this check evaluated to (0 < 0) == (42 > 42), which is false == false, resulting in true. This caused the generated SPIR-V to incorrectly branch to an OpUnreachable instruction, preventing the result from being stored.
Fixes#24281.
* resinator: Only preprocess when the input is an .rc file
* resinator: Fix include directory detection when cross-compiling from certain host archs
Previously, resinator would use the host arch as the target arch when looking for windows-gnu include directories. However, Zig only thinks it can provide a libc for targets specified in the `std.zig.target.available_libcs` array, which only includes a few for windows-gnu. Therefore, when cross-compiling from a host architecture that doesn't have a windows-gnu target in the available_libcs list, resinator would fail to detect the MinGW include directories.
Now, the custom option `/:target` is passed to `zig rc` which is intended for the COFF object file target, but can be re-used for the include directory target as well. For the include directory target, resinator will convert the MachineType to the relevant arch, or fail if there is no equivalent arch/no support for detecting the includes for the MachineType (currently 64-bit Itanium and EBC).
Fixes the `windows_resources` standalone test failing when the host is, for example, `riscv64-linux`.
Previously, resinator would use the host arch as the target arch when looking for windows-gnu include directories. However, Zig only thinks it can provide a libc for targets specified in the `std.zig.target.available_libcs` array, which only includes a few for windows-gnu. Therefore, when cross-compiling from a host architecture that doesn't have a windows-gnu target in the available_libcs list, resinator would fail to detect the MinGW include directories.
Now, the custom option `/:target` is passed to `zig rc` which is intended for the COFF object file target, but can be re-used for the include directory target as well. For the include directory target, resinator will convert the MachineType to the relevant arch, or fail if there is no equivalent arch/no support for detecting the includes for the MachineType (currently 64-bit Itanium and EBC).
Fixes the `windows_resources` standalone test failing when the host is, for example, `riscv64-linux`.
* Fix warning WasmMut_toC not all control paths return a value
This is a follow up to https://github.com/ziglang/zig/pull/24206 where
I had previously submitted different mechanisms to fix this warning.
This PR is a suggestion by Alex to return NULL instead and Andrew
confirmed this is his preference.
* c.darwin: define MSG for macos
* darwin: add series os name
* Update lib/std/c.zig
Co-authored-by: Alex Rønne Petersen <alex@alexrp.com>
---------
Co-authored-by: Alex Rønne Petersen <alex@alexrp.com>
Btrfs at least supports 16 EiB files (limited in practice to 8EiB by the
Linux VFS code which uses signed 64-bit offsets). So fix the fs.zig test
case to expect either a FileTooBig or success from truncating a file to
8EiB. And test that beyond that size the offset is interpreted as a
negative number.
Fixes#24242
musl and glibc both specify r0 as an output register because its value
may be overwritten by system calls. As with the updates for 64-bit
PowerPC in the previous commit, this commit brings Zig's syscall
functions for 32-bit PowerPC in line with musl and glibc by adding r0 to
the list of clobbers. (Listing r0 as both an input and a clobber is as
close as we can get to musl, which declares it as a "+r" read-write
output, since Zig doesn't support multiple outputs or the "+"
specifier.)
On powerpc64le Linux, the registers used for passing syscall parameters
(r4-r8, as well as r0 for the syscall number) are volatile, or
caller-saved. However, Zig's syscall wrappers for this architecture do
not include all such registers in the list of clobbers, leading the
compiler to assume these registers will maintain their values after the
syscall completes.
In practice, this resulted in a segfault when allocating memory with
`std.heap.SmpAllocator`, which calls `std.os.linux.sched_getaffinity`.
The third parameter to `sched_getaffinity` is a pointer to a `cpu_set_t`
and is stored in register r5. After the syscall, the code attempts to
access data in the `cpu_set_t`, but because the compiler doesn't realize
the value of r5 may have changed, it uses r5 as the memory address, which
in practice resulted in a memory access at address 0x8.
This commit adds all volatile registers to the list of clobbers.
This is not meant to be a long-term solution, but it's the easiest thing
to get working quickly at the moment. The main intention of this hack is
to allow more tests to be enabled. By the time the coff linker is far
enough along to be enabled by default, this will no longer be required.
e.g. `x86_64-windows.win10...win11_dt-gnu` -> `x86_64-windows-gnu`
When the OS version is the default this is redundant with checking the
default in the standard library.
* `futex2_waitv` always takes a 64-bit timespec. Perhaps the
`kernel_timespec` should be renamed `timespec64`? Its used in iouring,
too.
* Add `packed struct` for futex v2 flags and parameters.
* Add very basic "tests" for the futex v2 syscalls (just to ensure the
code compiles).
* Update the stale or broken comments. (I could also just delete these
they're not really documenting Zig-specific behavior.)
Given that the futex2 APIs are not used by Zig's library (they're a bit
too new), and the fact that these are very specialized syscalls, and they
currently provide no benefit over the existing v1 API, I wonder if instead
of fixing these up, we should just replace them with a stub that says 'use
a 3rd party library'.
This is necessary in two cases:
* On POSIX, the exe path (`argv[0]`) must contain a path separator
* Some programs might treat a file named e.g. `-foo` as a flag, which
can be avoided by passing `./-foo`
Rather than detecting these two cases, just always include the prefix;
there's no harm in it.
Also, if the cwd is specified, include it in the manifest. If the user
has set the cwd of a Run step, it is clearly because this affects the
behavior of the executable somehow, so that cwd path should be a part of
the step's manifest.
Resolves: #24216
Cmake by default adds the `/RTC1` compiler flag for debug builds.
However, this causes C code that conforms to the C standard and has
well-defined behavior to trap. Here I've updated CMAKE to use the more
lenient `/RTCs` by default which removes the uninitialized variable checks
but keeps the stack error checks.
* Use `packed struct` for flags arguments. So, instead of
`linux.FUTEX.WAIT` use `.{ .cmd = .WAIT, .private = true }`
* rename `futex_wait` and `futex_wake` which didn't actually specify
wait/wake, as `futex_3arg` and `futex_4arg` (as its the number
of parameters that is different, the `op` is whatever is specified.
* expose the full six-arg flavor of the syscall (for some of the advanced
ops), and add packed structs for their arguments.
* Use a `packed union` to support the 4th parameter which is sometimes a
`timespec` pointer, and sometimes a `u32`.
* Add tests that make sure the structure layout is correct and that the
basic argument passing is working (no actual futexes are contended).
Also add a standalone test which covers the `-fentry` case. It does this
by performing two reproducible compilations which are identical other
than having different entry points, and checking whether the emitted
binaries are identical (they should *not* be).
Resolves: #23869
`std.Build.Step.ConfigHeader` emits a *directory* containing a config
header under a given sub path, but there's no good way to actually
access that directory as a `LazyPath` in the configure phase. This is
silly; it's perfectly valid to refer to that directory, perhaps to
explicitly pass as a "-I" flag to a different toolchain invoked via a
`Step.Run`. So now, instead of the `GeneratedFile` being the actual
*file*, it should be that *directory*, i.e. `cache/o/<digest>`. We can
then easily get the *file* if needed just by using `LazyPath.path` to go
"deeper", which there is a helper function for.
The legacy `getOutput` function is now a deprecated alias for
`getOutputFile`, and `getOutputDir` is introduced.
`std.Build.Module.IncludeDir.appendZigProcessFlags` needed a fix after
this change, so I took the opportunity to refactor it a little. I was
looking at this function while working on ziglang/translate-c yesterday
and realised it could be expressed much more simply -- particularly
after the `ConfigHeader` change here.
I had to update the test `standalone/cmakedefine/` -- it turns out this
test was well and truly reaching into build system internals, and doing
horrible not-really-allowed stuff like overriding the `makeFn` of a
`TopLevelStep`. To top it all off, the test forgot to set
`b.default_step` to its "test" step, so the test never even ran. I've
refactored it to follow accepted practices and to actually, like, work.
This function is sometimes used to assume a canonical representation of
a path. However, when the `Path` referred to `root_dir` itself, this
function previously resolved `sub_path` to ".", which is incorrect; per
doc comments, it should set `sub_path` to "".
This fix ultimately didn't solve what I was trying to solve, though I'm
still PRing it, because it's still *correct*. The background to this
commit is quite interesting and worth briefly discussing.
I originally worked on this to try and fix a bug in the build system,
where if the root package (i.e. the one you `zig build`) depends on
package X which itself depends back on the root package (through a
`.path` dependency), invalid dependency modules are generated. I hit
this case working on ziglang/translate-c, which wants to depend on
"examples" (similar to the Zig compiler's "standalone" test cases) which
themselves depend back on the translate-c package. However, after this
patch just turned that error into another, I realised that this case
simply cannot work, because `std.Build` needs to eagerly execute build
scripts at `dependency` calls to learn which artifacts, modules, etc,
exist.
...at least, that's how the build system is currently designed. One can
imagine a world where `dependency` sort of "queues" the call, `artifact`
and `module` etc just pretend that the thing exists, and all configure
functions are called non-recursively by the runner. The downside is that
it becomes impossible to query state set by a dependency's configure
script. For instance, if a dependency exposes an artifact, it would
become impossible to get that artifact's resolved target in the
configure phase. However, as well as allowing recursive package imports
(which are certainly kinda nifty), it would also make lazy dependencies
far more useful! Right now, lazy dependencies only really work if you
use options (`std.Build.option`) to block their usage, since any call to
`lazyDependency` causes the dependency to be fetched. However, if we
made this change, lazy dependencies could be made far more versatile by
only fetching them *if the final step plan requires them*. I'm not 100%
sure if this is a good idea or not, but I might open an issue for it
soon.
There will be more call sites to `preparePanicId` as we transition away
from safety checks in Sema towards safety checked instructions; it's
silly for them to all have this clunky usage.
This safety check was completely broken; it triggered unchecked illegal
behavior *in order to implement the safety check*. You definitely can't
do that! Instead, we must explicitly check the boundaries. This is a
tiny bit fiddly, because we need to make sure we do floating-point
rounding in the correct direction, and also handle the fact that the
operation truncates so the boundary works differently for min vs max.
Instead of implementing this safety check in Sema, there are now
dedicated AIR instructions for safety-checked intfromfloat (two
instructions; which one is used depends on the float mode). Currently,
no backend directly implements them; instead, a `Legalize.Feature` is
added which expands the safety check, and this feature is enabled for
all backends we currently test, including the LLVM backend.
The `u0` case is still handled in Sema, because Sema needs to check for
that anyway due to the comptime-known result. The old safety check here
was also completely broken and has therefore been rewritten. In that
case, we just check for 'abs(input) < 1.0'.
I've added a bunch of test coverage for the boundary cases of
`@intFromFloat`, both for successes (in `test/behavior/cast.zig`) and
failures (in `test/cases/safety/`).
Resolves: #24161
These conversion routines accept a `round` argument to control how the
result is rounded and return whether the result is exact. Most callers
wanted this functionality and had hacks around it being missing.
Also delete `std.math.big.rational` because it was only being used for
float conversion, and using rationals for that is a lot more complex
than necessary. It also required an allocator, whereas the new integer
routines only need to be passed enough memory to store the result.
If you write an if expression in mem.doNotOptimizeAway like
doNotOptimizeAway(if (ix < 0x00100000) x / 0x1p120 else x + 0x1p120);,
FCSEL instruction is used on AArch64.
FCSEL instruction selects one of the two registers according to
the condition and copies its value.
In this example, `x / 0x1p120` and `x + 0x1p120` are expressions
that raise different floating-point exceptions.
However, since both are actually evaluated before the FCSEL
instruction, the exception not intended by the programmer may
also be raised.
To prevent FCSEL instruction from being used here, this commit
splits doNotOptimizeAway in two.
and also rename `advancedPrint` to `bufferedPrint` in the zig init templates
These are left overs from my previous changes to zig init.
The new templating system removes LITNAME because the new restrictions on package names make it redundant with NAME, and the use of underscores for marking templated identifiers lets us template variable names while still keeping zig fmt happy.
Did you know that allocators reuse addresses? If not, then don't feel
bad, because apparently I don't either! This dumb mistake was probably
responsible for the CI failures on `master` yesterday.
I messed up atomic orderings on this variable because they changed in a
local refactor at some point. We need to always release on the store and
acquire on the loads so that a linker thread observing `.ready` sees the
stored MIR.
Because any `LazyPath` might be resolved to a relative path, it's
incorrect to pass that directly to a child process whose cwd might
differ. Instead, if the child has an overriden cwd, we need to convert
such paths to be relative to the child cwd using `std.fs.path.relative`.
File arguments added to `std.Build.Step.Run` with e.g. `addFileArg` are
not necessarily passed as absolute paths. It used to be the case that
they were as a consequence of an unnecessary path conversion done by the
frontend, but this no longer happens, at least not always, so these
tests were sometimes failing when run locally. Therefore, the standalone
tests must handle cwd-relative CLI paths correctly.
* Sema: allow binary operations and boolean not on vectors of bool
* langref: Clarify use of operators on vectors (`and` and `or` not allowed)
closes#24093
Looking at a compilation of 'test/behavior/x86_64/unary.zig' in
callgrind showed that a full 30% of the compiler runtime was spent in
this `stringToEnum` call, so optimizing it was low-hanging fruit.
We tried replacing it with nested `switch` statements using
`inline else`, but that generated too much code; it didn't emit huge
binaries or anything, but LLVM used a *ridiculous* amount of memory
compiling it in some cases. The core problem here is that only a small
subset of the cases are actually used (the rest fell through to an
"error" path), but that subset is computed at comptime, so we must rely
on the optimizer to eliminate the thousands of redundant cases. This
would be solved by #21507.
Instead, we pre-compute a lookup table at comptime. This table is pretty
big (I guess a couple hundred k?), but only the "valid" subset of
entries will be accessed in practice (unless a bug in the backend is
hit), so it's not too awful on the cache; and it performs much better
than the old `std.meta.stringToEnum` call.
Update the estimated total items for the codegen and link progress nodes
earlier. Rather than waiting for the main thread to dispatch the tasks,
we can add the item to the estimated total as soon as we queue the main
task. The only difference is we need to complete it even in error cases.
Without this cap, unlucky scheduling and/or details of what pipeline
stages perform best on the host machine could cause many gigabytes of
MIR to be stuck in the queue. At a certain point, pause the main thread
until some of the functions in flight have been processed.
This isn't really coherent to model as a `Feature`; this makes sense
because of zig1's specific environment. As such, I opted to check
`dev.env` directly.
Previously, `PerThread.populateTestFunctions` was analyzing the
`test_functions` declaration if it hadn't already been analyzed, so that
it could then populate it. However, the logic for doing this wasn't
actually correct, because it didn't trigger the necessary type
resolution. I could have tried to fix this, but there's actually a
simpler solution! If the `test_functions` declaration isn't referenced
or has a compile error, then we simply don't need to update it; either
it's unreferenced so its value doesn't matter, or we're going to get a
compile error anyway. Either way, we can just give up early. This avoids
doing semantic analysis after `performAllTheWork` finishes.
Also, get rid of the "Code Generation" progress node while updating the
test decl: this is a linking task.
The name of the ZCU object file emitted by the LLVM backend has been
changed in this branch from e.g. `foo.obj` to `foo_zcu.obj`. This is to
avoid name clashes. This commit just updates the stack trace tests which
started failing on windows because of the object name change.
The name of the ZCU object file emitted by the LLVM backend has been
changed in this branch from e.g. `foo.o` to `foo_zcu.o`. This is to
avoid name clashes. This commit just updates a link test which started
failing because the object name in a linker error changed.
glibc, freebsd, and netbsd all do caching manually, because of the fact
that they emit multiple files which they want to cache as a block.
Therefore, the individual sub-compilation on a cache miss should be
using `CacheMode.none` so that we can specify the output paths for each
sub-compilation as being in the shared output directory.
* "Flush" nodes ("LLVM Emit Object", "ELF Flush") appear under "Linking"
* "Code Generation" disappears when all analysis and codegen is done
* We only show one node under "Semantic Analysis" to accurately convey
that analysis isn't happening in parallel, but rather that we're
pausing one task to do another
Previously, various doc comments heavily disagreed with the
implementation on both what lives where on the filesystem at what time,
and how that was represented in code. Notably, the combination of emit
paths outside the cache and `disable_lld_caching` created a kind of
ad-hoc "cache disable" mechanism -- which didn't actually *work* very
well, 'most everything still ended up in this cache. There was also a
long-standing issue where building using the LLVM backend would put a
random object file in your cwd.
This commit reworks how emit paths are specified in
`Compilation.CreateOptions`, how they are represented internally, and
how the cache usage is specified.
There are now 3 options for `Compilation.CacheMode`:
* `.none`: do not use the cache. The paths we have to emit to are
relative to the compiler cwd (they're either user-specified, or
defaults inferred from the root name). If we create any temporary
files (e.g. the ZCU object when using the LLVM backend) they are
emitted to a directory in `local_cache/tmp/`, which is deleted once
the update finishes.
* `.whole`: cache the compilation based on all inputs, including file
contents. All emit paths are computed by the compiler (and will be
stored as relative to the local cache directory); it is a CLI error to
specify an explicit emit path. Artifacts (including temporary files)
are written to a directory under `local_cache/tmp/`, which is later
renamed to an appropriate `local_cache/o/`. The caller (who is using
`--listen`; e.g. the build system) learns the name of this directory,
and can get the artifacts from it.
* `.incremental`: similar to `.whole`, but Zig source file contents, and
anything else which incremental compilation can handle changes for, is
not included in the cache manifest. We don't need to do the dance
where the output directory is initially in `tmp/`, because our digest
is computed entirely from CLI inputs.
To be clear, the difference between `CacheMode.whole` and
`CacheMode.incremental` is unchanged. `CacheMode.none` is new
(previously it was sort of poorly imitated with `CacheMode.whole`). The
defined behavior for temporary/intermediate files is new.
`.none` is used for direct CLI invocations like `zig build-exe foo.zig`.
The other cache modes are reserved for `--listen`, and the cache mode in
use is currently just based on the presence of the `-fincremental` flag.
There are two cases in which `CacheMode.whole` is used despite there
being no `--listen` flag: `zig test` and `zig run`. Unless an explicit
`-femit-bin=xxx` argument is passed on the CLI, these subcommands will
use `CacheMode.whole`, so that they can put the output somewhere without
polluting the cwd (plus, caching is potentially more useful for direct
usage of these subcommands).
Users of `--listen` (such as the build system) can now use
`std.zig.EmitArtifact.cacheName` to find out what an output will be
named. This avoids having to synchronize logic between the compiler and
all users of `--listen`.
It turns out that LLD caching hasn't been in use for a while. On master,
it is currently only enabled when you compile via the build system,
passing `-fincremental`, using LLD (and so LLVM if there's a ZCU). That
case never happens, because `-fincremental` is only useful when you're
using a backend *other* than the LLVM backend. My previous commits
accidentally re-enabled this logic in some cases, exposing bugs; that
ultimately led to this realisation. So, let's just delete that logic --
less LLVM-related cruft to maintain.
Unfortunately, the self-hosted SPIR-V backend is quite tightly coupled
with the self-hosted SPIR-V linker through its `Object` concept (which
is much like `llvm.Object`). Reworking this would be too much work for
this branch. So, for now, I have introduced a special case (similar to
the LLVM backend's special case) to the codegen logic when using this
backend. We will want to delete this special case at some point, but it
need not block this work.
My original goal here was just to get the self-hosted Wasm backend
compiling again after the pipeline change, but it turned out that from
there it was pretty simple to entirely eliminate the shared state
between `codegen.wasm` and `link.Wasm`. As such, this commit not only
fixes the backend, but makes it the second backend (after CBE) to
support the new 1:N:1 threading model.
As of this commit, every backend other than self-hosted Wasm and
self-hosted SPIR-V compiles and (at least somewhat) functions again.
Those two backends are currently disabled with panics.
Note that `Zcu.Feature.separate_thread` is *not* enabled for the fixed
backends. Avoiding linker references from codegen is a non-trivial task,
and can be done after this branch.
The idea here is that instead of the linker calling into codegen,
instead codegen should run before we touch the linker, and after MIR is
produced, it is sent to the linker. Aside from simplifying the call
graph (by preventing N linkers from each calling into M codegen
backends!), this has the huge benefit that it is possible to
parallellize codegen separately from linking. The threading model can
look like this:
* 1 semantic analysis thread, which generates AIR
* N codegen threads, which process AIR into MIR
* 1 linker thread, which emits MIR to the binary
The codegen threads are also responsible for `Air.Legalize` and
`Air.Liveness`; it's more efficient to do this work here instead of
blocking the main thread for this trivially parallel task.
I have repurposed the `Zcu.Feature.separate_thread` backend feature to
indicate support for this 1:N:1 threading pattern. This commit makes the
C backend support this feature, since it was relatively easy to divorce
from `link.C`: it just required eliminating some shared buffers. Other
backends don't currently support this feature. In fact, they don't even
compile -- the next few commits will fix them back up.
Similar to the previous commit, this commit untangles LLD integration
from the self-hosted linkers. Despite the big network of functions which
were involved, it turns out what was going on here is quite simple. The
LLD linking logic is actually very self-contained; it requires a few
flags from the `link.File.OpenOptions`, but that's really about it. We
don't need any of the mutable state on `Elf`/`Coff`/`Wasm`, for
instance. There was some legacy code trying to handle support for using
self-hosted codegen with LLD, but that's not a supported use case, so
I've just stripped it out.
For now, I've just pasted the logic for linking the 3 targets we
currently support using LLD for into this new linker implementation,
`link.Lld`; however, it's almost certainly possible to combine some of
the logic and simplify this file a bit. But to be honest, it's not
actually that bad right now.
This commit ends up eliminating the distinction between `flush` and
`flushZcu` (formerly `flushModule`) in linkers, where the latter
previously meant something along the lines of "flush, but if you're
going to be linking with LLD, just flush the ZCU object file, don't
actually link"?. The distinction here doesn't seem like it was properly
defined, and most linkers seem to treat them as essentially identical
anyway. Regardless, all calls to `flushZcu` are gone now, so it's
deleted -- one `flush` to rule them all!
The end result of this commit and the preceding one is that LLVM and LLD
fit into the pipeline much more sanely:
* If we're using LLVM for the ZCU, that state is on `zcu.llvm_object`
* If we're using LLD to link, then the `link.File` is a `link.Lld`
* Calls to "ZCU link functions" (e.g. `updateNav`) lower to calls to the
LLVM object if it's available, or otherwise to the `link.File` if it's
available (neither is available under `-fno-emit-bin`)
* After everything is done, linking is finalized by calling `flush` on
the `link.File`; for `link.Lld` this invokes LLD, for other linkers it
flushes self-hosted linker state
There's one messy thing remaining, and that's how self-hosted function
codegen in a ZCU works; right now, we process AIR with a call sequence
something like this:
* `link.doTask`
* `Zcu.PerThread.linkerUpdateFunc`
* `link.File.updateFunc`
* `link.Elf.updateFunc`
* `link.Elf.ZigObject.updateFunc`
* `codegen.generateFunction`
* `arch.x86_64.CodeGen.generate`
So, we start in the linker, take a scenic detour through `Zcu`, go back
to the linker, into its implementation, and then... right back out, into
code which is generic over the linker implementation, and then dispatch
on the *backend* instead! Of course, within `arch.x86_64.CodeGen`, there
are some more places which switch on the `link` implementation being
used. This is all pretty silly... so it shall be my next target.
The main goal of this commit is to make it easier to decouple codegen
from the linkers by being able to do LLVM codegen without going through
the `link.File`; however, this ended up being a nice refactor anyway.
Previously, every linker stored an optional `llvm.Object`, which was
populated when using LLVM for the ZCU *and* linking an output binary;
and `Zcu` also stored an optional `llvm.Object`, which was used only
when we needed LLVM for the ZCU (e.g. for `-femit-llvm-bc`) but were not
emitting a binary.
This situation was incredibly silly. It meant there were N+1 places the
LLVM object might be instead of just 1, and it meant that every linker
had to start a bunch of methods by checking for an LLVM object, and just
dispatching to the corresponding method on *it* instead if it was not
`null`.
Instead, we now always store the LLVM object on the `Zcu` -- which makes
sense, because it corresponds to the object emitted by, well, the Zig
Compilation Unit! The linkers now mostly don't make reference to LLVM.
`Compilation` makes sure to emit the LLVM object if necessary before
calling `flush`, so it is ready for the linker. Also, all of the
`link.File` methods which act on the ZCU -- like `updateNav` -- now
check for the LLVM object in `link.zig` instead of in every single
individual linker implementation. Notably, the change to LLVM emit
improves this rather ludicrous call chain in the `-fllvm -flld` case:
* Compilation.flush
* link.File.flush
* link.Elf.flush
* link.Elf.linkWithLLD
* link.Elf.flushModule
* link.emitLlvmObject
* Compilation.emitLlvmObject
* llvm.Object.emit
Replacing it with this one:
* Compilation.flush
* llvm.Object.emit
...although we do currently still end up in `link.Elf.linkWithLLD` to do
the actual linking. The logic for invoking LLD should probably also be
unified at least somewhat; I haven't done that in this commit.
* The `codegen_nav`, `codegen_func`, `codegen_type` tasks are renamed to
`link_nav`, `link_func`, and `link_type`, to more accurately reflect
their purpose of sending data to the *linker*. Currently, `link_func`
remains responsible for codegen; this will change in an upcoming
commit.
* Don't go on a pointless detour through `PerThread` when linking ZCU
functions/`Nav`s; so, the `linkerUpdateNav` etc logic now lives in
`link.zig`. Currently, `linkerUpdateFunc` is an exception, because it
has broader responsibilities including codegen, but this will be
solved in an upcoming commit.
For directory arguments that need both prefix and suffix strings
appended.
Needed to unbreak ffmpeg package after fe855691f6f742a14678cb617422977c2a55be39
I'm not convinced that some of the possibilities that these regexes allowed are real. I've literally never seen or heard of "armhfel", nor of "thumb" ever showing up in `uname -m`, etc.
* trailing whitespace
* langref: undefined _is_ materialized in all safe modes
I am also not super happy about the clause that immediately follows. I
_believe_ what we want to say here is that, simultaneously:
* undefined is guaranteed to be matrerialized in in all safe modes.
A Zig implementation that elides `ptr.* = undefined` in ReleaseSafe
mode would be a non-conforming implementation.
* A Zig program that relies on undefined being materialized is buggy.
But I don't think it's the time to engage this level of language-lawering!
Currently, Zig semantically loads an array as a temporary when indexing
it. This means it cannot be guaranteed that only the requested element
is loaded; in particular, our self-hosted backends do not elide the load
of the full array, so this test case was crashing on self-hosted.
Representing this with a `GenZir` field is incredibly bug-prone.
Instead, just pass this data directly to the relevant expression in the
very few places which actually provide a name strategy.
Resolves: #22798
Note that `openLoadArchive` already has linker script support.
With this change I get a failure parsing a real archive in the self
hosted elf linker, rather than the previous behavior of getting an error
while trying to parse a pseudo archive that is actually a load script.
The addition of FreeBSD and NetBSD targets to the test matrix in #24013 seems to
be causing timeouts under load. We might need to exclude some of those from CI,
but start by bumping the timeout so we can get a sense of how much more time is
actually needed.
To my knowledge, the only platforms that actually *require* PIE are Fuchsia and
Android, and the latter *only* when building a dynamically-linked executable.
OpenBSD and macOS both strongly encourage using PIE by default, but it isn't
technically required. So for the latter platforms, we enable it by default but
don't enforce it.
Also, importantly, if we're building an object file or a static library, and the
user hasn't explicitly told us whether to build PIE or non-PIE code (and the
target doesn't require PIE), we should *not* default to PIE. Doing so produces
code that cannot be linked into non-PIE output. In other words, building an
object file or a static library as PIE is an optimization only to be done when
the user knows that it'll end up in a PIE executable in the end.
Closes#21837.
Linking it by default means that we produce binaries that, effectively, only run
on systems which have the Windows SDK installed because ucrtbased.dll is not
redistributable, and the Windows SDK is what actually installs ucrtbased.dll
into %SYSTEM32%. The resulting binaries also can't run under Wine because Wine
does not provide ucrtbased.dll.
It is also inconsistent with our behavior for *-windows-gnu where we always link
ucrtbase.dll. See #23983, #24019, and #24053 for more details.
So just use ucrtbase.dll regardless of mode. With this change, we can also drop
the implicit definition of the _DEBUG macro in zig cc, which has in some cases
been problematic for users.
Users who want to opt into the old behavior can do so, both for *-windows-msvc
and *-windows-gnu, by explicitly passing -lucrtbased and -D_DEBUG. We might
consider adding a more ergonomic flag like -fdebug-crt to the zig build-* family
of commands in the future.
Closes#24052.
We have no control over memory usage on arbitrary systems in the wild. But we
would still like to get the warnings so we can adjust the values based on
observations in the official ZSF CI.
Closes#23254.
Closes#23638.
This commit introduces a new flag to generate a new Zig project using
`zig init` without comments for users who are already familiar with the
Zig build system.
Additionally, the generated files are now different. Previously we would
generate a set of files that defined a static library and an executable,
which real-life experience has shown to cause confusion to newcomers.
The new template generates one Zig module and one executable both in
order to accommodate the two most common use cases, but also to suggest
that a library could use a CLI tool (e.g. a parser library could use a
CLI tool that provides syntax checking) and vice-versa a CLI tool might
want to expose its core functionality as a Zig module.
All references to C interoperability are removed from the template under
the assumption that if you're tall enough to do C interop, you're also
tall enough to find your way around the build system. Experienced users
will still be able to use the current template and adapt it with minimal
changes in order to perform more advanced operations. As an example, one
only needs to change `b.addExecutable` to `b.addLibrary` to switch from
generating an executable to a dynamic (or static) library.
For instance, the file 'cases/compile_errors/undeclared_identifier.zig'
now corresponds to test name 'compile_errors.undeclared_identifier'.
This is useful because you can now filter based on the case dirname
using `-Dtest-filter`.
`castTruncatedData` was a poorly worded error (all shrinking casts
"truncate bits", it's just that we assume those bits to be zext/sext of
the other bits!), and `negativeToUnsigned` was a pointless distinction
which forced the compiler to emit worse code (since two separate safety
checks were required for casting e.g. 'i32' to 'u16') and wasn't even
implemented correctly. This commit combines those safety panics into one
function, `integerOutOfBounds`. The name maybe isn't perfect, but that's
not hugely important; what matters is the new default message, which is
clearer than the old ones: "integer does not fit in destination type".
Runtime `@shuffle` has two cases which backends generally want to handle
differently for efficiency:
* One runtime vector operand; some result elements may be comptime-known
* Two runtime vector operands; some result elements may be undefined
The latter case happens if both vectors given to `@shuffle` are
runtime-known and they are both used (i.e. the mask refers to them).
Otherwise, if the result is not entirely comptime-known, we are in the
former case. `Sema` now diffentiates these two cases in the AIR so that
backends can easily handle them however they want to. Note that this
*doesn't* really involve Sema doing any more work than it would
otherwise need to, so there's not really a negative here!
Most existing backends have their lowerings for `@shuffle` migrated in
this commit. The LLVM backend uses new lowerings suggested by Jacob as
ones which it will handle effectively. The x86_64 backend has not yet
been migrated; for now there's a panic in there. Jacob will implement
that before this is merged anywhere.
This adds 4 `Legalize.Feature`s:
* `expand_intcast_safe`
* `expand_add_safe`
* `expand_sub_safe`
* `expand_mul_safe`
These do pretty much what they say on the tin. This logic was previously
in Sema, used when `Zcu.Feature.safety_checked_instructions` was not
supported by the backend. That `Zcu.Feature` has been removed in favour
of this legalization.
We don't seem to be getting non-deterministic hangs since 4f3b59f, and e28b402
cut the run times significantly on top of that. Runs now seem to take around 1-2
hours, so the default timeout should be plenty.
This defines a WinMain() function that can be potentially problematic when it
isn't wanted. If we add back support for this library in the future, it should
be built separately from mingw32.lib and on demand.
These are almost entirely identical, with these exceptions:
* lib/libc/include/csky-linux-{gnueabi,gnueabihf}
* gnu/{lib-names,stubs}.h will need manual patching for float ABI.
* lib/libc/include/{powerpc-linux-{gnueabi,gnueabihf},{powerpc64,powerpc64le}-linux-gnu}
* bits/long-double.h will need manual patching for long double ABI.
std tests are temporarily disabled for arm-freebsd-eabihf due to #23949.
I omitted x86-freebsd-none and powerpc-freebsd-none because these will be
dropped in FreeBSD 15.0 anyway, so there's no point in us spending resources on
those now.
There's not really any point in targeting *-windows-(gnu,msvc) when not linking
libc, so add entries for *-windows-(gnu,msvc) that actually link libc, and
change the old non-libc entries to *-windows-none.
Also add missing aarch64-windows-(none,msvc) and thumb-windows-(none,msvc)
entries. thumb-windows-gnu is disabled for now due to #24016.
Each target can opt into different sets of legalize features.
By performing these transformations before liveness, instructions
that become unreferenced will have up-to-date liveness information.
This is equivalent to `array_elem_val`, and doing that conversion in
Sema (seems reasonable since it's just a simple branch) is much easier
for the self-hosted x86_64 backend then actually handling this case.
Because we don't pass -fqemu and -fwasmtime on aarch64-linux, we're just
spending a bunch of time compiling all these module tests only to not even run
them. x86_64-linux already covers both compiling and running them.
Pointers to thread-local variables do not have their addresses known
until runtime, so it is nonsensical for them to be comptime-known. There
was logic in the compiler which was essentially attempting to treat them
as not being comptime-known despite the pointer being an interned value.
This was a bit of a mess, the check was frequent enough to actually show
up in compiler profiles, and it was very awkward for backends to deal
with, because they had to grapple with the fact that a "constant" they
were lowering might actually require runtime operations.
So, instead, do not consider these pointers to be comptime-known in
*any* way. Never intern such a pointer; instead, when the address of a
threadlocal is taken, emit an AIR instruction which computes the pointer
at runtime. This avoids lots of special handling for TLVs across
basically all codegen backends; of all somewhat-functional backends, the
only one which wasn't improved by this change was the LLVM backend,
because LLVM pretends this complexity around threadlocals doesn't exist.
This change simplifies Sema and codegen, avoids a potential source of
bugs, and potentially improves Sema performance very slightly by
avoiding a non-trivial check on a hot path.
In the case where a declaration has no type annotation, the interaction
between resolution of `nav_ty` and `nav_val` is a little fiddly because
of the fact that resolving `nav_val` actually implicitly resolves the
type as well. This means `nav_ty` never gets an opporunity to mark its
dependency on the `nav_val`. So, `ensureNavValUpToDate` needs to be the
one to do it. It can't do it too early, though; otherwise, our marking
of dependees as out-of-date/up-to-date will go wrong.
Resolves: #23959
In a compiler built with debug extensions, pass `--debug-incremental` to
spawn the "incremental debug server". This is a TCP server exposing a
REPL which allows querying a bunch of compiler state, some of which is
stored only when that flag is passed. Eventually, this will probably
move into `std.zig.Server`/`std.zig.Client`, but this is easier to work
with right now. The easiest way to interact with the server is `telnet`.
Reduced number of runners from 9 to 6.
This number is the total physical memory (251G) divided by the number of
runners we have active (6).
see previous commit 5b9e528bc550e7ea9e286fdd2324316f9895d5da
GitHub have introduced an absolutely baffling feature where users can
use Copilot to take their simple explanation of a bug, and reword it
into a multi-paragraph monologue with no interesting details and added
false information, while also potentially ignoring issue templates.
So far, GitHub has not provided a way to block this feature at the
repository or organisation level, so for now, this is the only way to
prevent users from filing LLM-generated slop.
Related: https://github.com/orgs/community/discussions/159749
The doc comment here agreed with the implementation, but not with *any*
`Step` which populates a `GeneratedFile`, where they are treated as
cwd-relative. This is the obvious correct choice, because these paths
usually come from joining onto a cache root, and those are cwd-relative
if not absolute.
This was a pre-existing bug, but #23836 caused it to trigger more often,
because the compiler now commonly passes the local cache directory to
the build runner process as a relative path where it was previously an
absolute path.
Resolves: #23954
37a9a4e accidentally turned paths `b/[hash]/` into `b[hash]/` in the
global cache. This doesn't technically break anything, but it pollutes
the global cache directory. Sorry about that one!
This was an unintentional regression in 23c8175 which meant that
backwards-incompatible ZIR changes would have caused compiler crashes if
old caches were present.
Right now, if you override the build root with `--build-root`, then
`Run` steps can fail to execute because of incorrect path handling in
the compiler: `std.process.Child` gets a cwd-relative path, but also has
its cwd set to the build root. The latter behavior is really weird; it
doesn't match my expectations, nor does it match how we spawn child
`zig` processes. So, this commit makes the child process inherit the
build runner's cwd, as `LazyPath.getPath2` *expects* it to.
After investigating, this behavior dates all the way back to 2017; it
was introduced in 4543413. So, there isn't any clear/documented reason
for this; it should be safe to revert, since under the modern `LazyPath`
system it is strictly a bug AFAICT.
* libc: implement common `abs` for various integer sizes
* libc: move imaxabs to inttypes.zig and don't use cInclude
* libc: delete `fabs` c implementations because already implemented in compiler_rt
* libc: export functions depending on the target libc
Previously all the functions that were exported were handled equally,
though some may exist and some not inside the same file. Moving the
checks inside the file allows handling different functions differently
* remove empty ifs in inttypes
Co-authored-by: Alex Rønne Petersen <alex@alexrp.com>
* remove empty ifs in stdlib
Co-authored-by: Alex Rønne Petersen <alex@alexrp.com>
* libc: use `@abs` for the absolute value calculation
---------
Co-authored-by: Alex Rønne Petersen <alex@alexrp.com>
Nothing interesting here; literally just the bare minimum so I can work on this
on and off in a branch without worrying about merge conflicts in the non-backend
code.
I only wanted to fix a bug originally, but this logic was kind of a
rat's nest. But now... okay, it still *is*, but it's now a slightly more
navigable nest, with cute little signs occasionally, painted by adorable
rats desparately trying to follow the specification.
Hopefully #3806 comes along at some point to simplify this logic a
little.
Resolves: #23139
This prevents symbols from these libraries from polluting the dynamic symbol
tables of binaries built with Zig. The downside is that we no longer deduplicate
the symbols at run time due to weak linkage.
Closes#7935.
Closes#13303.
Closes#19342.
This commit makes some big changes to how we track state for Zig source
files. In particular, it changes:
* How `File` tracks its path on-disk
* How AstGen discovers files
* How file-level errors are tracked
* How `builtin.zig` files and modules are created
The original motivation here was to address incremental compilation bugs
with the handling of files, such as #22696. To fix this, a few changes
are necessary.
Just like declarations may become unreferenced on an incremental update,
meaning we suppress analysis errors associated with them, it is also
possible for all imports of a file to be removed on an incremental
update, in which case file-level errors for that file should be
suppressed. As such, after AstGen, the compiler must traverse files
(starting from analysis roots) and discover the set of "live files" for
this update.
Additionally, the compiler's previous handling of retryable file errors
was not very good; the source location the error was reported as was
based only on the first discovered import of that file. This source
location also disappeared on future incremental updates. So, as a part
of the file traversal above, we also need to figure out the source
locations of imports which errors should be reported against.
Another observation I made is that the "file exists in multiple modules"
error was not implemented in a particularly good way (I get to say that
because I wrote it!). It was subject to races, where the order in which
different imports of a file were discovered affects both how errors are
printed, and which module the file is arbitrarily assigned, with the
latter in turn affecting which other files are considered for import.
The thing I realised here is that while the AstGen worker pool is
running, we cannot know for sure which module(s) a file is in; we could
always discover an import later which changes the answer.
So, here's how the AstGen workers have changed. We initially ensure that
`zcu.import_table` contains the root files for all modules in this Zcu,
even if we don't know any imports for them yet. Then, the AstGen
workers do not need to be aware of modules. Instead, they simply ignore
module imports, and only spin off more workers when they see a by-path
import.
During AstGen, we can't use module-root-relative paths, since we don't
know which modules files are in; but we don't want to unnecessarily use
absolute files either, because those are non-portable and can make
`error.NameTooLong` more likely. As such, I have introduced a new
abstraction, `Compilation.Path`. This type is a way of representing a
filesystem path which has a *canonical form*. The path is represented
relative to one of a few special directories: the lib directory, the
global cache directory, or the local cache directory. As a fallback, we
use absolute (or cwd-relative on WASI) paths. This is kind of similar to
`std.Build.Cache.Path` with a pre-defined list of possible
`std.Build.Cache.Directory`, but has stricter canonicalization rules
based on path resolution to make sure deduplicating files works
properly. A `Compilation.Path` can be trivially converted to a
`std.Build.Cache.Path` from a `Compilation`, but is smaller, has a
canonical form, and has a digest which will be consistent across
different compiler processes with the same lib and cache directories
(important when we serialize incremental compilation state in the
future). `Zcu.File` and `Zcu.EmbedFile` both contain a
`Compilation.Path`, which is used to access the file on-disk;
module-relative sub paths are used quite rarely (`EmbedFile` doesn't
even have one now for simplicity).
After the AstGen workers all complete, we know that any file which might
be imported is definitely in `import_table` and up-to-date. So, we
perform a single-threaded graph traversal; similar to what
`resolveReferences` plays for `AnalUnit`s, but for files instead. We
figure out which files are alive, and which module each file is in. If a
file turns out to be in multiple modules, we set a field on `Zcu` to
indicate this error. If a file is in a different module to a prior
update, we set a flag instructing `updateZirRefs` to invalidate all
dependencies on the file. This traversal also discovers "import errors";
these are errors associated with a specific `@import`. With Zig's
current design, there is only one possible error here: "import outside
of module root". This must be identified during this traversal instead
of during AstGen, because it depends on which module the file is in. I
tried also representing "module not found" errors in this same way, but
it turns out to be much more useful to report those in Sema, because of
use cases like optional dependencies where a module import is behind a
comptime-known build option.
For simplicity, `failed_files` now just maps to `?[]u8`, since the
source location is always the whole file. In fact, this allows removing
`LazySrcLoc.Offset.entire_file` completely, slightly simplifying some
error reporting logic. File-level errors are now directly built in the
`std.zig.ErrorBundle.Wip`. If the payload is not `null`, it is the
message for a retryable error (i.e. an error loading the source file),
and will be reported with a "file imported here" note pointing to the
import site discovered during the single-threaded file traversal.
The last piece of fallout here is how `Builtin` works. Rather than
constructing "builtin" modules when creating `Package.Module`s, they are
now constructed on-the-fly by `Zcu`. The map `Zcu.builtin_modules` maps
from digests to `*Package.Module`s. These digests are abstract hashes of
the `Builtin` value; i.e. all of the options which are placed into
"builtin.zig". During the file traversal, we populate `builtin_modules`
as needed, so that when we see this imports in Sema, we just grab the
relevant entry from this map. This eliminates a bunch of awkward state
tracking during construction of the module graph. It's also now clearer
exactly what options the builtin module has, since previously it
inherited some options arbitrarily from the first-created module with
that "builtin" module!
The user-visible effects of this commit are:
* retryable file errors are now consistently reported against the whole
file, with a note pointing to a live import of that file
* some theoretical bugs where imports are wrongly considered distinct
(when the import path moves out of the cwd and then back in) are fixed
* some consistency issues with how file-level errors are reported are
fixed; these errors will now always be printed in the same order
regardless of how the AstGen pass assigns file indices
* incremental updates do not print retryable file errors differently
between updates or depending on file structure/contents
* incremental updates support files changing modules
* incremental updates support files becoming unreferenced
Resolves: #22696
This function was broken, because it took ownership of the buffer on
error *sometimes*, in a way which the caller could not tell. Rather than
trying to be clever, it's easier to just follow the same interface as
all other `addFilePost` methods, and not take ownership of the path.
This is a breaking change. The next commits will apply it to the
compiler, which is the only user of this function in the ziglang/zig
repository.
This code applies to ~any POSIX OS where we don't link libc. For example, it'll
be useful for FreeBSD and NetBSD.
As part of this, move std.os.linux.pie to std.pie since there's really nothing
Linux-specific about what that file is doing.
* sysident_assym.h was manually expanded.
* The ELF_NOTE_MARCH_DESC and ELF_NOTE_MARCH_DESCSZ macros will be defined
by the compiler.
* Legacy .init/.fini stuff was removed.
* GCJ nonsense was removed.
* mips64/mips64el on NetBSD are soft float; we have no support for this yet.
* powerpc64 does not appear to be a thing.
* riscv32/riscv64 have not seen official releases yet.
We want the latest unversioned inclusion that fits the target version. This
theoretically matters because it might have a different global vs weak linkage
compared to an older inclusion.
Evaluate all child processes in the temporary directory, and use
`std.fs.path.relative` to make every other path relative to that child
cwd instead of our cwd.
Resolves: #22119
It's incorrect to ever set `include_reference_trace` here, because the
compiler has already given or not given reference traces depending on
the `-freference-trace` option propagated to the compiler process by
`std.Build.Step.Compile`.
Perhaps in future we could make the compiler always return the reference
trace when communicating over the compiler protocol; that'd be more
versatile than the current behavior, because the build runner could, for
instance, show a reference trace on-demand without having to even invoke
the compiler. That seems really useful, since the reference trace is
*often* unnecessary noise, but *sometimes* essential. However, we don't
live in that world right now, so passing the option here doesn't make
sense.
Resolves: #23415
To an average user, it may be unclear why these notes are not just in
the reference trace; that's because they are more important, because
they are inline calls through which comptime values may propagate. There
are now 3 possible wordings for this note:
* "called at comptime here"
* "called inline here"
* "generic function instantiated here"
An alternative could be these wordings:
* "while analyzing comptime call here"
* "while analyzing inline call here"
* "while analyzing generic instantiation here"
I'm not sure which is better -- but this commit is certainly better than
status quo.
Inline calls which happened in the erroring `AnalUnit` still show as
error notes, because they tend to make very important context (e.g. to
see how comptime values propagate through them). However, "earlier"
inline calls are still useful to see to understand how something is
being referenced, so we should include them in the reference trace.
When `-freference-trace` is not passed, we want to show exactly one
reference trace. Previously, we set the reference trace root in `Sema`
iff there were no other failed analyses. However, this results in an
arbitrary error being the one with the reference trace after error
sorting. It is also incompatible with incremental compilation, where
some errors might be unreferenced. Instead, set the field on all
analysis errors, and decide in `Compilation.getAllErrorsAlloc` which
reference trace[s] to actually show.
Error messages never contain periods or grave accents.
Get rid of the periods and use apostrophes instead in
probably the only two error messages that had them.
* ucontext_t ptr is 8-byte aligned instead of 16-byte aligned which @alignCast() expects
* Retrieve pc address from ucontext_t since unwind_state is null
* Work around __mcontext_data being written incorrectly by the kernel
These symbols are defined in the statically-linked startup code. The real
libc.so.7 contains strong references to them, so they need to be put into the
dynamic symbol table.
Textual PTX is just assembly language like any other. And if we do ever add
support for emitting PTX object files after reverse engineering the bytecode
format, we'd be emitting ELF files like the CUDA toolchain. So there's really no
need for a special ObjectFormat tag here, nor linker code that treats it as a
distinct format.
* NT_FREEBSD_ABI_TAG was manually adjusted from using a hardcoded value to using
__FreeBSD_version which will be defined by the compiler.
* GCJ stuff was removed.
* HAVE_CTORS definitions were removed.
* Introduce common `bzero` libc implementation.
* Update test name according to review
Co-authored-by: Linus Groh <mail@linusgroh.de>
* address code review
- import common implementation when musl or wasi is included
- don't use `c_builtins`, use `@memset`
* bzero calling conv to .c
* Apply review
Co-authored-by: Veikka Tuominen <git@vexu.eu>
---------
Co-authored-by: Linus Groh <mail@linusgroh.de>
Co-authored-by: Veikka Tuominen <git@vexu.eu>
For C code the macros SIGRTMIN and SIGRTMAX provide these values. In
practice what looks like a constant is actually provided by a libc call.
So the Zig implementations are explicitly function calls.
glibc (and Musl) export a run-time minimum "real-time" signal number,
based on how many signals are reserved for internal implementation details
(generally threading). In practice, on Linux, sigrtmin() is 35 on glibc
with the older LinuxThread and 34 with the newer NPTL-based
implementation. Musl always returns 35. The maximum "real-time" signal
number is NSIG - 1 (64 on most Linux kernels, but 128 on MIPS).
When not linking a C Library, Zig can report the full range of "rt"
signals (none are reserved by Zig).
Fixes#21189
If clang encountered bad imports, the depfile will not be generated. It
doesn't make sense to warn the user in this case. In fact,
`FileNotFound` is never worth warning about here; it just means that
the file we were deleting to save space isn't there in the first place!
If the missing file actually affected the compilation (e.g. another
process raced to delete it for some reason) we would already error in
the normal code path which reads these files, so we can safely omit the
warning in the `FileNotFound` case always, only warning when the file
might still exist.
To see what this fixes, create the following file...
```c
#include <nonexist>
```
...and run `zig build-obj` on it. Before this commit, you will get a
redundant warning; after this commit, that warning is gone.
Most of these are gated by -Dtest-extra-targets because:
* We don't really have CI resources to spare at the moment.
* They're relatively niche if you're not on a musl distro.
* And the few musl distros that exist don't support all these targets.
* Quite a few of them are broken and need investigating.
x86_64-linux-musl and aarch64-linux-musl are not gated as they're the most
common targets that people will be running dynamic musl on, so we'll want to
have some bare minimum coverage of those.
It remains 1 everywhere else.
Also remove some code that allowed setting the libc++ ABI version on the
Compilation since there are no current plans to actually expose this in the CLI.
* When storing a zero-bit type, we should short-circuit almost
immediately. Zero-bit stores do not need to do any work.
* The bit size computation for arrays is incorrect; the `abiSize` will
already be appropriately aligned, but the logic to do so here
incorrectly assumes that zero-bit types have an alignment of 0. They
don't; their alignment is 1.
Resolves: #21202Resolves: #21508Resolves: #23307
There were several bugs with the synchronization here; most notably an
ABA problem which was causing #21663. I fixed that and some other
issues, and took the opportunity to get rid of the `.seq_cst` orderings
from this file. I'm at least relatively sure my new orderings are correct.
Co-authored-by: achan1989 <achan1989@gmail.com>
Resolves: #21663
This is generally ill-advised, but can be useful in some niche situations where
the caveats don't apply. It might also be useful when providing a libc.txt that
points to Eyra.
I changed to `wasm/abi.zig`, this design is certainly better than the previous one. Still there is some conflict of interest between llvm and self-hosted backend, better design will appear when abi tests will be tested with self-hosted.
Resolves: #23304Resolves: #23305
Dunno why the MIPS signal numbers are different, or why Zig had them
already special cased, but wrong.
We have the technology to test these constants. We should use it.
All the existing code that manipulates `ucontext_t` expects there to be a
glibc-compatible sigmask (1024-bit). The `ucontext_t` struct need to be
cleaned up so the glibc-dependent format is only used when linking
glibc/musl library, but that is a more involved change.
In practice, no Zig code looks at the sigset field contents, so it just
needs to be the right size.
By returning an initialized sigset (instead of taking the set as an output
parameter), these functions can be used to directly initialize the `mask`
parameter of a `Sigaction` instance.
When linking a libc, Zig should defer to the C library for sigset
operations. The pre-filled constants signal sets (empty_sigset,
filled_sigset) are not compatible with C library initialization, so remove
them and use the runtime `sigemptyset` and `sigfillset` methods to
initialize any sigset.
Unify the C library sigset_t and Linux native sigset_t and the accessor
operations.
Add tests that the various sigset_t operations are working. And clean up
existing tests a bit.
The kernel ABI sigset_t is smaller than the glibc one. Define the
right-sized sigset_t and fixup the sigaction() wrapper to leverage it.
The Sigaction wrapper here is not an ABI, so relax it (drop the "extern"
and the "restorer" fields), the existing `k_sigaction` is the ABI
sigaction struct.
Linux defines `sigset_t` with a c_ulong, so it can be 32-bit or 64-bit,
depending on the platform. This can make a difference on big-endian
systems.
Patch up `ucontext_t` so that this change doesn't impact its layout.
AFAICT, its currently the glibc layout.
Export the sigset_t ops (sigaddset, etc) from the C library. Don't rely
on the linux.zig defintions (which will be defined to use the kernel ABI).
Move Darwin sigset and NSIG declarations into darwin.zig. Remove
extraneous (?) sigaddset. The C library sigaddset can reject some signals
being added, so need to defer to it.
* Indexing zero-bit types should not produce AIR indexing instructions
* Getting a runtime-known element pointer from a many-pointer should
check that the many-pointer is not comptime-only
Resolves: #23405
`writeCValue` already emits a cast; including another here is, in fact,
invalid, and emits errors under MSVC. Probably this code was originally
added to work around the incorrect `.Initializer` location which was
fixed in the previous commit.
The last Intel Quark MCU was released in 2015. Quark was announced to be EOL in
2019, and stopped shipping entirely in 2022.
The OS tag was only meaningful for Intel's weird fork of Linux 3.8.7 with a
special ABI that differs from the regular i386 System V ABI; beyond that, the
CPU itself is just a plain old P54C (i586). We of course keep support for the
CPU itself, just not Intel's Linux fork.
These backends are completely unusable at the moment; they can produce neither
assembly files nor object files. So give a nicer error when users try to use
them.
Aside from adding comments to document the logic in `Cache.Manifest.hit`
better, this commit fixes two serious bugs.
The first, spotted by Andrew, is that when upgrading from a shared to an
exclusive lock on the manifest file, we do not seek it back to the
start. This is a simple fix.
The second is more subtle, and has to do with the computation of file
digests. Broadly speaking, the goal of the main loop in `hit` is to
iterate the files listed in the manifest file, and check if they've
changed, based on stat and a file hash. While doing this, the
`bin_digest` field of `std.Build.Cache.File`, which is initially
`undefined`, is populated for all files, either straight from the
manifest (if the stat matches) or recomputed from the file on-disk. This
file digest is then used to update `man.hash.hasher`, which is building
the final hash used as, for instance, the output directory name when the
compiler emits into the cache directory. When `hit` returns a cache
miss, it is expected that `man.hash.hasher` includes the digests of all
"initial files"; that is, those which have been already added with e.g.
`addFilePath`, but not those which will later be added with
`addFilePost` (even though the manifest file has told us about some such
files). Previously, `hit` was using the `unhit` function to do this in a
few cases. However, this is incorrect, because `hit` assumes that all
files already have their `bin_digest` field populated; this function is
only valid to call *after* `hit` returns. Instead, we need to actually
compute the hashes which haven't yet been populated. Even if this logic
has been working, there was still a bug here, because we called `unhit`
when upgrading from a shared to an exclusive lock, writing the
(potentially `undefined`) file digests, but the loop itself writes the
file digests *again*! All in all, the hashing logic here was actually
incredibly broken.
I've taken the opportunity to restructure this section of the code into
what I think is a more readable format. A new function,
`hitWithCurrentLock`, uses the open manifest file to try and find a
cache hit. It returns a tagged union which, in the miss case, tells the
caller (`hit`) how many files already have their hash populated. This
avoids redundant work recomputing the same hash multiple times in
situations where the lock needs upgrading. This also eliminates the
outer loop from `hit`, which was a little confusing because it iterated
no more than twice!
The bugs fixed here could manifest in several different ways depending
on how contended file locks were satisfied. Most notably, on a cache
miss, the Zig compiler might have written the compilation output to the
incorrect directory (because it incorrectly constructed a hash using
`undefined` or repeated file digests), resulting in all future hits on
this manifest causing `error.FileNotFound`. This is #23110. I have been
able to reproduce #23110 on `master`, and have not been able to after
this commit, so I am relatively sure this commit resolves that issue.
Resolves: #23110
This allows emitting object files for s390x-zos (GOFF) and powerpc(64)-aix
(XCOFF).
Note that GOFF emission in LLVM is still being worked on upstream for LLVM 21;
the resulting object files are useless right now. Also, -fstrip is required, or
LLVM will SIGSEGV during DWARF emission.
* Accept -fsanitize-c=trap|full in addition to the existing form.
* Accept -f(no-)sanitize-trap=undefined in zig cc.
* Change type of std.Build.Module.sanitize_c to std.zig.SanitizeC.
* Add some missing Compilation.Config fields to the cache.
Closes#23216.
* This has not seen meaningful development for about a decade.
* The Linux kernel port was never upstreamed.
* The glibc port was never upstreamed.
* GCC 15.1 recently deprecated support it.
It may still make sense to support an ILP32 ABI on AArch64 more broadly (which
we already have the Abi.ilp32 tag for), but, to the extent that it even existed
in any "official" sense, the *GNU* ILP32 ABI is certainly dead.
This is fairly straightforward; the actual compiler changes are limited
to the CLI, since `Compilation` already supports this combination.
A new `std.Build` API is introduced to allow representing this. By
passing the `emit_object` option to `std.Build.addTest`, you get a
`Step.Compile` which emits an object file; you can then use that as you
would any other object, such as either installing it for external use,
or linking it into another step.
A standalone test is added to cover the build system API. It builds a
test into an object, and links it into a final executable, which it then
runs.
Using this build system mechanism prevents the build system from
noticing that you're running a `zig test`, so the build runner and test
runner do not communicate over stdio. However, that's okay, because the
real-world use cases for this feature don't want to do that anyway!
Resolves: #23374
Compile log output is now separated based on the `AnalUnit` which
perfomred the `@compileLog` call, so that we can omit the output for
unreferenced ("dead") units. The units are also sorted when collecting
the `ErrorBundle`, so that compile logs are always printed in a
consistent order, like compile errors are. This is important not only
for incremental compilation, but also for parallel analysis.
Resolves: #23609
* Fix compile error in Fuzzer web-ui
The error was:
```
error: expected type '?mem.Alignment', found 'comptime_int'
```
* Apply suggestions from code review
`.of` call is shorter and clearer
Co-authored-by: Alex Rønne Petersen <alex@alexrp.com>
---------
Co-authored-by: Alex Rønne Petersen <alex@alexrp.com>
Before:
❯ zig cc main.c -target x86_64-linux-musl && musl-ldd ./a.out
musl-ldd: ./a.out: Not a valid dynamic program
❯ zig cc main.c -target x86_64-linux-musl -static && musl-ldd ./a.out
musl-ldd: ./a.out: Not a valid dynamic program
❯ zig cc main.c -target x86_64-linux-musl -dynamic && musl-ldd ./a.out
musl-ldd: ./a.out: Not a valid dynamic program
After:
❯ zig cc main.c -target x86_64-linux-musl && musl-ldd ./a.out
musl-ldd: ./a.out: Not a valid dynamic program
❯ zig cc main.c -target x86_64-linux-musl -static && musl-ldd ./a.out
musl-ldd: ./a.out: Not a valid dynamic program
❯ zig cc main.c -target x86_64-linux-musl -dynamic && musl-ldd ./a.out
/lib/ld-musl-x86_64.so.1 (0x72c10019e000)
libc.so => /lib/ld-musl-x86_64.so.1 (0x72c10019e000)
Closes#11909.
They are, themselves, static libraries even if the resulting artifact strictly
speaking requires dynamic linking to the corresponding system DLLs to run. Note,
though, that there's no libc-provided dynamic linker on Windows like on POSIX,
so this isn't particularly problematic.
This matches x86_64-w64-mingw32-gcc behavior.
std.crypto: add constant-time codecs
Add constant-time hex/base64 codecs designed to process cryptographic
secrets, adapted from libsodium's implementations.
Introduce a `crypto.codecs` namespace for crypto-related encoders and
decoders. Move ASN.1 codecs to this namespace.
This will also naturally accommodate the proposed PEM codecs.
This lays the groundwork for #2879. This library will be built and linked when a
static libc is going to be linked into the compilation. Currently, that means
musl, wasi-libc, and MinGW-w64. As a demonstration, this commit removes the musl
C code for a few string functions and implements them in libzigc. This means
that those libzigc functions are now load-bearing for musl and wasi-libc.
Note that if a function has an implementation in compiler-rt already, libzigc
should not implement it. Instead, as we recently did for memcpy/memmove, we
should delete the libc copy and rely on the compiler-rt implementation.
I repurposed the existing "universal libc" code to do this. That code hadn't
seen development beyond basic string functions in years, and was only usable-ish
on freestanding. I think that if we want to seriously pursue the idea of Zig
providing a freestanding libc, we should do so only after defining clear goals
(and non-goals) for it. See also #22240 for a similar case.
The code was using u32 and usize interchangably, which doesn't work on
64-bit systems. This:
`pub const sigset_t = [1024 / 32]u32;`
is not consistent with this:
`const shift = @as(u5, @intCast(s & (usize_bits - 1)));`
However, normal signal numbers are less than 31, so the bad math doesn't matter much. Also, despite support for 1024 signals in the set, only setting signals between 1 and NSIG (which is mostly 65, but sometimes 128) is defined. The existing tests only exercised signal numbers in the first 31 bits so they didn't trip over this:
The C library `sigaddset` will return `EINVAL` if given an out of bounds signal number. I made the Zig code just silently ignore any out of bounds signal numbers.
Moved all the `sigset` related declarations next to each in the source, too.
The `filled_sigset` seems non-standard to me. I think it is meant to be used like `empty_sigset`, but it only contains 31 set signals, which seems wrong (should be 64 or 128, aka `NSIG`). It's also unused. The oddly named but similar `all_mask` is used (by posix.zig) but sets all 1024 bits (which I understood to be undefined behavior but seems to work just fine). For comparison the musl `sigfillset` fills in 65 bits or 128 bits.
* Oops, I accidentally disabled most of them.
* Cleanup some workarounds for now closed issues.
* Test binary operations with more scalar integer types.
Linux kernel syscalls expect to be given the number of bits of sigset that
they're built for, not the full 1024-bit sigsets that glibc supports.
I audited the other syscalls in here that use `sigset_t` and they're all
using `NSIG / 8`.
Fixes#12715
Translate-c didn't properly account for C macro functions having parameter names that are C keywords. So something like `#define FOO(float) ((float) + 10)` would've been interpreted as casting `+10` to a `float` type, instead of adding `10` to the parameter `float`.
An example of a real-world macro function like this is SDL3's `SDL_DEFINE_AUDIO_FORMAT` from `SDL_audio.h`, which uses `signed` as a parameter.
This started failing in LLVM 20:
test
+- test-stack-traces
+- check error union switch with call operand (ReleaseSafe llvm) failure
error:
========= expected this stdout: =========
error: TheSkyIsFalling
source.zig:3:5: [address] in [function]
return error.TheSkyIsFalling;
^
========= but found: ====================
error: TheSkyIsFalling
source.zig:13:27: [address] in [function]
error.NonFatal => return,
^
We build zig2.c and compiler_rt.c with -O0 but then proceed to link with -O3.
So zig2.o and compiler_rt.o will have references to ubsan-rt symbols, but the
-O3 causes the compiler to not link ubsan-rt. We don't actually need the safety
here, so just explicitly disable ubsan.
These started failing with LLVM 20 for unclear reasons:
test-std
└─ run test std-mips64-linux.4.19...6.13.4-gnuabi64.2.28-mips64r2-Debug-libc 2798/2878 passed, 2 failed, 78 skipped
error: 'posix.test.test.link with relative paths' failed: expected 2, found 0
/home/alexrp/Source/ziglang/zig-llvm20/lib/std/testing.zig:103:17: 0x1d9e5bf in expectEqualInner__anon_47031 (test)
return error.TestExpectedEqual;
^
/home/alexrp/Source/ziglang/zig-llvm20/lib/std/posix/test.zig:311:9: 0x3650f57 in test.link with relative paths (test)
try testing.expectEqual(@as(@TypeOf(nstat.nlink), 2), nstat.nlink);
^
error: 'posix.test.test.linkat with different directories' failed: expected 2, found 0
/home/alexrp/Source/ziglang/zig-llvm20/lib/std/testing.zig:103:17: 0x1d9e5bf in expectEqualInner__anon_47031 (test)
return error.TestExpectedEqual;
^
/home/alexrp/Source/ziglang/zig-llvm20/lib/std/posix/test.zig:355:9: 0x3653377 in test.linkat with different directories (test)
try testing.expectEqual(@as(@TypeOf(nstat.nlink), 2), nstat.nlink);
^
error: while executing test 'zig.system.darwin.macos.test.detect', the following test command failed:
qemu-mips64 -L /opt/glibc/mips64-linux-gnu-n64 /home/alexrp/Source/ziglang/zig-llvm20/.zig-cache/o/22a8c3762ea56ae3a674fa9ad15f6657/test --seed=0xa1dbb43c --cache-dir=/home/alexrp/Source/ziglang/zig-llvm20/.zig-cache --listen=-
test-std
└─ run test std-mips64-linux.4.19...6.13.4-gnuabi64.2.28-mips64r2-Debug-libc 2798/2878 passed, 1 failed, 79 skipped
error: 'posix.test.test.linkat with different directories' failed: expected 2, found 0
/home/alexrp/Source/ziglang/zig-llvm20/lib/std/testing.zig:103:17: 0x1d9e22f in expectEqualInner__anon_47031 (test)
return error.TestExpectedEqual;
^
/home/alexrp/Source/ziglang/zig-llvm20/lib/std/posix/test.zig:356:9: 0x3650b47 in test.linkat with different directories (test)
try testing.expectEqual(@as(@TypeOf(nstat.nlink), 2), nstat.nlink);
^
error: while executing test 'zig.system.darwin.macos.test.detect', the following test command failed:
qemu-mips64 -L /opt/glibc/mips64-linux-gnu-n64 /home/alexrp/Source/ziglang/zig-llvm20/.zig-cache/o/22a8c3762ea56ae3a674fa9ad15f6657/test --seed=0xa1dbb43c --cache-dir=/home/alexrp/Source/ziglang/zig-llvm20/.zig-cache --listen=-
Unfortunately, neither GDB nor LLDB want to play nice with qemu-mips64(el) at
the moment, so I can't easily debug these failures.
LLVM 20 started tail-calling it in some of our test cases, resulting in:
error: AndMyCarIsOutOfGas
/home/alexrp/Source/ziglang/zig-llvm20/repro.zig:2:5: 0x103ef9d in main (repro)
return error.TheSkyIsFalling;
^
/home/alexrp/Source/ziglang/zig-llvm20/repro.zig:6:5: 0x103efa5 in main (repro)
return error.AndMyCarIsOutOfGas;
^
/home/alexrp/Source/ziglang/zig-llvm20/lib/std/start.zig:656:37: 0x103ee83 in posixCallMainAndExit (repro)
const result = root.main() catch |err| {
^
instead of the expected:
error: AndMyCarIsOutOfGas
/home/alexrp/Source/ziglang/zig-llvm20/repro.zig:2:5: 0x103f00d in main (repro)
return error.TheSkyIsFalling;
^
/home/alexrp/Source/ziglang/zig-llvm20/repro.zig:6:5: 0x103f015 in main (repro)
return error.AndMyCarIsOutOfGas;
^
/home/alexrp/Source/ziglang/zig-llvm20/repro.zig:11:9: 0x103f01d in main (repro)
try bar();
^
Also fix a bunch of cases where we didn't toggle features off if the relevant
leaf isn't available, and switch XCR0 checks to a packed struct.
Closes#23385.
I'm not actually aware of any distro where the name is wine64, so just use wine
in all cases. As part of this, I also fixed the architecture checks to match
reality.
Closes#23411.
* If a function prototype is declarated inside a function, do not
translate it to a top-level extern function declaration. Similar to
extern local variable, just wrapped it into a block-local struct.
* Add a new extern_local_fn tag of aro_translate_c node for present
extern local function declaration.
* When a function body has a C function prototype declaration, it adds
an extern local function declaration. Subsequent function references
will look for this function declaration.
`wasm2c` uses an interesting mechanism to "fake" the existence of cache
directories. However, `wasi_snapshot_preview1_fd_seek` was not correctly
integrated with this system, so previously crashed when run on a file in
a cache directory due to trying to call `fseek` on a `FILE *` which was
`NULL`.
* When saving bigint limbs, we gave the iovec the wrong length, meaning
bigint data (and the following string and compile error data) was corrupted.
* When updating a stale ZOIR cache, we failed to truncate the file, so
just wrote more bytes onto the end of the stale cache.
This is actually completely well-defined. The resulting slice always has
0 elements. The only disallowed case is casting *to* a slice of a
zero-bit type, because in that case, you cna't figure out how many
destination elements to use (and there's *no* valid destination length
if the source slice corresponds to more than 0 bits).
When decoding the literals section of a compressed block, the length of
the regenerated size of the literals must be checked against the buffer
literals are decoded into.
`--fetch` flag now has additional optional parameter, which specifies
how lazy dependencies should be fetched:
* `needed` — lazy dependencies are fetched only if they are required
for current build configuration to work. Default and works same
as old `--fetch` flag.
* `all` — lazy dependencies are always fetched. If `--system` flag
is used after that, it's guaranteed that **any** build configuration
will not require additional download of dependencies during build.
Helpful for distro packagers and CI systems:
https://www.github.com/ziglang/zig/issues/14597#issuecomment-1426827495
If none is passed, behaviour is same as if `needed` was passed.
Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
We already do these on the x86_64-linux machines. They're fairly costly, and it
seems very unlikely to me that they'll uncover issues that wouldn't be uncovered
on x86_64-linux.
This change fixes false-positive cache hits for run steps that get run
with different sets of environment variables due the the environment map
being excluded from the cache hash.
Context:
- https://blog.rust-lang.org/2024/09/04/cve-2024-43402.html
- https://github.com/rust-lang/rust/pull/129962
Note that the Rust test case for this checks that it executes the batch file successfully with the proper mitigation in place, while the Zig test case expects a FileNotFound error. This is because of a PATHEXT optimization that Zig does, and that Rust doesn't do because Rust doesn't do PATHEXT appending (it only appends .exe specifically). See the added comment for more details.
Add a test for std.fs.File's `setEndPos` (which is a simple wrapper around
`std.posix.ftruncate`) to exercise some success and failure paths.
Explicitly check that the `ftruncate` length isn't negative when
interpreted as a signed value. This avoids having to decode overloaded
`EINVAL` errors.
Add errno handling to Windows path to map INVALID_PARAMETER to FileTooBig.
Fixes#22960
Adds a CreateProcessFlags packed struct for all the possible flags to
CreateProcessW on windows. In addition, propagates the existing
`start_suspended` option in std.process.Child which was previously only
used on Darwin. Also adds a `create_no_window` option to std.process.Child
which is a commonly used flag for launching console executables on
windows without causing a new console window to "pop up".
This PR consistently maps .ACCES into AccessDenied and .PERM into
PermissionDenied. AccessDenied is returned if the file mode bit
(user/group/other rwx bits) disallow access (errno was `EACCES`).
PermissionDenied is returned if something else denies access (errno was
`EPERM`) (immutable bit, SELinux, capabilities, etc). This somewhat
subtle distinction is a POSIX thing.
Most of the change is updating std.posix Error Sets to contain both
errors, and then propagating the pair up through caller Error Sets.
Fixes#16782
Use error.AccessDenied for permissions (rights) failures on Wasi
(`EACCES`) and error.PermissionDenied (`EPERM`) for systemic failures.
And pass-through underlying Wasi errors (PermissionDenied or AccessDenied)
without mapping.
Windows defines an `ACCESS_DENIED` error code. There is no
PERMISSION_DENIED (or its equivalent) which seems to only exist on POSIX
systems. Fix a couple Windows calls code to return `error.AccessDenied`
for `ACCESS_DENIED` and to stop mapping AccessDenied into
PermissionDenied.
The "musl" part of the Zig target triples `wasm32-wasi-musl` and
`wasm32-emscripten-musl` refers to the libc, not really the ABI.
For WASM, most LLVM-based tooling uses `wasm32-wasi`, which is
normalized into `wasm32-unknown-wasi`, with an implicit `-unknown` and
without `-musl`.
Similarly, Emscripten uses `wasm32-unknown-emscripten` without `-musl`.
By using `-unknown` instead of `-musl` we get better compatibility with
external tooling.
While it is not allowed for a function coercion to change whether a
function is generic, it *is* okay to make existing concrete parameters
of a generic function also generic, or vice versa. Either of these cases
implies that the result is a generic function, so comptime type checks
will happen when the function is ultimately called.
Resolves: #21099
Emscripten currently implements `emscripten_return_address()` by calling
out into JavaScript and parsing a stack trace, which introduces
significant overhead that we would prefer to avoid in release builds.
This is especially problematic for allocators because the generic parts
of `std.mem.Allocator` make frequent use of `@returnAddress`, even
though very few allocator implementations even observe the return
address, which makes allocators nigh unusable for performance-critical
applications like games if the compiler is unable to devirtualize the
allocator calls.
The old logic only decremented `remaining_prelink_tasks` if `bin_file`
was not `null`. This meant that on `-fno-emit-bin` builds with
registered prelink tasks (e.g. C source files), we exited from
`Compilation.performAllTheWorkInner` early, assuming a prelink error.
Instead, when `bin_file` is `null`, we still decrement
`remaining_prelink_tasks`; we just don't do any actual work.
Resolves: #22682
Both sliceTo and indexOfScalarPos use SIMD when available to speed up the search. On my x86_64 machine, this leads to getenvW being around 2-3x faster overall.
Additionally, any future improvements to sliceTo/indexOfScalarPos will benefit getenvW.
Too many bugs have been found with `truncate` at this point, so it was
rewritten from scratch.
Based on the doc comment, the utility of `convertToTwosComplement` over
`r.truncate(a, .unsigned, bit_count)` is unclear and it has a subtle
behavior difference that is almost certainly a bug, so it was deleted.
When determining the type of RC compiler, meson passes `/?` or `--version` and then reads from `stdout` looking for particular string(s) anywhere in the output.
So, by adding the string "Microsoft Resource Compiler" to the `/?` output, meson will recognize `zig rc` as rc.exe and give it the correct options, which works fine since `zig rc` is drop-in CLI compatible with rc.exe.
This allows using `zig rc` with meson for (cross-)compiling, by either:
- Setting WINDRES="zig rc" or putting windres = ['zig', 'rc'] in the cross-file
+ This will work like rc.exe, so it will output .res files. This will only link successfully if you are using a linker that can do .res -> .obj conversion (so something like zig cc, MSVC, lld)
- Setting WINDRES="zig rc /:output-format coff" or putting windres = ['zig', 'rc', '/:output-format', 'coff'] in the cross-file
+ This will make meson pass flags as if it were rc.exe, but it will cause the resulting .res file to actually be a COFF object file, meaning it will work with any linker that handles COFF object files
Example cross file that uses `zig cc` (which can link `.res` files, so `/:output-format coff` is not necessary) and `zig rc`:
```
[binaries]
c = ['zig', 'cc', '--target=x86_64-windows-gnu']
windres = ['zig', 'rc']
[target_machine]
system = 'windows'
cpu_family = 'x86_64'
cpu = 'x86_64'
endian = 'little'
```
LLD expects the library file name (minus extension) to be exactly libmingw32. By
calling it mingw32 previously, we prevented it from being detected as being in
LLD's list of libraries that are excluded from the MinGW-specific auto-export
mechanism.
b9d27ac252/lld/COFF/MinGW.cpp (L30-L56)
As a result, a DLL built for *-windows-gnu with Zig would export a bunch of
internal MinGW symbols. This sometimes worked out fine, but it could break at
link or run time when linking an EXE with a DLL, where both are targeting
*-windows-gnu and thus linking separate copies of mingw32.lib. In #23204, this
manifested as the linker getting confused about _gnu_exception_handler() because
it was incorrectly exported by the DLL while also being defined in the
mingw32.lib that was being linked into the EXE.
Closes#23204.
This code previously added 4 NUL code units, but that was likely due to a misinterpretation of this part of the CreateProcess documentation:
> A Unicode environment block is terminated by four zero bytes: two for the last string, two more to terminate the block.
(four zero *bytes* means *two* zero code units)
Additionally, the second zero code unit is only actually needed when the environment is empty due to a quirk of the CreateProcess implementation. In the case of a non-empty environment, there always ends up being two trailing NUL code units since one will come after the last environment variable in the block.
This commit reworks how Sema handles arithmetic on comptime-known
values, fixing many bugs in the process.
The general pattern is that arithmetic on comptime-known values is now
handled by the new namespace `Sema.arith`. Functions handling comptime
arithmetic no longer live on `Value`; this is because some of them can
emit compile errors, so some *can't* go on `Value`. Only semantic
analysis should really be doing arithmetic on `Value`s anyway, so it
makes sense for it to integrate more tightly with `Sema`.
This commit also implements more coherent rules surrounding how
`undefined` interacts with comptime and mixed-comptime-runtime
arithmetic. The rules are as follows.
* If an operation cannot trigger Illegal Behavior, and any operand is
`undefined`, the result is `undefined`. This includes operations like
`0 *| undef`, where the LHS logically *could* be used to determine a
defined result. This is partly to simplify the language, but mostly to
permit codegen backends to represent `undefined` values as completely
invalid states.
* If an operation *can* trigger Illegal Behvaior, and any operand is
`undefined`, then Illegal Behavior results. This occurs even if the
operand in question isn't the one that "decides" illegal behavior; for
instance, `undef / 1` is undefined. This is for the same reasons as
described above.
* An operation which would trigger Illegal Behavior, when evaluated at
comptime, instead triggers a compile error. Additionally, if one
operand is comptime-known undef, such that the other (runtime-known)
operand isn't needed to determine that Illegal Behavior would occur,
the compile error is triggered.
* The only situation in which an operation with one comptime-known
operand has a comptime-known result is if that operand is undefined,
in which case the result is either undefined or a compile error per
the above rules. This could potentially be loosened in future (for
instance, `0 * rt` could be comptime-known 0 with a runtime assertion
that `rt` is not undefined), but at least for now, defining it more
conservatively simplifies the language and allows us to easily change
this in future if desired.
This commit fixes many bugs regarding the handling of `undefined`,
particularly in vectors. Along with a collection of smaller tests, two
very large test cases are added to check arithmetic on `undefined`.
The operations which have been rewritten in this PR are:
* `+`, `+%`, `+|`, `@addWithOverflow`
* `-`, `-%`, `-|`, `@subWithOverflow`
* `*`, `*%`, `*|`, `@mulWithOverflow`
* `/`, `@divFloor`, `@divTrunc`, `@divExact`
* `%`, `@rem`, `@mod`
Other arithmetic operations are currently unchanged.
Resolves: #22743Resolves: #22745Resolves: #22748Resolves: #22749Resolves: #22914
The code did one useless thing and two wrong things:
- ref counting was basically a noop
- last_dir_fd was chosen from the wrong index and also under the wrong
condition
This caused regular crashes on macOS which are now gone.
On updates with failed files, we should refrain from doing any semantic
analysis, or even touching codegen/link. That way, incremental
compilation state is untouched for when the user fixes the AstGen
errors.
Resolves: #23205
* use `tmp.dir.realpathAlloc()` to get full path into tmpDir instances
* use `testing.allocator` where that simplifies things (vs. manual ArenaAllocator for 1 or 2 allocs)
* Trust `TmpDir.cleanup()` to clean up contained files and sub-trees
* Remove some unnecessary absolute paths (enabling WASI to run the tests)
* Drop some no-longer necessary `[_][]const u8` casts
* Add scopes to reduce `var` usage in favor of `const`
This reverts commit 7e0c25eccd8d9bc5b77953dbc9a39a26e383c550.
The `--git-dir` argument is relative to the `-C` argument, making this
patch OK after all.
I added a comment to go along with this since I found it confusing.
Apologies for the revert.
Sometimes Zig is built not from a git repository (e.g. from tarball), but inside another git repository (e.g. distro package repository). Make sure that the version check tries to parse a tag of Zig, and not of a parent directory.
This should be a lot easier to maintain. It's also a small step towards
eventually making the builder API parse the data layout string in order to
answer layout questions that we need to ask during code generation.
Clang's integrated Arm assembler doesn't understand -mabi yet, so this results
in "unused command line argument" warnings when building musl code and glibc
stubs, for example.
This commits adds the following distinct integer types to std.zig.Ast:
- OptionalTokenIndex
- TokenOffset
- OptionalTokenOffset
- Node.OptionalIndex
- Node.Offset
- Node.OptionalOffset
The `Node.Index` type has also been converted to a distinct type while
`TokenIndex` remains unchanged.
`Ast.Node.Data` has also been changed to a (untagged) union to provide
safety checks.
This function checks for various possibilities that are never produced
by the parser.
Given that lastToken is unsafe to call on an Ast with errors, I also
removed code paths that would be reachable on an Ast with errors.
[Incremental provided buffer
consumption](https://github.com/axboe/liburing/wiki/What's-new-with-io_uring-in-6.11-and-6.12#incremental-provided-buffer-consumption)
support is added in kernel 6.12.
IoUring.BufferGroup will now use incremental consumption whenever
kernel supports it.
Before, provided buffers are wholly consumed when picked. Each cqe
points to the different buffer. With this, cqe points to the part of the
buffer. Multiple cqe's can reuse same buffer.
Appropriate sizing of buffers becomes less important.
There are slight changes in BufferGroup interface (it now needs to track
current receive point for each buffer). Init requires allocator
instead of buffers slice, it will allocate buffers slice and head
pointers slice. Get and put now requires cqe becasue there we have
information will the buffer be reused.
ring.cmd_sock is generic socket operation. Two most common uses are
setsockopt and getsockopt. This provides same interface as posix
versions of this methods.
libring has also [sqe_set_flags](https://man7.org/linux/man-pages/man3/io_uring_sqe_set_flags.3.html)
method. Adding that in our io_uring_sqe. Adding sqe.link_next method for setting most common flag.
consttest_filters=b.option([]const[]constu8,"test-filter","Skip tests that do not match any filter")orelse&[0][]constu8{};
consttest_target_filters=b.option([]const[]constu8,"test-target-filter","Skip tests whose target triple do not match any filter")orelse&[0][]constu8{};
consttest_slow_targets=b.option(bool,"test-slow-targets","Enable running module tests for targets that have a slow compiler backend")orelsefalse;
consttest_extra_targets=b.option(bool,"test-extra-targets","Enable running module tests for additional targets")orelsefalse;
Error Return Traces are enabled by default in {#link|Debug#} and {#link|ReleaseSafe#} builds and disabled by default in {#link|ReleaseFast#} and {#link|ReleaseSmall#} builds.
Error Return Traces are enabled by default in {#link|Debug#} builds and disabled by default in {#link|ReleaseFast#}, {#link|ReleaseSafe#} and {#link|ReleaseSmall#} builds.
</p>
<p>
There are a few ways to activate this error return tracing feature:
This is usually <code>src/main.zig</code> but depends on what file is built.
<li>{#syntax#}@import("builtin"){#endsyntax#} - Target-specific information. The command <code>zig build-exe --show-builtin</code> outputs the source to stdout for reference.</li>
<li>{#syntax#}@import("root"){#endsyntax#} - Alias for the root module. In typical project structures, this means it refers back to <code>src/main.zig</code>.
<p>Returns the comptime-only "enum literal" type. This is the type of uncoerced {#link|Enum Literals#}. Values of this type can coerce to any {#link|enum#} with a matching field.</p>
<li>If a call to {#syntax#}@import{#endsyntax#} is analyzed, the file being imported is analyzed.</li>
<li>If a type (including a file) is analyzed, all {#syntax#}comptime{#endsyntax#}, {#syntax#}usingnamespace{#endsyntax#}, and {#syntax#}export{#endsyntax#} declarations within it are analyzed.</li>
<li>If a type (including a file) is analyzed, all {#syntax#}comptime{#endsyntax#} and {#syntax#}export{#endsyntax#} declarations within it are analyzed.</li>
<li>If a type (including a file) is analyzed, and the compilation is for a {#link|test|Zig Test#}, and the module the type is within is the root module of the compilation, then all {#syntax#}test{#endsyntax#} declarations within it are also analyzed.</li>
<li>If a reference to a named declaration (i.e. a usage of it) is analyzed, the declaration being referenced is analyzed. Declarations are order-independent, so this reference may be above or below the declaration being referenced, or even in another file entirely.</li>
</ul>
@ -7119,10 +7138,12 @@ coding style.
</p>
<ul>
<li>
If {#syntax#}x{#endsyntax#} is a {#syntax#}type{#endsyntax#}
then {#syntax#}x{#endsyntax#} should be {#syntax#}TitleCase{#endsyntax#}, unless it
is a {#syntax#}struct{#endsyntax#} with 0 fields and is never meant to be instantiated,
in which case it is considered to be a "namespace" and uses {#syntax#}snake_case{#endsyntax#}.
If {#syntax#}x{#endsyntax#} is a {#syntax#}struct{#endsyntax#} with 0 fields and is never meant to be instantiated
then {#syntax#}x{#endsyntax#} is considered to be a "namespace" and should be {#syntax#}snake_case{#endsyntax#}.
</li>
<li>
If {#syntax#}x{#endsyntax#} is a {#syntax#}type{#endsyntax#} or {#syntax#}type{#endsyntax#} alias
then {#syntax#}x{#endsyntax#} should be {#syntax#}TitleCase{#endsyntax#}.
</li>
<li>
If {#syntax#}x{#endsyntax#} is callable, and {#syntax#}x{#endsyntax#}'s return type is
<spanclass="tooltip-content">Sum across all threads of the time spent in this pipeline component</span>
</th>
<thscope="col"class="tooltip">Real Time
<spanclass="tooltip-content">Wall-clock time elapsed between the start and end of this compilation phase</span>
</th>
<thscope="col">Compilation Phase</th>
</tr>
</thead>
<tbody>
<tr>
<thscope="row"class="tooltip">Parsing
<spanclass="tooltip-content"><code>tokenize</code> converts a file of Zig source code into a sequence of tokens, which are then processed by <code>Parse</code> into an Abstract Syntax Tree (AST).</span>
<spanclass="tooltip-content">Tokenization, parsing, and lowering of Zig source files to a high-level IR.<br><br>Starting from module roots, every file theoretically accessible through a chain of <code>@import</code> calls is processed. Individual source files are processed serially, but different files are processed in parallel by a thread pool.<br><br>The results of this phase of compilation are cached on disk per source file, meaning the time spent here is typically only relevant to "clean" builds.</span>
</th>
</tr>
<tr>
<thscope="row"class="tooltip">AST Lowering
<spanclass="tooltip-content"><code>AstGen</code> converts a file's AST into a high-level SSA IR named Zig Intermediate Representation (ZIR). The resulting ZIR code is cached on disk to avoid, for instance, re-lowering all source files in the Zig standard library each time the compiler is invoked.</span>
</th>
<td><slotname="cpu-time-astgen"></slot></td>
</tr>
<tr>
<thscope="row"class="tooltip">Semantic Analysis
<spanclass="tooltip-content"><code>Sema</code> interprets ZIR to perform type checking, compile-time code execution, and type resolution, collectively termed "semantic analysis". When a runtime function body is analyzed, it emits Analyzed Intermediate Representation (AIR) code to be sent to the next pipeline component. Semantic analysis is currently entirely single-threaded.</span>
<spanclass="tooltip-content">Semantic analysis, code generation, and linking, at the granularity of individual declarations (as opposed to whole source files).<br><br>These components are run in parallel with one another. Semantic analysis is almost always the bottleneck, as it is complex and currently can only run single-threaded.<br><br>This phase completes when a work queue empties, but semantic analysis may add work by one declaration referencing another.<br><br>This is the main phase of compilation, typically taking significantly longer than File Lower (even in a clean build).</span>
</th>
</tr>
<tr>
<thscope="row"class="tooltip">Code Generation
<spanclass="tooltip-content"><code>CodeGen</code> converts AIR from <code>Sema</code> into machine instructions in the form of Machine Intermediate Representation (MIR). This work is usually highly parallel, since in most cases, arbitrarily many functions can be run through <code>CodeGen</code> simultaneously.</span>
</th>
<td><slotname="cpu-time-codegen"></slot></td>
</tr>
<tr>
<thscope="row"class="tooltip">Linking
<spanclass="tooltip-content"><code>link</code> converts MIR from <code>CodeGen</code>, as well as global constants and variables from <code>Sema</code>, and places them in the output binary. MIR is converted to a finished sequence of real instruction bytes.<br><br>When using the LLVM backend, most of this work is instead deferred to the "LLVM Emit" phase.</span>
</th>
<td><slotname="cpu-time-link"></slot></td>
</tr>
<trclass="llvm-only">
<thclass="empty-cell"></th>
<tdclass="empty-cell"></td>
<td><slotname="real-time-llvm-emit"></slot></td>
<thscope="row"class="tooltip">LLVM Emit
<spanclass="tooltip-content"><b>Only applicable when using the LLVM backend.</b><br><br>Conversion of generated LLVM bitcode to an object file, including any optimization passes.<br><br>When using LLVM, this phase of compilation is typically the slowest by a significant margin. Unfortunately, the Zig compiler implementation has essentially no control over it.</span>
</th>
</tr>
<tr>
<thclass="empty-cell"></th>
<tdclass="empty-cell"></td>
<td><slotname="real-time-link-flush"></slot></td>
<thscope="row"class="tooltip">Linker Flush
<spanclass="tooltip-content">Finalizing the emitted binary, and ensuring it is fully written to disk.<br><br>When using LLD, this phase represents the entire linker invocation. Otherwise, the amount of work performed here is dependent on details of Zig's linker implementation for the particular output format, but typically aims to be fairly minimal.</span>
</th>
</tr>
</tbody>
</table>
<detailsclass="section">
<summary>Files</summary>
<tableclass="time-stats">
<thead>
<tr>
<thscope="col">File</th>
<thscope="col">Semantic Analysis</th>
<thscope="col">Code Generation</th>
<thscope="col">Linking</th>
<thscope="col">Total</th>
</tr>
</thead>
<!-- HTML does not allow placing a 'slot' inside of a 'tbody' for backwards-compatibility
reasons, so we unfortunately must template on the `id` here. -->
<tbodyid="fileTableBody"></tbody>
</table>
</details>
<detailsclass="section">
<summary>Declarations</summary>
<tableclass="time-stats">
<thead>
<tr>
<thscope="col">File</th>
<thscope="col">Declaration</th>
<thscope="col"class="tooltip">Analysis Count
<spanclass="tooltip-content">The number of times the compiler analyzed some part of this declaration. If this is a function, <code>inline</code> and <code>comptime</code> calls to it are <i>not</i> included here. Typically, this value is approximately equal to the number of instances of a generic declaration.</span>
</th>
<thscope="col">Semantic Analysis</th>
<thscope="col">Code Generation</th>
<thscope="col">Linking</th>
<thscope="col">Total</th>
</tr>
</thead>
<!-- HTML does not allow placing a 'slot' inside of a 'tbody' for backwards-compatibility
reasons, so we unfortunately must template on the `id` here. -->
<tbodyid="declTableBody"></tbody>
</table>
</details>
<detailsclass="section llvm-only">
<summary>LLVM Pass Timings</summary>
<div><slotname="llvm-pass-timings"></slot></div>
</details>
</div>
<divid="runTestReport">
<tableclass="time-stats">
<thead>
<tr>
<thscope="col">Test Name</th>
<thscope="col">Duration</th>
</tr>
</thead>
<!-- HTML does not allow placing a 'slot' inside of a 'tbody' for backwards-compatibility
reasons, so we unfortunately must template on the `id` here. -->
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
