* CompileStep: Avoid calling producesPdbFile() to determine whether the
option should be respected. If the user asks for it, put it on the
command line and let the Zig CLI deal with it appropriately.
* Make the namespace of `std.dwarf.Format.dwarf32` no longer have a
redundant "dwarf" in it.
* Add `zig cc` integration for `-gdwarf32` and `-gdwarf64`.
* Toss in a bonus bug fix for `-gdwarf-2`, `-gdwarf-3`, etc.
* Avoid using default init values for struct fields unnecessarily.
* Add missing cache hash addition for the new option.
This commit enables producing 64-bit DWARF format for Zig executables
that are produced through the LLVM backend. This is achieved by exposing
both command-line flags and CompileStep flags. The production of the
64-bit format only affects binaries that use the DWARF format and it is
disabled on MacOS due to it being problematic. This commit, despite
generating the interface for the Zig user to be able to tell the compile
which format is wanted, is just implemented for the LLVM backend, so
clang and the self-hosted backends will need this to be implemented in a
future commit.
This is an effort to work around #7962, since the emission of the 64-bit
format automatically produces 64-bit relocations. Further investigation
will be needed to make DWARF 32-bit format to emit bigger relocations
when needed and not make the linker angry.
This fixes `.INVAL => unreachable` being triggered by the cache system
on macOS when multiple processes race to create the same compilation.
The problem is that when two processes race to create a file, it
sometimes returns ENOENT even though that error code is nonsensical for
this situation.
Commit 2b0929929d67e222ca6a9523a3a594ed456c4a51 purportedly solved this,
but it did not open the file with write permissions, leading to the
EINVAL panic later on. This commit remedies the situation by introducing
a loop and simply retrying when the ENOENT occurs.
The CI now runs C backend tests in addition to compiling them. It uses
-std=c99 -pedantic -Werror in order to catch non-conformant C code.
This necessitated disabling a test case that caused a C compile error,
in addition to disabling a handful of warnings that are already being
triggered by Zig's C backend output for the behavior tests.
The upshot is that I was able to, very cleanly, integrate the C backend
tests into the build system, so that it communicates via the test runner
protocol along with all the other behavior tests.
reverts regression introduced in
d2ad3f5074877475c8f0ec0fbbb323a05fe8cf78.
The commit correctly removed dest_builder from InstallArtifactStep, but
the change to installLibraryHeaders was incorrect since it affected
different steps than that one.
* remove setName, setFilter, and setTestRunner. Please set these
options directly when creating the CompileStep.
* removed unused field
* remove computeOutFileNames and inline the logic, making clear the
goal of avoiding state mutations after the build step is created.
These functions are problematic in light of dependencies because they
run and install, respectively, for the *owner* package rather than for
the *user* package. By removing these functions, the build script is
forced to provide the *Build object to associate the new step with,
making everything less surprising.
Unfortunately, this is a widely breaking change.
see #15079
This was used to ensure that an artifact would only be installed once,
but this is not only unnecessary, but actively harmful, in the face of
dependencies.
see #15079
Also clean up parsing of linker args - reuse `ArgsIterator`.
In MachO, ensure we add every symbol marked with `-u` as undefined
before proceeding with symbol resolution. Additionally, ensure those
symbols are never garbage collected.
MachO entry_in_dylib test: pass `-u _my_main` when linking executable
so that it is not incorrectly garbage collected by the linker.
* use the same hash function as the rest of the steps
* fix race condition due to a macOS oddity.
* fix race condition due to file truncation (rename into place instead)
* integrate with marking Step.result_cached. check if the file already
exists with fs.access before doing anything else.
* use a directory so that the file basename can be "options.zig"
instead of a hash digest.
* better error reporting in case of file system failures.
Previously this code asserted that a fifo's readable length was greater
than or equal to the length of its readable slice, which was an invalid
assertion.
This code avoids making that assumption.
The error was caught and created a Step failure rather than bubbling up
so that the interpreter logic could handle it. Fixes hundreds of test
failures on Windows.
There are no dir components, so you would think that this was
unreachable, however we have observed on macOS two processes racing
to do openat() with O_CREAT manifest in ENOENT.
* Remove some functions that are no longer needed since
EmulateableRunStep is gone.
* Add removeEnvironmentVariable function.
* Support printing environment variables in --verbose mode.
std.Build.addTest creates a CompileStep as before, however, this kind of
step no longer actually runs the unit tests. Instead it only compiles
it, and one must additionally create a RunStep from the CompileStep in
order to actually run the tests.
RunStep gains integration with the default test runner, which now
supports the standard --listen=- argument in order to communicate over
stdin and stdout. It also reports test statistics; how many passed,
failed, and leaked, as well as directly associating the relevant stderr
with the particular test name that failed.
This separation of CompileStep and RunStep means that
`CompileStep.Kind.test_exe` is no longer needed, and therefore has been
removed in this commit.
* build runner: show unit test statistics in build summary
* added Step.writeManifest since many steps want to treat it as a
warning and emit the same message if it fails.
* RunStep: fixed error message that prints the failed command printing
the original argv and not the adjusted argv in case an interpreter
was used.
* RunStep: fixed not passing the command line arguments to the
interpreter.
* move src/Server.zig to std.zig.Server so that the default test runner
can use it.
* the simpler test runner function which is used by work-in-progress
backends now no longer prints to stderr, which is necessary in order
for the build runner to not print the stderr as a warning message.
This commit extracts out server code into src/Server.zig and uses it
both in the main CLI as well as `zig objcopy`.
std.Build.ObjCopyStep now adds `--listen=-` to the CLI for `zig objcopy`
and observes the protocol for progress and other kinds of integrations.
This fixes the last two test failures of this branch when I run
`zig build test` locally.
* std.zig.ErrorBundle: support rendering options for whether to include
the reference trace, whether to include the source line, and TTY
configuration.
* build runner: don't print progress in dumb terminals
* std.Build.CompileStep:
- add a way to expect compilation errors via the new `expect_errors`
field. This is an advanced setting that can change the intent of
the CompileStep. If this slice has nonzero length, it means that
the CompileStep exists to check for compile errors and return
*success* if they match, and failure otherwise.
- remove the object format parameter from `checkObject`. The object
format is known based on the CompileStep's target.
- Avoid passing -L and -I flags for nonexistent directories within
search_prefixes. This prevents a warning, that should probably be
upgraded to an error in Zig's CLI parsing code, when the linker
sees an -L directory that does not exist.
* std.Build.Step:
- When spawning the zig compiler process, takes advantage of the new
`std.Progress.Node.setName` API to avoid ticking up a meaningless
number at every progress update.
CLI tests are now ported over to the new std.Build API and thus work
properly with concurrency.
* add `std.Build.addCheckFile` for creating a
`std.Build.CheckFileStep`.
* add `std.Build.makeTempPath`. This function is intended to be called
in the `configure` phase only. It returns an absolute directory path,
which is potentially going to be a source of API breakage in the
future, so keep that in mind when using this function.
* add `std.Build.CheckFileStep.setName`.
* `std.Build.CheckFileStep`: better error message when reading the
input file fails.
* `std.Build.RunStep`: add a `has_side_effects` flag for when you need
to override the autodetection.
* `std.Build.RunStep`: add the ability to obtain a FileSource for the
directory that contains the written files.
* `std.Build.WriteFileStep`: add a way to write bytes to an arbitrary
path - absolute or relative to the package root. Be careful with this
because it updates source files. This should not be used as part of
the normal build process, but as a utility occasionally run by a
developer with intent to modify source files and then commit those
changes to version control. A file added this way is not available
with `getFileSource`.
* RunStep: ability to set stdin
* RunStep: ability to capture stdout and stderr as a FileSource
* RunStep: add setName method
* RunStep: hash the stdio checks
The problem is that one may execute too many subprocesses concurrently
that, together, exceed an RSS value that causes the OOM killer to kill
something problematic such as the window manager. Or worse, nothing, and
the system freezes.
This is a real world problem. For example when building LLVM a simple
`ninja install` will bring your system to its knees if you don't know
that you should add `-DLLVM_PARALLEL_LINK_JOBS=1`.
In particular: compiling the zig std lib tests takes about 2G each,
which at 16x at once (8 cores + hyperthreading) is using all 32GB of my
RAM, causing the OOM killer to kill my window manager
The idea here is that you can annotate steps that might use a high
amount of system resources with an upper bound. So for example I could
mark the std lib tests as having an upper bound peak RSS of 3 GiB.
Then the build system will do 2 things:
1. ulimit the child process, so that it will fail if it would exceed
that memory limit.
2. Notice how much system RAM is available and avoid running too many
concurrent jobs at once that would total more than that.
This implements (1) not with an operating system enforced limit, but by
checking the maxrss after a child process exits.
However it does implement (2) correctly.
The available memory used by the build system defaults to the total
system memory, regardless of whether it is used by other processes at
the time of spawning the build runner. This value can be overridden with
the new --maxrss flag to `zig build`. This mechanism will ensure that
the sum total of upper bound RSS memory of concurrent tasks will not
exceed this value.
This system makes it so that project maintainers can annotate
problematic subprocesses, avoiding bug reports from users, who can
blissfully execute `zig build` without worrying about the project's
internals.
Nobody's computer crashes, and the build system uses as much parallelism
as possible without risking OOM. Users do not need to unnecessarily
resort to -j1 when the build system can figure this out for them.
* remove std.Build.updateFile. I noticed some people use it from
build.zig (declare phase) when it is intended only for use in the
make phase.
- This also was incorrectly reporting errors with std.log.
* std.Build.InstallArtifactStep
- report better errors on failure
- report whether the step was cached or not
* std.Build.InstallDirStep: report better error on failure
* std.Build.InstallFileStep: report better error on failure
