std.Build.addTest creates a CompileStep as before, however, this kind of
step no longer actually runs the unit tests. Instead it only compiles
it, and one must additionally create a RunStep from the CompileStep in
order to actually run the tests.
RunStep gains integration with the default test runner, which now
supports the standard --listen=- argument in order to communicate over
stdin and stdout. It also reports test statistics; how many passed,
failed, and leaked, as well as directly associating the relevant stderr
with the particular test name that failed.
This separation of CompileStep and RunStep means that
`CompileStep.Kind.test_exe` is no longer needed, and therefore has been
removed in this commit.
* build runner: show unit test statistics in build summary
* added Step.writeManifest since many steps want to treat it as a
warning and emit the same message if it fails.
* RunStep: fixed error message that prints the failed command printing
the original argv and not the adjusted argv in case an interpreter
was used.
* RunStep: fixed not passing the command line arguments to the
interpreter.
* move src/Server.zig to std.zig.Server so that the default test runner
can use it.
* the simpler test runner function which is used by work-in-progress
backends now no longer prints to stderr, which is necessary in order
for the build runner to not print the stderr as a warning message.
This commit extracts out server code into src/Server.zig and uses it
both in the main CLI as well as `zig objcopy`.
std.Build.ObjCopyStep now adds `--listen=-` to the CLI for `zig objcopy`
and observes the protocol for progress and other kinds of integrations.
This fixes the last two test failures of this branch when I run
`zig build test` locally.
Instead of using `zig test` to build a special version of the compiler
that runs all the test-cases, the zig build system is now used as much
as possible - all with the basic steps found in the standard library.
For incremental compilation tests (the ones that look like foo.0.zig,
foo.1.zig, foo.2.zig, etc.), a special version of the compiler is
compiled into a utility executable called "check-case" which checks
exactly one sequence of incremental updates in an independent
subprocess. Previously, all incremental and non-incremental test cases
were done in the same test runner process.
The compile error checking code is now simpler, but also a bit
rudimentary, and so it additionally makes sure that the actual compile
errors do not include *extra* messages, and it makes sure that the
actual compile errors output in the same order as expected. It is also
based on the "ends-with" property of each line rather than the previous
logic, which frankly I didn't want to touch with a ten-meter pole. The
compile error test cases have been updated to pass in light of these
differences.
Previously, 'error' mode with 0 compile errors was used to shoehorn in a
different kind of test-case - one that only checks if a piece of code
compiles without errors. Now there is a 'compile' mode of test-cases,
and 'error' must be only used when there are greater than 0 errors.
link test cases are updated to omit the target object format argument
when calling checkObject since that is no longer needed.
The test/stage2 directory is removed; the 2 files within are moved to be
directly in the test/ directory.
* std.zig.ErrorBundle: support rendering options for whether to include
the reference trace, whether to include the source line, and TTY
configuration.
* build runner: don't print progress in dumb terminals
* std.Build.CompileStep:
- add a way to expect compilation errors via the new `expect_errors`
field. This is an advanced setting that can change the intent of
the CompileStep. If this slice has nonzero length, it means that
the CompileStep exists to check for compile errors and return
*success* if they match, and failure otherwise.
- remove the object format parameter from `checkObject`. The object
format is known based on the CompileStep's target.
- Avoid passing -L and -I flags for nonexistent directories within
search_prefixes. This prevents a warning, that should probably be
upgraded to an error in Zig's CLI parsing code, when the linker
sees an -L directory that does not exist.
* std.Build.Step:
- When spawning the zig compiler process, takes advantage of the new
`std.Progress.Node.setName` API to avoid ticking up a meaningless
number at every progress update.
Rework std.Build.Step to have an `owner: *Build` field. This
simplified the implementation of installation steps, as well as provided
some much-needed common API for the new parallelized build system.
--verbose is now defined very concretely: it prints to stderr just
before spawning a child process.
Child process execution is updated to conform to the new
parallel-friendly make() function semantics.
DRY up the failWithCacheError handling code. It now integrates properly
with the step graph instead of incorrectly dumping to stderr and calling
process exit.
In the main CLI, fix `zig fmt` crash when there are no errors and stdin
is used.
Deleted steps:
* EmulatableRunStep - this entire thing can be removed in favor of a
flag added to std.Build.RunStep called `skip_foreign_checks`.
* LogStep - this doesn't really fit with a multi-threaded build runner
and is effectively superseded by the new build summary output.
build runner:
* add -fsummary and -fno-summary to override the default behavior,
which is to print a summary if any of the build steps fail.
* print the dep prefix when emitting error messages for steps.
std.Build.FmtStep:
* This step now supports exclude paths as well as a check flag.
* The check flag decides between two modes, modify mode, and check
mode. These can be used to update source files in place, or to fail
the build, respectively.
Zig's own build.zig:
* The `test-fmt` step will do all the `zig fmt` checking that we expect
to be done. Since the `test` step depends on this one, we can simply
remove the explicit call to `zig fmt` in the CI.
* The new `fmt` step will actually perform `zig fmt` and update source
files in place.
std.Build.RunStep:
* expose max_stdio_size is a field (previously an unchangeable
hard-coded value).
* rework the API. Instead of configuring each stream independently,
there is a `stdio` field where you can choose between
`infer_from_args`, `inherit`, or `check`. These determine whether the
RunStep is considered to have side-effects or not. The previous
field, `condition` is gone.
* when stdio mode is set to `check` there is a slice of any number of
checks to make, which include things like exit code, stderr matching,
or stdout matching.
* remove the ill-defined `print` field.
* when adding an output arg, it takes the opportunity to give itself a
better name.
* The flag `skip_foreign_checks` is added. If this is true, a RunStep
which is configured to check the output of the executed binary will
not fail the build if the binary cannot be executed due to being for
a foreign binary to the host system which is running the build graph.
Command-line arguments such as -fqemu and -fwasmtime may affect
whether a binary is detected as foreign, as well as system
configuration such as Rosetta (macOS) and binfmt_misc (Linux).
- This makes EmulatableRunStep no longer needed.
* Fix the child process handling to properly integrate with the new
bulid API and to avoid deadlocks in stdout/stderr streams by polling
if necessary.
std.Build.RemoveDirStep now uses the open build_root directory handle
instead of an absolute path.
This makes progress be exposed to the top-level caller of update().
I tossed in a bonus change: when the `zig build` subcommand sees exit
code 2, it omits the "following command failed" line, and the build
runner uses exit code 2 when there are compile errors. This tidies up
the output on build failure by a little bit.
The compiler now provides a server protocol for an interactive session
with another process. The build runner uses this protocol to communicate
compilation errors semantically from zig compiler subprocesses to the
build runner.
The protocol is exposed via stdin/stdout, or on a network socket,
depending on whether the CLI flag `--listen=-` or e.g.
`--listen=127.0.0.1:1337` is used.
Additionally:
* add the zig version string to the build runner cache prefix
* remove --prominent-compile-errors CLI flag because it no longer does
anything. Compilation errors are now unconditionally displayed at the
bottom of the build summary output when using the terminal-based
build runner.
* Remove the color field from std.Build. The build steps are no longer
supposed to interact with stderr directly. Instead they communicate
semantically back to the build runner, which has its own logic about
TTY configuration.
* Use the cleanExit() pattern in the build runner.
* Build steps can now use error.MakeFailed when they have already
properly reported an error, or they can fail with any other error
code in which case the build runner will create a simple message
based on this error code.
Introduces std.zig.ErrorBundle which is a trivially serializeable set
of compilation errors. This is in the standard library so that both
the compiler and the build runner can use it. The idea is they will
use it to communicate compilation errors over a binary protocol.
The binary encoding of ErrorBundle is a bit problematic - I got a little
too aggressive with compaction. I need to change it in a follow-up
commit to use some indirection in the error message list, otherwise
iteration is too unergonomic. In fact it's so problematic right now that
the logic getAllErrorsAlloc() actually fails to produce a viable
ErrorBundle because it puts SourceLocation data in between the root
level ErrorMessage data.
This commit has a simplification - redundant logic for rendering AST
errors to stderr has been removed in favor of moving the logic for
lowering AST errors into AstGen. So even if we get parse errors, the
errors will get lowered into ZIR before being reported. I believe this
will be useful when working on --autofix. Either way, some redundant
brittle logic was happily deleted.
In Compilation, updateSubCompilation() is improved to properly perform
error reporting when a sub-compilation object fails. It no longer dumps
directly to stderr; instead it populates an ErrorBundle object, which
gets added to the parent one during getAllErrorsAlloc().
In package fetching code, instead of dumping directly to stderr, it now
populates an ErrorBundle object, and gets properly reported at the CLI
layer of abstraction.
- improve fn prototypes of process_vm_writev
- make the memory writable in the ELF file
- force the linker to always append the function
- write updates with process_vm_writev
Previously, if you had a pointer to multiple array elements and tried to
write to it at comptime, it was incorrectly treated as a pointer to one
specific array value, leading to an assertion down the line. If we try
to mutate a value at an elem_ptr larger than the element type, we need
to perform a modification to multiple array elements.
This solution isn't ideal, since it will result in storePtrVal
serializing the whole array, modifying the relevant parts, and storing
it back. Ideally, it would only take the required elements. However,
this change would have been more complex, and this is a fairly rare
operation (nobody ever ran into the bug before after all), so it doesn't
matter all that much.
* There was an edge case where the arena could be destroyed twice on
error: once from the arena itself and once from the decl destruction.
* The type of the created decl was incorrect (it should have been the
pointer child type), but it's not required anyway, so it's now just
initialized to anyopaque (which more accurately reflects what's
actually at that memory, since e.g. [*]T may correspond to nothing).
* A runtime bitcast of the pointer was performed, meaning @extern didn't
work at comptime. This is unnecessary: the decl_ref can just be
initialized with the correct pointer type.
