13 Commits

Author SHA1 Message Date
Andrew Kelley
58edefc6d1 zig build: many enhancements related to parallel building
Rework std.Build.Step to have an `owner: *Build` field. This
simplified the implementation of installation steps, as well as provided
some much-needed common API for the new parallelized build system.

--verbose is now defined very concretely: it prints to stderr just
before spawning a child process.

Child process execution is updated to conform to the new
parallel-friendly make() function semantics.

DRY up the failWithCacheError handling code. It now integrates properly
with the step graph instead of incorrectly dumping to stderr and calling
process exit.

In the main CLI, fix `zig fmt` crash when there are no errors and stdin
is used.

Deleted steps:
 * EmulatableRunStep - this entire thing can be removed in favor of a
   flag added to std.Build.RunStep called `skip_foreign_checks`.
 * LogStep - this doesn't really fit with a multi-threaded build runner
   and is effectively superseded by the new build summary output.

build runner:
 * add -fsummary and -fno-summary to override the default behavior,
   which is to print a summary if any of the build steps fail.
 * print the dep prefix when emitting error messages for steps.

std.Build.FmtStep:
 * This step now supports exclude paths as well as a check flag.
 * The check flag decides between two modes, modify mode, and check
   mode. These can be used to update source files in place, or to fail
   the build, respectively.

Zig's own build.zig:
 * The `test-fmt` step will do all the `zig fmt` checking that we expect
   to be done. Since the `test` step depends on this one, we can simply
   remove the explicit call to `zig fmt` in the CI.
 * The new `fmt` step will actually perform `zig fmt` and update source
   files in place.

std.Build.RunStep:
 * expose max_stdio_size is a field (previously an unchangeable
   hard-coded value).
 * rework the API. Instead of configuring each stream independently,
   there is a `stdio` field where you can choose between
   `infer_from_args`, `inherit`, or `check`. These determine whether the
   RunStep is considered to have side-effects or not. The previous
   field, `condition` is gone.
 * when stdio mode is set to `check` there is a slice of any number of
   checks to make, which include things like exit code, stderr matching,
   or stdout matching.
 * remove the ill-defined `print` field.
 * when adding an output arg, it takes the opportunity to give itself a
   better name.
 * The flag `skip_foreign_checks` is added. If this is true, a RunStep
   which is configured to check the output of the executed binary will
   not fail the build if the binary cannot be executed due to being for
   a foreign binary to the host system which is running the build graph.
   Command-line arguments such as -fqemu and -fwasmtime may affect
   whether a binary is detected as foreign, as well as system
   configuration such as Rosetta (macOS) and binfmt_misc (Linux).
   - This makes EmulatableRunStep no longer needed.
 * Fix the child process handling to properly integrate with the new
   bulid API and to avoid deadlocks in stdout/stderr streams by polling
   if necessary.

std.Build.RemoveDirStep now uses the open build_root directory handle
instead of an absolute path.
2023-03-15 10:48:13 -07:00
Andrew Kelley
b4997d0890 std.Build.RunStep: better default step name
Now it renames itself when an output argument is added.
2023-03-15 10:48:13 -07:00
Andrew Kelley
0e078790fe multiplex compiler progress messages into the build runner 2023-03-15 10:48:13 -07:00
Andrew Kelley
986a30e373 integrate the build runner and the compiler server
The compiler now provides a server protocol for an interactive session
with another process. The build runner uses this protocol to communicate
compilation errors semantically from zig compiler subprocesses to the
build runner.

The protocol is exposed via stdin/stdout, or on a network socket,
depending on whether the CLI flag `--listen=-` or e.g.
`--listen=127.0.0.1:1337` is used.

Additionally:

 * add the zig version string to the build runner cache prefix

 * remove --prominent-compile-errors CLI flag because it no longer does
   anything. Compilation errors are now unconditionally displayed at the
   bottom of the build summary output when using the terminal-based
   build runner.

 * Remove the color field from std.Build. The build steps are no longer
   supposed to interact with stderr directly. Instead they communicate
   semantically back to the build runner, which has its own logic about
   TTY configuration.

 * Use the cleanExit() pattern in the build runner.

 * Build steps can now use error.MakeFailed when they have already
   properly reported an error, or they can fail with any other error
   code in which case the build runner will create a simple message
   based on this error code.
2023-03-15 10:48:13 -07:00
Andrew Kelley
02381c0372 std.Build: improve debugging of misconfigured steps
* Step.init() now takes an options struct
 * Step.init() now captures a small stack trace and stores it in the
   Step so that it can be accessed when printing user-friendly debugging
   information, including the lines of code that created the step in
   question.
2023-03-15 10:48:12 -07:00
Andrew Kelley
adfc019d60
Merge pull request #11982 from marler8997/ignoreSigpipe
ignore SIGPIPE by default
2023-02-19 13:55:12 -05:00
Andrew Kelley
5e7b09ce9f std.Build.RunStep: fix default caching logic
RunStep is supposed to auto-detect whether the intend is for
side-effects or for producing an output file. The auto-detection logic
was incorrect, and this commit fixes it.

I tested this manually locally. Automated testing will require a more
significant investment in the test harness, which I will work on in a
future enhancement.

closes #14666
2023-02-18 19:33:47 -05:00
Jonathan Marler
0a8fe34b11 add test to ignore sigpipe 2023-02-17 15:25:36 -07:00
Andrew Kelley
d97042ad2e std.Build: start using the cache system with RunStep
* Use std.Build.Cache.Directory instead of a string for storing the
   cache roots and build roots.
 * Set up a std.Build.Cache in build_runner.zig and use it in
   std.Build.RunStep for avoiding redundant work.
2023-02-13 06:42:26 -07:00
Andrew Kelley
2654d0c668 std.Build.RunStep: introduce addOutputFileArg API
This provides file path as a command line argument to the command being
run, and returns a FileSource which can be used as inputs to other APIs
throughout the build system.

Unfortunately, it is implemented by pooping a ton of temporary files
into zig-cache/tmp for the time being. I think one of the very next
improvements to the build system should be moving the compiler's cache
system to the standard library and using it in the build system. I had a
look at the dependencies and it is already pretty untangled.
2023-02-13 06:42:25 -07:00
Andrew Kelley
90e48d4b34 std.Build: avoid use of catch unreachable
Usage of `catch unreachable` in build scripts is completely harmless
because build scripts are always run in Debug mode, however, it sets a
poor example for beginners to learn from.
2023-01-31 15:09:35 -07:00
Andrew Kelley
16cdd1297e rename std.Build.LibExeObjStep to std.Build.CompileStep
This matches the nomenclature internally: a Compilation is the main type
that represents a single invokation of the compiler.
2023-01-31 15:09:35 -07:00
Andrew Kelley
36e2d992dd combine std.build and std.build.Builder into std.Build
I've been wanting to do this for along time.
2023-01-31 15:09:35 -07:00