12 Commits

Author SHA1 Message Date
Andrew Kelley
0b8736f5ed re-enable CLI tests
CLI tests are now ported over to the new std.Build API and thus work
properly with concurrency.

 * add `std.Build.addCheckFile` for creating a
   `std.Build.CheckFileStep`.
 * add `std.Build.makeTempPath`. This function is intended to be called
   in the `configure` phase only. It returns an absolute directory path,
   which is potentially going to be a source of API breakage in the
   future, so keep that in mind when using this function.
 * add `std.Build.CheckFileStep.setName`.
 * `std.Build.CheckFileStep`: better error message when reading the
   input file fails.
 * `std.Build.RunStep`: add a `has_side_effects` flag for when you need
   to override the autodetection.
 * `std.Build.RunStep`: add the ability to obtain a FileSource for the
   directory that contains the written files.
 * `std.Build.WriteFileStep`: add a way to write bytes to an arbitrary
   path - absolute or relative to the package root. Be careful with this
   because it updates source files. This should not be used as part of
   the normal build process, but as a utility occasionally run by a
   developer with intent to modify source files and then commit those
   changes to version control. A file added this way is not available
   with `getFileSource`.
2023-03-15 10:48:13 -07:00
Andrew Kelley
a24af8e400 re-integrate stack trace tests with the new std.Build API
* RunStep: ability to set stdin
 * RunStep: ability to capture stdout and stderr as a FileSource
 * RunStep: add setName method
 * RunStep: hash the stdio checks
2023-03-15 10:48:13 -07:00
Andrew Kelley
405bf1b091 std.Build.ConfigHeaderStep: integrate with the cache system 2023-03-15 10:48:13 -07:00
Andrew Kelley
dcec4d55e3 eliminate stderr usage in std.Build make() functions
* Eliminate all uses of `std.debug.print` in make() functions, instead
  properly using the step failure reporting mechanism.
* Introduce the concept of skipped build steps. These do not cause the
  build to fail, and they do allow their dependants to run.
* RunStep gains a new flag, `skip_foreign_checks` which causes the
  RunStep to be skipped if stdio mode is `check` and the binary cannot
  be executed due to it being a foreign executable.
  - RunStep is improved to automatically use known interpreters to
    execute binaries if possible (integrating with flags such as
    -fqemu and -fwasmtime). It only does this after attempting a native
    execution and receiving a "exec file format" error.
  - Update RunStep to use an ArrayList for the checks rather than this
    ad-hoc reallocation/copying mechanism.
  - `expectStdOutEqual` now also implicitly adds an exit_code==0 check
    if there is not already an expected termination. This matches
    previously expected behavior from older API and can be overridden by
    directly setting the checks array.
* Add `dest_sub_path` to `InstallArtifactStep` which allows choosing an
  arbitrary subdirectory relative to the prefix, as well as overriding
  the basename.
  - Delete the custom InstallWithRename step that I found deep in the
    test/ directory.
* WriteFileStep will now update its step display name after the first
  file is added.
* Add missing stdout checks to various standalone test case build
  scripts.
2023-03-15 10:48:13 -07:00
Andrew Kelley
58edefc6d1 zig build: many enhancements related to parallel building
Rework std.Build.Step to have an `owner: *Build` field. This
simplified the implementation of installation steps, as well as provided
some much-needed common API for the new parallelized build system.

--verbose is now defined very concretely: it prints to stderr just
before spawning a child process.

Child process execution is updated to conform to the new
parallel-friendly make() function semantics.

DRY up the failWithCacheError handling code. It now integrates properly
with the step graph instead of incorrectly dumping to stderr and calling
process exit.

In the main CLI, fix `zig fmt` crash when there are no errors and stdin
is used.

Deleted steps:
 * EmulatableRunStep - this entire thing can be removed in favor of a
   flag added to std.Build.RunStep called `skip_foreign_checks`.
 * LogStep - this doesn't really fit with a multi-threaded build runner
   and is effectively superseded by the new build summary output.

build runner:
 * add -fsummary and -fno-summary to override the default behavior,
   which is to print a summary if any of the build steps fail.
 * print the dep prefix when emitting error messages for steps.

std.Build.FmtStep:
 * This step now supports exclude paths as well as a check flag.
 * The check flag decides between two modes, modify mode, and check
   mode. These can be used to update source files in place, or to fail
   the build, respectively.

Zig's own build.zig:
 * The `test-fmt` step will do all the `zig fmt` checking that we expect
   to be done. Since the `test` step depends on this one, we can simply
   remove the explicit call to `zig fmt` in the CI.
 * The new `fmt` step will actually perform `zig fmt` and update source
   files in place.

std.Build.RunStep:
 * expose max_stdio_size is a field (previously an unchangeable
   hard-coded value).
 * rework the API. Instead of configuring each stream independently,
   there is a `stdio` field where you can choose between
   `infer_from_args`, `inherit`, or `check`. These determine whether the
   RunStep is considered to have side-effects or not. The previous
   field, `condition` is gone.
 * when stdio mode is set to `check` there is a slice of any number of
   checks to make, which include things like exit code, stderr matching,
   or stdout matching.
 * remove the ill-defined `print` field.
 * when adding an output arg, it takes the opportunity to give itself a
   better name.
 * The flag `skip_foreign_checks` is added. If this is true, a RunStep
   which is configured to check the output of the executed binary will
   not fail the build if the binary cannot be executed due to being for
   a foreign binary to the host system which is running the build graph.
   Command-line arguments such as -fqemu and -fwasmtime may affect
   whether a binary is detected as foreign, as well as system
   configuration such as Rosetta (macOS) and binfmt_misc (Linux).
   - This makes EmulatableRunStep no longer needed.
 * Fix the child process handling to properly integrate with the new
   bulid API and to avoid deadlocks in stdout/stderr streams by polling
   if necessary.

std.Build.RemoveDirStep now uses the open build_root directory handle
instead of an absolute path.
2023-03-15 10:48:13 -07:00
Andrew Kelley
0e078790fe multiplex compiler progress messages into the build runner 2023-03-15 10:48:13 -07:00
Andrew Kelley
02381c0372 std.Build: improve debugging of misconfigured steps
* Step.init() now takes an options struct
 * Step.init() now captures a small stack trace and stores it in the
   Step so that it can be accessed when printing user-friendly debugging
   information, including the lines of code that created the step in
   question.
2023-03-15 10:48:12 -07:00
Andrew Kelley
d3c9bfada6 std.Build.WriteFileStep: integrate with cache system
And additionally support writing files to source files. This means a
custom build step in zig's own build.zig is no longer needed for copying
zig.h because it is handled by WriteFileStep.
2023-02-23 01:08:23 -05:00
Andrew Kelley
d97042ad2e std.Build: start using the cache system with RunStep
* Use std.Build.Cache.Directory instead of a string for storing the
   cache roots and build roots.
 * Set up a std.Build.Cache in build_runner.zig and use it in
   std.Build.RunStep for avoiding redundant work.
2023-02-13 06:42:26 -07:00
Andrew Kelley
b29e3fa2cd std.Build: enhancements to ConfigHeaderStep
Breaking API change to std.Build.addConfigHeader. It now uses an options
struct.

Introduce std.Build.CompileStep.installConfigHeader which also accepts
an options struct. This is used to add a generated config file into the
set of installed header files for a particular compilation artifact.

std.Build.ConfigHeaderStep now additionally supports a "blank" style
where a header is generated from scratch. It no longer exposes
`output_dir`. Instead it exposes a FileSource via `output_file`.
It now additionally accepts an `include_path` option which affects the
include path of CompileStep when using the `#include` directive, as well
as affecting the default installation subdirectory for header
installation purposes.

The hash used for the directory to store the generated config file now
includes the contents of the generated file. This fixes possible race
conditions when generating multiple header files simultaneously.

The values hash table is now an array hash map, to preserve order for
the "blank" use case.

I also took the opportunity to remove output_dir from TranslateCStep and
WriteFileStep. This is technically a breaking change, but it was always
naughty to access these fields.
2023-02-05 06:26:30 -07:00
Andrew Kelley
90e48d4b34 std.Build: avoid use of catch unreachable
Usage of `catch unreachable` in build scripts is completely harmless
because build scripts are always run in Debug mode, however, it sets a
poor example for beginners to learn from.
2023-01-31 15:09:35 -07:00
Andrew Kelley
36e2d992dd combine std.build and std.build.Builder into std.Build
I've been wanting to do this for along time.
2023-01-31 15:09:35 -07:00