79 Commits

Author SHA1 Message Date
Andrew Kelley
1bcf674a43 build runner: make step_stack a map to remove redundant steps
This prevents compilation errors from being emitted twice.
2023-03-15 10:48:13 -07:00
Andrew Kelley
01299864e2 build runner: fix unicode tree printing 2023-03-15 10:48:13 -07:00
Andrew Kelley
0e078790fe multiplex compiler progress messages into the build runner 2023-03-15 10:48:13 -07:00
Andrew Kelley
81376e7205 fix UAF in build runner 2023-03-15 10:48:13 -07:00
Andrew Kelley
7da34bd9e8 build runner: print a fancy tree with build results on failure 2023-03-15 10:48:13 -07:00
Andrew Kelley
79440d2b47 std.Build.CompileStep: obtain the build output dir from protocol
Now building successfully works again.
2023-03-15 10:48:13 -07:00
Andrew Kelley
7a3dabdc47 build runner: account for debug builds in cleanExit
build runner is always compiled in debug mode, so the switch on
optimization here was silly.
2023-03-15 10:48:13 -07:00
Andrew Kelley
986a30e373 integrate the build runner and the compiler server
The compiler now provides a server protocol for an interactive session
with another process. The build runner uses this protocol to communicate
compilation errors semantically from zig compiler subprocesses to the
build runner.

The protocol is exposed via stdin/stdout, or on a network socket,
depending on whether the CLI flag `--listen=-` or e.g.
`--listen=127.0.0.1:1337` is used.

Additionally:

 * add the zig version string to the build runner cache prefix

 * remove --prominent-compile-errors CLI flag because it no longer does
   anything. Compilation errors are now unconditionally displayed at the
   bottom of the build summary output when using the terminal-based
   build runner.

 * Remove the color field from std.Build. The build steps are no longer
   supposed to interact with stderr directly. Instead they communicate
   semantically back to the build runner, which has its own logic about
   TTY configuration.

 * Use the cleanExit() pattern in the build runner.

 * Build steps can now use error.MakeFailed when they have already
   properly reported an error, or they can fail with any other error
   code in which case the build runner will create a simple message
   based on this error code.
2023-03-15 10:48:13 -07:00
Andrew Kelley
cb09470063 zig build: add a -j<N> option for limiting concurrency 2023-03-15 10:48:12 -07:00
Andrew Kelley
96d798db8b update to new for loop syntax 2023-03-15 10:48:12 -07:00
Andrew Kelley
26486c7f23 build runner: show stderr even on successful steps run 2023-03-15 10:48:12 -07:00
Andrew Kelley
c5edd8b7f8 std.Build: better handling of stderr of child processes
With this commit, the build runner now communicates progress towards
completion of the step graph to the terminal, while also printing the
stderr of child processes as soon as possible, without clobbering each
other, and without clobbering the CLI progress output.
2023-03-15 10:48:12 -07:00
Andrew Kelley
9580fbcf35 build system: capture stderr and report it later
Instead of dumping directly to stderr. This prevents processes running
simultaneously from racing their stderr against each other.

For now it only reports at the end, but an improvement would be to
report as soon as a failed step occurs.
2023-03-15 10:48:12 -07:00
Andrew Kelley
1fa1484288 build runner: proper threaded dependency management
After sorting the step stack so that dependencies can be popped before
their dependants are popped, there is still a situation left to handle
correctly:

Example:

A depends on:
  B
  C
D depends on:
  E
  F

They will be ordered like this:

A B C D E F

If there are 6+ cores, then all of them will be evaluated at once,
incorrectly evaluating A and D before their dependencies.

Starting evaluation of F and then E is correct, but waiting until they
are done is not correct because it should start working on B and C as
well.

This commit solves the problem by computing dependants in the dependency
loop checking logic, and then having workers queue up their dependants
when they finish their own work.
2023-03-15 10:48:12 -07:00
Andrew Kelley
cff86cf7a1 build_runner now executes the step graph in parallel 2023-03-15 10:48:12 -07:00
Andrew Kelley
0b744d7d67 build runner: untangle dependency loop checking from making 2023-03-15 10:48:12 -07:00
Andrew Kelley
c6a895f667 extract some logic from std.Build to build_runner.zig 2023-03-15 10:48:12 -07:00
Auguste Rame
25b83188d0
Add --build-runner zig build option (#14742) 2023-03-01 13:20:01 -05:00
Andrew Kelley
3f8f63b132 std.Build: make cache_root and global_cache_root relative to cwd
This makes it so that when there is a tree of std.Build objects, only
one zig-cache is used (the top-level application) instead of polluting
package directories with zig-cache folders.
2023-02-13 06:42:26 -07:00
Andrew Kelley
d97042ad2e std.Build: start using the cache system with RunStep
* Use std.Build.Cache.Directory instead of a string for storing the
   cache roots and build roots.
 * Set up a std.Build.Cache in build_runner.zig and use it in
   std.Build.RunStep for avoiding redundant work.
2023-02-13 06:42:26 -07:00
dweiller
ca0528209f zig build: add -l, --list-steps option to build_runner.zig 2023-02-11 20:50:57 +02:00
Andrew Kelley
5129fae4e8 std.Build: accept host Target in create()
And only detect native target in LibExeObjStep once, in create().
2023-01-31 15:09:35 -07:00
Andrew Kelley
36e2d992dd combine std.build and std.build.Builder into std.Build
I've been wanting to do this for along time.
2023-01-31 15:09:35 -07:00
Andrew Kelley
d703270545 zig build system: add LibExeObjStep.installLibraryHeaders
This function is needed when a library exposes one of its own library
dependency's headers as part of its own public API.

Also, improve error message when a file system error occurs during
install file step.
2023-01-25 02:45:57 -05:00
Andrew Kelley
cfcf9771c1 zig build: support dependencies
The `zig build` command now makes `@import("@dependencies")` available
to the build runner package. It contains all the dependencies in a
generated file that looks something like this:

```zig
pub const imports = struct {
    pub const foo = @import("foo");
    pub const @"bar.baz" = @import("bar.baz");
};
pub const build_root = struct {
    pub const foo = "<path>";
    pub const @"bar.baz" = "<path>";
};
```

The build runner exports this import so that `std.build.Builder` can
access it. `std.build.Builder` uses it to implement the new `dependency`
function which can be used like so:

```zig
const libz_dep = b.dependency("libz", .{});
const libmp3lame_dep = b.dependency("libmp3lame", .{});
// ...
lib.linkLibrary(libz_dep.artifact("z"));
lib.linkLibrary(libmp3lame_dep.artifact("mp3lame"));
```

The `dependency` function calls the build.zig file of the dependency as
a child Builder, and then can be ransacked for its build steps via the
`artifact` function.

This commit also renames `dependency.id` to `dependency.name` in the
`build.zig.ini` file.
2023-01-11 15:39:49 -08:00
Andrew Kelley
28514476ef remove -fstage1 option
After this commit, the self-hosted compiler does not offer the option to
use stage1 as a backend anymore.
2022-12-06 12:15:04 -07:00
Veikka Tuominen
e4fd9acc2a CLI: allow using --debug-compile-errors with zig build 2022-11-30 19:14:04 +02:00
Veikka Tuominen
31daea74d2 stage2: implement referenced by trace for error messages
Closes #7668
Closes #12141
2022-09-15 00:50:18 +03:00
Andrew Kelley
ec95e00e28 flatten lib/std/special and improve "pkg inside another" logic
stage2: change logic for detecting whether the main package is inside
the std package. Previously it relied on realpath() which is not portable.
This uses resolve() which is how imports already work.

 * stage2: fix cleanup bug when creating Module
 * flatten lib/std/special/* to lib/*
   - this was motivated by making main_pkg_is_inside_std false for
     compiler_rt & friends.
 * rename "mini libc" to "universal libc"
2022-05-06 22:41:00 -07:00