433 Commits

Author SHA1 Message Date
Andrew Kelley
a3c74aca99 add --debug-rt CLI arg to the compiler + bonus edits
The flag makes compiler_rt and libfuzzer be in debug mode.

Also:
* fuzzer: override debug logs and disable debug logs for frequently
  called functions
* std.Build.Fuzz: fix bug of rerunning the old unit test binary
* report errors from rebuilding the unit tests better
* link.Elf: additionally add tsan lib and fuzzer lib to the hash
2024-07-25 18:52:21 -07:00
Andrew Kelley
b501adccbe std.Build.Fuzz: fix progress node hierarchy 2024-07-25 18:52:21 -07:00
Andrew Kelley
bce3b1efb0 build runner sends a start_fuzzing message to test runner 2024-07-25 18:52:21 -07:00
Andrew Kelley
711ed56ce3 build runner: extract logic to std.Build.Fuzz 2024-07-25 18:52:21 -07:00
Andrew Kelley
047640383e add --fuzz CLI argument to zig build
This flag makes the build runner rebuild unit tests after the pipeline
finishes, if it finds any unit tests.

I did not make this integrate with file system watching yet.

The test runner is updated to detect which tests are fuzz tests.

Run step is updated to track which test indexes are fuzz tests.
2024-07-25 18:52:20 -07:00
Andrew Kelley
94f4f9c4ef add sub-compilation cache inputs to parents in whole mode
closes #20782
2024-07-24 19:40:54 -07:00
Andrew Kelley
26d2a7960e default "trace pc guard" coverage off
* Add -f(no-)sanitize-coverage-trace-pc-guard CLI flag which defaults to
  off. This value lowers to TracePCGuard = true (LLVM backend) and -Xclang
  -fsanitize-coverage-trace-pc-guard. These settings are not
  automatically included with -ffuzz.
* Add `Build.Step.Compile` flag for sanitize_coverage_trace_pc_guard
  with appropriate documentation.
* Add `zig cc` integration for the respective flags.
* Avoid crashing in ELF linker code when -ffuzz -femit-llvm-ir used
  together.
2024-07-23 17:30:53 -07:00
Alex Rønne Petersen
be1e1fa180 std.Build.Step.Run: Fix invocation syntax for Wasmtime 14+.
https://github.com/bytecodealliance/wasmtime/issues/7384
2024-07-23 12:39:40 -07:00
Andrew Kelley
33c7984183 add std.testing.random_seed
closes #17609
2024-07-23 11:43:12 -07:00
Andrew Kelley
1741b821c3 add the build system API for enabling fuzzing 2024-07-22 13:07:02 -07:00
Eric Joldasov
d263f1ec0e zig build: respect PKG_CONFIG environment variable
`PKG_CONFIG` environment variable is used to override path to
pkg-config executable, for example when it's name is prepended by
target triple for cross-compilation purposes:

```
PKG_CONFIG=/usr/bin/aarch64-unknown-linux-gnu-pkgconf zig build
```

Signed-off-by: Eric Joldasov <bratishkaerik@landless-city.net>
2024-07-21 02:32:33 -07:00
Andrew Kelley
eb4028bf30 add std.fmt.hex
converts an unsigned integer into an array
2024-07-20 01:06:29 -07:00
Andrew Kelley
445bd7a06f build runner: update watch caption to include subprocesses 2024-07-14 22:27:51 -07:00
Andrew Kelley
987f63208e build runner: handle compiler subprocess failures gracefully
Compilation errors now report a failure on rebuilds triggered by file
system watches.

Compiler crashes now report failure correctly on rebuilds triggered by
file system watches.

The compiler subprocess is restarted if a broken pipe is encountered on
a rebuild.
2024-07-14 22:17:29 -07:00
Andrew Kelley
f6c1b71c22 build system: update std.Progress.Node for long-lived children 2024-07-14 21:38:55 -07:00
Andrew Kelley
716b128a24 frontend: add -fincremental, -fno-incremental flag
Remove --debug-incremental

This flag is also added to the build system. Importantly, this tells
Compile step whether or not to keep the compiler running between
rebuilds. It defaults off because it is currently crashing
zirUpdateRefs.
2024-07-14 21:18:09 -07:00
Andrew Kelley
abf8955951 make zig compiler processes live across rebuilds
Changes the `make` function signature to take an options struct, which
additionally includes `watch: bool`. I intentionally am not exposing
this information to configure phase logic.

Also adds global zig cache to the compiler cache prefixes.

Closes #20600
2024-07-14 19:51:16 -07:00
kcbanner
373e53d7c5 Compile: Pass the default --zig-lib-dir along to child processes
main: print the self_exe_path when `findZigLibDirFromSelfExe` fails in all cases
2024-07-13 19:04:55 -04:00
Andrew Kelley
1d20ff11d7
Merge pull request #20580 from ziglang/watch
introduce file system watching features to the zig build system
2024-07-12 16:56:17 -07:00
Andrew Kelley
5efcc2e9e7 build runner: refactor fs watch logic for OS abstraction
Makes the build runner compile successfully for non-linux targets;
printing an error if you ask for --watch rather than making build
scripts fail to compile.
2024-07-12 14:20:20 -07:00
Andrew Kelley
4f9a8b6843 update build system unit test
need to add another field to initialize now
2024-07-12 11:00:52 -07:00
xtex
0d79aa0176
std.Build.Step.Run: support prefixed artifact args
Just like how addPrefixedFileArg and addPrefixedDirectoryArg works, we can make it for artifacts.

Signed-off-by: Bingwu Zhang <xtexchooser@duck.com>
2024-07-12 11:06:06 +02:00
Andrew Kelley
5a34e6c3e6 frontend: add file system inputs for incremental cache mode
These are also used for whole cache mode in the case that any compile
errors are emitted.
2024-07-12 00:15:11 -07:00
Andrew Kelley
a3c20dffae integrate Compile steps with file watching
Updates the build runner to unconditionally require a zig lib directory
parameter. This parameter is needed in order to correctly understand
file system inputs from zig compiler subprocesses, since they will refer
to "the zig lib directory", and the build runner needs to place file
system watches on directories in there.

The build runner's fanotify file watching implementation now accounts
for when two or more Cache.Path instances compare unequal but ultimately
refer to the same directory in the file system.

Breaking change: std.Build no longer has a zig_lib_dir field. Instead,
there is the Graph zig_lib_directory field, and individual Compile steps
can still have their zig lib directories overridden. I think this is
unlikely to break anyone's build in practice.

The compiler now sends a "file_system_inputs" message to the build
runner which shares the full set of files that were added to the cache
system with the build system, so that the build runner can watch
properly and redo the Compile step. This is implemented for whole cache
mode but not yet for incremental cache mode.
2024-07-12 00:14:08 -07:00
Andrew Kelley
fd4d366009 std.Build.Cache.Path: fix the format method
This function previously wrote a trailing directory separator, but
that's not correct if the path refers to a file.
2024-07-12 00:14:08 -07:00
Andrew Kelley
818f9cb5a0 std.Build.Step.ObjCopy: remove random bytes from cache hash
The cache hash already has the zig version in there, so it's not really
needed.
2024-07-12 00:14:08 -07:00
Andrew Kelley
dad07fb6f3 std.Build.Cache.Path: fix hash impl on windows 2024-07-12 00:14:08 -07:00
Andrew Kelley
2e42969786 std.Build.Step.Run: integrate with --watch 2024-07-12 00:14:08 -07:00
Andrew Kelley
6fcb1897d2 std.Build.Step.WriteFile: remove random bytes from cache hash
The cache hash already has the zig version in there, so it's not really
needed.
2024-07-12 00:14:08 -07:00
Andrew Kelley
a966eee090 std.Build.Step.WriteFile: fix handling of directories
and add file system watching integration.

`addDirectoryWatchInput` now returns a `bool` which helps remind the
caller to
1. call addDirectoryWatchInputFromPath on any derived paths
2. but only if the dependency is not already captured by a step
   dependency edge.

The make function now recursively walks all directories and adds the
found files to the cache hash rather than incorrectly only adding the
directory name to the cache hash.

closes #20571
2024-07-12 00:14:08 -07:00
Andrew Kelley
f2856403c6 introduce std.Build.Cache.Manifest.addFilePath
and deprecate `addFile`. Part of an effort to move towards using
`std.Build.Cache.Path` abstraction in more places, which makes it easier
to avoid absolute paths and path resolution.
2024-07-12 00:14:08 -07:00
Andrew Kelley
5c3fae3a32 td.Build.Step.InstallDir: leave hint for wrong cached status
Since I spent a couple minutes debugging this, hopefully this saves
someone some future trouble doing the same.
2024-07-12 00:14:08 -07:00
Andrew Kelley
61d2234743 std.Build.Watch: add ONDIR to fanotify event mask
This makes mkdir/rmdir events show up.
2024-07-12 00:14:08 -07:00
Andrew Kelley
2ebf021061 build runner: don't pass a dirfd + null to fanotify_mark
Otherwise it reports EBADF.
2024-07-12 00:14:08 -07:00
Andrew Kelley
7bccef3e4e std.Build.Watch: introduce special file "." to watch entire dir
And use it to implement InstallDir Step watch integration.

I'm not seeing any events triggered when I run `mkdir` in the watched
directory, however, and I have not yet figured out why.
2024-07-12 00:14:08 -07:00
Andrew Kelley
dcbb3aa1f3 std.Build.Cache.Path: fix format function for absolute paths 2024-07-12 00:14:08 -07:00
Andrew Kelley
d1c14f2f52 std.Build.Step.WriteFile: extract UpdateSourceFiles
This has been planned for quite some time; this commit finally does it.

Also implements file system watching integration in the make()
implementation for UpdateSourceFiles and fixes the reporting of step
caching for both.

WriteFile does not yet have file system watching integration.
2024-07-12 00:14:08 -07:00
Andrew Kelley
0cc492a272 make more build steps integrate with the watch system 2024-07-12 00:14:08 -07:00
Andrew Kelley
956f1ebc70 std.Build.Watch: gracefully handle fanotify queue overflow 2024-07-12 00:14:08 -07:00
Andrew Kelley
001ff7b3b2 std.Build.Watch: make dirty steps invalidate each other
and make failed steps always be invalidated
and make steps that don't need to be reevaluated marked as cached
2024-07-12 00:14:07 -07:00
Andrew Kelley
6f89824c22 build system: make debounce interval CLI-configurable 2024-07-12 00:14:07 -07:00
Andrew Kelley
5ee3971b18 proof-of-concept --watch implementation based on fanotify
So far, only implemented for InstallFile steps.

Default debounce interval bumped to 50ms. I think it should be
configurable.

Next I have an idea to simplify the fanotify implementation, but other
OS implementations might want to refer back to this commit before I make
those changes.
2024-07-12 00:14:07 -07:00
Andrew Kelley
bbd90a562e build runner: implement --watch (work-in-progress)
I'm still learning how the fanotify API works but I think after playing
with it in this commit, I finally know how to implement it, at least on
Linux. This commit does not accomplish the goal but I want to take the
code in a different direction and still be able to reference this point
in time by viewing a source control diff.

I think the move is going to be saving the file_handle for the parent
directory, which combined with the dirent names is how we can correlate
the events back to the Step instances that have registered file system
inputs. I predict this to be similar to implementations on other
operating systems.
2024-07-12 00:14:07 -07:00
Andrew Kelley
deea36250f std.Build.Cache.Path: add subPathOpt and TableAdapter
Helpful methods when using one of these structs as a hash table key.
2024-07-12 00:14:07 -07:00
Andrew Kelley
26d506c0f8 std.Build: remove the "push installed file" mechanism
Tracked by #14943
2024-07-12 00:14:07 -07:00
Andrew Kelley
6e025fc2e2 build system: add --watch flag and report source file in InstallFile
This direction is not quite right because it mutates shared state in a
threaded context, so the next commit will need to fix this.
2024-07-12 00:14:07 -07:00
Andrew Kelley
908c2c902a std.Build.Cache.Path: add eql method 2024-07-12 00:14:07 -07:00
Krzysztof Wolicki
0cc42d090f
std.fs.Dir: Rename OpenDirOptions to OpenOptions (#20542)
* std.fs.Dir: Rename OpenDirOptions to OpenOptions
https://ziglang.org/documentation/master/#Avoid-Redundant-Names-in-Fully-Qualified-Namespaces

* std.fs.Dir: Add deprecated alias `OpenDirOptions`
2024-07-09 13:36:38 -07:00
Krzysztof Wolicki
7205756a69 Step.TranslateC: fix defineCMacro 2024-07-07 12:28:28 +02:00
Krzysztof Wolicki
ae919915f6 std.Build.Cache.Path: fix makeOpenPath signature 2024-07-07 12:10:19 +02:00