950 Commits

Author SHA1 Message Date
Andrew Kelley
904fcda736 Compilation: fix -femit-docs 2024-08-07 00:48:32 -07:00
Andrew Kelley
ff503edc04 Compilation: fix not showing sub-errors for autodocs 2024-08-07 00:48:32 -07:00
Alex Rønne Petersen
27507ad971 compiler: Enable -Werror=date-time for C/C++ code in release builds.
We advertise reproducible builds for release modes, so let's help users achieve
that in C/C++ code. Users can still override this manually if they really want.
2024-07-29 11:27:38 -07:00
Alex Rønne Petersen
642cd730c8 link: Accept -Brepro linker option and pass it to LLD.
Enable it by default when building Zig code in release modes.

Contributes to #9432.
2024-07-28 14:37:03 +02:00
Jakub Konka
91c17979f1
Merge pull request #20807 from Rexicon226/riscv
riscv: more backend progress
2024-07-28 11:34:23 +02:00
David Rubin
a7498a6a53
comp: enable compilation of zig_libc 2024-07-26 14:02:48 -07:00
Andrew Kelley
208baa37ca frontend: add missed cache hash on --debug-rt
Makes adding --debug-rt correctly invalidate the cache for compiler_rt
and libfuzzer.
2024-07-26 08:42:40 -07:00
Andrew Kelley
a3c74aca99 add --debug-rt CLI arg to the compiler + bonus edits
The flag makes compiler_rt and libfuzzer be in debug mode.

Also:
* fuzzer: override debug logs and disable debug logs for frequently
  called functions
* std.Build.Fuzz: fix bug of rerunning the old unit test binary
* report errors from rebuilding the unit tests better
* link.Elf: additionally add tsan lib and fuzzer lib to the hash
2024-07-25 18:52:21 -07:00
Andrew Kelley
94f4f9c4ef add sub-compilation cache inputs to parents in whole mode
closes #20782
2024-07-24 19:40:54 -07:00
Andrew Kelley
3844b91db7 Compilation: build compiler_rt and fuzzer in parallel
With the rest of the pipeline.

Tracked by #9188
2024-07-24 18:24:00 -07:00
Andrew Kelley
26d2a7960e default "trace pc guard" coverage off
* Add -f(no-)sanitize-coverage-trace-pc-guard CLI flag which defaults to
  off. This value lowers to TracePCGuard = true (LLVM backend) and -Xclang
  -fsanitize-coverage-trace-pc-guard. These settings are not
  automatically included with -ffuzz.
* Add `Build.Step.Compile` flag for sanitize_coverage_trace_pc_guard
  with appropriate documentation.
* Add `zig cc` integration for the respective flags.
* Avoid crashing in ELF linker code when -ffuzz -femit-llvm-ir used
  together.
2024-07-23 17:30:53 -07:00
Andrew Kelley
33d4742456 LLVM: more fine-grained sancov emit options
Exposes sanitizer coverage flags to the target machine emit function.
Makes it easier to change sancov options without rebuilding the C++
files.

This also enables PCTable = true for sancov which is needed by AFL, and
adds the corresponding Clang flag.
2024-07-23 16:04:58 -07:00
Andrew Kelley
6f3e9939d0
Merge pull request #20725 from ziglang/fuzz
initial support for integrated fuzzing
2024-07-23 11:39:19 -07:00
Andrew Kelley
1cb994899d Compilation: fix regression in addCCArgs
`-fno-sanitize=function` must come after `-fsanitize=undefined` or it
has no effect.
2024-07-22 14:26:17 -07:00
Andrew Kelley
54b7e144b1 initial support for integrated fuzzing
* Add the `-ffuzz` and `-fno-fuzz` CLI arguments.
* Detect fuzz testing flags from zig cc.
* Set the correct clang flags when fuzz testing is requested. It can be
  combined with TSAN and UBSAN.
* Compilation: build fuzzer library when needed which is currently an
  empty zig file.
* Add optforfuzzing to every function in the llvm backend for modules
  that have requested fuzzing.
* In ZigLLVMTargetMachineEmitToFile, add the optimization passes for
  sanitizer coverage.
* std.mem.eql uses a naive implementation optimized for fuzzing when
  builtin.fuzz is true.

Tracked by #20702
2024-07-22 13:07:02 -07:00
Jakub Konka
cba3389d90 macho: redo input file parsing in prep for multithreading 2024-07-22 12:05:56 +02:00
Andrew Kelley
b5f3d12164
Merge pull request #20688 from ziglang/incr-test
introduce a new tool for testing incremental compilation
2024-07-20 13:04:49 -07:00
Jacob Young
2e65244cae dev: fix llvm backend checks 2024-07-20 07:43:40 -04:00
Andrew Kelley
eb4028bf30 add std.fmt.hex
converts an unsigned integer into an array
2024-07-20 01:06:29 -07:00
Jacob Young
4f742c4cfc dev: introduce dev environments that enable compiler feature sets 2024-07-19 22:35:33 -07:00
Andrew Kelley
8cb09db4e3 update some Module references to Zcu instead
I ended up reverting my real change, so here's a consolation prize
instead.
2024-07-15 19:05:55 -07:00
Andrew Kelley
60318a1e39 frontend: move updateZirRefs to be single-threaded
for simplicity's sake. This makes it O(M) instead of O(N*M) where N is
tracked insts and M is number of changed source files.
2024-07-15 18:54:41 -07:00
Andrew Kelley
716b128a24 frontend: add -fincremental, -fno-incremental flag
Remove --debug-incremental

This flag is also added to the build system. Importantly, this tells
Compile step whether or not to keep the compiler running between
rebuilds. It defaults off because it is currently crashing
zirUpdateRefs.
2024-07-14 21:18:09 -07:00
Andrew Kelley
abf8955951 make zig compiler processes live across rebuilds
Changes the `make` function signature to take an options struct, which
additionally includes `watch: bool`. I intentionally am not exposing
this information to configure phase logic.

Also adds global zig cache to the compiler cache prefixes.

Closes #20600
2024-07-14 19:51:16 -07:00
Andrew Kelley
d404d8a363
Merge pull request #20593 from jacobly0/more-races
InternPool: fix more races
2024-07-14 17:32:51 -07:00
Andrew Kelley
d8c1c9ea6e frontend: report correct paths for C objects
oops, the src_path field of CObject is not relative to the module
owner's root directory.
2024-07-13 02:13:03 -07:00
Jacob Young
ad55fb7a20 Compilation: restore saveState 2024-07-13 04:47:38 -04:00
Jacob Young
2ff49751aa Compilation: introduce work stages for better work distribution 2024-07-13 04:47:38 -04:00
Jacob Young
a1053e8e1d InternPool: add and use a mutate mutex for each list
This allows the mutate mutex to only be locked during actual grows,
which are rare. For the lists that didn't previously have a mutex, this
change has little effect since grows are rare and there is zero
contention on a mutex that is only ever locked by one thread.  This
change allows `extra` to be mutated without racing with a grow.
2024-07-13 04:47:38 -04:00
Andrew Kelley
67e3e4989d Compilation: fix rebase conflict 2024-07-12 00:18:29 -07:00
Andrew Kelley
5a34e6c3e6 frontend: add file system inputs for incremental cache mode
These are also used for whole cache mode in the case that any compile
errors are emitted.
2024-07-12 00:15:11 -07:00
Andrew Kelley
a3c20dffae integrate Compile steps with file watching
Updates the build runner to unconditionally require a zig lib directory
parameter. This parameter is needed in order to correctly understand
file system inputs from zig compiler subprocesses, since they will refer
to "the zig lib directory", and the build runner needs to place file
system watches on directories in there.

The build runner's fanotify file watching implementation now accounts
for when two or more Cache.Path instances compare unequal but ultimately
refer to the same directory in the file system.

Breaking change: std.Build no longer has a zig_lib_dir field. Instead,
there is the Graph zig_lib_directory field, and individual Compile steps
can still have their zig lib directories overridden. I think this is
unlikely to break anyone's build in practice.

The compiler now sends a "file_system_inputs" message to the build
runner which shares the full set of files that were added to the cache
system with the build system, so that the build runner can watch
properly and redo the Compile step. This is implemented for whole cache
mode but not yet for incremental cache mode.
2024-07-12 00:14:08 -07:00
Jacob Young
c79d3e4aab Compilation: fix leak 2024-07-10 22:05:52 -04:00
Jacob Young
c2316c5228 InternPool: make global_error_set thread-safe 2024-07-10 21:39:55 -04:00
Jacob Young
afa66fa392 InternPool: make tracked_insts thread-safe 2024-07-10 14:42:50 -04:00
Jacob Young
f290b54f89 InternPool: make files more thread-safe 2024-07-10 11:20:08 -04:00
Jacob Young
3d2dfbe828 InternPool: add FileIndex to *File mapping 2024-07-10 11:20:08 -04:00
Jacob Young
65ced4a334 Compilation: put supported codegen backends on a separate thread
(There are no supported backends.)
2024-07-08 11:00:38 -04:00
Jacob Young
bdae01ab04 InternPool: implement and use thread-safe list for extra and limbs 2024-07-07 22:59:52 -04:00
Jacob Young
92ddb959a7 InternPool: implement and use thread-safe list for items 2024-07-07 22:59:52 -04:00
Jacob Young
8293ff94cf InternPool: implement and use thread-safe list for strings 2024-07-07 22:59:52 -04:00
Jacob Young
cda716ecc4 InternPool: implement thread-safe hash map 2024-07-07 22:59:52 -04:00
Jacob Young
ca02266157 Zcu: pass PerThread to intern pool string functions 2024-07-07 22:59:52 -04:00
Jacob Young
525f341f33 Zcu: introduce PerThread and pass to all the functions 2024-07-07 22:59:52 -04:00
Andrew Kelley
74346b0f79 frontend: TrackedInst stores FileIndex instead of path digest
The purpose of using path digest was to reference a file in a
serializable manner. Now that there is a stable index associated with
files, it is a superior way to accomplish that goal, since removes one
layer of indirection, and makes TrackedInst 8 bytes instead of 20.

The saved Zig Compiler State file for "hello world" goes from 1.3M to
1.2M with this change.
2024-07-04 17:56:01 -07:00
Andrew Kelley
30ec43a6c7 Zcu: extract permanent state from File
Primarily, this commit removes 2 fields from File, relying on the data
being stored in the `files` field, with the key as the path digest, and
the value as the struct decl corresponding to the File. This table is
serialized into the compiler state that survives between incremental
updates.

Meanwhile, the File struct remains ephemeral data that can be
reconstructed the first time it is needed by the compiler process, as
well as operated on by independent worker threads.

A key outcome of this commit is that there is now a stable index that
can be used to refer to a File. This will be needed when serializing
error messages to survive incremental compilation updates.
2024-07-04 17:51:35 -07:00
Andrew Kelley
29512f0edd Compilation: don't give len=0 bufs to pwritev
The OS returns EFAULT for undefined pointers, even when len=0.
2024-07-04 17:12:45 -07:00
Andrew Kelley
790b8428a2
Merge pull request #20494 from mlugg/the-great-decl-split
refactors ad infinitum
2024-07-04 20:12:05 -04:00
Jakub Konka
d2cace58bd Compilation: rename tsan_static_lib to tsan_lib 2024-07-04 22:09:57 +02:00
Jacob Young
00da182e68
cbe: fix for export changes 2024-07-04 21:01:42 +01:00