This commit introduces `--debug-incremental` so that we can start
playing around with incremental compilation while it is still being
developed, and before it is enabled by default.
Currently it saves InternPool data, and has TODO comments for the
remaining things. Deserialization is not implemented yet, which will
require some post-processing such as to build a string map out of
null-terminated string table bytes.
The saved compiler state is stored in a file called <root-name>.zcs
alongside <root-name>.o, <root-name>.pdb, <root-name>.exe, etc. In case
of using the zig build system, these files are all in a zig-cache
directory.
For the self-hosted compiler, here is one data point on the performance
penalty of saving this data:
```
Benchmark 1 (3 runs): zig build-exe ...
measurement mean ± σ min … max outliers delta
wall_time 51.1s ± 354ms 50.7s … 51.4s 0 ( 0%) 0%
peak_rss 3.91GB ± 354KB 3.91GB … 3.91GB 0 ( 0%) 0%
cpu_cycles 212G ± 3.17G 210G … 216G 0 ( 0%) 0%
instructions 274G ± 57.5M 274G … 275G 0 ( 0%) 0%
cache_references 13.1G ± 97.6M 13.0G … 13.2G 0 ( 0%) 0%
cache_misses 1.12G ± 24.6M 1.10G … 1.15G 0 ( 0%) 0%
branch_misses 1.53G ± 1.46M 1.53G … 1.53G 0 ( 0%) 0%
Benchmark 2 (3 runs): zig build-exe ... --debug-incremental
measurement mean ± σ min … max outliers delta
wall_time 51.8s ± 271ms 51.5s … 52.1s 0 ( 0%) + 1.3% ± 1.4%
peak_rss 3.91GB ± 317KB 3.91GB … 3.91GB 0 ( 0%) - 0.0% ± 0.0%
cpu_cycles 213G ± 398M 212G … 213G 0 ( 0%) + 0.3% ± 2.4%
instructions 275G ± 79.1M 275G … 275G 0 ( 0%) + 0.1% ± 0.1%
cache_references 13.1G ± 26.9M 13.0G … 13.1G 0 ( 0%) - 0.1% ± 1.2%
cache_misses 1.12G ± 5.66M 1.11G … 1.12G 0 ( 0%) - 0.6% ± 3.6%
branch_misses 1.53G ± 1.75M 1.53G … 1.54G 0 ( 0%) + 0.2% ± 0.2%
```
At the end of each compilation with `--debug-incremental`, we end up
with a 43 MiB `zig.zcs` file that contains all of the InternPool data
serialized.
Of course, it will necessarily be more expensive to save the state than
to not save the state. However, this data point shows just how cheap the
save state operation is, with all of the groundwork laid for using a
serialization-friendly in-memory data layout.
When compiling for *-windows-msvc, find the native libc_installation and
add the lib dirs to lib_dirs, so that system libs can be found.
Previously, `version` and `ole32` were detected via the mingw.libExists logic,
even on .msvc, which was a false positive. This detection logic for mingw doesn't
find uuid.lib, which was the failure that triggered this bugfix.
Only build the issue_5825 test if the native target is x86_64-windows-msvc,
since it requires the .msvc abi.
The include directories used when preprocessing .rc files are now separate from the target, and by default will use the system MSVC include paths if the MSVC + Windows SDK are present, otherwise it will fall back to the MinGW includes distributed with Zig. This default behavior can be overridden by the `-rcincludes` option (possible values: any (the default), msvc, gnu, or none).
This behavior is useful because Windows resource files may `#include` files that only exist with in the MSVC include dirs (e.g. in `<MSVC install directory>/atlmfc/include` which can contain other .rc files, images, icons, cursors, etc). So, by defaulting to the `any` behavior (MSVC if present, MinGW fallback), users will by default get behavior that is most-likely-to-work.
It also should be okay that the include directories used when compiling .rc files differ from the include directories used when compiling the main binary, since the .res format is not dependent on anything ABI-related. The only relevant differences would be things like `#define` constants being different values in the MinGW headers vs the MSVC headers, but any such differences would likely be a MinGW bug.
The new `@depedencies` module contains generated code like the
following (where strings like "abc123" represent hashes):
```zig
pub const root_deps = [_]struct { []const u8, []const u8 }{
.{ "foo", "abc123" },
};
pub const packages = struct {
pub const abc123 = struct {
pub const build_root = "/home/mlugg/.cache/zig/blah/abc123";
pub const build_zig = @import("abc123");
pub const deps = [_]struct { []const u8, []const u8 }{
.{ "bar", "abc123" },
.{ "name", "ghi789" },
};
};
};
```
Each package contains a build root string, the build.zig import, and a
mapping from dependency names to package hashes. There is also such a
mapping for the root package dependencies.
In theory, we could now remove the `dep_prefix` field from `std.Build`,
since its main purpose is now handled differently. I believe this is a
desirable goal, as it doesn't really make sense to assign a single FQN
to any package (because it may appear in many different places in the
package hierarchy). This commit does not remove that field, as it's used
non-trivially in a few places in the build runner and compiler tests:
this will be a future enhancement.
Resolves: #16354Resolves: #17135
Regressed in 2006add8496c47804ee3b6c562f420871cb4ea0a.
References to native_darwin_sdk are no longer kept in the frontend.
Instead the darwin SDK is detected as part of NativePaths and as part of
LibCInstallation.
The `null` value here was missed in
0a4d4eb252a73555a568a532902951a13284d2ef. I hope it is the cause of the
CI failures on Windows.
The fact that libc++ depends on libc is not important for the CLI and
Compilation.create already handles that logic.
chicken out and make -l match the status quo behavior, where it looks
for dynamic libraries and then falls back to static libraries.
library resolution is still done in the CLI now though, and these
options are added:
-search_static_first Search for static libs in all library search
paths, then dynamic libs.
-search_dylibs_only Only search for dynamic libs.
-search_static_only Only search for static libs.
this matches the already existing options below:
-search_paths_first For each library search path, check for dynamic
lib then static lib before proceeding to next path.
-search_dylibs_first Search for dynamic libs in all library search
So, it is still possible to get the strict behavior by passing
`-search_dylibs_only` or `-search_static_only`.
This commit also makes -dynamic and -static affect the preferred link
mode and search strategy.
First, system_libs are collected into a list. This is the same as
before.
Next, system_libs are filtered into external_system_libs, which is the
same list but without any libc, compiler_rt, etc.
At this point, if there are any external system libs, native library
directory paths are detected and added to lib_dirs.
Finally, extern_system_libs are filtered into resolved_system_libs,
which has full paths to all of the libraries. This is the list passed
into Compilation.
This makes the required changes noted by @ifreund in the code review for
this branch.
search_strategy is no longer passed to Compilation at all; instead it is
used in the CLI code only.
When using Zig CLI mode, `-l` no longer has the ability to link
statically; use positional arguments for this.
The CLI has a small abstraction around library resolution handling which
is used to remove some code duplication regarding static libraries, as
well as handle the difference between zig cc CLI mode and zig CLI mode.
Thanks to this, system libraries are now included in the cache hash, and
thus changes to them will correctly cause cache misses.
In the future, lib_dirs should no longer be passed to Compilation at
all, because it is a frontend-only concept.
Previously, -search_paths_first and -search_dylibs_first were
Darwin-only arguments; they now work the same for all targets. Same
thing with --sysroot.
Improved the error reporting for failure to find a system library. An
example error now looks like this:
```
$ zig build-exe test.zig -lfoo -L. -L/a -target x86_64-macos --sysroot /home/andy/local
error: unable to find Dynamic system library 'foo' using strategy 'no_fallback'. search paths:
./libfoo.tbd
./libfoo.dylib
./libfoo.so
/home/andy/local/a/libfoo.tbd
/home/andy/local/a/libfoo.dylib
/home/andy/local/a/libfoo.so
/a/libfoo.tbd
/a/libfoo.dylib
/a/libfoo.so
```
closes#14963
This mirrors the behavior of the progress indicator for the actual
compilation. Not doing this was causing sporadic CI failures due to the
(non-existent) fetches taking long enough to appear in stderr.
* build.zig: introduce `-Dflat` option which makes the installation
match what we want to ship for our download tarballs. This allows
deleting a bunch of shell script logic from the CI.
- for example it puts the executable directly in prefix/zig rather
than prefix/bin/zig and it additionally includes prefix/LICENSE.
* build.zig: by default also install std lib documentation to doc/std/
- this can be disabled by `-Dno-autodocs` similar to how there is
already `-Dno-langref`.
* build.zig: add `std-docs` and `langref` steps which build and install
the std lib autodocs and langref to prefix/doc/std and
prefix/doc/langref.html, respectively.
* std.Build: implement proper handling of `-femit-docs` using the
LazyPath system. This is a breaking change.
- this is a partial implementation of #16351
* frontend: fixed the handling of Autodocs with regards to caching and
putting the artifacts in the proper location to integrate with the
build system.
- closes#15864
* CI: delete the logic for autodocs since it is now handled by build.zig
and is enabled by default.
- in the future we should strive to have nearly all the CI shell
script logic deleted in favor of `zig build` commands.
* CI: pass `-DZIG_NO_LIB=ON`/`-Dno-lib` except for the one command where
we want to actually generate the langref and autodocs. Generating the
langref takes 14 minutes right now (why?!) so we don't want to do that
more times than necessary.
* Autodoc: fixed use of a global variable. It works fine as a local
variable instead.
- note that in the future we will want to make Autodoc run
simultaneously using the job system, but for now the principle of
YAGNI dictates that we don't have an init()/deinit() API and instead
simply call the function that does the things.
* Autodoc: only do it when there are no compile errors
When targeting WebAssembly, we default to building a single-threaded build
as threads are still experimental. The user however can enable a multi-
threaded build by specifying '-fno-single-threaded'. It's a compile-error
to enable this flag, but not also enable shared-memory.
This flag allows the user to force export the memory to the host
environment. This is useful when the memory is imported from the
host but must also be exported. This is (currently) required
to pass the memory validation for runtimes when using threads.
In this future this may become an error instead.
.res files are compiled Windows resource files that get linked into executables/libraries. The linker knows what to do with them, but previously you had to trick Zig into thinking it was an object file (by renaming it to have the .obj extension, for example).
After this commit, the following works:
zig build-exe main.zig resource.res
or, in build.zig:
exe.addObjectFile("resource.res");
Closes#6488
Most of this migration was performed automatically with `zig fmt`. There
were a few exceptions which I had to manually fix:
* `@alignCast` and `@addrSpaceCast` cannot be automatically rewritten
* `@truncate`'s fixup is incorrect for vectors
* Test cases are not formatted, and their error locations change
