writeFile was deprecated in favor of writeFile2 in f645022d16361865e24582d28f1e62312fbc73bb. This commit renames writeFile2 to writeFile and makes writeFile2 a compile error.
Previously, when multiple modules had builtin modules with identical
sources, two distinct `Module`s and `File`s were created pointing at the
same file path. This led to a bug later in the frontend. These modules
are now deduplicated with a simple hashmap on the builtin source.
At a minimum required glibc is v2.17, as earlier versions do not define
some symbols (e.g., getauxval()) used by the Zig standard library.
Additionally, glibc only supports some architectures at more recent
versions (e.g., riscv64 support came in glibc v2.27). So add a
`glibc_min` field to `available_libcs` for architectures with stronger
version requirements.
Extend the existing `canBuildLibC` function to check the target against
the Zig minimum, and the architecture/os minimum.
Also filter the list shown by `zig targets`, too:
$ zig targets | jq -c '.glibc'
["2.17.0","2.18.0","2.19.0","2.20.0","2.21.0","2.22.0","2.23.0","2.24.0","2.25.0","2.26.0","2.27.0","2.28.0","2.29.0","2.30.0","2.31.0","2.32.0","2.33.0","2.34.0","2.35.0","2.36.0","2.37.0","2.38.0"]
Fixes#17034Fixes#17769
The scope of libc_nonshared.a was greatly changed in glibc 2.33 and
2.34, but only the change from 2.34 was reflected so far. Glibc 2.33
finally switched to versioned symbols for stat functions, meaning that
libc_nonshared.a no longer contains them since 2.33. Relevant files were
therefore reverted to 2.32 versions and renamed accordingly.
This commit also removes errno.c, which was probably added to
libc_nonshared.a based on a wrong assumption that glibc/include/errno.h
requires glibc/csu/errno.c. In reality errno.h should refer to
__libc_errno (not to be confused with the public __errno_location),
which should be imported from libc.so. The inclusion of errno.c resulted
in wrong compile options as well; this commit fixes them as well.
Fixes#16152
Instead of making its own inside create. 10 out of 10 calls to create()
had already an arena in scope, so this commit means that 10 instances of
Compilation now reuse an existing arena with the same lifetime rather
than creating a redundant one.
In other words, this very slightly optimizes initialization of the
frontend in terms of memory allocation.
Use inline to vastly simplify the exposed API. This allows a
comptime-known endian parameter to be propogated, making extra functions
for a specific endianness completely unnecessary.
Finish the work started in 4c4fb839972f66f55aa44fc0aca5f80b0608c731.
Now the compiler compiles again.
Wire up dependency tree fetching code in the CLI for `zig build`.
Everything is hooked up except for `createDependenciesModule` is not yet
implemented.
Most of this migration was performed automatically with `zig fmt`. There
were a few exceptions which I had to manually fix:
* `@alignCast` and `@addrSpaceCast` cannot be automatically rewritten
* `@truncate`'s fixup is incorrect for vectors
* Test cases are not formatted, and their error locations change
* move `ptrBitWidth` from Arch to Target since it needs to know about the abi
* double isn't always 8 bits
* AVR uses 1-byte alignment for everything in GCC
This makes progress be exposed to the top-level caller of update().
I tossed in a bonus change: when the `zig build` subcommand sees exit
code 2, it omits the "following command failed" line, and the build
runner uses exit code 2 when there are compile errors. This tidies up
the output on build failure by a little bit.
Introduces std.zig.ErrorBundle which is a trivially serializeable set
of compilation errors. This is in the standard library so that both
the compiler and the build runner can use it. The idea is they will
use it to communicate compilation errors over a binary protocol.
The binary encoding of ErrorBundle is a bit problematic - I got a little
too aggressive with compaction. I need to change it in a follow-up
commit to use some indirection in the error message list, otherwise
iteration is too unergonomic. In fact it's so problematic right now that
the logic getAllErrorsAlloc() actually fails to produce a viable
ErrorBundle because it puts SourceLocation data in between the root
level ErrorMessage data.
This commit has a simplification - redundant logic for rendering AST
errors to stderr has been removed in favor of moving the logic for
lowering AST errors into AstGen. So even if we get parse errors, the
errors will get lowered into ZIR before being reported. I believe this
will be useful when working on --autofix. Either way, some redundant
brittle logic was happily deleted.
In Compilation, updateSubCompilation() is improved to properly perform
error reporting when a sub-compilation object fails. It no longer dumps
directly to stderr; instead it populates an ErrorBundle object, which
gets added to the parent one during getAllErrorsAlloc().
In package fetching code, instead of dumping directly to stderr, it now
populates an ErrorBundle object, and gets properly reported at the CLI
layer of abstraction.
In glibc.zig, there were a few instances where the zig lib dir path name
incorrectly made its way into the cache namespace for various build
artifacts, resulting in unnecessary rebuilds of glibc.
Closes#13619
Before, cache manifest files would have absolute file paths. This is
problematic for two reasons:
* Absolute file paths are not portable. Some operating systems such as
WASI have trouble with them. The files themselves are less portable;
they cannot be migrated from one user's home directory to another's.
And finally they can break due to file paths exceeding maximum path
component size.
* They would prevent some advanced use cases of Zig, where the lib dir
has a different path in a different invocation but is ultimately the
same Zig version and lib directory as before.
This commit adds a new column that specifies the prefix directory for
each file. 0 is an escape hatch and has the previous behavior. The other
two prefixes introduced are zig lib directory, and the cache directory.
This means files in zig-cache manifests can reference files local to
these directories.
In practice, this means it is possible to use a different file path for
the zig lib directory in a subsequent run of zig and have it still take
advantage of the global cache, provided that the files inside remain
unchanged.
closes#13050
Before, the code for building glibc stubs used a special case of the
Cache API that did not add any file inputs, and did not use
writeManifest(). This is not really how the Cache API is designed to
work and it shows because there was a race condition.
This commit adds as an input file the abilists file that comes with
Zig's installation, which has the added benefit of making glibc stub
caching properly detect cache invalidation when the user decides to
overwrite their abilists file. This harmonizes with the rest of how Zig
works, which intentionally allows you to hack the installation files and
have it behave properly with the cache system.
Finally, because of having any file inputs, the normal API flow of the
Cache system can be used, eliminating the one place that used the Cache
API in a special way. In other words, it uses writeManifest() now and
properly obeys the cache hit/miss semantics.
closes#13160
When linking libc and compiling natively, Zig tries to integrate with
the system C compiler. However, this caused Zig to fail when no system C
compiler is installed, despite the fact that Zig is perfectly capable of
compiling & linking libc without one.
This commit makes Zig fall back to using its own ability to provide libc
in the case that no C compiler is installed. For glibc, it means
sometimes getting the warning "zig cannot build new glibc version abc,
providing instead xyz".
Ideally, Zig would do some more validation about the system libraries
being linked against, and report an error in case it could not provide
the exact correct libc version of the system libraries (or that the
system libraries themselves conflict with each other), however, I think
it is fair to call that a separate enhancement.
Rename all references of sparcv9 to sparc64, to make Zig align more with
other projects. Also, added new function to convert glibc arch name to Zig
arch name, since it refers to the architecture as sparcv9.
This is based on the suggestion by @kubkon in PR 11847.
(https://github.com/ziglang/zig/pull/11487#pullrequestreview-963761757)
also use the common naming convention for glibc versions ("2.33" rather
than "2-33").
I also verified that these files are exactly identical to the previous
files from before zig updated to glibc 2.34.
__libc_start_main() from glibc.2.33.so or older needs to have a __libc_csu_init function callback parameter.
glibc-2.34 on the other hand has a different __libc_start_main() that does not use it,
and the start.S file from glibc-2.34 no longer construct the init function and pass null when calling __libc_start_main.
So, When targetting an older glibc, use the start.s files as they were in glibc-2.33 and construct the __libc_csu_init function.
fixes#10386#10512
This commit fixes two problems:
* `zig build-obj` regressed from the cache-mode branch. It would crash
because it assumed that dirname on the emit bin path would not be
null. This assumption was invalid when outputting to the current
working directory - a pretty common use case for `zig build-obj`.
* When using the LLVM backend, `-fno-emit-bin` combined with any other
kind of emitting, such as `-femit-asm`, emitted nothing.
Both issues are now fixed.
The two CacheMode values are `whole` and `incremental`.
`incremental` is what we had before; `whole` is new.
Whole cache mode uses everything as inputs to the cache hash;
and when a hit occurs it skips everything including linking.
This is ideal for when source files change rarely and for backends that
do not have good incremental compilation support, for example
compiler-rt or libc compiled with LLVM with optimizations on.
This is the main motivation for the additional mode, so that we can have
LLVM-optimized compiler-rt/libc builds, without waiting for the LLVM
backend every single time Zig is invoked.
Incremental cache mode hashes only the input file path and a few target
options, intentionally relying on collisions to locate already-existing
build artifacts which can then be incrementally updated.
The bespoke logic for caching stage1 backend build artifacts
is removed since we now have a global caching mechanism for
when we want to cache the entire compilation, *including* linking.
Previously we had to get "creative" with libs.txt and a special
byte in the hash id to communicate flags, so that when the cached
artifacts were re-linked, we had this information from stage1
even though we didn't actually run it. Now that `CacheMode.whole`
includes linking, this extra information does not need to be
preserved for cache hits. So although this changeset introduces
complexity, it also removes complexity.
The main trickiness here comes from the inherent differences between the
two modes: `incremental` wants a directory immediately to operate on,
while `whole` doesn't know the output directory until the compilation is
complete. This commit deals with this problem mostly inside `update()`,
where, on a cache miss, it replaces `zig_cache_artifact_directory` with a
temporary directory, and then renames it into place once the compilation is
complete.
Items remaining before this branch can be merged:
* [ ] make sure these things make it into the cache manifest:
- @import files
- @embedFile files
- we already add dep files from c but make sure the main .c files make
it in there too, not just the included files
* [ ] double check that the emit paths of other things besides the binary
are working correctly.
* [ ] test `-fno-emit-bin` + `-fstage1`
* [ ] test `-femit-bin=foo` + `-fstage1`
* [ ] implib emit directory copies bin_file_emit directory in create() and needs
to be adjusted to be overridden as well.
* [ ] make sure emit-h is handled correctly in the cache hash
* [ ] Cache: detect duplicate files added to the manifest
Some preliminary performance measurements of wall clock time and
peak RSS used:
stage1 behavior (1077 tests), llvm backend, release build:
* cold global cache: 4.6s, 1.1 GiB
* warm global cache: 3.4s, 980 MiB
stage2 master branch behavior (575 tests), llvm backend, release build:
* cold global cache: 0.62s, 191 MiB
* warm global cache: 0.40s, 128 MiB
stage2 this branch behavior (575 tests), llvm backend, release build:
* cold global cache: 0.62s, 179 MiB
* warm global cache: 0.27s, 90 MiB
Upstream, some of the nonshared functions moved to be different for hurd
and for linux. Since our glibc is linux-only we update to use the
linux-specific files.
This fixes std lib tests for x86_64 when linking glibc.
This commit introduces tools/update_glibc.zig to update the start files
for next time.
Some notable changes in recent glibc:
* abi-note.S has been changed to abi-note.c but we resist the change to
keep it easier to compile the start files.
* elf-init.c has been deleted upstream. Further testing should be done
to verify that binaries against glibc omitting elf-init.c still run
properly on oldel glibc linux systems.
Closes#4926
This commit upgrades glibc shared library stub-creating code to use the
new abilists file which is generated by the new glibc-abi-tool project:
https://github.com/ziglang/glibc-abi-tool/
The abilists file is different in these ways:
* It additionally encodes whether a symbol is a function or an object,
and if it is an object, it additionally encodes the size in bytes.
* It additionally encodes migrations of symbols from one library to
another between glibc versions.
* It is binary data instead of ascii.
* It is one file instead of three.
* It is 165 KB instead of 200 KB.
This solves three bugs:
Fixes#7667Fixes#8714Fixes#8896
