also use the common naming convention for glibc versions ("2.33" rather
than "2-33").
I also verified that these files are exactly identical to the previous
files from before zig updated to glibc 2.34.
__libc_start_main() from glibc.2.33.so or older needs to have a __libc_csu_init function callback parameter.
glibc-2.34 on the other hand has a different __libc_start_main() that does not use it,
and the start.S file from glibc-2.34 no longer construct the init function and pass null when calling __libc_start_main.
So, When targetting an older glibc, use the start.s files as they were in glibc-2.33 and construct the __libc_csu_init function.
fixes#10386#10512
This commit fixes two problems:
* `zig build-obj` regressed from the cache-mode branch. It would crash
because it assumed that dirname on the emit bin path would not be
null. This assumption was invalid when outputting to the current
working directory - a pretty common use case for `zig build-obj`.
* When using the LLVM backend, `-fno-emit-bin` combined with any other
kind of emitting, such as `-femit-asm`, emitted nothing.
Both issues are now fixed.
The two CacheMode values are `whole` and `incremental`.
`incremental` is what we had before; `whole` is new.
Whole cache mode uses everything as inputs to the cache hash;
and when a hit occurs it skips everything including linking.
This is ideal for when source files change rarely and for backends that
do not have good incremental compilation support, for example
compiler-rt or libc compiled with LLVM with optimizations on.
This is the main motivation for the additional mode, so that we can have
LLVM-optimized compiler-rt/libc builds, without waiting for the LLVM
backend every single time Zig is invoked.
Incremental cache mode hashes only the input file path and a few target
options, intentionally relying on collisions to locate already-existing
build artifacts which can then be incrementally updated.
The bespoke logic for caching stage1 backend build artifacts
is removed since we now have a global caching mechanism for
when we want to cache the entire compilation, *including* linking.
Previously we had to get "creative" with libs.txt and a special
byte in the hash id to communicate flags, so that when the cached
artifacts were re-linked, we had this information from stage1
even though we didn't actually run it. Now that `CacheMode.whole`
includes linking, this extra information does not need to be
preserved for cache hits. So although this changeset introduces
complexity, it also removes complexity.
The main trickiness here comes from the inherent differences between the
two modes: `incremental` wants a directory immediately to operate on,
while `whole` doesn't know the output directory until the compilation is
complete. This commit deals with this problem mostly inside `update()`,
where, on a cache miss, it replaces `zig_cache_artifact_directory` with a
temporary directory, and then renames it into place once the compilation is
complete.
Items remaining before this branch can be merged:
* [ ] make sure these things make it into the cache manifest:
- @import files
- @embedFile files
- we already add dep files from c but make sure the main .c files make
it in there too, not just the included files
* [ ] double check that the emit paths of other things besides the binary
are working correctly.
* [ ] test `-fno-emit-bin` + `-fstage1`
* [ ] test `-femit-bin=foo` + `-fstage1`
* [ ] implib emit directory copies bin_file_emit directory in create() and needs
to be adjusted to be overridden as well.
* [ ] make sure emit-h is handled correctly in the cache hash
* [ ] Cache: detect duplicate files added to the manifest
Some preliminary performance measurements of wall clock time and
peak RSS used:
stage1 behavior (1077 tests), llvm backend, release build:
* cold global cache: 4.6s, 1.1 GiB
* warm global cache: 3.4s, 980 MiB
stage2 master branch behavior (575 tests), llvm backend, release build:
* cold global cache: 0.62s, 191 MiB
* warm global cache: 0.40s, 128 MiB
stage2 this branch behavior (575 tests), llvm backend, release build:
* cold global cache: 0.62s, 179 MiB
* warm global cache: 0.27s, 90 MiB
Upstream, some of the nonshared functions moved to be different for hurd
and for linux. Since our glibc is linux-only we update to use the
linux-specific files.
This fixes std lib tests for x86_64 when linking glibc.
This commit introduces tools/update_glibc.zig to update the start files
for next time.
Some notable changes in recent glibc:
* abi-note.S has been changed to abi-note.c but we resist the change to
keep it easier to compile the start files.
* elf-init.c has been deleted upstream. Further testing should be done
to verify that binaries against glibc omitting elf-init.c still run
properly on oldel glibc linux systems.
Closes#4926
This commit upgrades glibc shared library stub-creating code to use the
new abilists file which is generated by the new glibc-abi-tool project:
https://github.com/ziglang/glibc-abi-tool/
The abilists file is different in these ways:
* It additionally encodes whether a symbol is a function or an object,
and if it is an object, it additionally encodes the size in bytes.
* It additionally encodes migrations of symbols from one library to
another between glibc versions.
* It is binary data instead of ascii.
* It is one file instead of three.
* It is 165 KB instead of 200 KB.
This solves three bugs:
Fixes#7667Fixes#8714Fixes#8896
* Add missing Linux headers. Closes#9837
* Update existing headers to latest Linux.
* Consolidate headers that are the same for multiple Zig target CPU
architectures. For example, Linux has only an x86 directory for both
x86_64 and x86 CPU architectures. Now Zig only ships an x86 directory
for Linux headers, and will emit the proper corresponding -isystem
flags.
* tools/update-linux-headers.zig is now available for upgrading to
newer Linux headers, and the update process is now documented on the
wiki.
* There is now a main_pkg in addition to root_pkg. They are usually the
same. When using `zig test`, main_pkg is the user's source file and
root_pkg has the test runner.
* scanDecl no longer looks for test decls outside the package being
tested. honoring `--test-filter` is still TODO.
* test runner main function has a void return value rather than
`anyerror!void`
* Sema is improved to generate better AIR for for loops on slices.
* Sema: fix incorrect capacity calculation in zirBoolBr
* Sema: add compile errors for trying to use slice fields as an lvalue.
* Sema: fix type coercion for error unions
* Sema: fix analyzeVarRef generating garbage AIR
* C codegen: fix renderValue for error unions with 0 bit payload
* C codegen: implement function pointer calls
* CLI: fix usage text
Adds 4 new AIR instructions:
* slice_len, slice_ptr: to get the ptr and len fields of a slice.
* slice_elem_val, ptr_slice_elem_val: to get the element value of
a slice, and a pointer to a slice.
AstGen gains a new functionality:
* One of the unused flags of struct decls is now used to indicate
structs that are known to have non-zero size based on the AST alone.
closes#9034
These options were listed under the
"Debug Options (Zig Compiler Development)" heading. Anything in this
section should be considered unstable and can be modified at any time
at any developer's discretion.
Our glibc stub assembly file looked something like this:
```
.globl _Exit_2_2_5
.type _Exit_2_2_5, %function;
.symver _Exit_2_2_5, _Exit@@GLIBC_2.2.5
.hidden _Exit_2_2_5
_Exit_2_2_5:
```
With clang 12, the shared objects this produced stopped having any
exported symbols. When I removed the `.hidden` directive, it resolved
the issue, however, there are now unwanted exports:
```
$ readelf -W --dyn-syms libc.so.6 | grep sys_errlist
139: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 _sys_errlist_GLIBC_2_3
147: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 _sys_errlist_GLIBC_2_4
395: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 _sys_errlist_GLIBC_2_2_5
487: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 sys_errlist_GLIBC_2_2_5
1266: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 _sys_errlist@@GLIBC_2.12
1267: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 _sys_errlist@GLIBC_2.2.5
1268: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 _sys_errlist@GLIBC_2.3
1269: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 _sys_errlist@GLIBC_2.4
2137: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 sys_errlist@@GLIBC_2.12
2138: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 sys_errlist@GLIBC_2.2.5
2139: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 sys_errlist@GLIBC_2.3
2140: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 sys_errlist@GLIBC_2.4
2156: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 sys_errlist_GLIBC_2_3
2161: 000000000001ee08 0 FUNC GLOBAL DEFAULT 7 sys_errlist_GLIBC_2_4
```
Every line here without an `@` symbol is an unwanted export. Before, the
unwanted ones had LOCAL HIDDEN linkage.
As a mitigation, I did two things:
* Added `_GLIBC_` to the unwanted exports so that they would not
conflict with anything.
* Made the default export (the `@@` one) the bare symbol name. This
appears to reduce the unwanted exports to only symbols that have more
than one symbol (which is still quite many).
This will unblock progress on this branch, however, there is now a new
issue to solve, that the provided glibc stub .so files have too many
symbols exported. We will have to find a way to avoid this.
* CLI: change to -mred-zone and -mno-red-zone to match gcc/clang.
* build.zig: remove the double negative and make it an optional bool.
This follows precedent from other flags, allowing the compiler CLI to
be the decider of what is default instead of duplicating the default
value into the build system code.
* Compilation: make it an optional `want_red_zone` instead of a
`no_red_zone` bool. The default is decided by a call to
`target_util.hasRedZone`.
* When creating a Clang command line, put -mred-zone on the command
line if we are forcing it to be enabled.
* Update update_clang_options.zig with respect to the recent {s}/{} format changes.
* `zig cc` integration with red zone preference.
Previously Zig would need to recompile runtime libs if you changed the
values of --strip or -O. Now, unless the `debug_compiler_runtime_libs`
flag is set (which is currently not exposed to the CLI), Zig will always
choose ReleaseFast or ReleaseSmall for compiler runtime libraries.
When the main application chooses ReleaseFast or ReleaseSmall, that
value is propagated to compiler runtime libraries. Otherwise a decision
is made based on the target, which is currently ReleaseSmall for
freestanding WebAssembly and ReleaseFast for everything else.
Ultimately the purpose of this commit is to have Debug and ReleaseSafe
builds of applications still get optimized builds of, e.g. libcxx and
libunwind, as well as to spend less time unnecessarily rebuilding compiler
runtime libraries.
* rename is_compiler_rt_or_libc to skip_linker_dependencies
and set it to `true` for all sub-Compilations. I believe
this resolves the deadlock we were experiencing on Drone
CI and on some users' computers. I will remove the CI workaround in
a follow-up commit.
* enabling TSAN automatically causes the Compilation to link against
libc++ even if not requested, because TSAN depends on libc++.
* add -fno-rtti flags where appropriate when building TSAN objects.
Thanks Firefox317 for pointing this out.
* TSAN support: resolve all the undefined symbols. We are still seeing
a dependency on __gcc_personality_v0 but will resolve this one in a
follow-up commit.
* static libs do not try to build libc++ or libc++abi.
std.crypto.random
* cross platform, even freestanding
* can't fail. on initialization for some systems requires calling
os.getrandom(), in which case there are rare but theoretically
possible errors. The code panics in these cases, however the
application may choose to override the default seed function and then
handle the failure another way.
* thread-safe
* supports the full Random interface
* cryptographically secure
* no syscall required to initialize on Linux (AT_RANDOM)
* calls arc4random on systems that support it
`std.crypto.randomBytes` is removed in favor of `std.crypto.random.bytes`.
I moved some of the Random implementations into their own files in the
interest of organization.
stage2 no longer requires passing a RNG; instead it uses this API.
Closes#6704
