When entries are inserted and removed into a hash map at an equivalent rate (maintaining a mostly-consistent total count of entries), it should never need to be resized. But `HashMapUnmanaged.available` does not presently count tombstoned slots as "available", so this put/remove pattern eventually panics (assertion failure) when `available` reaches `0`.
The solution implemented here is to count tombstoned slots as "available". Another approach (which hashbrown (b3eaf32e60/src/raw/mod.rs (L1455-L1542)) takes) would be to rehash all entries in place when there are too many tombstones. This is more complex but avoids an `O(n)` bad case when the hash map is full of many tombstones.
This number tracks the glibc version in the oldest still-active LTS
version of Debian, which is Jessie, extended LTS expiring in June 2022,
at which point this number can be bumped again.
Before this commit, glibc headers did the following mapping:
* (zig) mipsel-linux-gnu => (glibc) mipsel-linux-gnu
* (zig) mipsel-linux-gnu-soft => (glibc) (none)
* (zig) mips-linux-gnu => (glibc) mips-linux-gnu
* (zig) mips-linux-gnu-soft => (glibc) (none)
While the glibc ABI stubs used the (zig) gnueabi and gnueabihf ABIs,
and the stage2 available_libcs array listed:
* (zig) mipsel-linux-gnu
* (zig) mips-linux-gnu
The problem is the mismatch between the ABI component of the headers and
the stubs.
This commit makes the following clarifications:
* (zig) mips-linux-gnueabi means soft-float
* (zig) mipsel-linux-gnueabi means soft-float
* (zig) mips-linux-gnueabihf means hard-float
* (zig) mipsel-linux-gnueabihf means hard-float
Consequently, the glibc headers now do this mapping:
* (zig) mips-linux-gnueabihf => (glibc) mips-linux-gnu
* (zig) mipsel-linux-gnueabihf => (glibc) mipsel-linux-gnu
* (zig) mips-linux-gnueabi => (glibc) mips-linux-gnu-soft
* (zig) mipsel-linux-gnueabi => (glibc) mipsel-linux-gnu-soft
The glibc ABI stubs are unchanged, and the stage2 available_libcs
array's 2 entries are modified and it gains 2 more:
* (zig) mipsel-linux-gnueabi
* (zig) mipsel-linux-gnueabihf
* (zig) mips-linux-gnueabi
* (zig) mips-linux-gnueabihf
Now everything is consistent. Zig no longer recognizes a `mips-linux-gnu`
triple; one must use `mips-linux-gnueabi` (soft float) or
`mips-linux-gnueabihf` (hard float).
Upstream, some of the nonshared functions moved to be different for hurd
and for linux. Since our glibc is linux-only we update to use the
linux-specific files.
This fixes std lib tests for x86_64 when linking glibc.
This commit introduces tools/update_glibc.zig to update the start files
for next time.
Some notable changes in recent glibc:
* abi-note.S has been changed to abi-note.c but we resist the change to
keep it easier to compile the start files.
* elf-init.c has been deleted upstream. Further testing should be done
to verify that binaries against glibc omitting elf-init.c still run
properly on oldel glibc linux systems.
Closes#4926
execve can return EBADLIB on Linux. I observed this when passing
an x86_64 interpreter path to qemu-i386.
This error code is Linux and Solaris-only. I came up with an improved
pattern for dealing with OS-specific error codes.
Since `BoundedArray.insert` internally reserves space for the element
to be inserted, it can support inserting at the position that is
the current length of the array. Change the check for the insertion position
to allow this.
- adds __cmpsi2, __cmpdi2, __cmpti2
- adds __ucmpsi2, __ucmpdi2, __ucmpti2
- use 2 if statements with 2 temporaries and a constant
- tests: MIN, MIN+1, MIN/2, -1, 0, 1, MAX/2, MAX-1, MAX if applicable
See #1290
- use negXi2.zig to prevent confusion with negXf2.zig
- used for size optimized builds and machines without carry instruction
- tests: special cases 0, -INT_MIN
* use divTrunc range and shift with constant offsets
See #1290
Notating a symbol to be exported in code will only tell the linker
where to find this symbol, so other object files can find it. However, this does not mean
said symbol will also be exported to the host environment. Currently, we 'fix' this by force
exporting every single symbol that is visible. This creates bigger binaries and means host environments
have access to symbols that they perhaps shouldn't have. Now, users can tell Zig which symbols
are to be exported, meaning all other symbols that are not specified will not be exported.
Another change is we now support `-rdynamic` in the wasm linker as well, meaning all symbols will
be put in the dynamic symbol table. This is the same behavior as with ELF. This means there's a 3rd strategy
users will have to build their wasm binary.
When the Zig compiler is statically linked, it inspects the
/usr/bin/env ELF file to determine the native glibc version, by checking
the DT_RUNPATH, and then calling readlink() on the libc.so file, because
typically the symlink will have e.g. libc-2.33.so in the name, revealing
the glibc version.
Fortunately, this information is also in readlink() of ld.so, which is
available as the "INTERP" file path. This commit looks for e.g.
`ld-2.33.so` on the symlink data for the dynamic linker.
In theory a more complete solution would also look at `/etc/ld.so.cache`
if necessary, and finally fall back to some hard coded paths, in order
to resolve the location of libc.so, in order to do this readlink() trick
on the resulting path. You can find that flow chart with `man ld.so`.
But I think this logic will be enough to get a correct answer in all real
world cases.
This has been tested on Debian Buster and glibc-based Void Linux.
Fixes#6469
This commit upgrades glibc shared library stub-creating code to use the
new abilists file which is generated by the new glibc-abi-tool project:
https://github.com/ziglang/glibc-abi-tool/
The abilists file is different in these ways:
* It additionally encodes whether a symbol is a function or an object,
and if it is an object, it additionally encodes the size in bytes.
* It additionally encodes migrations of symbols from one library to
another between glibc versions.
* It is binary data instead of ascii.
* It is one file instead of three.
* It is 165 KB instead of 200 KB.
This solves three bugs:
Fixes#7667Fixes#8714Fixes#8896
- each byte gets masked, shifted and combined
- use boring masks instead of comptime for readability
- tests: bit patterns with reverse operation, if applicable
See #1290
This is the result of the work on tools/gen_stubs.zig. It now uses the
preprocessor to emit different symbols and sizes depending on the
architecture. The data is collected directly from multiple libc.so files
on disk built with upstream musl.
Closes#8178
Addresses #8896 for musl
There is still room for further improvement to this, which is to
put `.ds` directives after symbols that are not followed by aliases, to
avoid the potential problem of a linker believing that all symbols are
aliases of each other.
tools/gen_stubs.zig now cuts out the middle man and operates directly on
the libc.so ELF file. it outputs accurate .size directives for objects.
std.elf gains an STV enum.
All Zig code is eligible to `@import("builtin")` which is mapped to a
generated file, build.zig, based on the target and other settings.
Zig invocations which share the same target settings will generate the
same builtin.zig file and thus the path to builtin.zig is in a shared
cache folder, and different projects can sometimes use the same file.
Before this commit, this led to race conditions where multiple
invocations of `zig` would race to write this file. If one process
wanted to *read* the file while the other process *wrote* the file, the
reading process could observe a truncated or partially written
builtin.zig file.
This commit makes the following improvements:
- limitations:
- avoid clobbering the inode, mtime in the hot path
- avoid creating a partially written file
- builtin.zig needs to be on disk for debug info / stack trace purposes
- don't mark the task as complete until the file is finished being populated
(possibly by an external process)
- strategy:
- create the `@import("builtin")` `Module.File` during the AstGen
work, based on generating the contents in memory rather than
loading from disk.
- write builtin.zig in a separate task that doesn't have
to complete until the end of the AstGen work queue so that it
can be done in parallel with everything else.
- when writing the file, first stat the file path. If it exists, we are done.
- otherwise, write the file to a temp file in the same directory and atomically
rename it into place (clobbering the inode, mtime in the cold path).
- summary:
- all limitations respected
- hot path: one stat() syscall that happens in a worker thread
This required adding a missing function to the standard library:
`std.fs.Dir.statFile`. In this commit, it does open() and then fstat()
which is two syscalls. It should be improved in a future commit to only
make one.
Fixes#9439.
Not sure why this warning is being emitted; let's reexamine it on the
next libunwind upgrade. I triggered it with this:
zig c++ -o hello hello.cpp -target x86_64-windows
The console test# label [test#/#tests] was being generated inside
refreshWithHeldLock (in lib/std/Progress.zig), using the number of
completed items. This was being incremented by 1 when displayed,
which is not required.
