7195 Commits

Author SHA1 Message Date
Jakub Konka
362eccf539 stage2: handle RIP relative addressing in MI, RM and MR 2021-12-22 23:07:46 +01:00
Jakub Konka
b40e5adf54 stage2: add lowering for FD/TD encodings 2021-12-22 22:36:56 +01:00
Jakub Konka
b9a6f81d1a stage2: add lowering fn for OI encoding
Implement movabs using OI generic encoding.
2021-12-22 19:01:43 +01:00
Jakub Konka
2b5de9403d stage2: create generic lowering fns for MI, RM, and MR encodings
This way I am hopeful they can be reused for every MIR lowering
function which follows a given encoding. Currently, support MI,
RM and MR encodings without SIB scaling.
2021-12-22 18:10:52 +01:00
Jakub Konka
5156ccd552 stage2: merge MOV back with arith instructions
* turns out MOV and other arithmetic instructions such as ADD can
  naturally share the same lowering codepath (for the same variants)
* there are variants that are specific to ADD, or MOV which will be
  implemented as standalone MIR tags
* tweak Isel tests to generate corresponding test cases for all
  arithmetic instructions in comptime
2021-12-20 23:29:41 +01:00
Jakub Konka
71c5eebd32 stage2: remove obsolete MOV variant 0b11
* variant `0b11` when both `reg1 != .none` and `reg2 != .none` is
  identical to `0b00` therefore it can safely be removed
* fix proper destination register size calculation when setting register from
  another source register
2021-12-20 18:20:06 +01:00
Jakub Konka
4a40c0a80c stage2: refactor errors thrown for size mismatch in mirMovImpl
* introduce `EmitResult` wrapper struct for easier manipulation of
  intermediate emit results - this is mainly to track errors such as
  size mismatch between operands
* create an informative `ErrorMsg` directly at the callsite
2021-12-20 17:59:47 +01:00
Jakub Konka
70bff2f4d5 stage2: fix MOV MIR -> Isel lowering logic
* ensure that every callsite of basic MOV MIR instruction follows the
  Intel syntax (dst <- src)
* add extensive unit tests for MOV MIR -> Isel lowering
* leave TODOs for cases that are currently not handled and/or missing
* fix any ABI size mismatch between operands
2021-12-20 17:59:37 +01:00
Lee Cannon
d54ba76e40 TracyAllocator: correct order of free and alloc 2021-12-16 19:09:57 -08:00
Andrew Kelley
9257669cd4 glibc: do not pass -DSHARED to any of the libc_nonshared.a sources
This essentially reverts 6a12dce207114842e2e49a3aeb18af01ab207f0b but it
leaves breadcrumbs in the source explaining why things are the way they
are.
2021-12-16 03:34:20 -07:00
Andrew Kelley
e1436873a3 glibc: don't pass -frounding-math
glibc wants this flag but clang (our C compiler) does not support it.
2021-12-16 03:01:13 -07:00
Andrew Kelley
37fa6f955d glibc: add stat_t64_cp.c to libnonshared.a
Fixes 32-bit architectures.
2021-12-16 03:01:13 -07:00
Andrew Kelley
afb1652c50 glibc: i386: make glibc not use invalid inline asm syntax 2021-12-16 03:01:13 -07:00
Andrew Kelley
6a12dce207 glibc: fix some inconsistent flags with upstream
This more correctly matches what glibc does to build the objects
inside libnonshared.a.
2021-12-16 03:01:13 -07:00
Andrew Kelley
c9863c0a0c CLI: helpful error message when libc requested but not provided 2021-12-16 03:01:13 -07:00
Andrew Kelley
c8af00c66e glibc: fix inconsistency of powerpc ABI mapping
See the commit message of 5b6d26e97bb97d79782f3c77b02a997e361a1497 for
an explanation. This is the same thing but for powerpc instead of mips.
2021-12-16 03:01:13 -07:00
Andrew Kelley
5b6d26e97b glibc: fix inconsistency of mips ABI mapping
Before this commit, glibc headers did the following mapping:

 * (zig) mipsel-linux-gnu      => (glibc) mipsel-linux-gnu
 * (zig) mipsel-linux-gnu-soft => (glibc) (none)
 * (zig) mips-linux-gnu        => (glibc) mips-linux-gnu
 * (zig) mips-linux-gnu-soft   => (glibc) (none)

While the glibc ABI stubs used the (zig) gnueabi and gnueabihf ABIs,
and the stage2 available_libcs array listed:

 * (zig) mipsel-linux-gnu
 * (zig) mips-linux-gnu

The problem is the mismatch between the ABI component of the headers and
the stubs.

This commit makes the following clarifications:

 * (zig) mips-linux-gnueabi     means soft-float
 * (zig) mipsel-linux-gnueabi   means soft-float
 * (zig) mips-linux-gnueabihf   means hard-float
 * (zig) mipsel-linux-gnueabihf means hard-float

Consequently, the glibc headers now do this mapping:

 * (zig) mips-linux-gnueabihf   => (glibc) mips-linux-gnu
 * (zig) mipsel-linux-gnueabihf => (glibc) mipsel-linux-gnu
 * (zig) mips-linux-gnueabi     => (glibc) mips-linux-gnu-soft
 * (zig) mipsel-linux-gnueabi   => (glibc) mipsel-linux-gnu-soft

The glibc ABI stubs are unchanged, and the stage2 available_libcs
array's 2 entries are modified and it gains 2 more:

 * (zig) mipsel-linux-gnueabi
 * (zig) mipsel-linux-gnueabihf
 * (zig) mips-linux-gnueabi
 * (zig) mips-linux-gnueabihf

Now everything is consistent. Zig no longer recognizes a `mips-linux-gnu`
triple; one must use `mips-linux-gnueabi` (soft float) or
`mips-linux-gnueabihf` (hard float).
2021-12-15 19:09:50 -07:00
Andrew Kelley
e977455f7c glibc: improve RISC-V support
* omit crti.o / crtn.o for this architecture
 * add missing entry.h header from upstream
2021-12-15 18:34:27 -07:00
Andrew Kelley
4d48948b52 glibc: pass -D__GLIBC_MINOR__=XX
instead of hard-coding it. This means that C code has accurate glibc
version information in the preprocessor.
2021-12-15 17:52:27 -07:00
Andrew Kelley
3dcd3612dc glibc: use linux-specific files for nonshared
Upstream, some of the nonshared functions moved to be different for hurd
and for linux. Since our glibc is linux-only we update to use the
linux-specific files.

This fixes std lib tests for x86_64 when linking glibc.
2021-12-15 15:23:56 -07:00
Andrew Kelley
19ca2415f2 update glibc start files to 2.34
This commit introduces tools/update_glibc.zig to update the start files
for next time.

Some notable changes in recent glibc:

 * abi-note.S has been changed to abi-note.c but we resist the change to
   keep it easier to compile the start files.
 * elf-init.c has been deleted upstream. Further testing should be done
   to verify that binaries against glibc omitting elf-init.c still run
   properly on oldel glibc linux systems.

Closes #4926
2021-12-15 14:30:03 -07:00
Jakub Konka
0478446377 stage2: fix register size selection
This actually needs proper rework, and I'll get to that when refactoring
MIR.
2021-12-15 17:28:48 +01:00
Jakub Konka
510357355a stage2: fix premature atom deinit in macho linker 2021-12-15 17:28:48 +01:00
Jakub Konka
4b5f8bca5e stage2: clean up tests
* move darwin tests into respective architecture test files: `x86_64`
and `aarch64`
* run majority of `x86_64` tests on macOS
2021-12-15 17:28:48 +01:00
Jakub Konka
3ff05b79b9 macho: handle TLS imported from dylib
This is a missing feature which requires `__thread_ptrs` section
to be synthesised for any extern reference to a global TLS variable.
2021-12-15 08:59:20 +01:00
Jakub Konka
2e7a48d6bf macho: fix section size calculation when no prealloc
Add test for common symbol alignment.
2021-12-15 08:59:20 +01:00
Jakub Konka
ab328aca33 macho: put LC_* consts in a typed enum(u32) LC
repeat for `PLATFORM_*` and `TOOL_*` sets
2021-12-15 08:59:20 +01:00
Luuk de Gram
50201e1c30 wasm-linker: Allow specifying symbols to be exported
Notating a symbol to be exported in code will only tell the linker
where to find this symbol, so other object files can find it. However, this does not mean
said symbol will also be exported to the host environment. Currently, we 'fix' this by force
exporting every single symbol that is visible. This creates bigger binaries and means host environments
have access to symbols that they perhaps shouldn't have. Now, users can tell Zig which symbols
are to be exported, meaning all other symbols that are not specified will not be exported.

Another change is we now support `-rdynamic` in the wasm linker as well, meaning all symbols will
be put in the dynamic symbol table. This is the same behavior as with ELF. This means there's a 3rd strategy
users will have to build their wasm binary.
2021-12-14 14:02:23 -08:00
Stephen Gutekanst
e563b166b2 Fix compilation cache updating bug leading to slow builds
While investigating slow build times with [a large project](https://github.com/hexops/mach/issues/124),
I found that the compiler was reading from disk nearly every C source file in my project
when rebuilding despite no changes having been made. This accounted for several seconds of
time (approx. 20-30% of running `zig build` without any changes to the sources.)

The cause of this was that comparisons of file mtimes would _always_ fail (the mtime of the file on
disk was always newer than that stored in the cache manifest), and so the cache logic would always
fall back to byte-for-byte file content comparisons with what is on disk vs. in the cache-reading every
C source file in my project from disk during each rebuild. Because file contents were the same, a cache
hit occurred, and _despite the mtime being different the cache manifest would not be updated._

One can reproduce this by building a Zig project so the cache is populated, and then changing mtimes
of their C source files to be newer than what is in the cache (without altering file contents.)

The fix is rather simple: we should always write the updated cache manifest regardless of
whether or not a cache hit occurred (a cache hit doesn't indicate if a manifest is dirty) Luckily,
`writeManifest` already contains logic to determine if a manifest is dirty and becomes no-op if no
change to the manifest file is necessary-so we merely need to ensure it is invoked.

Signed-off-by: Stephen Gutekanst <stephen@hexops.com>
2021-12-14 13:56:35 -08:00
Andrew Kelley
1442aa7dc0 stage2: improved glibc stubs
This commit upgrades glibc shared library stub-creating code to use the
new abilists file which is generated by the new glibc-abi-tool project:
https://github.com/ziglang/glibc-abi-tool/

The abilists file is different in these ways:
 * It additionally encodes whether a symbol is a function or an object,
   and if it is an object, it additionally encodes the size in bytes.
 * It additionally encodes migrations of symbols from one library to
   another between glibc versions.
 * It is binary data instead of ascii.
 * It is one file instead of three.
 * It is 165 KB instead of 200 KB.

This solves three bugs:

Fixes #7667
Fixes #8714
Fixes #8896
2021-12-13 15:07:53 -08:00
Jakub Konka
4ce6208750 macho: if lib or framework not found, wait until syms resolved
This way, we will inform the user that there are unresolved symbols
in addition to missing library/framework as requested on the linker
line. If all symbols were resolved on the other hand, we still
flag up that the library/framework cannot be found.

Example behaviour:

```
$ zig cc hello.c -framework MyFoundation --verbose
warning(link): framework not found for '-framework MyFoundation'
warning(link): Framework search paths:
warning(link):   /Library/Frameworks
warning(link):   /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks
thread 1079397 panic: attempt to unwrap error: FrameworkNotFound
...stack trace...
```

and

```
❯ zig cc hello.c -lWAT --verbose
warning(link): library not found for '-lWAT'
warning(link): Library search paths:
warning(link):   /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib
warning(link):   /usr/local/lib
thread 1079824 panic: attempt to unwrap error: LibraryNotFound
...stack trace...
```
2021-12-12 13:53:59 +01:00
Jakub Konka
1abae06432 Merge branch 'framework-not-found' of git://github.com/motiejus/zig into motiejus-framework-not-found 2021-12-12 13:31:26 +01:00
Jakub Konka
dbfcebf8d8 macho: allow undefined symbols in dylibs
We now respect both `-fallow-shlib-undefined` and
`-Wl,"-undefined=dynamic_lookup"` flags. This is the first step
towards solving issues #8180 and #3000. We currently do not expose
any other ld64 equivalent flag for `-undefined` flag - we basically
throw an error should the user specify a different flag. Support for
those is conditional on closing #8180. As a result of this change,
it is now possible to generate a valid native Node.js addon with Zig
for macOS.
2021-12-12 00:32:03 +01:00
Motiejus Jakštys
82a7069344 macho: fail if requested -framework is not found
If `-framework` is requested, but not found, the linker will err
instead of creating a strange executable.

https://github.com/ziglang/zig/issues/10299#issuecomment-990404953

Refs #9542
Refs #10299
Refs #10158
2021-12-11 10:40:57 +02:00
Isaac Freund
7bb6393b59
stage1: implement @prefetch() builtin 2021-12-11 00:29:31 +01:00
Isaac Freund
175463d75d
AstGen: implement @prefetch() builtin 2021-12-10 23:09:02 +01:00
Isaac Freund
47c309c34a
AstGen: increase zig fmt off/on granularity
This enables automatic formatting for a significant amount of code that
currently doesn't deviate from the standard zig fmt enforced style.
2021-12-10 23:09:01 +01:00
Jakub Konka
828f61e8df macho: move all helpers from commands.zig into std.macho
This way we will finally be able to share common parsing logic
between different Zig components and 3rd party packages.
2021-12-10 18:18:28 +01:00
Jakub Konka
81e7d8505c macho: move helper functions to libstd
Helper functions such as `commands.sectionName`, etc. should really
belong in `std.macho.section_64` extern struct.
2021-12-10 11:56:51 +01:00
Andrew Kelley
3e618f8432 Cache: use mutex to protect recent_problematic_timestamp
The previous commit tried to use atomics but not many CPUs support
128-bit atomics. So we use a mutex. In order to avoid contention, we
also store `recent_problematic_timestamp` locally on the `Manifest`
which is only ever accessed from a single thread at a time, and only
consult the global one if the local one is problematic.

This commit was tested by running `zig build test-behavior` in two
separate terminals at the same time.
2021-12-09 22:10:38 -07:00
Andrew Kelley
fdbb329d10 Cache: fix data race with is_problematic_timestamp
Previously `recent_problematic_timestamp` was unprotected and accessed
potentially with multiple worker threads simultaneously.

This commit protects it with atomics and also introduces a flag to
prevent multiple timestamp checks from within the same call to hit().

Unfortunately the compiler-rt function __sync_val_compare_and_swap_16 is
not yet implemented, so I will have to take a different strategy in a
follow-up commit.
2021-12-09 22:10:38 -07:00
Andrew Kelley
4da83feccb Cache: improvements to previous commit
* put `recent_problematic_timestamp` onto `Cache` so that it can be
   shared by multiple Manifest instances.
 * make `isProblematicTimestamp` return true on any filesystem error.
 * save 1 syscall by using truncate=true in createFile instead of
   calling `setEndPos`.
2021-12-09 22:10:31 -07:00
Travis Martin
72ee042ab0 Cache: fix two issues with isProblematicTimestamp
1. It was looking for trailing zero bits when it should be looking for
   trailing decimal zeros.
2. Clock timestamps had more precision than the actual file timestamps

The fix is to grab a timestamp from a 'just now changed' temp file.
This timestamp is "problematic". Any file timestamp greater than or equal
to this timestamp is considered problematic. File timestamps **prior** to
this **can** be trusted.

Downside is that it causes a disk I/O to write to and then read the
timestamp from this file ~1ms on my system. This is partially mitigated by
keeping track of the most recent problematic timestamp, and only checking
for a new problematic timestamp when checking a timestamp that is equal to
or larger than the last problematic one.

This fixes #6082.
2021-12-09 18:34:17 -07:00
Andrew Kelley
01cb0bdb83
Merge pull request #10304 from ziglang/musl-stubgrade
Improve musl libc.so stubs
2021-12-09 14:12:51 -08:00
Andrew Kelley
f69f55c807 stage2: upgrade musl libc stub file
This is the result of the work on tools/gen_stubs.zig. It now uses the
preprocessor to emit different symbols and sizes depending on the
architecture. The data is collected directly from multiple libc.so files
on disk built with upstream musl.

Closes #8178
Addresses #8896 for musl

There is still room for further improvement to this, which is to
put `.ds` directives after symbols that are not followed by aliases, to
avoid the potential problem of a linker believing that all symbols are
aliases of each other.
2021-12-09 01:39:07 -07:00
Andrew Kelley
8031783539 stage1: fix regression of shift by negative value error
The previous commit (38b2d6209239f0dad7cb38e656d9d38506f126ca) regressed
the compile error test case for when doing saturating shift left of a
comptime-known negative RHS.

This commit additionally fixes the error for regular shifts in addition
to saturating shifts.
2021-12-08 19:09:37 -07:00
Andrew Kelley
38b2d62092 stage1: saturating shl operates using LHS type
Saturating shift left (`<<|`) previously used the `ir_analyze_bin_op_math`
codepath rather than the `ir_analyze_bit_shift` codepath, leading to it
doing peer type resolution (incorrect) instead of using the LHS type as
the number of bits to do the saturating against.

This required implementing SIMD vector support for `@truncate`.

Additionall, this commit adds a compile error for saturating shift left
on a comptime_int.

stage2 does not pass these new behavior tests yet.

closes #10298
2021-12-08 15:25:31 -07:00
Jakub Konka
64e2bfaa23 macho: write out LC_FUNCTION_START data 2021-12-08 23:24:40 +01:00
Jakub Konka
98e36ea6b9 macho: handle non-extern UNSIGNED relocs on arm64 2021-12-08 20:48:12 +01:00
Jakub Konka
f0ec70edd1 macho: do not write null symbols into undef symbols section
Also, skip creating stub entries for resolved globally defined
regular symbols.
2021-12-08 16:58:29 +01:00