21861 Commits

Author SHA1 Message Date
Jakub Konka
f7266e03a8 macho: identifier string in code signature should be just basename 2022-12-16 18:05:58 +01:00
Jakub Konka
bd6d951f63 macho: fix 32bit build 2022-12-16 18:05:58 +01:00
Jakub Konka
b1f874c78a enable testing for stage3 and stage4 being byte-for-byte identical 2022-12-16 18:05:58 +01:00
Jakub Konka
585c21e54d macho: move parallel file hashing back to CodeSignature
I need to think some more how to calculate UUID in parallel, if
it is even possible, to preserve UUID's determinism.
2022-12-16 18:05:58 +01:00
Jakub Konka
1928ed7dab macho: RingBuffer experiment to crack down the bug 2022-12-16 18:05:58 +01:00
Jakub Konka
79b92f9eb8 macho: remove temp debugging routines 2022-12-16 18:05:58 +01:00
Jakub Konka
d00094dd45 macho: exclude all content of the binary that could cause non-deterministic UUID 2022-12-16 18:05:58 +01:00
Jakub Konka
b323e14b1c macho: exclude linkedit and symtab/dysymtab load commands from the uuid calc 2022-12-16 18:05:58 +01:00
Jakub Konka
660270b7a9 macho: calculate UUID excluding stabs and part of contributing strtab 2022-12-16 18:05:58 +01:00
Jakub Konka
09dee74414 macho: store LC headers to often updated LINKEDIT sections 2022-12-16 18:05:58 +01:00
Jakub Konka
3af6a4e887 macho: generate stabs last to aid in deterministic uuid calculation 2022-12-16 18:05:58 +01:00
Evin Yulo
8c8f6bfa64 Add test for #13366
Closes #13366
2022-12-16 18:21:56 +02:00
Evin Yulo
b5d1df091b Add test for #12571
Closes #12571
2022-12-16 18:19:32 +02:00
Ganesan Rajagopal
aa3964477f c.zig: Add definition for pthread_key_t
Add missing definition for pthread_key_t

Closes #13950
2022-12-16 18:14:12 +02:00
Motiejus Jakštys
b3f4e0d091 Elf: switch link order of libcompiler_rt and libc
Given `main.go`:

    package main
    import  _ "os/user"
    func main() {}

Compiling it to linux/arm64:

    $ CGO_CFLAGS='-O0' GOOS=linux GOARCH=arm64 CGO_ENABLED=1 CC="zig cc -target aarch64-linux-gnu.2.28" go build main.go

Results in this error:

    runtime/cgo(.text): unknown symbol memset in callarm64
    runtime/cgo(.text): unknown symbol memset in callarm64
    runtime/cgo(.text): relocation target memset not defined

In the midst of intermediate compilations files we can see this commmand:

    ld.lld -o _cgo_.o <...> /tmp/go-build206961058/b043/_x009.o <...> ~/.cache/zig/.../libcompiler_rt.a <...> ~/.cache/.../libc.so.6

`_x009.o` needs memset:

    $ readelf -Ws ./b043/_x009.o | grep memset
        22: 0000000000000000     0 NOTYPE  GLOBAL DEFAULT  UND memset

Both `libcompiler_rt.a` and `libc.so.6` provide it:

    $ readelf -Ws ~/.cache/zig/.../libcompiler_rt.a | grep memset
       870: 0000000000000000   318 FUNC    WEAK   DEFAULT  519 memset

    $ readelf -Ws ~/.cache/zig/.../libc.so.6 | grep -w memset
       476: 000000000001d34c     0 FUNC    GLOBAL DEFAULT    7 memset@@GLIBC_2.2.5

Since `libcompiler_rt.a` comes before libc in the linker line, the
resulting `_cgo_.o` still links to a weak, unversioned memset:

    $ readelf -Ws ./b043/_cgo_.o | grep -w memset
        40: 000000000022c07c   160 FUNC    WEAK   DEFAULT   14 memset
       719: 000000000022c07c   160 FUNC    WEAK   DEFAULT   14 memset

Since the final linking step is done by Golang's linker, it does not
know of `libcompiler_rt.a`, and fails to link with the error message
above. However, Go linker does recognize memset from glibc. If we
specify an `-lc` equivalent before the `libcompiler_rt.a`, it will link
to memset from libc:

    $ readelf -Wa ./b043/_x009.o |grep memset
        14: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND memset@GLIBC_2.17 (2)
       157: 0000000000000000     0 FUNC    GLOBAL DEFAULT  UND memset@GLIBC_2.17

... and then `main.go` will compile+link successfully.

Why doesn't Go linker take memset from glibc? An educated guess: Go
determines whether to link with glibc from what the program asks (I
presume `.dynsym`). Since `memset` is no longer attributed to glibc, Go
skips linking to glibc altogether.

Bonus question: curious why `-O0` is necessary? Because when
optimizations are enabled (the default), the C compiler replaces
`memset` function call with plain `stp` instructions (on aarch64).
2022-12-16 07:46:47 -05:00
Andrew Kelley
cf85462a78
Merge pull request #13966 from r00ster91/stage1things
compiler_rt: re-enable tests and remove old workarounds
2022-12-16 06:11:16 -05:00
Travis Staloch
1ebb761244 codegen - lower str_lit to vector 2022-12-16 06:08:10 -05:00
Veikka Tuominen
8a0a6b7387 port packed vector elem ptr logic from stage1
Closes #12812
Closes #13925
2022-12-15 21:06:35 -05:00
r00ster91
171e6c7aa7 compiler_rt: remove workaround for past LLVM optimizer bug 2022-12-16 00:41:30 +01:00
r00ster91
41a7670a38 compiler_rt: remove stage1 switch case 2022-12-16 00:34:30 +01:00
r00ster91
20c5b6ad65 compiler_rt: no longer skip f80 conversion test
This is working again.

Closes #11408
2022-12-16 00:18:25 +01:00
Jakub Konka
6b99aab0eb macho: do not assume exe or dylib when flushing module 2022-12-16 00:01:04 +01:00
Jakub Konka
79457fc76a macho: generalize parallel hasher; impl parallel MD5-like hash
By pulling out the parallel hashing setup from `CodeSignature.zig`,
we can now reuse it different places across MachO linker (for now;
I can totally see its usefulness beyond MachO, eg. in COFF or ELF too).
The parallel hasher is generic over actual hasher such as Sha256 or MD5.
The implementation is kept as it was.

For UUID calculation, depending on the linking mode:
* incremental - since it only supports debug mode, we don't bother with MD5
  hashing of the contents, and populate it with random data but only once
  per a sequence of in-place binary patches
* traditional - in debug, we use random string (for speed); in release,
  we calculate the hash, however we use LLVM/LLD's trick in that we
  calculate a series of MD5 hashes in parallel and then one an MD5 of MD5
  final hash to generate digest.
2022-12-16 00:01:04 +01:00
Jakub Konka
ec40c6b28f macho: calculate UUID as a streaming MD5 hash of the file contents 2022-12-16 00:01:04 +01:00
Jakub Konka
db2052bc35 macho: dedup LC emitting logic
Fix path written to `LC_ID_DYLIB` to include the current CWD (if any).
2022-12-16 00:01:04 +01:00
Andrew Kelley
0d92fcf6a5 std.debug: disable sporadically failing test
see tracking issue #13963
2022-12-15 15:56:13 -07:00
Meghan
1704971666
std: make builtin.Type.{Int,Float}.bits a u16 instead of comptime_int 2022-12-15 23:08:51 +02:00
Luuk de Gram
ae106db889
wasm-linker: Fix debug info relocations 2022-12-15 20:28:30 +01:00
Ivan Velickovic
88b49ed00d std.leb128: Re-enable test for riscv64
These were previously disabled due to a LLVM 14 regression, see
https://github.com/ziglang/zig/issues/12031 for more details. This has been fixed
in LLVM 15.
2022-12-15 09:00:42 -05:00
Ryan Liptak
077b003c50 std.compress: Improve tests, remove reliance on openDirAbsolute
- Previously, some of the compress tests used `@src()` in combination with `dirname` and `openDirAbsolute` to read test files at runtime, which both excludes platforms that `openDirAbsolute` is not implemented for (WASI) and platforms that `SourceLocation.file` is not absolute (this was true for me locally on Windows). Instead of converting the tests to use `fs.cwd().openDir`, they have been converted to use `@embedFile` to avoid any potential problems with the runtime cwd.
- In order to use `@embedFile`, some of the `[]u8` parameters needed to be changed to `[]const u8`; none of them needed to be non-const anyway
- The tests now use `expectEqual` and `expectEqualSlices` where appropriate for better diagnostics
2022-12-15 05:13:21 -05:00
Stevie Hryciw
35750cd54f Resolve lazy value before comparing 2022-12-15 00:56:27 -08:00
Stevie Hryciw
e57e835904 Sema: elide integer comparisons with guaranteed outcomes 2022-12-15 00:56:27 -08:00
Stevie Hryciw
477038abe9 test: observe and check note count in error output 2022-12-15 00:56:27 -08:00
Stevie Hryciw
69b784ff52 std: add CompareOperator.reverse 2022-12-15 00:56:26 -08:00
Ryan Liptak
88d927a511 std.debug.TTY: Fix colors not resetting on Windows
This fixes a regression introduced in #12298 where colors would never reset in a Windows console because the attributes would be queried on every `setColor` call, and then try to 'reset' the attributes to what it just queried (i.e. it was essentially doing a complicated no-op on .Reset).

This fixes the problem while (I think) keeping with the spirit of the changes in #12298--that is, `TTY.Config` is not specifically tied to stderr like it was before #12298. To that end, detectTTYConfig now takes a `File` and that's what gets used to query the initial attributes to reset to.

(for context, before #12298, the first `setColor` call is where the reset attributes would get queried and it would always use stderr to do it)
2022-12-15 03:49:26 -05:00
Ryan Liptak
83e0e23f8a ArrayList.toOwnedSlice: Fix potential for leaks when using errdefer
#13666 introduced a footgun when using `toOwnedSlice` with `errdefer array_list.deinit()`, since `toOwnedSlice` could retain capacity if `resize` failed, meaning it would leak without `deinit` being called. This meant that the only correct way to use `toOwnedSlice` was with `defer` instead of `errdefer` to ensure that the ArrayList would get cleaned up.

Now, toOwnedSlice will now behave similarly to how it did before #13666, in that it will always clear the ArrayList's capacity if the resize/realloc succeeds.

This also reverts commit 05890a12f532ba9d58904a14381ec174b9efe473, which was contingent on the modified toOwnedSlice behavior.

Closes #13946
2022-12-15 02:56:24 -05:00
Andrew Kelley
8ff9284c46 glibc: avoid poisoning the cache namespace with zig lib dir
In glibc.zig, there were a few instances where the zig lib dir path name
incorrectly made its way into the cache namespace for various build
artifacts, resulting in unnecessary rebuilds of glibc.

Closes #13619
2022-12-15 00:07:53 -05:00
Andrew Kelley
c2f5848fe4
Merge pull request #11594 from KaneRoot/master
Dir.statFile now uses fstatat (fewer syscalls)
2022-12-14 20:54:04 -05:00
Andrew Kelley
b911b54aeb CI: fix path to langref when running tidy 2022-12-14 14:26:02 -07:00
Andrew Kelley
32d76f0e4a stage2: remove pub from a private function 2022-12-14 14:26:02 -07:00
Andrew Kelley
bac4a5c196 std: remove a solved TODO comment 2022-12-14 14:26:02 -07:00
Andrew Kelley
353121d9d5 std.c.fstatat64: add noalias attributes 2022-12-14 14:26:02 -07:00
Andrew Kelley
a62c8d36d5 std.fs.Dir.statFile rework
* revert changes to Module because the error set is consistent across
   operating systems.
 * remove duplicated Stat.fromSystem code and use a less redundant name.
 * make fs.Dir.statFile follow symlinks, and avoid pointless control
   flow through the posix layer.
2022-12-14 14:26:02 -07:00
Philippe Pittoli
f65cdef7c8 std.fs.Dir.statFile: use fstatat
This avoids extra syscalls.
2022-12-14 14:11:59 -07:00
Andrew Kelley
4af305b30a
Merge pull request #13891 from Vexu/compile-errors
Organize and implement remaining stage1 compile errors
2022-12-14 12:08:08 -05:00
Andrew Kelley
3bf97bfd46 std: remove OOM workarounds
No longer needed since stage1 is deleted.
2022-12-14 11:42:28 -05:00
Jakub Konka
b27b17e253
Merge pull request #13935 from ziglang/mach-tasks
darwin: expose more Mach primitives, expose wrapped ptrace, and allow starting suspended for posix_spawn
2022-12-14 17:42:02 +01:00
Luuk de Gram
f95549ddc7
wasm-linker: Export data symbols as global
When a data symbol is required to be exported, we instead generate
a global that will be exported. This global is immutable and contains
the address of the data symbol.
2022-12-14 16:55:23 +01:00
Veikka Tuominen
014009a730 parser: fix usage of incorrect error tag
Closes #13921
2022-12-14 14:08:22 +02:00
Veikka Tuominen
bc97a5662d Sema: display cimport errors from clang 2022-12-14 14:08:22 +02:00