179 Commits

Author SHA1 Message Date
Andrew Kelley
37cdb5dbf9 Merge remote-tracking branch 'origin/master' into llvm15 2022-09-09 10:26:17 -07:00
Jakub Konka
56b96cd61b
Merge pull request #12772 from ziglang/coff-basic-imports
coff: implement enough of the incremental linker to pass behavior and incremental tests on Windows
2022-09-09 13:08:58 +02:00
Motiejus Jakštys
a833bdcd7e [ld] add --print-* for diagnostics
This adds the following for passthrough to lld:
- `--print-gc-sections`
- `--print-icf-sections`
- `--print-map`

I am not adding these to the cache manifest, since it does not change
the produced artifacts.

Tested with an example from #11398: it successfully prints the resulting
map and the GC'd sections.
2022-09-08 22:30:32 -04:00
Jakub Konka
a35f156cf6 coff: re-enable default entrypoint for Windows 2022-09-07 22:42:55 +02:00
Andrew Kelley
1e21876de2 Merge remote-tracking branch 'origin/master' into llvm15 2022-08-30 13:02:43 -07:00
Andrew Kelley
7377dce368 avoid exposing supportsTailCall in the standard library
This is problematic because in practice it depends on whether the
compiler backend supports it too, as evidenced by the TODO comment about
LLVM not supporting some architectures that in fact do support tail
calls.

Instead this logic is organized strategically in src/target.zig, part of
the internal compiler source code, and the behavior tests in question
duplicate some logic for deciding whether to proceed with the test.

The proper place to expose this flag is in `@import("builtin")` - the
generated source file - so that third party compilers can advertise
whether they support tail calls.
2022-08-30 12:50:15 -07:00
Jakub Konka
90b3599c68 coff: reorganize the linker 2022-08-30 10:42:21 +02:00
Andrew Kelley
de7270028d Merge remote-tracking branch 'origin/master' into llvm15 2022-08-29 14:10:59 -07:00
Motiejus Jakštys
9038528187 copy_file_range: fix zigification of TXTBSY
From `copy_file_range(2)` errors:

    ETXTBSY
           Either fd_in or fd_out refers to an active swap file.

Same error will be used in the upcoming `ioctl_ficlonerange(2)`:

    ETXTBSY
           One of the files is a swap file.  Swap files cannot share storage.
2022-08-26 15:36:40 +03:00
Jakub Konka
d5233ee85c add ability to pass force undefined symbols to the linker
This commit enables `-u <symbol>` for ELF and `-include:<symbol>` for
COFF linkers for use internally. This means we do not expose these
flags to the users just yet, however, we make use of them internally
whenever required. One such use case is forcing inclusion of
`_tls_index` when linking for Windows with mingw and LTO and dead
code stripping enabled. This ensures we add `_tls_index` to the symbol
resolver as an undefined symbol and force the linker to include an atom
that provides it marking it a dead-code-stripping root - meaning it will
not be garbage collected by the linker no matter what.
2022-08-25 16:23:01 -04:00
Andrew Kelley
507aae4a1a make self-hosted the default compiler
stage1 is available behind the -fstage1 flag.

closes #89
2022-08-19 16:45:15 -07:00
Andrew Kelley
c0b7f20893 stage2: implement stack protectors
This is one of the final remaining TODOs for the LLVM backend.
2022-08-19 03:41:13 -07:00
Andrew Kelley
b975f7a56f std.Target gains ObjectFormat field 2022-08-18 18:58:28 -07:00
Jakub Konka
b12992cb01 macho: do not open file handle when building static archive
Firstly, opening a file handle is not really needed since we won't even
use it, and secondly, this can cause AccessDenied errors on Windows
when trying to move a directory from zig-cache/tmp/ to zig-cache/o/
since, without POSIX semantics, it is illegal to move directories
with open handles to any of its resources.
2022-08-18 15:54:09 -04:00
Andrew Kelley
a82d7c2063 std: add missing error to windows.WriteFile
I encountered this error today when testing the self-hosted compiler on
Windows.
2022-08-11 06:22:05 -04:00
Andrew Kelley
70d3912390 update Target, CPU, OS, ABI, etc. to LLVM 15 2022-07-31 18:10:28 -07:00
Jakub Konka
9eb7e5182b macho: rework symbol handling to match zld/ELF
Now, each object file will store a mutable table of symbols that it
defines. Upon symbol resolution between object files, the symbol
will be updated with a globally allocated section ordinal and address
in virtual memory. If the object defines a globally available symbol,
its location only (comprising of the symbol index and object index)
will be stored in the globals map for easy access when relocating, etc.
This approach cleans up the symbol management significantly, and matches
the status quo used in zld/ELF.

Additionally, this makes scoping symbol stabs easier too as they are
now naturally contained within each object file.
2022-07-22 16:58:20 +02:00
Luuk de Gram
8033767082
wasm-linker: Implement linker tests (#12006)
* test/link: initial wasm support

This adds basic parsing and dumping of wasm section so they
can be tested using the new linker-test infrastructure.

* test/link: all wasm sections parsing and dumping

We now parse and dump all sections for the wasm binary format.
Currently, this only dumps the name of a custom section.
Later this should also dump symbol table, name, linking metadata and relocations.
All of those live within the custom sections.

* Add wasm linker test

This also fixes a parser mistake in reading the flags.

* test/link: implement linker tests wasm & fixes

Adds several test cases to test the wasm self-hosted linker.
This also introduces fixes that were caught during the implementation
of those tests.

* test-runner: obey omit_stage2 for standalone

When a standalone test requires stage2, but stage2 is omit
from the compiler, such test case will not be included as part
of the test suite that is being ran. This is to support CI's
where we omit stage2 to lower the memory usage.
2022-07-12 14:36:33 +02:00
Andrew Kelley
2b99182e25 stage2: cleanups to --compress-debug-sections
* make the setting in the linker backend be non-optional; by this time
   all defaults are supposed to be resolved.
 * integrate with `zig cc`
 * change the CLI parsing to match C compiler parsing, allowing
   `--compress-debug-sections` alone to choose a default encoding of
   zlib.
2022-07-11 14:18:56 -07:00
Motiejus Jakštys
1f410b500c ELF: understand -Wl,--compress-debug-sections
This argument is both a compiler and a linker flag. The linker flag was
not understood; now it is. Go likes to use it as a linker flag.

Tested with sqlite3. The size difference is significant, and I confirmed
gdb understands both binaries.

zlib: 3.66MB
------------

    CC="zig cc"
    CFLAGS="-Wl,--compress-debug-sections=zlib -O2"
    ./configure --disable-tcl
    make

        FILE SIZE        VM SIZE
     --------------  --------------
      39.1%  1.43Mi  88.4%  1.43Mi    .text
      19.6%   734Ki   0.0%       0    .debug_info
      16.4%   613Ki   0.0%       0    .debug_loc
      13.1%   492Ki   0.0%       0    .debug_line
       4.2%   157Ki   9.5%   157Ki    .rodata
       2.3%  87.6Ki   0.0%       0    .debug_ranges
       1.5%  56.2Ki   0.0%       0    .symtab
       1.1%  40.2Ki   0.0%       0    .strtab
       1.0%  38.2Ki   0.0%       0    .debug_str
       0.7%  26.2Ki   0.0%       0    .debug_frame
       0.4%  15.3Ki   0.9%  15.3Ki    .data
       0.1%  4.71Ki   0.3%  4.71Ki    .dynsym
       0.1%  3.65Ki   0.2%  3.26Ki    [16 Others]
       0.1%  2.55Ki   0.2%  2.55Ki    .rela.plt
       0.1%  2.12Ki   0.0%       0    [ELF Section Headers]
       0.0%       0   0.1%  2.02Ki    .bss
       0.0%  1.84Ki   0.1%  1.84Ki    .dynstr
       0.0%  1.72Ki   0.1%  1.72Ki    .plt
       0.0%  1.58Ki   0.1%  1.58Ki    .hash
       0.0%  1.17Ki   0.0%       0    .debug_abbrev
       0.0%  1.01Ki   0.1%  1.01Ki    .rela.dyn
     100.0%  3.66Mi 100.0%  1.62Mi    TOTAL

none: 8.56MB
------------

    CC="zig cc" CFLAGS="-O2" ./configure --disable-tcl
    make

        FILE SIZE        VM SIZE
     --------------  --------------
      41.1%  3.52Mi   0.0%       0    .debug_loc
      18.5%  1.59Mi   0.0%       0    .debug_info
      16.7%  1.43Mi  88.4%  1.43Mi    .text
      11.8%  1.01Mi   0.0%       0    .debug_line
       5.9%   515Ki   0.0%       0    .debug_ranges
       1.8%   157Ki   9.5%   157Ki    .rodata
       1.3%   118Ki   0.0%       0    .debug_frame
       1.3%   110Ki   0.0%       0    .debug_str
       0.6%  56.2Ki   0.0%       0    .symtab
       0.5%  40.2Ki   0.0%       0    .strtab
       0.2%  15.3Ki   0.9%  15.3Ki    .data
       0.1%  4.71Ki   0.3%  4.71Ki    .dynsym
       0.0%  3.64Ki   0.2%  3.26Ki    [16 Others]
       0.0%  2.98Ki   0.0%       0    .debug_abbrev
       0.0%  2.55Ki   0.2%  2.55Ki    .rela.plt
       0.0%  2.12Ki   0.0%       0    [ELF Section Headers]
       0.0%       0   0.1%  2.02Ki    .bss
       0.0%  1.84Ki   0.1%  1.84Ki    .dynstr
       0.0%  1.72Ki   0.1%  1.72Ki    .plt
       0.0%  1.58Ki   0.1%  1.58Ki    .hash
       0.0%  1.01Ki   0.1%  1.01Ki    .rela.dyn
     100.0%  8.56Mi 100.0%  1.62Mi    TOTAL
2022-07-11 13:55:29 -07:00
Andrew Kelley
2ee864ca5e CLI: add support for -fno-builtin 2022-07-05 15:21:20 -07:00
Andrew Kelley
d65e248ed1 stage2: ELF: improve error reporting when libc is missing
Future improvement: make plain error notes actually render as notes
rather than errors, but keep them as errors for the case of
sub-compilation errors, e.g. when compiler-rt has compilation errors.
2022-07-04 14:06:20 -07:00
Jakub Konka
c02dc17618 link: cache weak libraries 2022-06-28 20:52:31 +02:00
Jakub Konka
f353b59efa macho: discriminate between normal and weak dylibs
Parse `-weak-lx` and `-weak_framework x` in the CLI.
2022-06-28 09:18:54 +02:00
Jakub Konka
0dd28920da macho: implement and handle -needed-* and -needed_* family of flags
MachO linker now handles `-needed-l<name>`, `-needed_library=<name>`
and `-needed_framework=<name>`. While on macOS `-l` is equivalent
to `-needed-l`, and `-framework` to `-needed_framework`, it can be
used to the same effect as on Linux if combined with `-dead_strip_dylibs`.

This commit also adds handling for `-needed_library` which is macOS
specific flag only (in addition to `-needed-l`).

Finally, in order to leverage new linker testing harness, this commit
added ability to specify lowering to those flags via `build.zig`:
`linkSystemLibraryNeeded` (and related), and `linkFrameworkNeeded`.
2022-06-27 19:53:38 +02:00
Jakub Konka
efc5c97bff macho: implement -dead_strip_dylibs linker flag 2022-06-27 19:53:38 +02:00
Jakub Konka
589bf67635 macho: implement -headerpad_max_install_names 2022-06-25 18:04:40 +02:00
Jakub Konka
8c1feef4cd macho: implement -headerpad_size option
Includes both traditiona and incremental codepaths with one caveat that
in incremental case, the requested size cannot be smaller than the
default padding size due to prealloc required due to incremental nature
of linking.

Also parse `-headerpad_max_install_names`, however, not actionable just yet -
missing implementation.
2022-06-25 17:59:08 +02:00
Jakub Konka
0df7ed79d3 macho: implement -search_dylibs_first linker option 2022-06-24 20:25:16 +02:00
Luuk de Gram
7c87f9c828 link:clarification & enable MachO getGlobalSymbol
This adds clarification to the getGlobalSymbol doc comments,
as well as renames the `addExternFn` function for MachO to `getGlobalSymbol`.
This function will now be called from 'src/link.zig' as well.

Finally, this also enables compiling zig's libc using LLVM even though
the `fno-LLVM` flag is given.
2022-06-24 08:12:17 +02:00
Luuk de Gram
8d03e4fc6b link: Implement API to get global symbol index 2022-06-24 08:12:17 +02:00
Motiejus Jakštys
98138ba78c [MachO] add -pagezero_size
Pass `-pagezero_size` to the MachO linker. This is the final
"unsupported linker arg" that I could chase that CGo uses. After this
and #11874 we may be able to fail on an "unsupported linker arg" instead
of emiting a warning.

Test case:

    zig=/code/zig/build/zig
    CGO_ENABLED=1 GOOS=darwin GOARCH=amd64 CC="$zig cc -target x86_64-macos" CXX="$zig c++ -target x86_64-macos" go build -a -ldflags "-s -w" cgo.go

I compiled a trivial CGo program and executed it on an amd64 Darwin
host.

To be honest, I am not entirely sure what this is doing. This feels
right after reading what this argument does in LLVM sources, but I am by
no means qualified to make MachO pull requests. Will take feedback.
2022-06-20 13:39:33 +02:00
Jakub Konka
47c834e477 macho: unify flushing object path with other linkers 2022-06-17 16:38:59 -07:00
Jakub Konka
62023c60b4 stage2: correctly work out dirname for ar 2022-06-11 00:23:10 +02:00
Isaac Freund
5816d3eaec linker: remove -z noexecstack option
Note that the current documentation for the `-z noexecstack` is
incorrect. This indicates that an object *does not* require an
executable stack.

This is actually the default of LLD, and there has never been a way to
override this default by passing `-z execstack` to LLD.

This commit removes the redundant `-z noexecstack` option from
zig build-exe/build-lib/build-obj and ignores the option if passed
to zig cc for compatibility.

As far as I can tell, there is no reason for code to require an
executable stack. This option only exists because the stack was
originally executable by default and some programs came to depend
on that behavior. Instead, mprotect(2) may be used to make memory
pages executable.
2022-06-09 18:51:43 -04:00
Andrew Kelley
59219e7e91 stage2: add support for -fbuild-id,-fno-build-id
closes #3047
2022-05-31 13:36:33 -07:00
Motiejus Jakštys
1d532f12b5 [Elf] add -z nocopyreloc
Warnings about non-implemented `-z nocopyreloc` are common when
compiling go code (including Go's tests themselves). Let's just
make it stop complaining.
2022-05-19 20:21:07 -04:00
Luuk de Gram
8e1c220be2 wasm: Add basic debug info references 2022-05-09 18:51:46 +02:00
Andrew Kelley
f7596ae942 stage2: use indexes for Decl objects
Rather than allocating Decl objects with an Allocator, we instead allocate
them with a SegmentedList. This provides four advantages:
 * Stable memory so that one thread can access a Decl object while another
   thread allocates additional Decl objects from this list.
 * It allows us to use u32 indexes to reference Decl objects rather than
   pointers, saving memory in Type, Value, and dependency sets.
 * Using integers to reference Decl objects rather than pointers makes
   serialization trivial.
 * It provides a unique integer to be used for anonymous symbol names,
   avoiding multi-threaded contention on an atomic counter.
2022-04-20 17:37:35 -07:00
Andrew Kelley
a7c05c06be stage2: expose progress bar API to linker backends
This gives us insight as to what is happening when we are waiting for
things such as LLVM emit object and LLD linking.
2022-04-17 04:09:35 -07:00
Jakub Konka
88d87d6506 stage2,macho: swap out inodes before checking for intermediary basename
This way we avoid the infamous SIGKILL on arm64 macos.
2022-04-16 12:06:58 +02:00
Andrew Kelley
2587474717 stage2: progress towards stage3
* The `@bitCast` workaround is removed in favor of `@ptrCast` properly
   doing element casting for slice element types. This required an
   enhancement both to stage1 and stage2.
 * stage1 incorrectly accepts `.{}` instead of `{}`. stage2 code that
   abused this is fixed.
 * Make some parameters comptime to support functions in switch
   expressions (as opposed to making them function pointers).
 * Avoid relying on local temporaries being mutable.
 * Workarounds for when stage1 and stage2 disagree on function pointer
   types.
 * Workaround recursive formatting bug with a `@panic("TODO")`.
 * Remove unreachable `else` prongs for some inferred error sets.

All in effort towards #89.
2022-04-14 10:12:45 -07:00
jagt
b9d86c6bc8 fix link.renameTmpIntoCache on windows when dest dir exists.
Previously it would fail as `renameW` do not ever fail with
`PathAlreadyExists`.

As a workaround we check for dest dir existence before rename
on Windows.
2022-04-12 06:01:25 -04:00
Jakub Konka
4ca9b4c44a dwarf: move DbgInfoTypeRelocsTable into Dwarf module 2022-03-27 20:53:06 +02:00
Andrew Kelley
593130ce0a stage2: lazy @alignOf
Add a `target` parameter to every function that deals with Type and
Value.
2022-03-22 15:45:58 -07:00
Jakub Konka
0376fd09bc macho: extend CodeSignature to accept entitlements
With this change, we can now bake in entitlements into the binary.
Additionally, I see this as the first step towards full code signature
support which includes baking in Apple issued certificates for
redistribution, etc.
2022-03-22 07:06:39 +01:00
Jakub Konka
68c224d6ec macho: simplify writing atoms for stage2
Also, fix premature exit in `link.File.makeWritable` in case we
are running M1 but executing binaries using Rosetta2.
2022-03-13 14:15:26 +01:00
Andrew Kelley
c6160fa3a5 LLVM: add compile unit to debug info
This commit also adds a bunch of bindings for debug info.
2022-03-08 14:58:53 -07:00
Jakub Konka
ba17552b4e dwarf: move all dwarf into standalone module
Hook up Elf and MachO linkers to the new solution.
2022-03-08 09:46:27 +01:00
Luuk de Gram
5a45fe2dba
wasm: Call generateSymbol for updateDecl
To unify the wasm backend with the other backends, we will now call `generateSymbol` to
lower a Decl into bytes. This means we also have to change some function signatures
to comply with the linker interface.

Since the general purpose generateSymbol is less featureful than wasm's, some tests are
temporarily disabled.
2022-03-06 19:38:50 +01:00