489 Commits

Author SHA1 Message Date
Andrew Kelley
73bbd1069a build: remove the option to omit stage2 2022-08-19 16:45:15 -07:00
Andrew Kelley
c0b7f20893 stage2: implement stack protectors
This is one of the final remaining TODOs for the LLVM backend.
2022-08-19 03:41:13 -07:00
Andrew Kelley
b975f7a56f std.Target gains ObjectFormat field 2022-08-18 18:58:28 -07:00
Andrew Kelley
9656ec271c better default for use_stage1 when -ofmt=c is provided 2022-08-10 16:24:04 -07:00
Meghan Denny
0879cbeed2 remove 'builtin.stage2_arch', Sema is smart enough now 2022-08-07 16:07:08 -07:00
Meghan
02acde99a1 stage2: ensure 'std', 'builtin', and 'root' is always available to @import 2022-07-28 15:19:17 -07:00
InKryption
a0d3a87ce1 std.fmt: require specifier for unwrapping ?T and E!T 2022-07-26 11:25:49 -07:00
r00ster
cff5d9c805
std.mem: add first method to SplitIterator and SplitBackwardsIterator 2022-07-25 22:04:30 +03:00
Andrew Kelley
934573fc5d Revert "std.fmt: require specifier for unwrapping ?T and E!T."
This reverts commit 7cbd586ace46a8e8cebab660ebca3cfc049305d9.

This is causing a fail to build from source:

```
./lib/std/fmt.zig:492:17: error: cannot format optional without a specifier (i.e. {?} or {any})
                @compileError("cannot format optional without a specifier (i.e. {?} or {any})");
                ^
./src/link/MachO/Atom.zig:544:26: note: called from here
                log.debug("  RELA({s}) @ {x} => %{d} in object({d})", .{
                         ^
```

I looked at the code to fix it but none of those args are optionals.
2022-07-24 11:50:10 -07:00
InKryption
7cbd586ace
std.fmt: require specifier for unwrapping ?T and E!T.
Co-authored-by: Veikka Tuominen <git@vexu.eu>
2022-07-24 12:01:56 +03:00
Andrew Kelley
7d636f0f9d delete the stage1 implementation of autodoc 2022-07-19 19:10:12 -07:00
Andrew Kelley
e5d21e10bc Autodoc: skip docs when compile errors occur 2022-07-19 19:10:12 -07:00
Loris Cro
0135d22716 autodocs: add support for indirect decl references 2022-07-19 19:10:10 -07:00
Loris Cro
652e13e7c0 autodoc: init work 2022-07-19 19:10:10 -07:00
Veikka Tuominen
b2486fbc5e
Merge pull request #12121 from Vexu/span
Stage2 point to error location using spans
2022-07-16 12:22:53 +03:00
r00ster91
8f943b3d33 style: add missing comma
This is really minor but the issue this fixes is that if you copy-paste this output of `--show-builtin` into your `build.zig` for example then the formatter will format
```
pub const os = std.Target.Os{
    .tag = .freestanding,
    .version_range = .{ .none = {} }
};
```
to
```
pub const os = std.Target.Os{ .tag = .freestanding, .version_range = .{ .none = {} } };
```
which doesn't match the output.
With this comma, the output will stay the way it is after a `zig fmt`.
2022-07-16 09:03:04 +00:00
Veikka Tuominen
1463144fc8 Compilation: point caret in error message at the main token 2022-07-15 15:11:43 +03:00
Veikka Tuominen
b5a838247b stage2: point to error location using spans 2022-07-14 22:18:58 +03:00
Andrew Kelley
7d2e142679
Merge pull request #12044 from Vexu/stage2-compile-errors
Sema: add detailed error notes to `coerceInMemoryAllowed`
2022-07-11 20:45:27 -04:00
Andrew Kelley
8b3f15f218
Merge pull request #11863 from motiejus/compress-debug-sections
ELF: understand --compress-debug-sections
2022-07-11 17:23:02 -04:00
Andrew Kelley
2b99182e25 stage2: cleanups to --compress-debug-sections
* make the setting in the linker backend be non-optional; by this time
   all defaults are supposed to be resolved.
 * integrate with `zig cc`
 * change the CLI parsing to match C compiler parsing, allowing
   `--compress-debug-sections` alone to choose a default encoding of
   zlib.
2022-07-11 14:18:56 -07:00
r00ster91
da75eb0d79
Compilation: indent multiline error messages properly
Co-authored-by: Veikka Tuominen <git@vexu.eu>
2022-07-12 00:10:39 +03:00
Motiejus Jakštys
1f410b500c ELF: understand -Wl,--compress-debug-sections
This argument is both a compiler and a linker flag. The linker flag was
not understood; now it is. Go likes to use it as a linker flag.

Tested with sqlite3. The size difference is significant, and I confirmed
gdb understands both binaries.

zlib: 3.66MB
------------

    CC="zig cc"
    CFLAGS="-Wl,--compress-debug-sections=zlib -O2"
    ./configure --disable-tcl
    make

        FILE SIZE        VM SIZE
     --------------  --------------
      39.1%  1.43Mi  88.4%  1.43Mi    .text
      19.6%   734Ki   0.0%       0    .debug_info
      16.4%   613Ki   0.0%       0    .debug_loc
      13.1%   492Ki   0.0%       0    .debug_line
       4.2%   157Ki   9.5%   157Ki    .rodata
       2.3%  87.6Ki   0.0%       0    .debug_ranges
       1.5%  56.2Ki   0.0%       0    .symtab
       1.1%  40.2Ki   0.0%       0    .strtab
       1.0%  38.2Ki   0.0%       0    .debug_str
       0.7%  26.2Ki   0.0%       0    .debug_frame
       0.4%  15.3Ki   0.9%  15.3Ki    .data
       0.1%  4.71Ki   0.3%  4.71Ki    .dynsym
       0.1%  3.65Ki   0.2%  3.26Ki    [16 Others]
       0.1%  2.55Ki   0.2%  2.55Ki    .rela.plt
       0.1%  2.12Ki   0.0%       0    [ELF Section Headers]
       0.0%       0   0.1%  2.02Ki    .bss
       0.0%  1.84Ki   0.1%  1.84Ki    .dynstr
       0.0%  1.72Ki   0.1%  1.72Ki    .plt
       0.0%  1.58Ki   0.1%  1.58Ki    .hash
       0.0%  1.17Ki   0.0%       0    .debug_abbrev
       0.0%  1.01Ki   0.1%  1.01Ki    .rela.dyn
     100.0%  3.66Mi 100.0%  1.62Mi    TOTAL

none: 8.56MB
------------

    CC="zig cc" CFLAGS="-O2" ./configure --disable-tcl
    make

        FILE SIZE        VM SIZE
     --------------  --------------
      41.1%  3.52Mi   0.0%       0    .debug_loc
      18.5%  1.59Mi   0.0%       0    .debug_info
      16.7%  1.43Mi  88.4%  1.43Mi    .text
      11.8%  1.01Mi   0.0%       0    .debug_line
       5.9%   515Ki   0.0%       0    .debug_ranges
       1.8%   157Ki   9.5%   157Ki    .rodata
       1.3%   118Ki   0.0%       0    .debug_frame
       1.3%   110Ki   0.0%       0    .debug_str
       0.6%  56.2Ki   0.0%       0    .symtab
       0.5%  40.2Ki   0.0%       0    .strtab
       0.2%  15.3Ki   0.9%  15.3Ki    .data
       0.1%  4.71Ki   0.3%  4.71Ki    .dynsym
       0.0%  3.64Ki   0.2%  3.26Ki    [16 Others]
       0.0%  2.98Ki   0.0%       0    .debug_abbrev
       0.0%  2.55Ki   0.2%  2.55Ki    .rela.plt
       0.0%  2.12Ki   0.0%       0    [ELF Section Headers]
       0.0%       0   0.1%  2.02Ki    .bss
       0.0%  1.84Ki   0.1%  1.84Ki    .dynstr
       0.0%  1.72Ki   0.1%  1.72Ki    .plt
       0.0%  1.58Ki   0.1%  1.58Ki    .hash
       0.0%  1.01Ki   0.1%  1.01Ki    .rela.dyn
     100.0%  8.56Mi 100.0%  1.62Mi    TOTAL
2022-07-11 13:55:29 -07:00
Veikka Tuominen
2a41b1449b Compilation: do not repeat AstGen error source line for notes 2022-07-11 14:16:42 +03:00
Veikka Tuominen
e644a2ab6a Compilation: do not repeat same source line for notes 2022-07-10 23:47:56 +03:00
Andrew Kelley
fc7c0e07be stage2: propagate use_stage1 to sub-compilation
This makes it so that -fno-stage1 also affects compiler-rt for example.
2022-07-05 15:45:48 -07:00
Andrew Kelley
2ee864ca5e CLI: add support for -fno-builtin 2022-07-05 15:21:20 -07:00
Andrew Kelley
d65e248ed1 stage2: ELF: improve error reporting when libc is missing
Future improvement: make plain error notes actually render as notes
rather than errors, but keep them as errors for the case of
sub-compilation errors, e.g. when compiler-rt has compilation errors.
2022-07-04 14:06:20 -07:00
Andrew Kelley
152462e2e1 stage2: object format affects whether LLVM can be used 2022-06-30 18:33:02 -07:00
Ken Micklas
22490892fa Pass -O0 rather than -Og to Clang for Debug builds 2022-06-28 14:16:22 -04:00
Jakub Konka
0dd28920da macho: implement and handle -needed-* and -needed_* family of flags
MachO linker now handles `-needed-l<name>`, `-needed_library=<name>`
and `-needed_framework=<name>`. While on macOS `-l` is equivalent
to `-needed-l`, and `-framework` to `-needed_framework`, it can be
used to the same effect as on Linux if combined with `-dead_strip_dylibs`.

This commit also adds handling for `-needed_library` which is macOS
specific flag only (in addition to `-needed-l`).

Finally, in order to leverage new linker testing harness, this commit
added ability to specify lowering to those flags via `build.zig`:
`linkSystemLibraryNeeded` (and related), and `linkFrameworkNeeded`.
2022-06-27 19:53:38 +02:00
Jakub Konka
efc5c97bff macho: implement -dead_strip_dylibs linker flag 2022-06-27 19:53:38 +02:00
Jakub Konka
589bf67635 macho: implement -headerpad_max_install_names 2022-06-25 18:04:40 +02:00
Jakub Konka
8c1feef4cd macho: implement -headerpad_size option
Includes both traditiona and incremental codepaths with one caveat that
in incremental case, the requested size cannot be smaller than the
default padding size due to prealloc required due to incremental nature
of linking.

Also parse `-headerpad_max_install_names`, however, not actionable just yet -
missing implementation.
2022-06-25 17:59:08 +02:00
Jakub Konka
dfdb807543 cache setting macho search strategy flags 2022-06-25 10:50:00 +02:00
Jakub Konka
0df7ed79d3 macho: implement -search_dylibs_first linker option 2022-06-24 20:25:16 +02:00
Luuk de Gram
7c87f9c828 link:clarification & enable MachO getGlobalSymbol
This adds clarification to the getGlobalSymbol doc comments,
as well as renames the `addExternFn` function for MachO to `getGlobalSymbol`.
This function will now be called from 'src/link.zig' as well.

Finally, this also enables compiling zig's libc using LLVM even though
the `fno-LLVM` flag is given.
2022-06-24 08:12:17 +02:00
Luuk de Gram
a6747d328c stage2: Enable compiler-rt when LLVM is existant
Rather than checking if the user wants to use LLVM for the current compilation,
check for the existance of LLVM as part of the compiler. This is temporarily,
until other backends gain the ability to compiler LLVM themselves.
This means that when a user passed `-fno-LLVM` we will use the native
backend for the user's code, but use LLVM for compiler-rt.

This also fixes emitting names for symbols in the Wasm linker,
by deduplicating symbol names when multiple symbols point the same object.
2022-06-24 08:12:17 +02:00
Jakub Konka
ea9b7a0626 macho: round down pagezero size to page size
If page aligned requested pagezero size is 0, skip generating
__PAGEZERO segment.

Add misc improvements to the pipeline, and correctly transfer the
requested __PAGEZERO size to the linker.
2022-06-20 13:39:33 +02:00
Motiejus Jakštys
98138ba78c [MachO] add -pagezero_size
Pass `-pagezero_size` to the MachO linker. This is the final
"unsupported linker arg" that I could chase that CGo uses. After this
and #11874 we may be able to fail on an "unsupported linker arg" instead
of emiting a warning.

Test case:

    zig=/code/zig/build/zig
    CGO_ENABLED=1 GOOS=darwin GOARCH=amd64 CC="$zig cc -target x86_64-macos" CXX="$zig c++ -target x86_64-macos" go build -a -ldflags "-s -w" cgo.go

I compiled a trivial CGo program and executed it on an amd64 Darwin
host.

To be honest, I am not entirely sure what this is doing. This feels
right after reading what this argument does in LLVM sources, but I am by
no means qualified to make MachO pull requests. Will take feedback.
2022-06-20 13:39:33 +02:00
Andrew Kelley
e4092d4442 stage2: rip out multi-compilation-unit compiler-rt
After doing performance testing, it seems that multi-compilation-unit
compiler-rt did not bring the performance improvements that we expected
it to. The idea is that it makes linking faster, however, it incurred a
cost in the frontend that was not offset by any gains in linking.

Furthermore, the single-object compiler-rt (with -ffunction-sections and
--gc-sections) ends up being fewer bytes on disk and so it's actually
the same or faster linking speed than the multi-compilation-unit
version.

So we are planning to keep using single-compilation-unit compiler-rt for
the foreseeable future, but may experiment with this again in the
future, in which case this commit can be reverted.
2022-06-17 18:34:11 -07:00
Andrew Kelley
e798a3a779 compiler-rt: disable separate compilation units 2022-06-17 16:38:59 -07:00
Andrew Kelley
5cd548e530 Compilation: multi-thread compiler-rt
compiler_rt_lib and compiler_rt_obj are extracted from the generic
JobQueue into simple boolean flags, and then handled explicitly inside
performAllTheWork().

Introduced generic handling of allocation failure and made
setMiscFailure not return a possible error.

Building the compiler-rt static library now takes advantage of
Compilation's ThreadPool. This introduced a problem, however, because
now each of the object files of compiler-rt all perform AstGen for the
full standard library and compiler-rt files. Even though all of them end
up being cache hits except for the first ones, this is wasteful - O(N*M)
where N is number of compilation units inside compiler-rt and M is the
number of .zig files in the standard library and compiler-rt combined.

More importantly, however, it causes a deadlock, because each thread
interacts with a file system lock for doing AstGen on files, and threads
end up waiting for each other. This will need to be handled with a
process-level file caching system, or some other creative solution.
2022-06-17 16:38:59 -07:00
Jakub Konka
2259d629d3 compiler_rt: use single cache for libcompiler_rt.a static lib 2022-06-17 16:38:59 -07:00
Jakub Konka
80790be309 compiler_rt: compile each unit separately for improved archiving 2022-06-17 16:38:59 -07:00
Andrew Kelley
fcfeafe99a
Merge pull request #11819 from ziglang/std.debug.Trace
introduce std.debug.Trace and use it to debug a LazySrcLoc in stage2 that is set to a bogus value
2022-06-10 05:26:59 -04:00
Isaac Freund
5816d3eaec linker: remove -z noexecstack option
Note that the current documentation for the `-z noexecstack` is
incorrect. This indicates that an object *does not* require an
executable stack.

This is actually the default of LLD, and there has never been a way to
override this default by passing `-z execstack` to LLD.

This commit removes the redundant `-z noexecstack` option from
zig build-exe/build-lib/build-obj and ignores the option if passed
to zig cc for compatibility.

As far as I can tell, there is no reason for code to require an
executable stack. This option only exists because the stack was
originally executable by default and some programs came to depend
on that behavior. Instead, mprotect(2) may be used to make memory
pages executable.
2022-06-09 18:51:43 -04:00
Andrew Kelley
fd32f6890d stage2: fold redundant error notes 2022-06-09 15:37:16 -07:00
Isaac Freund
3381779426
linker: Enable full RELRO by default
Full RELRO is a hardening feature that makes it impossible to perform
certian attacks involving overwriting parts of the Global Offset Table
to invoke arbitrary code.

It requires all symbols to be resolved before execution of the program
starts which may have an impact on startup time. However most if
not all popular Linux distributions enable full RELRO by default for
all binaries and this does not seem to make a noticeable difference
in practice.

"Partial RELRO" is equivalent to `-z relro -z lazy`.
"Full RELRO" is equivalent to `-z relro -z now`.

LLD defaults to `-z relro -z lazy`, which means Zig's current `-z relro`
option has no effect on LLD's behavior.

The changes made by this commit are as follows:

- Document that `-z relro` is the default and add `-z norelro`.
- Pass `-z now` to LLD by default to enable full RELRO by default.
- Add `-z lazy` to disable passing `-z now`.
2022-06-08 14:33:11 +02:00
Andrew Kelley
cbb806da6e stage2: -fbuild-id causes default linker to be LLD
until zig's self-hosted linker gains this functionality.
2022-05-31 22:56:39 -07:00