401 Commits

Author SHA1 Message Date
Andrew Kelley
934573fc5d Revert "std.fmt: require specifier for unwrapping ?T and E!T."
This reverts commit 7cbd586ace46a8e8cebab660ebca3cfc049305d9.

This is causing a fail to build from source:

```
./lib/std/fmt.zig:492:17: error: cannot format optional without a specifier (i.e. {?} or {any})
                @compileError("cannot format optional without a specifier (i.e. {?} or {any})");
                ^
./src/link/MachO/Atom.zig:544:26: note: called from here
                log.debug("  RELA({s}) @ {x} => %{d} in object({d})", .{
                         ^
```

I looked at the code to fix it but none of those args are optionals.
2022-07-24 11:50:10 -07:00
InKryption
7cbd586ace
std.fmt: require specifier for unwrapping ?T and E!T.
Co-authored-by: Veikka Tuominen <git@vexu.eu>
2022-07-24 12:01:56 +03:00
Jakub Konka
7ec9a4f382 cli: support --gc-sections and --no-gc-sections for Zig sources 2022-07-22 16:58:20 +02:00
Jakub Konka
44ee42c6bc cli: parse -dead_strip MachO linker flag 2022-07-22 16:58:20 +02:00
Veikka Tuominen
b2486fbc5e
Merge pull request #12121 from Vexu/span
Stage2 point to error location using spans
2022-07-16 12:22:53 +03:00
Veikka Tuominen
da94227f78
Merge pull request #12060 from Vexu/IterableDir
std.fs: split `Dir` into `IterableDir`
2022-07-16 12:18:21 +03:00
Luuk de Gram
6ba2fb3db7 zig cc: support -Wl,--export=<symbol> 2022-07-15 16:59:23 +02:00
Veikka Tuominen
1463144fc8 Compilation: point caret in error message at the main token 2022-07-15 15:11:43 +03:00
Veikka Tuominen
2b67f56c35 std.fs: split Dir into IterableDir
Also adds safety check for attempting to iterate directory not opened with `iterate = true`.
2022-07-15 13:04:21 +03:00
Veikka Tuominen
b5a838247b stage2: point to error location using spans 2022-07-14 22:18:58 +03:00
Veikka Tuominen
2a3f3766a4 stage2: handle parser notes in a more general way 2022-07-12 12:50:59 +03:00
Veikka Tuominen
5d22204d2d parser: add helpful error for C style container declarations
```zig
// a.zig
struct Foo {
    a: u32,
};
```

before:
```
a.zig:1:1: error: expected test, comptime, var decl, or container field, found 'struct'
struct Foo {
^
```
after:
```
a.zig:1:8: error: 'struct Foo' is invalid
struct Foo {
       ^
a.zig:1:8: note: to declare a container do 'const Foo = struct'
struct Foo {
       ^
```
2022-07-12 12:50:59 +03:00
Andrew Kelley
2b99182e25 stage2: cleanups to --compress-debug-sections
* make the setting in the linker backend be non-optional; by this time
   all defaults are supposed to be resolved.
 * integrate with `zig cc`
 * change the CLI parsing to match C compiler parsing, allowing
   `--compress-debug-sections` alone to choose a default encoding of
   zlib.
2022-07-11 14:18:56 -07:00
Motiejus Jakštys
1f410b500c ELF: understand -Wl,--compress-debug-sections
This argument is both a compiler and a linker flag. The linker flag was
not understood; now it is. Go likes to use it as a linker flag.

Tested with sqlite3. The size difference is significant, and I confirmed
gdb understands both binaries.

zlib: 3.66MB
------------

    CC="zig cc"
    CFLAGS="-Wl,--compress-debug-sections=zlib -O2"
    ./configure --disable-tcl
    make

        FILE SIZE        VM SIZE
     --------------  --------------
      39.1%  1.43Mi  88.4%  1.43Mi    .text
      19.6%   734Ki   0.0%       0    .debug_info
      16.4%   613Ki   0.0%       0    .debug_loc
      13.1%   492Ki   0.0%       0    .debug_line
       4.2%   157Ki   9.5%   157Ki    .rodata
       2.3%  87.6Ki   0.0%       0    .debug_ranges
       1.5%  56.2Ki   0.0%       0    .symtab
       1.1%  40.2Ki   0.0%       0    .strtab
       1.0%  38.2Ki   0.0%       0    .debug_str
       0.7%  26.2Ki   0.0%       0    .debug_frame
       0.4%  15.3Ki   0.9%  15.3Ki    .data
       0.1%  4.71Ki   0.3%  4.71Ki    .dynsym
       0.1%  3.65Ki   0.2%  3.26Ki    [16 Others]
       0.1%  2.55Ki   0.2%  2.55Ki    .rela.plt
       0.1%  2.12Ki   0.0%       0    [ELF Section Headers]
       0.0%       0   0.1%  2.02Ki    .bss
       0.0%  1.84Ki   0.1%  1.84Ki    .dynstr
       0.0%  1.72Ki   0.1%  1.72Ki    .plt
       0.0%  1.58Ki   0.1%  1.58Ki    .hash
       0.0%  1.17Ki   0.0%       0    .debug_abbrev
       0.0%  1.01Ki   0.1%  1.01Ki    .rela.dyn
     100.0%  3.66Mi 100.0%  1.62Mi    TOTAL

none: 8.56MB
------------

    CC="zig cc" CFLAGS="-O2" ./configure --disable-tcl
    make

        FILE SIZE        VM SIZE
     --------------  --------------
      41.1%  3.52Mi   0.0%       0    .debug_loc
      18.5%  1.59Mi   0.0%       0    .debug_info
      16.7%  1.43Mi  88.4%  1.43Mi    .text
      11.8%  1.01Mi   0.0%       0    .debug_line
       5.9%   515Ki   0.0%       0    .debug_ranges
       1.8%   157Ki   9.5%   157Ki    .rodata
       1.3%   118Ki   0.0%       0    .debug_frame
       1.3%   110Ki   0.0%       0    .debug_str
       0.6%  56.2Ki   0.0%       0    .symtab
       0.5%  40.2Ki   0.0%       0    .strtab
       0.2%  15.3Ki   0.9%  15.3Ki    .data
       0.1%  4.71Ki   0.3%  4.71Ki    .dynsym
       0.0%  3.64Ki   0.2%  3.26Ki    [16 Others]
       0.0%  2.98Ki   0.0%       0    .debug_abbrev
       0.0%  2.55Ki   0.2%  2.55Ki    .rela.plt
       0.0%  2.12Ki   0.0%       0    [ELF Section Headers]
       0.0%       0   0.1%  2.02Ki    .bss
       0.0%  1.84Ki   0.1%  1.84Ki    .dynstr
       0.0%  1.72Ki   0.1%  1.72Ki    .plt
       0.0%  1.58Ki   0.1%  1.58Ki    .hash
       0.0%  1.01Ki   0.1%  1.01Ki    .rela.dyn
     100.0%  8.56Mi 100.0%  1.62Mi    TOTAL
2022-07-11 13:55:29 -07:00
Andrew Kelley
2ee864ca5e CLI: add support for -fno-builtin 2022-07-05 15:21:20 -07:00
Andrew Kelley
3ab5912eee LLD: the driver functions now return "false" to mean "error" 2022-07-01 22:13:30 -07:00
Andrew Kelley
1fa4a58d2c stage1: update to LLVM 14 API 2022-07-01 19:19:13 -07:00
Jakub Konka
7cc4176448 clang: add Zig equivalent for -headerpad_max_install_names cli flag 2022-06-29 17:21:32 +02:00
Jakub Konka
c2c1998269 clang: update cmdline options to include weak libs and frameworks
Clang accepts `-weak-lx`, `-weak_library x` and `-weak_framework x`.
2022-06-28 21:16:23 +02:00
Jakub Konka
f353b59efa macho: discriminate between normal and weak dylibs
Parse `-weak-lx` and `-weak_framework x` in the CLI.
2022-06-28 09:18:54 +02:00
Jakub Konka
1188415f4c cli: typo --needed_library should be -needed_library on macos 2022-06-28 07:30:13 +02:00
Jakub Konka
0dd28920da macho: implement and handle -needed-* and -needed_* family of flags
MachO linker now handles `-needed-l<name>`, `-needed_library=<name>`
and `-needed_framework=<name>`. While on macOS `-l` is equivalent
to `-needed-l`, and `-framework` to `-needed_framework`, it can be
used to the same effect as on Linux if combined with `-dead_strip_dylibs`.

This commit also adds handling for `-needed_library` which is macOS
specific flag only (in addition to `-needed-l`).

Finally, in order to leverage new linker testing harness, this commit
added ability to specify lowering to those flags via `build.zig`:
`linkSystemLibraryNeeded` (and related), and `linkFrameworkNeeded`.
2022-06-27 19:53:38 +02:00
Jakub Konka
efc5c97bff macho: implement -dead_strip_dylibs linker flag 2022-06-27 19:53:38 +02:00
Jakub Konka
589bf67635 macho: implement -headerpad_max_install_names 2022-06-25 18:04:40 +02:00
Jakub Konka
8c1feef4cd macho: implement -headerpad_size option
Includes both traditiona and incremental codepaths with one caveat that
in incremental case, the requested size cannot be smaller than the
default padding size due to prealloc required due to incremental nature
of linking.

Also parse `-headerpad_max_install_names`, however, not actionable just yet -
missing implementation.
2022-06-25 17:59:08 +02:00
Jakub Konka
eadac47631 cli: let the linker resolve libs into dynamic and static
Unlike targeting ELF-based OSes such as Linux, resolving system libs
on Darwin should follow one of two strategies: `-search_paths_first`
or `-search_dylibs_first` and hence we defer always forcing linking
a static library to the linker.
2022-06-24 22:02:19 +02:00
Jakub Konka
0df7ed79d3 macho: implement -search_dylibs_first linker option 2022-06-24 20:25:16 +02:00
Motiejus Jakštys
d589047e80 zld: ignore -search_paths_first
Ignore MachO-specific flag -search_paths_first, since it is the default
in zld and ld64.

Also see Jakub's comment[1]:

    Changing topic slightly, @motiejus dunno if you noticed, with this
    change building arm64 Zig binary fails on macos - it complains about
    unknown -search_paths_first flag. This one is an easy fix: we should
    ignore it since this is the default behaviour in both ld64 and zld.

[1]: https://github.com/ziglang/zig/pull/11906#issuecomment-1163545849
2022-06-24 19:03:15 +02:00
Jakub Konka
8752db3285 macho: -pagezero_size is always in hex
This matches the behavior of other linkers out there including
`ld64` and `lld`.
2022-06-20 14:26:59 +02:00
Jakub Konka
ea9b7a0626 macho: round down pagezero size to page size
If page aligned requested pagezero size is 0, skip generating
__PAGEZERO segment.

Add misc improvements to the pipeline, and correctly transfer the
requested __PAGEZERO size to the linker.
2022-06-20 13:39:33 +02:00
Motiejus Jakštys
98138ba78c [MachO] add -pagezero_size
Pass `-pagezero_size` to the MachO linker. This is the final
"unsupported linker arg" that I could chase that CGo uses. After this
and #11874 we may be able to fail on an "unsupported linker arg" instead
of emiting a warning.

Test case:

    zig=/code/zig/build/zig
    CGO_ENABLED=1 GOOS=darwin GOARCH=amd64 CC="$zig cc -target x86_64-macos" CXX="$zig c++ -target x86_64-macos" go build -a -ldflags "-s -w" cgo.go

I compiled a trivial CGo program and executed it on an amd64 Darwin
host.

To be honest, I am not entirely sure what this is doing. This feels
right after reading what this argument does in LLVM sources, but I am by
no means qualified to make MachO pull requests. Will take feedback.
2022-06-20 13:39:33 +02:00
Motiejus Jakštys
d506275a06 [elf] understand -no-pie
This passes -Wl,-no-pie linker arg. Golang uses that. From the `ld(1)`
man page:

   Create a position dependent executable.  This is the default.

Not adding to the help text, because this is the default.
2022-06-16 15:27:30 -04:00
Isaac Freund
5816d3eaec linker: remove -z noexecstack option
Note that the current documentation for the `-z noexecstack` is
incorrect. This indicates that an object *does not* require an
executable stack.

This is actually the default of LLD, and there has never been a way to
override this default by passing `-z execstack` to LLD.

This commit removes the redundant `-z noexecstack` option from
zig build-exe/build-lib/build-obj and ignores the option if passed
to zig cc for compatibility.

As far as I can tell, there is no reason for code to require an
executable stack. This option only exists because the stack was
originally executable by default and some programs came to depend
on that behavior. Instead, mprotect(2) may be used to make memory
pages executable.
2022-06-09 18:51:43 -04:00
Isaac Freund
3381779426
linker: Enable full RELRO by default
Full RELRO is a hardening feature that makes it impossible to perform
certian attacks involving overwriting parts of the Global Offset Table
to invoke arbitrary code.

It requires all symbols to be resolved before execution of the program
starts which may have an impact on startup time. However most if
not all popular Linux distributions enable full RELRO by default for
all binaries and this does not seem to make a noticeable difference
in practice.

"Partial RELRO" is equivalent to `-z relro -z lazy`.
"Full RELRO" is equivalent to `-z relro -z now`.

LLD defaults to `-z relro -z lazy`, which means Zig's current `-z relro`
option has no effect on LLD's behavior.

The changes made by this commit are as follows:

- Document that `-z relro` is the default and add `-z norelro`.
- Pass `-z now` to LLD by default to enable full RELRO by default.
- Add `-z lazy` to disable passing `-z now`.
2022-06-08 14:33:11 +02:00
Andrew Kelley
59219e7e91 stage2: add support for -fbuild-id,-fno-build-id
closes #3047
2022-05-31 13:36:33 -07:00
Ali Chraghi
0e6285c8fc math: make cast return optional instead of an error 2022-05-27 16:43:33 -04:00
Motiejus Jakštys
1d532f12b5 [Elf] add -z nocopyreloc
Warnings about non-implemented `-z nocopyreloc` are common when
compiling go code (including Go's tests themselves). Let's just
make it stop complaining.
2022-05-19 20:21:07 -04:00
Jakub Konka
5f97652da8 x64: remove verbose_mir functionality
Originally I thought interleaving AIR with MIR will be useful, however
as it stands, I have used it very sporadically, and recently, not at
all, and I do not think anyone else is actually using it. If there is
a simple error such as a wrong instruction emitted,
`objdump` is perfectly capable of narrowing it down, while if there's
something more subtle happening, regardless of having `--verbose-mir`
functionality or not, you still gotta go via the debugger which
offers a better view at interleaved source program with the emitted
machine code. Finally, I believe `-femit-asm` when we add it will offer a
more generic substitute.
2022-05-15 18:21:50 +02:00
Kirk Scheibelhut
24633b5613 zig fmt: make --exclude ignore missing dirs 2022-05-13 16:45:00 -04:00
Andrew Kelley
ec95e00e28 flatten lib/std/special and improve "pkg inside another" logic
stage2: change logic for detecting whether the main package is inside
the std package. Previously it relied on realpath() which is not portable.
This uses resolve() which is how imports already work.

 * stage2: fix cleanup bug when creating Module
 * flatten lib/std/special/* to lib/*
   - this was motivated by making main_pkg_is_inside_std false for
     compiler_rt & friends.
 * rename "mini libc" to "universal libc"
2022-05-06 22:41:00 -07:00
Andrew Kelley
ba127058d1 CLI: detect MinGW-flavored static libraries
Ideally on Windows, static libraries look like "foo.lib". However, CMake
and other build systems will unfortunately produce static libraries that
instead look like "libfoo.a". This patch makes Zig's CLI resolve "-lfoo"
arguments into static libraries that match this other pattern.

This patch fixes an issue with zig-bootstrap where it won't find the
LLVM, Clang, and LLD libraries.
2022-05-04 19:15:31 -07:00
Andrew Kelley
1387d2f5ac zig build: use CacheMode.whole for build_runner.zig
Maybe after we have incremental compilation metadata serialization
and non-LLVM backends, it will make sense to switch this back. For now,
however, this makes successive `zig build` commands much faster.
2022-05-01 15:43:52 -07:00
Jimmi Holst Christensen
a0a2ce92ca std: Do not allocate the result for ChildProcess.init
Instead, just return ChildProcess directly. This structure does not
require a stable address, so we can put it on the stack just fine. If
someone wants it on the heap they should do.

  const proc = try allocator.create(ChildProcess);
  proc.* = ChildProcess.init(args, allocator);
2022-04-29 22:50:34 -04:00
Andrew Kelley
f7596ae942 stage2: use indexes for Decl objects
Rather than allocating Decl objects with an Allocator, we instead allocate
them with a SegmentedList. This provides four advantages:
 * Stable memory so that one thread can access a Decl object while another
   thread allocates additional Decl objects from this list.
 * It allows us to use u32 indexes to reference Decl objects rather than
   pointers, saving memory in Type, Value, and dependency sets.
 * Using integers to reference Decl objects rather than pointers makes
   serialization trivial.
 * It provides a unique integer to be used for anonymous symbol names,
   avoiding multi-threaded contention on an atomic counter.
2022-04-20 17:37:35 -07:00
Andrew Kelley
0c5ad335d2 build system: add -fstage1/-fno-stage1 to zig build
So that people can start experimenting with compiling their projects
with the self-hosted compiler.

I expect this commit to be reverted after #89 is closed.
2022-04-19 14:40:27 -07:00
Cody Tapscott
bb9cd6db1c stage2: Move WASI/Zig-specific selfExePath to introspect.zig 2022-04-18 23:06:49 -07:00
Andrew Kelley
b75d86027d stage2: avoid binary bloat from GeneralPurposeAllocator
In the case of not using it.
2022-04-18 16:41:54 -07:00
Jakub Konka
b03345f32a
Merge pull request #11024 from topolarity/wasi-stage2
stage2: Add limited WASI support for selfExePath and globalCacheDir
2022-04-18 23:53:41 +02:00
Cody Tapscott
692ccd01b4 stage2: Initialize WASI preopens on startup 2022-04-18 10:20:20 -07:00
Wojtek Mach
b2344cc18e Support --subsystem=x instead of --subsystem,x 2022-04-18 11:43:17 +02:00