88 Commits

Author SHA1 Message Date
Isaac Freund
307d98dc35
stage2: support dynamically linking musl libc 2020-12-13 00:40:35 +01:00
Andrew Kelley
6ab5bebed1 stage2: proper file extension stripping
Previously it used mem.split on "." and took the first iterated item.
Now it uses fs.path.extension and strips off that number of bytes.

Closes #7404
2020-12-11 17:42:13 -07:00
Andrew Kelley
c4f53d1ef6 fix deadlock with build-exe on an object for windows
The steps to repro this issue are:

zig build-obj hello.zig -target x86_64-windows-msvc
zig build-exe hello.obj -target x86_64-windows-msvc --subsystem console
-lkernel32 -lntdll

What was happening is that the main Compilation added a work item to
produce kernel32.lib. Then it added a sub-Compilation to build zig's
libc, which ended up calling a function with extern "kernel32", which
caused the sub-Compilation to also try to produce kernel32.lib. The main
Compilation and sub-Compilation do not coordinate about the set of
import libraries that they will be trying to build, so this caused a
deadlock.

This commit solves the problem by disabling the extern "foo" feature
from working when building compiler_rt or libc. Zig's linker code is now
responsible for putting the appropriate import libs on the linker line,
if any for compiler_rt and libc.

Related: #5825
2020-12-11 18:34:34 -05:00
Andrew Kelley
64a590a311 stage2: detect redundant C/C++ source files
Cache exposes BinDigest.

Compilation gains a set of a BinDigest for every C/C++ source file. We
detect when the same source/flags have already been added and emit a
compile error. This prevents a deadlock in the caching system.

Closes #7308
2020-12-11 02:33:09 -05:00
Andrew Kelley
7dd4afb224 stage2: link: properly implement passthrough mode for LLD child proc
passthrough mode does not mean always exit - it just means to pass
through stdio and exit if the child process exits, without doing any
special error reporting.
2020-12-08 22:37:01 -07:00
Andrew Kelley
24cbf1c706 always use codeview (pdb) when object_format is PE/COFF
Previously, when mixing Zig and C/C++ code for windows-gnu targets, zig
would get codeview format but the C/C++ code would not get any debug
info. Now, C/C++ code properly emits debug info in codeview format and
everything just works.
2020-12-08 13:31:39 -07:00
Andrew Kelley
2ed1ed9b32 stage2: introduce Module.failed_root_source_file
Use case:

zig build-exe non_existent_file.zig

Previous behavior:

error.FileNotFound, followed by an error return trace

Behavior after this commit:

error: unable to read non_existent_file.zig: FileNotFound
(end of stderr, exit code 1)

This turns AllErrors.Message into a tagged union which now has the
capability to represent both "plain" errors as well as source-based
errors (with file, line, column, byte offset). The "no entry point found"
error has moved to be a plain error message.
2020-12-04 17:21:55 -07:00
Jakub Konka
c4853ba7ea Version-gate appending -syslibroot flag to lld
This commit version-gates appending `-syslibroot` flag to lld. This
is predicated upon the fact that for versions of macOS lower than
11, lld would fail to find and link against frameworks with this
flag specified.

Co-authored-by: Andrew Kelley <andrew@ziglang.org>
2020-12-04 11:37:55 -08:00
Andrew Kelley
ddaf915389 add @cImport files to Compilation cache manifest
closes #7007
2020-12-03 16:07:38 -07:00
LemonBoy
c91c4dc256 stage2: Create cache manifest before calling final
If we enter the `if` because `comp.disable_c_depfile` is false the `man`
object has no manifest and calling .final on it will trip an assertion.

Closes #7096
2020-12-01 16:52:28 -07:00
Andrew Kelley
5b5097a22a stage2: add -femit-foo=bar args to the cache hash
Closes #6979
Closes #7036
2020-12-01 16:35:27 -07:00
Alexandros Naskos
8d6eff5cb9 Add package names and paths to the zig root module cache hash.
Add package names to the stage1 cache hash and package files to the
stage1 manifest file.
2020-12-01 13:56:36 -08:00
Timon Kruiper
8cbcc61c4d Make sure to include the root_name in the cache.
This fixes a bug where the caching system did not notice when the
--name flag changed.
2020-12-01 10:38:33 -08:00
Andrew Kelley
21565ca991 restore -target wasm32-freestanding-musl for C headers
See #5854

Some tiny tweaks too:
 * Use `wasm-freestanding-musl` instead of `wasm32-freestanding-musl`,
   making it pointer-size-agnostic.
 * Fix trying to build non-existent wasm musl start files.
2020-11-30 20:44:36 -07:00
Andrew Kelley
429a219f42 stage2: fix not detecting all dynamic libraries
Positional shared library arguments were not being detected as causing
dynamic linking, resulting in invalid linker lines. LLD did not have an
error message for this when targeting x86_64-linux but it did emit an
error message when targeting aarch64-linux, which is how I noticed the
problem.

This surfaced an error having to do with fifo.pipe() in the cat example
which I did not diagnose but solved the issue by doing the revamp that
was already overdue for that example.

It appears that the zig-window project was exploiting the previous
behavior for it to function properly, so this prompts the question, is
there some kind of static/dynamic executable hybrid that the compiler
should recognize? Unclear - but we can discuss that in #7240.
2020-11-30 20:25:28 -07:00
Andrew Kelley
8c9919ec7b fix regression on wasm targets
The previous commit broke wasm targets because the linking step would
look for the compiler-rt lib in the wrong place. Fixed in this commit.
2020-11-30 17:46:10 -07:00
Andrew Kelley
ff9798eb26 rework the bundle compiler-rt feature
* it is now -fcompiler-rt and -fno-compiler-rt to override the (quite
   reasonable) default of bundling compiler-rt only for executables and
   dynamic libraries.
   - the build.zig API is still called bundle_compiler_rt however it is
     now an optional bool instead of a bool. leaving it as `null` means
     to use the compiler default.
 * renamed some internal identifiers to make the source more readable
 * additionally support -fcompiler-rt when doing build-obj for ELF files
   since that target already supports linking multiple objects into one.
   - includes an error message when attempting this for non-ELF. in the
     future this could additionally be supported with a more advanced
     implementation that does not rely on the linker.
 * properly populate the linker cache hash
2020-11-30 16:35:00 -07:00
Alexandros Naskos
2fae28b6af Added bundle-compiler-rt flag 2020-11-30 14:31:41 -07:00
Jakub Konka
375bab8460 stage2 elf: refactor override_soname to soname 2020-11-27 15:42:39 -07:00
Andrew Kelley
abc717f203 modernize the PIE patch for the latest master branch
This is the part of #3960 that has to be rewritten to apply to latest
master branch code.
2020-11-22 17:28:11 -07:00
Andrew Kelley
6904cd828e add missing -m<os>-version-min CLI args to clang
This fixes some code generation issues when targeting macOS and
compiling C/C++ code.
2020-11-10 15:24:10 -07:00
Andrew Kelley
4d67c3d887 macOS: depend on bundled headers even for native
This is an alternate fix for #6773. Reverts
872bc787b56f71e53c80f4681523bc8356915b71.
2020-11-08 18:51:12 -07:00
Jakub Konka
ab69b89d52 Address review comments 2020-11-06 11:57:53 +01:00
Jakub Konka
b7c3ebcb9e Rely on ZIG_SYSTEM_LINKER_HACK instead of input flags 2020-11-06 11:34:51 +01:00
Jakub Konka
e0e3ceac19 Re-enable system linker hack
It is now possible to force linking with system linker `ld` instead
of the LLVM `lld` linker when building natively on the target. This
can be done at each stage by specifying `--system-linker-hack` flag,
and can be useful on platforms where `lld` fails to operate properly
such as macOS 11 Big Sur on ARM64 where every binary/dylib is expected
to be codesigned.

Some example invocations for each stage of compilation of Zig
toolchain:

```
cmake .. -DCMAKE_PREFIX_PATH=/path/to/llvm -DSYSTEM_LINKER_HACK=1
```

```
build/zig build test --system-linker-hack
```

```
build/zig build --prefix $(pwd)/stage2 -Denable-llvm
--system-linker-hack
```

```
build/zig build-exe hello.zig --system-linker-hack
```
2020-11-05 18:35:51 +01:00
Jakub Konka
6ecefd5903 Add std.Target.current.isDarwin() to exclude non-macs in comptime 2020-11-02 19:31:07 +01:00
Jakub Konka
f42ebd1b0a
Update src/Compilation.zig
Co-authored-by: Andrew Kelley <andrew@ziglang.org>
2020-11-02 18:56:08 +01:00
Jakub Konka
7f5c96378d Refactor the code according to Andrew's suggestions 2020-11-02 18:38:09 +01:00
Jakub Konka
317c555a5c Fix linking issues on BigSur
This commit fixes linking issue on macOS 11 BigSur by appending
a prefix path to all lib and framework search paths known as
`-syslibroot`.

The reason this is needed is that in macOS 11, the system libraries
and frameworks are no longer readily available in the filesystem.
Instead, the new macOS ships with a built-in dynamic linker cache
of all system-provided libraries, and hence, when linking with either
`lld.ld64` or `ld64`, it is required to pass in `-syslibroot [dir]`.
The latter can usually be obtained by invoking `xcrun --show-sdk-path`.
With this commit, Zig will do this automatically when compiling natively
on macOS. However, it also provides a flag `-syslibroot` which can be
used to overwrite the automtically populated value.

To summarise, with this change, the user of Zig is not required to
generate and append their own syslibroot path. Standard invocations
such as `zig build-exe hello.zig` or `zig build` for projects will
work out of the box. The only missing bit is `zig cc` and `zig c++`
since the addition of the `-syslibroot` option would be a mismatch
between the values provided by `clang` itself and Zig's wrapper.
2020-11-02 17:06:09 +01:00
Vexu
a1d7f0053d
stage2: support imports inside packages 2020-10-30 15:58:11 +02:00
mlarouche
e3fed3c814 Temp Fix for deadlock in C compilation, found and tested on Windows 2020-10-25 21:33:46 -04:00
Jakub Konka
872bc787b5 cc: fix regression on macOS after adding libc headers
Signed-off-by: Jakub Konka <kubkon@jakubkonka.com>
2020-10-24 17:32:50 +02:00
LemonBoy
d44486b274 std: Add libssp implementation for GNU/Windows targets
Unlike glibc and musl, MinGW provides no libssp symbols leading to
countless compile errors if FORTIFY_SOURCE is defined.

Add a (incomplete) implementation of libssp written in Zig so that
linking succeeds.

Closes #6492
2020-10-16 21:22:14 -04:00
Andrew Kelley
2f52f95b92
Merge pull request #6669 from ifreund/color-fixes
std/build: support --color
2020-10-14 21:35:43 -04:00
Frank Denis
c0e9d3fb86 Classify .tbd files as shared libraries
On macOS, a .tbd ("text-based dylib definition") file is a shared library
stub, allowing symbols to be defined only once for all the architectures
the library was compiled for.

.tbd files can be linked like .dylib files.
2020-10-14 18:19:57 -07:00
Isaac Freund
6ba1fdf7e0
stage2: use meta.stringToEnum for Color parsing
This requires renaming the variants to be snake_case, which is the new
recommended style anyways.
2020-10-13 20:17:30 +02:00
Andrew Kelley
9f8f446435 fixups to previous commit
* std.fs.Dir.readFile: add doc comments to explain what it means when
   the returned slice has the same length as the supplied buffer.
 * introduce readSmallFile / writeSmallFile to abstract over the
   decision to use symlink or file contents to store data.
2020-10-09 16:45:39 -07:00
mlarouche
57912964af Use regular file for caching stage 1 hash digest instead of symlink, fix zig build caching on Windows
Fix #6500
2020-10-09 16:50:43 -04:00
Josh Wolfe
bc6904eccc include compiler_rt and c for wasm static libraries 2020-10-09 01:54:42 -04:00
travisstaloch
dd4771a5d2
cache-hash: add test_filter and test_name_prefix (#6583) 2020-10-06 19:36:14 -04:00
Andrew Kelley
55ac973953 fix each-lib-rpath functionality
It was regressed in 2 ways from the merge of #6250:
 * it was not being enabled by default when the target OS is native.
 * we were testing the libfoo.so file path existence with bogus format
   string ('{}' instead of '{s}') and so it ended up being something
   like "libstd.HashMap(K,V,...).Entry.so" instead of "libfoo.so". Using
   {} rather than {s} is a footgun, be careful!

Previous functionality is now restored.

closes #6523
2020-10-04 23:16:46 -07:00
Felix (xq) Queißner
dce74c31cc Depending on system libs only enforces libraries to require dynamic linking, but neither static libs nor object files. 2020-10-05 00:27:42 -04:00
Andrew Kelley
1d777e9958 add --image-base support
Based on #6121 by Jay Petacat.
2020-10-04 17:59:44 -07:00
Andrew Kelley
302a69f127
Merge pull request #6295 from Vexu/stage2
Stage2: basic imports
2020-10-04 18:00:21 -04:00
Alexandros Naskos
52ba2c3a43 Reintroduce progress bar when compiling C files 2020-10-03 22:11:29 -04:00
Nathan Bourgeois
4566b27373 Patch in emit relocs support 2020-10-01 17:04:04 -04:00
Vexu
6d3858dc8a
stage2: use directory handles for imports 2020-09-30 18:03:10 +03:00
Andrew Kelley
7bffa6f1fb clean up call to std.fs.rename
Thanks Ryan Liptak!
2020-09-29 21:47:37 -07:00
Andrew Kelley
1572cd4d76 different strategy to fix compiler_rt.zig and c.zig
with respect to std.builtin.link_libc.

The commit 27e008eb292038c5a6b9a13b64c7b69d1525f690 did not solve the
problem because although it got std.builtin.link_libc to be true for
compiler_rt.zig and c.zig, it had other unintentional side effects which
broke the build for -lc -target foo-linux-musl.

This commit introduces a new flag to Compilation to allow setting this
comptime flag to true without introducing other side effects to
compilation and linking.
2020-09-29 18:32:41 -07:00
Andrew Kelley
27e008eb29 fix not passing std.builtin.link_libc to compiler_rt.zig and c.zig
comment reproduced here:

This is so that compiler_rt and libc.zig libraries know whether they
will eventually be linked with libc. They make different decisions
about what to export depending on whether another libc will be linked
in. For example, compiler_rt will not export the __chkstk symbol if it
knows libc will provide it, and likewise c.zig will not export memcpy.
2020-09-29 17:24:21 -07:00