99 Commits

Author SHA1 Message Date
Alex Cameron
9849e894d5 Add emit_h path to compilation hash. 2020-12-23 01:14:35 +11:00
Noam Preil
15a148db01 Fix memory leak 2020-12-23 01:14:35 +11:00
Alex Cameron
c87da2f45a Remove redundant emit_h member in Compilation struct. 2020-12-23 01:14:35 +11:00
Alex Cameron
40f0275e7c Implement emit-h 2020-12-23 01:14:35 +11:00
Andrew Kelley
1d94a68936 add an option to compile zig in single-threaded mode
And enable it for Drone CI. I hate to do this, but I need to make
progress on other fronts.
2020-12-20 15:37:58 -07:00
Andrew Kelley
aa6ef10cc6 std.Progress: make the API thread-safe
We generally get away with atomic primitives, however a lock is required
around the refresh function since it traverses the Node graph, and we
need to be sure no references to Nodes remain after end() is called.
2020-12-20 15:08:59 -07:00
Andrew Kelley
0d1cd0d482 use kprotty's ThreadPool implementation (v5) 2020-12-20 15:08:59 -07:00
Andrew Kelley
01d33855c7 stage2: protect mutable state from data races in updateCObject 2020-12-20 15:08:59 -07:00
Andrew Kelley
013efaf139 std: introduce a thread-local CSPRNG for general use
std.crypto.random

* cross platform, even freestanding
* can't fail. on initialization for some systems requires calling
  os.getrandom(), in which case there are rare but theoretically
  possible errors. The code panics in these cases, however the
  application may choose to override the default seed function and then
  handle the failure another way.
* thread-safe
* supports the full Random interface
* cryptographically secure
* no syscall required to initialize on Linux (AT_RANDOM)
* calls arc4random on systems that support it

`std.crypto.randomBytes` is removed in favor of `std.crypto.random.bytes`.

I moved some of the Random implementations into their own files in the
interest of organization.

stage2 no longer requires passing a RNG; instead it uses this API.

Closes #6704
2020-12-18 12:22:46 -07:00
Andrew Kelley
8975fa5b38 stage2: add test_evented_io to cache hash 2020-12-16 19:22:48 -07:00
Isaac Freund
1d8f33ca98
stage2: link musl dynamically by default if native
If targeting the native OS and the system libc is musl, link against it
dynamically by default.
2020-12-13 00:40:35 +01:00
Isaac Freund
307d98dc35
stage2: support dynamically linking musl libc 2020-12-13 00:40:35 +01:00
Andrew Kelley
6ab5bebed1 stage2: proper file extension stripping
Previously it used mem.split on "." and took the first iterated item.
Now it uses fs.path.extension and strips off that number of bytes.

Closes #7404
2020-12-11 17:42:13 -07:00
Andrew Kelley
c4f53d1ef6 fix deadlock with build-exe on an object for windows
The steps to repro this issue are:

zig build-obj hello.zig -target x86_64-windows-msvc
zig build-exe hello.obj -target x86_64-windows-msvc --subsystem console
-lkernel32 -lntdll

What was happening is that the main Compilation added a work item to
produce kernel32.lib. Then it added a sub-Compilation to build zig's
libc, which ended up calling a function with extern "kernel32", which
caused the sub-Compilation to also try to produce kernel32.lib. The main
Compilation and sub-Compilation do not coordinate about the set of
import libraries that they will be trying to build, so this caused a
deadlock.

This commit solves the problem by disabling the extern "foo" feature
from working when building compiler_rt or libc. Zig's linker code is now
responsible for putting the appropriate import libs on the linker line,
if any for compiler_rt and libc.

Related: #5825
2020-12-11 18:34:34 -05:00
Andrew Kelley
64a590a311 stage2: detect redundant C/C++ source files
Cache exposes BinDigest.

Compilation gains a set of a BinDigest for every C/C++ source file. We
detect when the same source/flags have already been added and emit a
compile error. This prevents a deadlock in the caching system.

Closes #7308
2020-12-11 02:33:09 -05:00
Andrew Kelley
7dd4afb224 stage2: link: properly implement passthrough mode for LLD child proc
passthrough mode does not mean always exit - it just means to pass
through stdio and exit if the child process exits, without doing any
special error reporting.
2020-12-08 22:37:01 -07:00
Andrew Kelley
24cbf1c706 always use codeview (pdb) when object_format is PE/COFF
Previously, when mixing Zig and C/C++ code for windows-gnu targets, zig
would get codeview format but the C/C++ code would not get any debug
info. Now, C/C++ code properly emits debug info in codeview format and
everything just works.
2020-12-08 13:31:39 -07:00
Andrew Kelley
2ed1ed9b32 stage2: introduce Module.failed_root_source_file
Use case:

zig build-exe non_existent_file.zig

Previous behavior:

error.FileNotFound, followed by an error return trace

Behavior after this commit:

error: unable to read non_existent_file.zig: FileNotFound
(end of stderr, exit code 1)

This turns AllErrors.Message into a tagged union which now has the
capability to represent both "plain" errors as well as source-based
errors (with file, line, column, byte offset). The "no entry point found"
error has moved to be a plain error message.
2020-12-04 17:21:55 -07:00
Jakub Konka
c4853ba7ea Version-gate appending -syslibroot flag to lld
This commit version-gates appending `-syslibroot` flag to lld. This
is predicated upon the fact that for versions of macOS lower than
11, lld would fail to find and link against frameworks with this
flag specified.

Co-authored-by: Andrew Kelley <andrew@ziglang.org>
2020-12-04 11:37:55 -08:00
Andrew Kelley
ddaf915389 add @cImport files to Compilation cache manifest
closes #7007
2020-12-03 16:07:38 -07:00
LemonBoy
c91c4dc256 stage2: Create cache manifest before calling final
If we enter the `if` because `comp.disable_c_depfile` is false the `man`
object has no manifest and calling .final on it will trip an assertion.

Closes #7096
2020-12-01 16:52:28 -07:00
Andrew Kelley
5b5097a22a stage2: add -femit-foo=bar args to the cache hash
Closes #6979
Closes #7036
2020-12-01 16:35:27 -07:00
Alexandros Naskos
8d6eff5cb9 Add package names and paths to the zig root module cache hash.
Add package names to the stage1 cache hash and package files to the
stage1 manifest file.
2020-12-01 13:56:36 -08:00
Timon Kruiper
8cbcc61c4d Make sure to include the root_name in the cache.
This fixes a bug where the caching system did not notice when the
--name flag changed.
2020-12-01 10:38:33 -08:00
Andrew Kelley
21565ca991 restore -target wasm32-freestanding-musl for C headers
See #5854

Some tiny tweaks too:
 * Use `wasm-freestanding-musl` instead of `wasm32-freestanding-musl`,
   making it pointer-size-agnostic.
 * Fix trying to build non-existent wasm musl start files.
2020-11-30 20:44:36 -07:00
Andrew Kelley
429a219f42 stage2: fix not detecting all dynamic libraries
Positional shared library arguments were not being detected as causing
dynamic linking, resulting in invalid linker lines. LLD did not have an
error message for this when targeting x86_64-linux but it did emit an
error message when targeting aarch64-linux, which is how I noticed the
problem.

This surfaced an error having to do with fifo.pipe() in the cat example
which I did not diagnose but solved the issue by doing the revamp that
was already overdue for that example.

It appears that the zig-window project was exploiting the previous
behavior for it to function properly, so this prompts the question, is
there some kind of static/dynamic executable hybrid that the compiler
should recognize? Unclear - but we can discuss that in #7240.
2020-11-30 20:25:28 -07:00
Andrew Kelley
8c9919ec7b fix regression on wasm targets
The previous commit broke wasm targets because the linking step would
look for the compiler-rt lib in the wrong place. Fixed in this commit.
2020-11-30 17:46:10 -07:00
Andrew Kelley
ff9798eb26 rework the bundle compiler-rt feature
* it is now -fcompiler-rt and -fno-compiler-rt to override the (quite
   reasonable) default of bundling compiler-rt only for executables and
   dynamic libraries.
   - the build.zig API is still called bundle_compiler_rt however it is
     now an optional bool instead of a bool. leaving it as `null` means
     to use the compiler default.
 * renamed some internal identifiers to make the source more readable
 * additionally support -fcompiler-rt when doing build-obj for ELF files
   since that target already supports linking multiple objects into one.
   - includes an error message when attempting this for non-ELF. in the
     future this could additionally be supported with a more advanced
     implementation that does not rely on the linker.
 * properly populate the linker cache hash
2020-11-30 16:35:00 -07:00
Alexandros Naskos
2fae28b6af Added bundle-compiler-rt flag 2020-11-30 14:31:41 -07:00
Jakub Konka
375bab8460 stage2 elf: refactor override_soname to soname 2020-11-27 15:42:39 -07:00
Andrew Kelley
abc717f203 modernize the PIE patch for the latest master branch
This is the part of #3960 that has to be rewritten to apply to latest
master branch code.
2020-11-22 17:28:11 -07:00
Andrew Kelley
6904cd828e add missing -m<os>-version-min CLI args to clang
This fixes some code generation issues when targeting macOS and
compiling C/C++ code.
2020-11-10 15:24:10 -07:00
Andrew Kelley
4d67c3d887 macOS: depend on bundled headers even for native
This is an alternate fix for #6773. Reverts
872bc787b56f71e53c80f4681523bc8356915b71.
2020-11-08 18:51:12 -07:00
Jakub Konka
ab69b89d52 Address review comments 2020-11-06 11:57:53 +01:00
Jakub Konka
b7c3ebcb9e Rely on ZIG_SYSTEM_LINKER_HACK instead of input flags 2020-11-06 11:34:51 +01:00
Jakub Konka
e0e3ceac19 Re-enable system linker hack
It is now possible to force linking with system linker `ld` instead
of the LLVM `lld` linker when building natively on the target. This
can be done at each stage by specifying `--system-linker-hack` flag,
and can be useful on platforms where `lld` fails to operate properly
such as macOS 11 Big Sur on ARM64 where every binary/dylib is expected
to be codesigned.

Some example invocations for each stage of compilation of Zig
toolchain:

```
cmake .. -DCMAKE_PREFIX_PATH=/path/to/llvm -DSYSTEM_LINKER_HACK=1
```

```
build/zig build test --system-linker-hack
```

```
build/zig build --prefix $(pwd)/stage2 -Denable-llvm
--system-linker-hack
```

```
build/zig build-exe hello.zig --system-linker-hack
```
2020-11-05 18:35:51 +01:00
Jakub Konka
6ecefd5903 Add std.Target.current.isDarwin() to exclude non-macs in comptime 2020-11-02 19:31:07 +01:00
Jakub Konka
f42ebd1b0a
Update src/Compilation.zig
Co-authored-by: Andrew Kelley <andrew@ziglang.org>
2020-11-02 18:56:08 +01:00
Jakub Konka
7f5c96378d Refactor the code according to Andrew's suggestions 2020-11-02 18:38:09 +01:00
Jakub Konka
317c555a5c Fix linking issues on BigSur
This commit fixes linking issue on macOS 11 BigSur by appending
a prefix path to all lib and framework search paths known as
`-syslibroot`.

The reason this is needed is that in macOS 11, the system libraries
and frameworks are no longer readily available in the filesystem.
Instead, the new macOS ships with a built-in dynamic linker cache
of all system-provided libraries, and hence, when linking with either
`lld.ld64` or `ld64`, it is required to pass in `-syslibroot [dir]`.
The latter can usually be obtained by invoking `xcrun --show-sdk-path`.
With this commit, Zig will do this automatically when compiling natively
on macOS. However, it also provides a flag `-syslibroot` which can be
used to overwrite the automtically populated value.

To summarise, with this change, the user of Zig is not required to
generate and append their own syslibroot path. Standard invocations
such as `zig build-exe hello.zig` or `zig build` for projects will
work out of the box. The only missing bit is `zig cc` and `zig c++`
since the addition of the `-syslibroot` option would be a mismatch
between the values provided by `clang` itself and Zig's wrapper.
2020-11-02 17:06:09 +01:00
Vexu
a1d7f0053d
stage2: support imports inside packages 2020-10-30 15:58:11 +02:00
mlarouche
e3fed3c814 Temp Fix for deadlock in C compilation, found and tested on Windows 2020-10-25 21:33:46 -04:00
Jakub Konka
872bc787b5 cc: fix regression on macOS after adding libc headers
Signed-off-by: Jakub Konka <kubkon@jakubkonka.com>
2020-10-24 17:32:50 +02:00
LemonBoy
d44486b274 std: Add libssp implementation for GNU/Windows targets
Unlike glibc and musl, MinGW provides no libssp symbols leading to
countless compile errors if FORTIFY_SOURCE is defined.

Add a (incomplete) implementation of libssp written in Zig so that
linking succeeds.

Closes #6492
2020-10-16 21:22:14 -04:00
Andrew Kelley
2f52f95b92
Merge pull request #6669 from ifreund/color-fixes
std/build: support --color
2020-10-14 21:35:43 -04:00
Frank Denis
c0e9d3fb86 Classify .tbd files as shared libraries
On macOS, a .tbd ("text-based dylib definition") file is a shared library
stub, allowing symbols to be defined only once for all the architectures
the library was compiled for.

.tbd files can be linked like .dylib files.
2020-10-14 18:19:57 -07:00
Isaac Freund
6ba1fdf7e0
stage2: use meta.stringToEnum for Color parsing
This requires renaming the variants to be snake_case, which is the new
recommended style anyways.
2020-10-13 20:17:30 +02:00
Andrew Kelley
9f8f446435 fixups to previous commit
* std.fs.Dir.readFile: add doc comments to explain what it means when
   the returned slice has the same length as the supplied buffer.
 * introduce readSmallFile / writeSmallFile to abstract over the
   decision to use symlink or file contents to store data.
2020-10-09 16:45:39 -07:00
mlarouche
57912964af Use regular file for caching stage 1 hash digest instead of symlink, fix zig build caching on Windows
Fix #6500
2020-10-09 16:50:43 -04:00
Josh Wolfe
bc6904eccc include compiler_rt and c for wasm static libraries 2020-10-09 01:54:42 -04:00