This is only relevant for ELF files.
I also fixed a bug where passing a zig source file to `zig cc` would
incorrectly punt to clang because it thought there were no positional
arguments.
Augment relocation tracking mechanism to de-duplicate potential
creation of base as well as composite types while unrolling
composite types in the linker - there is still potential for
further space optimisation by moving all type information into
a separate section `.debug_types` and providing references to
entries within that section whenever required (e.g., `ref4` form).
Currently, we duplicate type definitions on a per-decl basis.
Anyhow, with this patch, an example function signature of the following
type:
```zig
fn byPtrPtr(ptr_ptr_x: **u32, ptr_x: *u32) void {
ptr_ptr_x.* = ptr_x;
}
```
will generate the following `.debug_info` for formal parameters:
```
<1><1aa>: Abbrev Number: 3 (DW_TAG_subprogram)
<1ab> DW_AT_low_pc : 0x8000197
<1b3> DW_AT_high_pc : 0x2c
<1b7> DW_AT_name : byPtrPtr
<2><1c0>: Abbrev Number: 7 (DW_TAG_formal_parameter)
<1c1> DW_AT_location : 1 byte block: 55 (DW_OP_reg5 (rdi))
<1c3> DW_AT_type : <0x1df>
<1c7> DW_AT_name : ptr_ptr_x
<2><1d1>: Abbrev Number: 7 (DW_TAG_formal_parameter)
<1d2> DW_AT_location : 1 byte block: 54 (DW_OP_reg4 (rsi))
<1d4> DW_AT_type : <0x1e4>
<1d8> DW_AT_name : ptr_x
<2><1de>: Abbrev Number: 0
<1><1df>: Abbrev Number: 5 (DW_TAG_pointer_type)
<1e0> DW_AT_type : <0x1e4>
<1><1e4>: Abbrev Number: 5 (DW_TAG_pointer_type)
<1e5> DW_AT_type : <0x1e9>
<1><1e9>: Abbrev Number: 4 (DW_TAG_base_type)
<1ea> DW_AT_encoding : 7 (unsigned)
<1eb> DW_AT_byte_size : 4
<1ec> DW_AT_name : u32
```
This commit fixes two problems:
* `zig build-obj` regressed from the cache-mode branch. It would crash
because it assumed that dirname on the emit bin path would not be
null. This assumption was invalid when outputting to the current
working directory - a pretty common use case for `zig build-obj`.
* When using the LLVM backend, `-fno-emit-bin` combined with any other
kind of emitting, such as `-femit-asm`, emitted nothing.
Both issues are now fixed.
Doc comments reproduced here:
This function is called by the frontend before flush(). It communicates that
`options.bin_file.emit` directory needs to be renamed from
`[zig-cache]/tmp/[random]` to `[zig-cache]/o/[digest]`.
The frontend would like to simply perform a file system rename, however,
some linker backends care about the file paths of the objects they are linking.
So this function call tells linker backends to rename the paths of object files
to observe the new directory path.
Linker backends which do not have this requirement can fall back to the simple
implementation at the bottom of this function.
This function is only called when CacheMode is `whole`.
This solves stack trace regressions on Windows and macOS because the
linker backends do not observe object file paths until flush().
The two CacheMode values are `whole` and `incremental`.
`incremental` is what we had before; `whole` is new.
Whole cache mode uses everything as inputs to the cache hash;
and when a hit occurs it skips everything including linking.
This is ideal for when source files change rarely and for backends that
do not have good incremental compilation support, for example
compiler-rt or libc compiled with LLVM with optimizations on.
This is the main motivation for the additional mode, so that we can have
LLVM-optimized compiler-rt/libc builds, without waiting for the LLVM
backend every single time Zig is invoked.
Incremental cache mode hashes only the input file path and a few target
options, intentionally relying on collisions to locate already-existing
build artifacts which can then be incrementally updated.
The bespoke logic for caching stage1 backend build artifacts
is removed since we now have a global caching mechanism for
when we want to cache the entire compilation, *including* linking.
Previously we had to get "creative" with libs.txt and a special
byte in the hash id to communicate flags, so that when the cached
artifacts were re-linked, we had this information from stage1
even though we didn't actually run it. Now that `CacheMode.whole`
includes linking, this extra information does not need to be
preserved for cache hits. So although this changeset introduces
complexity, it also removes complexity.
The main trickiness here comes from the inherent differences between the
two modes: `incremental` wants a directory immediately to operate on,
while `whole` doesn't know the output directory until the compilation is
complete. This commit deals with this problem mostly inside `update()`,
where, on a cache miss, it replaces `zig_cache_artifact_directory` with a
temporary directory, and then renames it into place once the compilation is
complete.
Items remaining before this branch can be merged:
* [ ] make sure these things make it into the cache manifest:
- @import files
- @embedFile files
- we already add dep files from c but make sure the main .c files make
it in there too, not just the included files
* [ ] double check that the emit paths of other things besides the binary
are working correctly.
* [ ] test `-fno-emit-bin` + `-fstage1`
* [ ] test `-femit-bin=foo` + `-fstage1`
* [ ] implib emit directory copies bin_file_emit directory in create() and needs
to be adjusted to be overridden as well.
* [ ] make sure emit-h is handled correctly in the cache hash
* [ ] Cache: detect duplicate files added to the manifest
Some preliminary performance measurements of wall clock time and
peak RSS used:
stage1 behavior (1077 tests), llvm backend, release build:
* cold global cache: 4.6s, 1.1 GiB
* warm global cache: 3.4s, 980 MiB
stage2 master branch behavior (575 tests), llvm backend, release build:
* cold global cache: 0.62s, 191 MiB
* warm global cache: 0.40s, 128 MiB
stage2 this branch behavior (575 tests), llvm backend, release build:
* cold global cache: 0.62s, 179 MiB
* warm global cache: 0.27s, 90 MiB
The status quo for the `build.zig` build system is preserved in
the sense that, if the user does not explicitly override
`dylib.setInstallName(...);` in their build script, the default
of `@rpath/libname.dylib` applies. However, should they want to
override the default behaviour, they can either:
1) unset it with
```dylib.setIntallName(null);```
2) set it to an explicit string with
```dylib.setInstallName("somename.dylib");```
When it comes to the command line however, the default is not to
use `@rpath` for the install name when creating a dylib. The user
will now be required to explicitly specify the `@rpath` as part
of the desired install name should they choose so like so:
1) with `build-lib`
```
zig build-lib -dynamic foo.zig -install_name @rpath/libfoo.dylib
```
2) with `cc`
```
zig cc -shared foo.c -o libfoo.dylib -Wl,"-install_name=@rpath/libfoo.dylib"
```
Notating a symbol to be exported in code will only tell the linker
where to find this symbol, so other object files can find it. However, this does not mean
said symbol will also be exported to the host environment. Currently, we 'fix' this by force
exporting every single symbol that is visible. This creates bigger binaries and means host environments
have access to symbols that they perhaps shouldn't have. Now, users can tell Zig which symbols
are to be exported, meaning all other symbols that are not specified will not be exported.
Another change is we now support `-rdynamic` in the wasm linker as well, meaning all symbols will
be put in the dynamic symbol table. This is the same behavior as with ELF. This means there's a 3rd strategy
users will have to build their wasm binary.
Since we are already detecting the path to the native SDK,
if available, also fetch SDK's version and route that to the linker.
The linker can then use it to correctly populate LC_BUILD_VERSION
load command.
Due to a deficiency in LLD, we need to special-case BPF to a simple
file copy when generating relocatables. Normally, we would expect
`lld -r` to work. However, because LLD wants to resolve BPF relocations
which it shouldn't, it fails before even generating the relocatable.
Co-authored-by: Matthew Knight <mattnite@protonmail.com>
In 7e23b3245a9bf6e002009e6c18c10a9995671afa I made -O flags to the
linker emit a warning that the argument does nothing. That was not
correct however; LLD does have some logic that does different things
depending on -O0, -O1, and -O2. It defaults to -O1, and it does less
optimizations with -O0 and more with -O2.
With this commit, e.g. `-Wl,-O1` is supported by the `zig cc` frontend,
and by default we pass `-O0` to LLD in debug mode, and `-O3` in release
modes.
I also fixed a bug in the LLD ELF linker line which was incorrectly
passing `-O` flags instead of `--lto-O` flags for LTO.
* Improve the logic for determining whether emitting an import lib is
eligible, and improve the error message when the user provides
contradictory arguments.
* Integrate with the EmitLoc / Emit system that already exists, and use
the `-femit-implib[=path]`/`-fno-emit-implib` convention that already
exists.
* Proper integration with the caching system.
* CLI: fix bug in error reporting for resolving EmitLoc values for
other parameters.
This mechanism for sending arbitrary linker args to LLD has no place in
the Zig frontend, because our goal is for the frontend to understand all
the arguments and not treat linker args like a black box.
For example we have self-hosted linking in addition to LLD, so we want to
have the options make sense to both linking codepaths, not just the LLD one.
Passing -O linker args will now result in a warning that the arg does
nothing.
* Introduce a mechanism into Sema for emitting a compile error when an
integer is too big and we need it to fit into a usize.
* Add `@intCast` where necessary
* link/MachO: fix an unnecessary allocation when all that was happening
was appending zeroes to an ArrayList.
* Add `error.Overflow` as a possible error to some codepaths, allowing
usage of `math.intCast`.
closes#9710
After this change, the default for dynamic libraries (`-l` or
`--library`) is to only link them if they end up being actually used.
With the Zig CLI, the new options `-needed-l` or `--needed-library` can
be used to force link against a dynamic library.
With `zig cc`, this behavior can be overridden with `-Wl,--no-as-needed`
(and restored with `-Wl,--as-needed`).
Closes#10164
--import-memory import memory from the environment
--initial-memory=[bytes] initial size of the linear memory
--max-memory=[bytes] maximum size of the linear memory
--global-base=[addr] where to start to place global data
See #8633
Because ArrayList.initCapacity uses 'precise' capacity allocation, this should save memory on average, and definitely will save memory in cases where ArrayList is used where a regular allocated slice could have also be used.
Add an option to allow the '-z notext' option to be passed to the linker
via. the compiler frontend, which is a flag that tells the linker that
relocations in read-only sections are permitted. Certain targets such as
Solana BPF rely on this flag.
Expose all linker options i.e. '-z nodelete', '-z now', '-z relro' in
the compiler frontend. Usage documentation has been updated accordingly.
Expose the '-z notext' flag in the standard library build runner.
Each element of the output JSON has the VM address of the generated
binary nondecreasing (some elements might occupy the same VM address
for example the atom and the relocation might coincide in the address
space).
The generated JSON can be inspected manually or via a preview tool
`zig-snapshots` that I am currently working on and will allow the user
to inspect interactively the state of the linker together with the
positioning of sections, symbols, atoms and relocations within each
snapshot state, and in the future, between snapshots too. This should
allow for quicker debugging of the linker which is nontrivial when
run in the incremental mode.
Note that the state will only be dumped if the compiler is built with
`-Dlink-snapshot` flag on, and then the compiler is passed `--debug-link-snapshot`
flag upon compiling a source/project.
* do not add linkage scope to aliased exported symbols - this is
not respected on macOS
* special-case `MachO.openPath` in `link.File.openPath` as on macOS
we always link with zld
* redirect to `MachO.flushObject` when linking relocatable objects
in MachO linker whereas move the entire linking logic into
`MachO.flushModule`
* LLVM backend: respect `sub_path` just like the other stage2 backends
do.
* Compilation has some new logic to only emit work queue jobs for
building stuff when it believes itself to be capable. The linker
backends no longer have duplicate logic; instead they respect the
optional bit on the respective asset.
Previously, linker backends or machine code backends were able to hold
on to references to inside Sema's temporary arena. However there can
be large objects stored there that we want to free after machine code is
generated.
The primary change in this commit is to use a temporary arena for Sema
of function bodies that gets freed after machine code backend finishes
handling `updateFunc` (at the same time that Air and Liveness get freed).
The other changes in this commit are fixing issues that fell out from
the primary change.
* The C linker backend is rewritten to handle updateDecl and updateFunc
separately. Also, all Decl updates get access to typedefs and
fwd_decls, not only functions.
* The C linker backend is updated to the new API that does not depend
on allocateDeclIndexes and does not have to handle garbage collected
decls.
* The C linker backend uses an arena for Type/Value objects that
`typedefs` references. These can be garbage collected every so often
after flush(), however that garbage collection code is not
implemented at this time. It will be pretty simple, just allocate a
new arena, copy all the Type objects to it, update the keys of the
hash map, free the old arena.
* Sema: fix a handful of instances of not copying Type/Value objects
from the temporary arena into the appropriate Decl arena.
* Type: fix some function types not reporting hasCodeGenBits()
correctly.
* Added doc comments for `std.Target.ObjectFormat` enum
* `std.Target.oFileExt` is removed because it is incorrect for Plan-9
targets. Instead, use `std.Target.ObjectFormat.fileExt` and pass a
CPU architecture.
* Added `Compilation.Directory.joinZ` for when a null byte is desired.
* Improvements to `Compilation.create` logic for computing `use_llvm`
and reporting errors in contradictory flags. `-femit-llvm-ir` and
`-femit-llvm-bc` will now imply `-fLLVM`.
* Fix compilation when passing `.bc` files on the command line.
* Improvements to the stage2 LLVM backend:
- cleaned up error messages and error reporting. Properly bubble up
some errors rather than dumping to stderr; others turn into panics.
- properly call ZigLLVMCreateTargetMachine and
ZigLLVMTargetMachineEmitToFile and implement calculation of the
respective parameters (cpu features, code model, abi name, lto,
tsan, etc).
- LLVM module verification only runs in debug builds of the compiler
- use LLVMDumpModule rather than printToString because in the case
that we incorrectly pass a null pointer to LLVM it may crash during
dumping the module and having it partially printed is helpful in
this case.
- support -femit-asm, -fno-emit-bin, -femit-llvm-ir, -femit-llvm-bc
- Support LLVM backend when used with Mach-O and WASM linkers.
Portable Executable is an executable format, not an object format.
Everywhere in the entire zig codebase, we treated coff and pe as if they
were the same. Remove confusion by not including pe in the
std.Target.ObjectFormat enum.
to the link infrastructure, instead of being stored with Module.Fn. This
moves towards a strategy to make more efficient use of memory by not
storing Air or Liveness data in the Fn struct, but computing it on
demand, immediately sending it to the backend, and then immediately
freeing it.
Backends which want to defer codegen until flush() such as SPIR-V
must move the Air/Liveness data upon `updateFunc` being called and keep
track of that data in the backend implementation itself.
* rename files to adhere to conventions
* remove unnecessary function / optionality
* fix merge conflict
* better panic message
* remove unnecessary TODO comment
* proper namespacing of declarations
* clean up documentation comments
* no copyright header needed for a brand new zig file that is not
copied from anywhere
