AstGen:
* rename the known_has_bits flag to known_non_opv to make it better
reflect what it actually means.
* add a known_comptime_only flag.
* make the flags take advantage of identifiers of primitives and the
fact that zig has no shadowing.
* correct the known_non_opv flag for function bodies.
Sema:
* Rename `hasCodeGenBits` to `hasRuntimeBits` to better reflect what it
does.
- This function got a bit more complicated in this commit because of
the duality of function bodies: on one hand they have runtime bits,
but on the other hand they require being comptime known.
* WipAnonDecl now takes a LazySrcDecl parameter and performs the type
resolutions that it needs during finish().
* Implement comptime `@ptrToInt`.
Codegen:
* Improved handling of lowering decl_ref; make it work for
comptime-known ptr-to-int values.
- This same change had to be made many different times; perhaps we
should look into merging the implementations of `genTypedValue`
across x86, arm, aarch64, and riscv.
There are some restrictions here.
- We either need C11 or a compiler that supports the aligned attribute
- We cannot provide align less than the type's natural C alignment.
If there is a big atom available for re-use in the free list, and
it's the last atom in section, it's ideal capacity might span the
entire section in which case we do not want to calculate the actual
end VM addr of the symbol since it may overflow. Instead, we just take
the max capacity available as end VM addr estimate. In this case,
the max capacity equals `std.math.maxInt(u64)`.
Prior to this change, `__DATA,__bss` and `__DATA,__thread_bss` would
get actually, physically written out to the output file, unnecessarily
filling the output file with 0s.
When a constant will be passed by reference, such as a struct, we will call into genTypedValue
to lower the constant to bytes and store them into the `rodata` section. We will then return the address
of this constant as a `WValue`.
This change means we will have all constants lowered during compilation time, and no longer have
to sacrifice runtime to lower them onto the stack.
This allows us to get rid of unused fields when generating code for non-function decls.
We can now create seperate instances of `DeclGen` which in turn can then be used
to generate the code for a constant.
Besides those reasons, it will be much easier to switch to the generic purpose `codegen.zig` that any
backend should use. Allowing us to deduplicate this code.
The backend can create annonymous local symbols. This can be used for constants
that will be passed by reference so it will not have to be lowered to the stack, and then
stored into the data section. This also means it's valid to return a pointer to a constant array.
Those local symbols that are created, will be managed by the parent decl. Free'ing the parent decl,
will also free all of its locals.
When a local symbol was created, the index of said symbol will be returned and saved in the `memory`
tag of a `WValue` which is then memoized. This means that each 'emit' of this WValue will create a relocation
for that constant/symbol and the actual pointer value will be set after relocation phase.
* rename `entry` to `entry_symbol_name` for the zig build API
* integrate with `zig cc` command line options
* integrate with COFF linking with LLD
* integrate with self-hosted ELF linker
* don't put it in the hash for MachO since it is ignored
This reverts commit d48e4245b68bf25c7f41804a5012ac157a5ee546.
I have no idea why this is failing Drone CI, but in a branch, reverting
this commit solved the problem.
Handle `__DATA,.rustc` section containing `rustc` metadata - this
is required to get crates like `serde_derive` link properly.
Note to self: this special section has to be copied __verbatim__
from the relocatable object file - this includes preserving its size
even though unpadded according the section's required alignment.
By placing the stack at the start of the memory section, we prevent the runtime
from silently overwriting the global declarations and instead trap.
We do however, allow users to overwrite this behavior by setting the global-base,
which puts the stack at the end of the memory section and the static data at the base that was specified.
The reason a user would want to do this, is when they are sure the stack will not overflow and they want
to decrease the binary size as the offsets to the static memory are generally smaller.
(Having the stack in front, means that accessing the memory after the stack has a bigger offset when loading/storing from memory).
* stage2: put decls in different MachO sections
Use `getDeclVAddrWithReloc` when targeting MachO backend rather than
`getDeclVAddr` - this fn returns a zero vaddr and instead creates a
relocation on the linker side which will get automatically updated
whenever the target decl is moved in memory. This fn also records
a rebase of the target pointer so that its value is correctly slid
in presence of ASLR.
This commit enables `zig test` on x86_64-macos.
* stage2: fix output section selection for type,val pairs
This exposes a function from stage2 to stage1 to append symbols to automatically export them.
This happends under the following conditions:
- Target is wasm
- User has not provided --export/--rdynamic flags themselves.
This allows the `acid` debugger on
plan9 to be used to debug a zig source
file without patching `acid`!
The patch adds a second `z` symbol. This z
symbol has a value of 0, which means that it
pops the history stack. We put a very large
number for the value of the second symbol because
it has to be at least as large as the linecount of
the file. The debuginfo format is meant to be used
with c files, where the stack would look something
like this:
```
-> Line: 0x1 (1) Name: 0x1/0x2/0x3/0xe/0x13/0x1b (/sys/src/libc/port/malloc.c)
-> Line: 0x2 (2) Name: 0x1/0x6/0x7/0x8 (/amd64/include/u.h)
-> Line: 0x4f (79) Name: ()
-> Line: 0x50 (80) Name: 0x1/0x2/0x7/0x9 (/sys/include/libc.h)
-> Line: 0x358 (856) Name: ()
-> Line: 0x359 (857) Name: 0x1/0x2/0x7/0x1c (/sys/include/pool.h)
-> Line: 0x392 (914) Name: ()
-> Line: 0x393 (915) Name: 0x1/0x2/0x7/0x1d (/sys/include/tos.h)
-> Line: 0x3ab (939) Name: ()
-> Line: 0x4eb (1259) Name: ()
```
however in zig, we do not use includes and .h files,
so we only need the first and last items in the stack:
the source file that the symbols belong to, and the pop
symbol with a null name and a value of the total linecount of the
preprocessed source. Since there is no preprocessing in zig, we
just make the linecount very large. There do not appear to be
any downsides to this approach. If this causes a bug in the future,
a simple fix would be to make the pop symbol just have the value
of how many newlines are in the source file.
This commit fixes two problems:
* `zig build-obj` regressed from the cache-mode branch. It would crash
because it assumed that dirname on the emit bin path would not be
null. This assumption was invalid when outputting to the current
working directory - a pretty common use case for `zig build-obj`.
* When using the LLVM backend, `-fno-emit-bin` combined with any other
kind of emitting, such as `-femit-asm`, emitted nothing.
Both issues are now fixed.
Doc comments reproduced here:
This function is called by the frontend before flush(). It communicates that
`options.bin_file.emit` directory needs to be renamed from
`[zig-cache]/tmp/[random]` to `[zig-cache]/o/[digest]`.
The frontend would like to simply perform a file system rename, however,
some linker backends care about the file paths of the objects they are linking.
So this function call tells linker backends to rename the paths of object files
to observe the new directory path.
Linker backends which do not have this requirement can fall back to the simple
implementation at the bottom of this function.
This function is only called when CacheMode is `whole`.
This solves stack trace regressions on Windows and macOS because the
linker backends do not observe object file paths until flush().
The two CacheMode values are `whole` and `incremental`.
`incremental` is what we had before; `whole` is new.
Whole cache mode uses everything as inputs to the cache hash;
and when a hit occurs it skips everything including linking.
This is ideal for when source files change rarely and for backends that
do not have good incremental compilation support, for example
compiler-rt or libc compiled with LLVM with optimizations on.
This is the main motivation for the additional mode, so that we can have
LLVM-optimized compiler-rt/libc builds, without waiting for the LLVM
backend every single time Zig is invoked.
Incremental cache mode hashes only the input file path and a few target
options, intentionally relying on collisions to locate already-existing
build artifacts which can then be incrementally updated.
The bespoke logic for caching stage1 backend build artifacts
is removed since we now have a global caching mechanism for
when we want to cache the entire compilation, *including* linking.
Previously we had to get "creative" with libs.txt and a special
byte in the hash id to communicate flags, so that when the cached
artifacts were re-linked, we had this information from stage1
even though we didn't actually run it. Now that `CacheMode.whole`
includes linking, this extra information does not need to be
preserved for cache hits. So although this changeset introduces
complexity, it also removes complexity.
The main trickiness here comes from the inherent differences between the
two modes: `incremental` wants a directory immediately to operate on,
while `whole` doesn't know the output directory until the compilation is
complete. This commit deals with this problem mostly inside `update()`,
where, on a cache miss, it replaces `zig_cache_artifact_directory` with a
temporary directory, and then renames it into place once the compilation is
complete.
Items remaining before this branch can be merged:
* [ ] make sure these things make it into the cache manifest:
- @import files
- @embedFile files
- we already add dep files from c but make sure the main .c files make
it in there too, not just the included files
* [ ] double check that the emit paths of other things besides the binary
are working correctly.
* [ ] test `-fno-emit-bin` + `-fstage1`
* [ ] test `-femit-bin=foo` + `-fstage1`
* [ ] implib emit directory copies bin_file_emit directory in create() and needs
to be adjusted to be overridden as well.
* [ ] make sure emit-h is handled correctly in the cache hash
* [ ] Cache: detect duplicate files added to the manifest
Some preliminary performance measurements of wall clock time and
peak RSS used:
stage1 behavior (1077 tests), llvm backend, release build:
* cold global cache: 4.6s, 1.1 GiB
* warm global cache: 3.4s, 980 MiB
stage2 master branch behavior (575 tests), llvm backend, release build:
* cold global cache: 0.62s, 191 MiB
* warm global cache: 0.40s, 128 MiB
stage2 this branch behavior (575 tests), llvm backend, release build:
* cold global cache: 0.62s, 179 MiB
* warm global cache: 0.27s, 90 MiB
- Previously the table index and function type index were switched.
This commit swaps them.
- This also emits the correct indirect function calls count when importing the function table
- Add method to easily create local for virtual stack
- Ensure function pointers are passed correctly
- Correctly handle slices as return types and values
- Fix wrapping error sets/payloads.
- Handle ptr-like optionals correctly, by using address '0' as null.
- Implement `array_to_slice`
- linker: Always emit a table, so call_indirect inside bodies do not fail if there's no table.
TODO: Only do this when we emit a call_indirect but the relocation cannot be resolved.
The linker will now emit names for all function, global and data segment symbols.
This increases the ability to debug wasm modules tremendously as tools like wasm2wat
can use this information to generate named functions, globals etc, rather than placeholders such as $f1.
Allocate a new program header and a new section to accomodate the read-only data
section ".rodata".
Separate TextBlock into multiple TextBlockList, to separate decl in different
sections.
If a Decl is not a function, it is added to the .rodata section.
The status quo for the `build.zig` build system is preserved in
the sense that, if the user does not explicitly override
`dylib.setInstallName(...);` in their build script, the default
of `@rpath/libname.dylib` applies. However, should they want to
override the default behaviour, they can either:
1) unset it with
```dylib.setIntallName(null);```
2) set it to an explicit string with
```dylib.setInstallName("somename.dylib");```
When it comes to the command line however, the default is not to
use `@rpath` for the install name when creating a dylib. The user
will now be required to explicitly specify the `@rpath` as part
of the desired install name should they choose so like so:
1) with `build-lib`
```
zig build-lib -dynamic foo.zig -install_name @rpath/libfoo.dylib
```
2) with `cc`
```
zig cc -shared foo.c -o libfoo.dylib -Wl,"-install_name=@rpath/libfoo.dylib"
```
Notating a symbol to be exported in code will only tell the linker
where to find this symbol, so other object files can find it. However, this does not mean
said symbol will also be exported to the host environment. Currently, we 'fix' this by force
exporting every single symbol that is visible. This creates bigger binaries and means host environments
have access to symbols that they perhaps shouldn't have. Now, users can tell Zig which symbols
are to be exported, meaning all other symbols that are not specified will not be exported.
Another change is we now support `-rdynamic` in the wasm linker as well, meaning all symbols will
be put in the dynamic symbol table. This is the same behavior as with ELF. This means there's a 3rd strategy
users will have to build their wasm binary.
This way, we will inform the user that there are unresolved symbols
in addition to missing library/framework as requested on the linker
line. If all symbols were resolved on the other hand, we still
flag up that the library/framework cannot be found.
Example behaviour:
```
$ zig cc hello.c -framework MyFoundation --verbose
warning(link): framework not found for '-framework MyFoundation'
warning(link): Framework search paths:
warning(link): /Library/Frameworks
warning(link): /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/System/Library/Frameworks
thread 1079397 panic: attempt to unwrap error: FrameworkNotFound
...stack trace...
```
and
```
❯ zig cc hello.c -lWAT --verbose
warning(link): library not found for '-lWAT'
warning(link): Library search paths:
warning(link): /Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/lib
warning(link): /usr/local/lib
thread 1079824 panic: attempt to unwrap error: LibraryNotFound
...stack trace...
```
