1761 Commits

Author SHA1 Message Date
Luuk de Gram
cec1e973b6
Merge pull request #16439 from Luukdegram/wasm-linker
wasm-linker: finish shared-memory & TLS implementation
2023-07-19 21:55:30 +02:00
Luuk de Gram
142dbc7b82
wasm-linker: create TLS Wasm globals correctly
Previously, they were only created when we had any TLS segment.
This meant that while the symbol existed, the global itself wouldn't.
The result of this was a crash during symbol names writing as it
would attempt to write the symbol name of a global that didn't exist.
Now we always create them, and instead update its `init` value during
`setupMemory`.

In the future, the entire symbol (and global) will be removed by
the garbage collector.
2023-07-19 17:22:46 +02:00
Luuk de Gram
3fd6e93f4f
wasm-linker: prevent double-free on parse failure 2023-07-19 17:22:46 +02:00
Luuk de Gram
2672f7d9e8
wasm-linker: shared-memory fixes
Implements the `start` section which will execute a given function
at startup of the program. After function execution, the _start
function will be called by the runtime. In the case of shared-memory
we set this section to the function `__wasm_init_memory` which will
initialize all memory on startup.

This also fixes the above mentioned function to ensure we correctly
lower the i32 values.

Lastly, this fixes a typo where we would retrieve a global, instead
of setting its value.
2023-07-19 17:22:46 +02:00
Luuk de Gram
376e1b4603
wasm-linker: implement TLS relocations 2023-07-19 17:22:46 +02:00
Luuk de Gram
388589987c
wasm-linker: allow explicit memory exports
Rather than verifying if importing memory is false, we now rely
on the option that was passed to the CLI (where export is defaulted
to `true` unless only import-memory is given).
2023-07-19 17:22:45 +02:00
Andrew Kelley
3f2a4720b1 compiler: fix branch regressions
* getOwnedFunctionIndex no longer checks if the value is actually a
   function.
 * The callsites to `intern` that I added want to avoid the `getCoerced`
   call, so I added `intern2`.
 * Adding to inferred error sets should not happen if the destination
   error set is not the inferred error set of the current Sema instance.
 * adhoc_inferred_error_set_type can be seen by the backend. Treat it
   like anyerror.
2023-07-18 19:02:06 -07:00
Andrew Kelley
f3dc53f6b5 compiler: rework inferred error sets
* move inferred error sets into InternPool.
   - they are now represented by pointing directly at the corresponding
     function body value.
 * inferred error set working memory is now in Sema and expires after
   the Sema for the function corresponding to the inferred error set is
   finished having its body analyzed.
 * error sets use a InternPool.Index.Slice rather than an actual slice
   to avoid lifetime issues.
2023-07-18 19:02:05 -07:00
Andrew Kelley
db33ee45b7 rework generic function calls
Abridged summary:

 * Move `Module.Fn` into `InternPool`.
 * Delete a lot of confusing and problematic `Sema` logic related to
   generic function calls.

This commit removes `Module.Fn` and replaces it with two new
`InternPool.Tag` values:

 * `func_decl` - corresponding to a function declared in the source
   code. This one contains line/column numbers, zir_body_inst, etc.

 * `func_instance` - one for each monomorphization of a generic
   function. Contains a reference to the `func_decl` from whence the
   instantiation came, along with the `comptime` parameter values (or
   types in the case of `anytype`)

Since `InternPool` provides deduplication on these values, these fields
are now deleted from `Module`:

 * `monomorphed_func_keys`
 * `monomorphed_funcs`
 * `align_stack_fns`

Instead of these, Sema logic for generic function instantiation now
unconditionally evaluates the function prototype expression for every
generic callsite. This is technically required in order for type
coercions to work. The previous code had some dubious, probably wrong
hacks to make things work, such as `hashUncoerced`. I'm not 100% sure
how we were able to eliminate that function and still pass all the
behavior tests, but I'm pretty sure things were still broken without
doing type coercion for every generic function call argument.

After the function prototype is evaluated, it produces a deduplicated
`func_instance` `InternPool.Index` which can then be used for the
generic function call.

Some other nice things made by this simplification are the removal of
`comptime_args_fn_inst` and `preallocated_new_func` from `Sema`, and the
messy logic associated with them.

I have not yet been able to measure the perf of this against master
branch. On one hand, it reduces memory usage and pointer chasing of the
most heavily used `InternPool` Tag - function bodies - but on the other
hand, it does evaluate function prototype expressions more than before.
We will soon find out.
2023-07-18 19:02:05 -07:00
Casey Banner
094cd92615
macho: fix __unwind_info sentinel entry not always being the upper bound (#16395)
macho: record highest address of unwind records before folding

---------

Co-authored-by: Jakub Konka <kubkon@jakubkonka.com>
2023-07-14 11:47:00 +00:00
Xavier Bouchoux
7a8002a5cf remove arbitrary stderr size limit when spawning a child process tool 2023-07-11 00:17:08 -07:00
Luuk de Gram
3819371163
wasm-ld: implement --export-memory flag
This flag allows the user to force export the memory to the host
environment. This is useful when the memory is imported from the
host but must also be exported. This is (currently) required
to pass the memory validation for runtimes when using threads.
In this future this may become an error instead.
2023-06-26 20:00:57 +02:00
Jakub Konka
852eb272bf macho: add fixes to __eh_frame parsing emitted by Nix C++ compiler 2023-06-25 14:33:00 -07:00
mlugg
f26dda2117 all: migrate code to new cast builtin syntax
Most of this migration was performed automatically with `zig fmt`. There
were a few exceptions which I had to manually fix:

* `@alignCast` and `@addrSpaceCast` cannot be automatically rewritten
* `@truncate`'s fixup is incorrect for vectors
* Test cases are not formatted, and their error locations change
2023-06-24 16:56:39 -07:00
Andrew Kelley
12813d5912
Merge pull request #16105 from jacobly0/intern-pool-opt
InternPool: various optimizations
2023-06-21 00:07:49 -07:00
Jakub Konka
d2b2567119 macho: do not skip scanning relocs if the sym is non-extern 2023-06-21 00:31:01 +02:00
Jakub Konka
7b5bd3a93f
Merge pull request #16097 from ziglang/macho-faster-uuid
macho: parallelize UUID hash calculation at the expense of full compatibility with ld64
2023-06-21 00:03:52 +02:00
Jacob Young
96cdd51c14 Type: delete legacy allocation functions 2023-06-20 14:02:09 -04:00
Jacob Young
51cbb081e9 wasm: fix decl alignment 2023-06-20 11:36:50 -04:00
Jakub Konka
eb1050b83a macho: fix 32bit compilation issues 2023-06-20 10:08:54 +02:00
Jakub Konka
22540e5402 macho: exclude code signature padding from uuid calculation 2023-06-19 22:50:08 +02:00
Eric Joldasov
50339f595a all: zig fmt and rename "@XToY" to "@YFromX"
Signed-off-by: Eric Joldasov <bratishkaerik@getgoogleoff.me>
2023-06-19 12:34:42 -07:00
Jakub Konka
ef9d6331fc macho: clean up hasher interface 2023-06-19 20:33:27 +02:00
Jakub Konka
8087c134db macho: calculate UUID chunk size based on available thread count 2023-06-19 12:53:26 +02:00
Jakub Konka
10aaf2983d macho: hash the entire file contents for UUID but calc in parallel 2023-06-19 11:26:56 +02:00
Jakub Konka
b3a2ab3fed macho: extract parallel hasher into a generic helper struct 2023-06-19 10:29:39 +02:00
Andrew Kelley
e23d48e61a
Merge pull request #15511 from mikdusan/netbsd-pkgsrc
netbsd: pkgsrc
2023-06-17 14:22:43 -07:00
r00ster91
6e84f46990 std: replace builtin.Version with SemanticVersion 2023-06-17 13:17:34 -07:00
Motiejus Jakštys
d41111d7ef mem: rename align*Generic to mem.align*
Anecdote 1: The generic version is way more popular than the non-generic
one in Zig codebase:

     git grep -w alignForward | wc -l
    56
     git grep -w alignForwardGeneric | wc -l
    149

     git grep -w alignBackward | wc -l
    6
     git grep -w alignBackwardGeneric | wc -l
    15

Anecdote 2: In my project (turbonss) that does much arithmetic and
alignment I exclusively use the Generic functions.

Anecdote 3: we used only the Generic versions in the Macho Man's linker
workshop.
2023-06-17 12:49:13 -07:00
Andrew Kelley
a10ddba921
Merge pull request #16064 from Luukdegram/wasm-linker
wasm/linker: symbol resolution improvements
2023-06-16 22:03:35 -07:00
Andrew Kelley
78c8cb4225
Merge pull request #16003 from g-w1/plan9-lazy-syms
Plan9: lots of fixes
2023-06-16 21:57:33 -07:00
r00ster91
2593156068 migration: std.math.{min, min3, max, max3} -> @min & @max 2023-06-16 13:44:09 -07:00
Luuk de Gram
44b322ce64
wasm-linker: correctly resolve undefined functions
We now resolve undefined symbols during incremental-compilation
where we discard the current symbol if we detect we found
an existing symbol which is not the one currently being updated.
The symbol will always be discarded in favor of the existing symbol
in such a case.
2023-06-16 17:16:56 +02:00
Luuk de Gram
729f822e31
wasm-linker: correctly resolve exported symbols
When compiling Zig code using the Wasm backend, we would previously
incorrectly resolve exported symbols as it would not correctly remove
existing symbols if they were to be overwritten. This meant that
undefined symbols could cause collisions although they should be
resolved by the exported symbol.
2023-06-16 17:16:55 +02:00
Jacob G-W
5343a2f566 plan9: revamp the relocation system to allow decl refs 2023-06-16 08:34:30 -04:00
Jacob G-W
153def1460 Elf: fix memory leak with lazy symbols 2023-06-16 08:34:30 -04:00
Jacob G-W
9e8c7b104e Plan9: Add support for lazy symbols
This includes a renaming from DeclBlock to Atom.
2023-06-16 08:34:30 -04:00
Andrew Kelley
2ad073ec6d link/Plan9: fix UAF of symbol names
Long term, linker backends will need to manage their own string tables
for things like this because my mandate is: no long-lived pointers
allowed in any of the codepaths touched by incremental compilation, so
that we can serialize and deserialize trivially.

Short term, I solved this with a couple calls to Allocator.dupe,
incurring some harmless leaks.
2023-06-12 19:09:30 -07:00
Jacob Young
d37ebfcf23 InternPool: avoid as many slices pointing to string_bytes as possible
These are frequently invalidated whenever a string is interned, so avoid
creating pointers to `string_bytes` wherever possible.  This is an
attempt to fix random CI failures.
2023-06-11 23:45:09 -07:00
Jacob Young
7507a76879 link: use Wasm.string_table offsets for Wasm.undefs keys
This avoids having dangling pointers into `InternPool.string_bytes`.
2023-06-11 03:01:31 -07:00
Andrew Kelley
69b7b91092 compiler: eliminate Decl.value_arena and Sema.perm_arena
The main motivation for this commit is eliminating Decl.value_arena.
Everything else is dominoes.

Decl.name used to be stored in the GPA, now it is stored in InternPool.
It ended up being simpler to migrate other strings to be interned as
well, such as struct field names, union field names, and a few others.
This ended up requiring a big diff, sorry about that. But the changes
are pretty nice, we finally start to take advantage of InternPool's
existence.

global_error_set and error_name_list are simplified. Now it is a single
ArrayHashMap(NullTerminatedString, void) and the index is the error tag
value.

Module.tmp_hack_arena is re-introduced (it was removed in
eeff407941560ce8eb5b737b2436dfa93cfd3a0c) in order to deal with
comptime_args, optimized_order, and struct and union fields. After
structs and unions get moved into InternPool properly, tmp_hack_arena
can be deleted again.
2023-06-10 20:47:58 -07:00
Jacob Young
9cd0ca9f48 Module: rename functions to make ownership checks explicit
This makes the difference between `decl.getOwnedFunction` and
`decl.val.getFunction` more clear when reading the code.
2023-06-10 20:47:55 -07:00
Jacob Young
6e0de1d116 InternPool: port most of value tags 2023-06-10 20:47:54 -07:00
Andrew Kelley
7e19c95668 Sema: move inferred_alloc_const/mut_type to InternPool
Now, all types are migrated to use `InternPool`. The `Type.Tag` enum is
deleted in this commit.
2023-06-10 20:47:54 -07:00
Andrew Kelley
9ff514b6a3 compiler: move error union types and error set types to InternPool
One change worth noting in this commit is that `module.global_error_set`
is no longer kept strictly up-to-date. The previous code reserved
integer error values when dealing with error set types, but this is no
longer needed because the integer values are not needed for semantic
analysis unless `@errorToInt` or `@intToError` are used and therefore
may be assigned lazily.
2023-06-10 20:47:53 -07:00
Andrew Kelley
7bf91fc79a compiler: eliminate legacy Type.Tag.pointer
Now pointer types are stored only in InternPool.
2023-06-10 20:47:53 -07:00
Andrew Kelley
17882162b3 stage2: move function types to InternPool 2023-06-10 20:47:53 -07:00
Andrew Kelley
d18881de1b stage2: move anon tuples and anon structs to InternPool 2023-06-10 20:47:52 -07:00
Andrew Kelley
88dbd62bcb stage2: move enum tag values into the InternPool
I'm seeing a new assertion trip: the call to `enumTagFieldIndex` in the
implementation of `@Type` is attempting to query the field index of an
union's enum tag, but the type of the enum tag value provided is not the
same as the union's tag type. Most likely this is a problem with type
coercion, since values are now typed.

Another problem is that I added some hacks in std.builtin because I
didn't see any convenient way to access them from Sema. That should
definitely be cleaned up before merging this branch.
2023-06-10 20:46:17 -07:00
Andrew Kelley
5881a2d637 stage2: move enum types into the InternPool
Unlike unions and structs, enums are actually *encoded* into the
InternPool directly, rather than using the SegmentedList trick. This
results in them being quite compact, and greatly improved the ergonomics
of using enum types throughout the compiler.

It did however require introducing a new concept to the InternPool which
is an "incomplete" item - something that is added to gain a permanent
Index, but which is then mutated in place. This was necessary because
enum tag values and tag types may reference the namespaces created by
the enum itself, which required constructing the namespace, decl, and
calling analyzeDecl on the decl, which required the decl value, which
required the enum type, which required an InternPool index to be
assigned and for it to be meaningful.

The API for updating enums in place turned out to be quite slick and
efficient - the methods directly populate pre-allocated arrays and
return the information necessary to output the same compilation errors
as before.
2023-06-10 20:42:30 -07:00