878 Commits

Author SHA1 Message Date
Andrew Kelley
a72292513e add std.Thread.Pool.spawnWg
This function accepts a WaitGroup parameter and manages the reference
counting therein. It also is infallible.

The existing `spawn` function is still handy when the job wants to
further schedule more tasks.
2024-05-03 20:58:02 -07:00
Ryan Liptak
b86c4bde64 Rename Dir.writeFile2 -> Dir.writeFile and update all callsites
writeFile was deprecated in favor of writeFile2 in f645022d16361865e24582d28f1e62312fbc73bb. This commit renames writeFile2 to writeFile and makes writeFile2 a compile error.
2024-05-03 13:29:22 -07:00
Jacob Young
3b5be9fb6e lld: use a response file on NameTooLong 2024-05-03 13:27:04 -07:00
Nameless
aecd9cc6d1 std.posix.iovec: use .base and .len instead of .iov_base and .iov_len 2024-04-28 00:20:30 -07:00
Travis Staloch
8af59d1f98 ComptimeStringMap: return a regular struct and optimize
this patch renames ComptimeStringMap to StaticStringMap, makes it
accept only a single type parameter, and return a known struct type
instead of an anonymous struct.  initial motivation for these changes
was to reduce the 'very long type names' issue described here
https://github.com/ziglang/zig/pull/19682.

this breaks the previous API.  users will now need to write:
`const map = std.StaticStringMap(T).initComptime(kvs_list);`

* move `kvs_list` param from type param to an `initComptime()` param
* new public methods
  * `keys()`, `values()` helpers
  * `init(allocator)`, `deinit(allocator)` for runtime data
  * `getLongestPrefix(str)`, `getLongestPrefixIndex(str)` - i'm not sure
     these belong but have left in for now incase they are deemed useful
* performance notes:
  * i posted some benchmarking results here:
    https://github.com/travisstaloch/comptime-string-map-revised/issues/1
  * i noticed a speedup reducing the size of the struct from 48 to 32
    bytes and thus use u32s instead of usize for all length fields
  * i noticed speedup storing KVs as a struct of arrays
  * latest benchmark shows these wall_time improvements for
    debug/safe/small/fast builds: -6.6% / -10.2% / -19.1% / -8.9%. full
    output in link above.
2024-04-22 15:31:41 -07:00
Ian Johnson
4fac5bd601 Autodoc: fix root module name in sources.tar
This was overlooked in #19458. Using the fully qualified name of each
module usually makes sense, but there is one module where it does not,
namely, the root module, since its name is `root`. The original Autodoc
tar creation logic used `comp.root_name` for the root module back when
it was the only module included in `sources.tar`, and that made sense.
Now, we get the best of both worlds, using the proper root name for the
root module while using the module name for the rest.
2024-04-12 23:43:57 -07:00
Jacob Young
7611d90ba0 InternPool: remove slice from byte aggregate keys
This deletes a ton of lookups and avoids many UAF bugs.

Closes #19485
2024-04-08 13:24:08 -04:00
Jacob Young
5a41704f7e cbe: rewrite CType
Closes #14904
2024-03-30 20:50:48 -04:00
Jacob Young
6f10b11658 cbe: fix bugs revealed by an upcoming commit
Closes #18023
2024-03-30 20:50:48 -04:00
Andrew Kelley
3661133f98
Merge pull request #19399 from ypsvlq/mingw
mingw: support -municode
2024-03-28 14:16:30 -07:00
Ian Johnson
424dd17b6c Autodoc: add all modules to sources.tar
Closes #19403

This commit adds all modules in the compilation to the generated
`sources.tar` when using `-femit-docs` (including `std` and `builtin`).
Additionally, it considers the main module when doing so, rather than
the root module, so the behavior when running `zig test -femit-docs
test.zig` is now correct.
2024-03-28 00:30:29 -04:00
Elaine Gibson
add74427b9 mingw: support -municode 2024-03-27 10:05:57 +00:00
mlugg
26a94e8481
Zcu: eliminate Decl.alive field
Legacy anon decls now have three uses:
* Type owner decls
* Function owner decls
* `@export` and `@extern`

Therefore, there are no longer any cases where we wish to explicitly
omit legacy anon decls from the binary. This means we can remove the
concept of an "alive" vs "dead" `Decl`, which also allows us to remove
the separate `anon_work_queue` in `Compilation`.
2024-03-26 13:48:06 +00:00
mlugg
9c3670fc93
compiler: implement analysis-local comptime-mutable memory
This commit changes how we represent comptime-mutable memory
(`comptime var`) in the compiler in order to implement the intended
behavior that references to such memory can only exist at comptime.

It does *not* clean up the representation of mutable values, improve the
representation of comptime-known pointers, or fix the many bugs in the
comptime pointer access code. These will be future enhancements.

Comptime memory lives for the duration of a single Sema, and is not
permitted to escape that one analysis, either by becoming runtime-known
or by becoming comptime-known to other analyses. These restrictions mean
that we can represent comptime allocations not via Decl, but with state
local to Sema - specifically, the new `Sema.comptime_allocs` field. All
comptime-mutable allocations, as well as any comptime-known const allocs
containing references to such memory, live in here. This allows for
relatively fast checking of whether a value references any
comptime-mtuable memory, since we need only traverse values up to
pointers: pointers to Decls can never reference comptime-mutable memory,
and pointers into `Sema.comptime_allocs` always do.

This change exposed some faulty pointer access logic in `Value.zig`.
I've fixed the important cases, but there are some TODOs I've put in
which are definitely possible to hit with sufficiently esoteric code. I
plan to resolve these by auditing all direct accesses to pointers (most
of them ought to use Sema to perform the pointer access!), but for now
this is sufficient for all realistic code and to get tests passing.

This change eliminates `Zcu.tmp_hack_arena`, instead using the Sema
arena for comptime memory mutations, which is possible since comptime
memory is now local to the current Sema.

This change should allow `Decl` to store only an `InternPool.Index`
rather than a full-blown `ty: Type, val: Value`. This commit does not
perform this refactor.
2024-03-25 14:49:41 +00:00
Andrew Kelley
2f4bbd6c63 std.Build.Cache: use an array hash map for files
Rather than an ArrayList. Provides deduplication.
2024-03-21 19:56:30 -07:00
Elaine Gibson
19b6995939 mingw: define _WIN32_WINNT when building CRT 2024-03-20 15:02:04 -07:00
Andrew Kelley
cd62005f19 extract std.posix from std.os
closes #5019
2024-03-19 11:45:09 -07:00
Andrew Kelley
0c61466771
Merge pull request #19174 from squeek502/lazy-resinator
Lazily compile the `zig rc` subcommand and use it during `zig build-exe`, etc
2024-03-11 17:18:09 -07:00
Tristan Ross
9d70d614ae
std.builtin: make link mode fields lowercase 2024-03-11 07:09:10 -07:00
Ryan Liptak
c32e0d3000 Fix progress when multiple zig rc child processes are building resinator 2024-03-11 05:06:17 -07:00
Ryan Liptak
8799f7466d Report the progress of lazily building zig rc
jitCmd now takes a `server` option that will emit progress/errors via std.zig.Server when enabled.
2024-03-11 05:06:17 -07:00
Ryan Liptak
7c05330287 Integrate resinator errors with Zig using std.zig.Server and ErrorBundle
This takes the code that was previously in src/Compilation.zig to turn resinator diagnostics into Zig error bundles and puts it in resinator/main.zig, and then makes resinator emit the resulting error bundles via std.zig.Server (which is used by the build runner, etc). Also adds support for turning Aro diagnostics into ErrorBundles.
2024-03-11 05:06:16 -07:00
Ryan Liptak
52de2802c4 Lazily compile the zig rc subcommand and use it during zig build-exe
This moves .rc/.manifest compilation out of the main Zig binary, contributing towards #19063

Also:
- Make resinator use Aro as its preprocessor instead of clang
- Sync resinator with upstream
2024-03-11 05:06:16 -07:00
Andrew Kelley
1397341540 add missing field to module creation
Fixes a merge conflict with one of mlugg's recent branches.
2024-03-10 18:22:53 -07:00
Andrew Kelley
6b8c7540a8 fix 32 bit compilation 2024-03-10 17:51:07 -07:00
Andrew Kelley
8b70e4f2c9 autodocs: fix root name and missing dir close 2024-03-10 17:51:07 -07:00
Andrew Kelley
261094ca85 -femit-docs: create main.wasm artifact 2024-03-10 17:51:07 -07:00
Andrew Kelley
ffd53a459e -femit-docs: creating sources.tar
It's always a good day when you get to use File.writeFileAll 😎
2024-03-10 17:51:07 -07:00
Andrew Kelley
91260459e3 frontend: skeleton for creating autodocs
This time, unlike the old implementation, it properly does autodoc
generation during the compilation pipeline, saving time.
2024-03-10 17:51:06 -07:00
Andrew Kelley
356e653f9b better to use AST than ZIR for doc generation 2024-03-10 17:51:05 -07:00
Andrew Kelley
97aa5f7b8a
Merge pull request #19190 from mlugg/struct-equivalence
compiler: namespace type equivalence based on AST node + captures
2024-03-07 18:41:45 -08:00
Dillen Meijboom
377ecc6afb feat: add support for --enable-new-dtags and --disable-new-dtags 2024-03-06 17:52:05 -08:00
mlugg
b41a0b4768
Package.Module: deduplicate identical builtin modules
Previously, when multiple modules had builtin modules with identical
sources, two distinct `Module`s and `File`s were created pointing at the
same file path. This led to a bug later in the frontend. These modules
are now deduplicated with a simple hashmap on the builtin source.
2024-03-06 21:26:39 +00:00
Jacob Young
b60fc16b4f compiler: audit debug mode checks
* Introduce `-Ddebug-extensions` for enabling compiler debug helpers
 * Replace safety mode checks with `std.debug.runtime_safety`
 * Replace debugger helper checks with `!builtin.strip_debug_info`

Sometimes, you just have to debug optimized compilers...
2024-03-01 17:42:54 -08:00
Andrew Kelley
f7d095d372 use hash.addListOfBytes where applicable 2024-02-28 20:38:24 -07:00
Andrew Kelley
30688f33a4 add missing export symbol names to whole mode wasm cache hash
Fixes false positive cache hit.
2024-02-28 20:38:24 -07:00
Andrew Kelley
240d0b68f6 make aro-based translate-c lazily built from source
Part of #19063.

Primarily, this moves Aro from deps/ to lib/compiler/ so that it can be
lazily compiled from source. src/aro_translate_c.zig is moved to
lib/compiler/aro_translate_c.zig and some of Zig CLI logic moved to a
main() function there.

aro_translate_c.zig becomes the "common" import for clang-based
translate-c.

Not all of the compiler was able to be detangled from Aro, however, so
it still, for now, remains being compiled with the main compiler
sources due to the clang-based translate-c depending on it. Once
aro-based translate-c achieves feature parity with the clang-based
translate-c implementation, the clang-based one can be removed from Zig.

Aro made it unnecessarily difficult to depend on with these .def files
and all these Zig module requirements. I looked at the .def files and
made these observations:

- The canonical source is llvm .def files.
- Therefore there is an update process to sync with llvm that involves
  regenerating the .def files in Aro.
- Therefore you might as well just regenerate the .zig files directly
  and check those into Aro.
- Also with a small amount of tinkering, the file size on disk of these
  generated .zig files can be made many times smaller, without
  compromising type safety in the usage of the data.

This would make things much easier on Zig as downstream project,
particularly we could remove those pesky stubs when bootstrapping.

I have gone ahead with these changes since they unblock me and I will
have a chat with Vexu to see what he thinks.
2024-02-28 13:21:05 -07:00
Andrew Kelley
a4380a30f5 move zig libc command to be lazily built
part of #19063

This is a prerequisite for doing the same for Resinator.
2024-02-27 22:55:00 -07:00
Andrew Kelley
d661f0f35b compiler: JIT zig fmt
See #19063
2024-02-26 22:26:19 -07:00
Andrew Kelley
7b37bc771b move Zir to std.zig.Zir
Part of an effort to ship more of the compiler in source form.
2024-02-26 21:35:30 -07:00
Jacob Young
d656c2a7ab test: rework how filtering works
* make test names contain the fully qualified name
 * make test filters match the fully qualified name
 * allow multiple test filters, where a test is skipped if it does not
   match any of the specified filters
2024-02-25 19:12:08 -08:00
Jakub Konka
d12c8db642
Merge pull request #18875 from ziglang/macho-zo-dwarf
macho: emit DWARF for ZigObject relocatable
2024-02-09 23:12:04 +01:00
Andrew Kelley
54bbc73f85
Merge pull request #18712 from Vexu/std.options
std: make options a struct instance instead of a namespace
2024-02-09 13:38:42 -08:00
Jakub Konka
32386a06ca builtin: enable panic handler on self-hosted macho
comp: toggle compiler-rt and zig-libc caps for macho
2024-02-08 23:51:21 +01:00
Matthew Lugg
0c80725068
Merge pull request #18814 from mlugg/incremental-dependencies
Begin re-implementing incremental compilation
2024-02-06 11:33:07 +00:00
Andrew Kelley
78f15bc714 compiler: rename value.zig to Value.zig
This commit only does the file rename to be friendlier to version
control conflicts.
2024-02-05 18:13:07 -07:00
mlugg
0784d38984
compiler: lock incremental dependency tracking behind --debug-incremental
This logic (currently) has a non-trivial cost (particularly in terms of
peak RSS) for tracking dependencies. Until incremental compilation is in
use in the wild, it doesn't make sense for users to pay that cost.
2024-02-04 19:17:20 +00:00
mlugg
0d8207c292
Zcu: refactor Decl.analysis field
* Functions failing codegen now set this failure on the function
  analysis state. Decl analysis `codegen_failure` is reserved for
  failures generating constant values.

* `liveness_failure` is consolidated into `codegen_failure`, as we do
  not need to distinguish these, and Liveness.Verify is just a debugging
  feature anyway.

* `sema_failure_retryable` and `codegen_failure_retryable` are removed.
  Instead, retryable failures are recorded in the new
  `Zcu.retryable_failures` list. On an incremental update, this list is
  flushed, and all elements are marked as outdated so that we re-attempt
  analysis and code generation.

Also remove the `generation` fields from `Zcu` and `Decl` as these are
not needed by our new strategy for incremental updates.
2024-02-04 18:38:40 +00:00
mlugg
269c1ae649
Zcu: incremental compilation improvements
* Mark root Decls for re-analysis separately
* Check for re-analysis of root Decls
* Remove `outdated` entry when analyzing fn body
* Remove legacy `outdated` field from Decl analysis state
2024-02-04 18:38:40 +00:00
mlugg
a0004cebc2
Zcu: more dependency tracking logic
* Invalidate `decl_val` dependencies
* Recursively mark and un-mark all dependencies correctly
* Queue analysis of outdated dependers in `Compilation.performAllTheWork`

Introduces logic to invalidate `decl_val` dependencies after
`Zcu.semaDecl` completes. Also, recursively un-mark dependencies as PO
where needed.

With this, all dependency invalidation logic is in place. The next step
is analyzing outdated dependencies and triggering appropriate
re-analysis.
2024-02-04 18:38:39 +00:00