1023 Commits

Author SHA1 Message Date
Jacob Young
7611d90ba0 InternPool: remove slice from byte aggregate keys
This deletes a ton of lookups and avoids many UAF bugs.

Closes #19485
2024-04-08 13:24:08 -04:00
Jacob Young
3f1f05011e haiku: fix abi and dynamic linker detection 2024-04-08 13:20:14 -04:00
Jacob Young
fcdb7027e9 haiku: fix linking issues 2024-04-08 13:20:14 -04:00
Carl Åstholm
4c393c7468 Update usages of fmtId/isValidId
`{}` for decls
`{p}` for enum fields
`{p_}` for struct fields and in contexts following a `.`

Elsewhere, `{p}` was used since it's equivalent to the old behavior.
2024-04-07 14:47:11 +02:00
Jakub Konka
23f729aec9
Merge pull request #19260 from mikdusan/macos-zippered
macos: add zippered support
2024-04-05 23:37:51 +02:00
Jacob Young
eb723a4070 Update uses of @fieldParentPtr to use RLS 2024-03-30 20:50:48 -04:00
Jacob Young
17673dcd6e AstGen: use RLS to infer the first argument of @fieldParentPtr 2024-03-30 20:50:48 -04:00
Jacob Young
e409afb79b Update uses of @fieldParentPtr to pass a pointer type 2024-03-30 20:50:48 -04:00
Jacob Young
71d878ba50 std.zig: remove extraneous newlines in log messages 2024-03-27 01:55:42 -04:00
mlugg
9c3670fc93
compiler: implement analysis-local comptime-mutable memory
This commit changes how we represent comptime-mutable memory
(`comptime var`) in the compiler in order to implement the intended
behavior that references to such memory can only exist at comptime.

It does *not* clean up the representation of mutable values, improve the
representation of comptime-known pointers, or fix the many bugs in the
comptime pointer access code. These will be future enhancements.

Comptime memory lives for the duration of a single Sema, and is not
permitted to escape that one analysis, either by becoming runtime-known
or by becoming comptime-known to other analyses. These restrictions mean
that we can represent comptime allocations not via Decl, but with state
local to Sema - specifically, the new `Sema.comptime_allocs` field. All
comptime-mutable allocations, as well as any comptime-known const allocs
containing references to such memory, live in here. This allows for
relatively fast checking of whether a value references any
comptime-mtuable memory, since we need only traverse values up to
pointers: pointers to Decls can never reference comptime-mutable memory,
and pointers into `Sema.comptime_allocs` always do.

This change exposed some faulty pointer access logic in `Value.zig`.
I've fixed the important cases, but there are some TODOs I've put in
which are definitely possible to hit with sufficiently esoteric code. I
plan to resolve these by auditing all direct accesses to pointers (most
of them ought to use Sema to perform the pointer access!), but for now
this is sufficient for all realistic code and to get tests passing.

This change eliminates `Zcu.tmp_hack_arena`, instead using the Sema
arena for comptime memory mutations, which is possible since comptime
memory is now local to the current Sema.

This change should allow `Decl` to store only an `InternPool.Index`
rather than a full-blown `ty: Type, val: Value`. This commit does not
perform this refactor.
2024-03-25 14:49:41 +00:00
Andrew Kelley
12191c8a22 std: promote tests to doctests
Now these show up as "example usage" in generated documentation.
2024-03-21 14:11:46 -07:00
Andrew Kelley
cd62005f19 extract std.posix from std.os
closes #5019
2024-03-19 11:45:09 -07:00
Jacob Young
c11b6adf13 Ast: fix comptime destructure
A preceding `comptime` keyword was being ignored if the first
destructure variable was an expression.
2024-03-17 15:23:16 -07:00
Jacob Young
d10c52c194 AstGen: disallow alignment on function types
A pointer type already has an alignment, so this information does not
need to be duplicated on the function type.  This already has precedence
with addrspace which is already disallowed on function types for this
reason.  Also fixes `@TypeOf(&func)` to have the correct addrspace and
alignment.
2024-03-17 03:06:17 +01:00
Stephen Gregoratto
9532f72937 Windows: Replace CreatePipe with ntdll implementation
This implementation is now a direct replacement for the `kernel32` one.
New bitflags for named pipes and other generic ones were added based on
browsing the ReactOS sources.

`UNICODE_STRING.Buffer` has also been changed to be nullable, as
this is what makes the implementation work.
This required some changes to places accesssing the buffer after a
`SUCCESS`ful return, most notably `QueryObjectName` which even referred
to it being nullable.
2024-03-16 23:37:50 +11:00
mlugg
00969062a9
compiler: detect duplicate test names in AstGen
There is no reason to perform this detection during semantic analysis.
In fact, doing so is problematic, because we wish to utilize detection
of existing decls in a namespace in incremental compilation.
2024-03-14 07:40:05 +00:00
Michael Dusan
22fd1851bd
zig libc: allow non-native targets
On macos, allow targets supported by the SDK. This then spawns `xcrun`
and correct paths are emitted for:

- x86_64-macos
- x86_64-ios
- x86_64-tvos
- x86_64-watchos
- x86_64-ios-macbi
- aarch64-macos
- aarch64-ios
- aarch64-tvos
- aarch64-watchos
- aarch64-ios-macbi

On platforms with android NDK, allow android targets. Example usage:

```
CC=/NDK/.../bin/aarch64-linux-android34-clang zig libc -target aarch64-linux-android
```
2024-03-13 02:17:28 -04:00
Andrew Kelley
0c61466771
Merge pull request #19174 from squeek502/lazy-resinator
Lazily compile the `zig rc` subcommand and use it during `zig build-exe`, etc
2024-03-11 17:18:09 -07:00
Tristan Ross
aab84a3dec
std.builtin: make float mode fields lowercase 2024-03-11 07:09:10 -07:00
Tristan Ross
099f3c4039
std.builtin: make container layout fields lowercase 2024-03-11 07:09:07 -07:00
Ryan Liptak
b16890e6dd ErrorBundle: Fix potential writes to stale/freed memory 2024-03-11 05:06:17 -07:00
Ryan Liptak
dc4b05894d ErrorBundle: Add support for reference traces in addOtherSourceLocation
Also adds a test for addBundleAsRoots, which uses addOtherSourceLocation.
2024-03-11 05:06:17 -07:00
Andrew Kelley
97aa5f7b8a
Merge pull request #19190 from mlugg/struct-equivalence
compiler: namespace type equivalence based on AST node + captures
2024-03-07 18:41:45 -08:00
Jacob Young
aa7d16aba1 grammar: remove gratuitous ambiguity
Previously, the following matched both ContainerField alternatives:
 * [IDENTIFIER]
 * [IDENTIFIER][COLON][TypeExpr]
2024-03-06 13:59:46 -08:00
mlugg
2c4ac44f25
compiler: treat decl_val/decl_ref of potentially generic decls as captures
This fixes an issue with the implementation of #18816. Consider the
following code:

```zig
pub fn Wrap(comptime T: type) type {
    return struct {
        pub const T1 = T;
        inner: struct { x: T1 },
    };
}
```

Previously, the type of `inner` was not considered to be "capturing" any
value, as `T1` is a decl. However, since it is declared within a generic
function, this decl reference depends on the context, and thus should be
treated as a capture.

AstGen has been augmented to tunnel references to decls through closure
when the decl was declared in a potentially-generic context (i.e. within
a function).
2024-03-06 21:26:38 +00:00
mlugg
a6ca20b9a1
compiler: change representation of closures
This changes the representation of closures in Zir and Sema. Rather than
a pair of instructions `closure_capture` and `closure_get`, the system
now works as follows:

* Each ZIR type declaration (`struct_decl` etc) contains a list of
  captures in the form of ZIR indices (or, for efficiency, direct
  references to parent captures). This is an ordered list; indexes into
  it are used to refer to captured values.
* The `extended(closure_get)` ZIR instruction refers to a value in this
  list via a 16-bit index (limiting this index to 16 bits allows us to
  store this in `extended`).
* `Module.Namespace` has a new field `captures` which contains the list
  of values captured in a given namespace. This is initialized based on
  the ZIR capture list whenever a type declaration is analyzed.

This change eliminates `CaptureScope` from semantic analysis, which is a
nice simplification; but the main motivation here is that this change is
a prerequisite for #18816.
2024-03-06 21:26:37 +00:00
mlugg
6a87e42c2e
AstGen: fix latent bug causing incorrect elision of dbg_stmt instructions
Thanks to jacobly0 for figuring this out. The chain of events causing
the failure this triggered is as follows.

* As of a recent commit, certain bodies no longer emit a redundant
  `block`, meaning there are more likely to be "interesting"
  instructions (i.e. not blocks) at the end of parent GenZir scopes.

* When emitting the first `dbg_stmt` in such a body, the elision logic
  incorrectly looks at a tag from an instruction in an enclosing scope.

* The tag of this instruction may be `undefined`, meaning that in unsafe
  builds it may be incorrectly identified as a `dbg_stmt` instruction.

* This instruction from another body is clobbered rather than emitting
  an actual `dbg_stmt` instruction. Note that this does not produce
  invalid ZIR, since the creator of the undefined instruction replaces
  the previously-undefined payload later.
2024-03-01 23:54:31 +00:00
mlugg
eefa60e376
AstGen: optimize ZIR for -1 literal 2024-03-01 06:01:53 +00:00
mlugg
07d8740882
AstGen: do not generate defers at unreachable end of block
Resolves: #8822
2024-02-29 23:38:17 +00:00
mlugg
f6abf022b7
AstGen: elide block instruction when already in empty body
In the code `if (cond) { ... }`, the "then body" of the `if` is
technically a block. However, we don't need to emit a real ZIR `block`
corresponding to it, because we are already within a condbr body; we
have a separate gz, and appropriate scoping for allocs and debug
variables. In this case, and many like it, we can trivially elide the
block here, instead emitting the block statements directly into the
current `GenZir`. This results in a significant decrease in ZIR bytes
for real code.
2024-02-29 23:38:17 +00:00
mlugg
f0a4bb6bd1
AstGen: avoid unnecessary coercion instructions
Coercions such as `@as(usize, 0)` can be trivially elided by matching
these cases and translating to fixed InternPool indices.
2024-02-29 23:38:17 +00:00
Andrew Kelley
a4380a30f5 move zig libc command to be lazily built
part of #19063

This is a prerequisite for doing the same for Resinator.
2024-02-27 22:55:00 -07:00
Ryan Liptak
ae7f3fc360 Eliminate error.InvalidHandle from OpenError and RealPathError
InvalidHandle in OpenError is no longer a possible error on any platform. In the past it was able to be returned in `openOptionsFromFlagsWasi`, but the implementation was changed in 7680c5330cbc9141b9a5444e30c512b6068ab50d to make it no longer possible.

InvalidHandle in RealPathError was a holdover from before d5312d53a066092ba9efd687e25b29a87eb6290c, which made realpath a compile error on WASI. However, InvalidHandle was also a possible error in the FreeBSD fallback implementation added in 537624734c4db9e0cdbdc0ebce57375d17172a70. This commit changes the FreeBSD fallback implementation to return FileNotFound instead of InvalidHandle which matches how EBADF is handled in all the other `realpath` implementations (including the FreeBSD non-fallback implementation).

Closes #19084
2024-02-27 15:59:50 -08:00
Andrew Kelley
dfe430e9f4 move lazily compiled source files to lib/compiler/ 2024-02-26 23:43:56 -07:00
Andrew Kelley
0157e1196c compiler: JIT zig reduce
See #19063
2024-02-26 23:43:42 -07:00
Andrew Kelley
d661f0f35b compiler: JIT zig fmt
See #19063
2024-02-26 22:26:19 -07:00
Andrew Kelley
b116063e02 move AstGen to std.zig.AstGen
Part of an effort to ship more of the compiler in source form.
2024-02-26 21:51:19 -07:00
Andrew Kelley
a2e87aba66 rearrange std.zig
This frees up std.zig.fmt to be used for the implementation of `zig
fmt`.
2024-02-26 21:35:33 -07:00
Andrew Kelley
7b37bc771b move Zir to std.zig.Zir
Part of an effort to ship more of the compiler in source form.
2024-02-26 21:35:30 -07:00
Jacob Young
2fdc9e6ae8 x86_64: implement @shuffle 2024-02-25 11:22:10 +01:00
Ryan Liptak
68b87918df Fix handling of Windows (WTF-16) and WASI (UTF-8) paths
Windows paths now use WTF-16 <-> WTF-8 conversion everywhere, which is lossless. Previously, conversion of ill-formed UTF-16 paths would either fail or invoke illegal behavior.

WASI paths must be valid UTF-8, and the relevant function calls have been updated to handle the possibility of failure due to paths not being encoded/encodable as valid UTF-8.

Closes #18694
Closes #1774
Closes #2565
2024-02-24 14:05:24 -08:00
Ryan Liptak
80508b98c2 Update deprecated std.unicode function usages 2024-02-24 14:04:59 -08:00
Andrew Kelley
9812bc7b10 std: map NETNAME_DELETED to error.ConnectionResetByPeer
This was observed in writing to a network stream that was closed on the
read end.
2024-02-24 04:16:53 -07:00
Tristan Ross
5c25ad0fda std.zig.system.linux: detect risc-v 2024-02-19 10:12:17 +02:00
Carl Åstholm
3cafb9655a zig fmt: Preserve trailing comma after single-item switch case 2024-02-18 12:36:04 +02:00
Andrew Kelley
07c1dd3d1d std.os.windows.OpenFile: add missing error
Encountered in a recent CI run on an aarch64-windows dev kit.

Pretty sure I disabled the virus scanner but it looks like it turned
itself back on with a Windows Update.

Rather than marking the new error code as unreachable in the places
where it is unexpected, this commit makes it return `error.Unexpected`.
2024-02-14 11:14:43 -08:00
Veikka Tuominen
a4f27e8987 remove std.io.Mode 2024-02-01 15:22:36 +02:00
Pavel Verigo
87733171b6 parser: fix "else" followed by "comptime" 2024-01-27 16:20:47 +02:00
Andrew Kelley
f37513aa68 NativePaths: support -L and -l arguments in NIX_LDFLAGS
A slightly more robust parsing of these flags in case of a separate
argument appearing after "-L" in the flags.
2024-01-23 18:01:59 -07:00
Winter
6f3ee587ba std.zig.system.NativePaths: ignore linkage directives in NIX_LDFLAGS
`NIX_LDFLAGS` typically contains just `-rpath` and `-L`, which we already
handle. However, at least one setup hook in Nixpkgs [0] adds a linkage
directive to it. To prevent library paths from being missed (as I've
observed myself with `NIX_LDFLAGS` being `-liconv ...`, making it so that
*all* paths are missed), let's just skip over them.

[0]: 08f615eb1b/pkgs/development/libraries/libiconv/setup-hook.sh
2024-01-23 17:49:25 -07:00