mirror of
https://github.com/ziglang/zig.git
synced 2025-12-06 14:23:09 +00:00
This commit makes some big changes to how we track state for Zig source files. In particular, it changes: * How `File` tracks its path on-disk * How AstGen discovers files * How file-level errors are tracked * How `builtin.zig` files and modules are created The original motivation here was to address incremental compilation bugs with the handling of files, such as #22696. To fix this, a few changes are necessary. Just like declarations may become unreferenced on an incremental update, meaning we suppress analysis errors associated with them, it is also possible for all imports of a file to be removed on an incremental update, in which case file-level errors for that file should be suppressed. As such, after AstGen, the compiler must traverse files (starting from analysis roots) and discover the set of "live files" for this update. Additionally, the compiler's previous handling of retryable file errors was not very good; the source location the error was reported as was based only on the first discovered import of that file. This source location also disappeared on future incremental updates. So, as a part of the file traversal above, we also need to figure out the source locations of imports which errors should be reported against. Another observation I made is that the "file exists in multiple modules" error was not implemented in a particularly good way (I get to say that because I wrote it!). It was subject to races, where the order in which different imports of a file were discovered affects both how errors are printed, and which module the file is arbitrarily assigned, with the latter in turn affecting which other files are considered for import. The thing I realised here is that while the AstGen worker pool is running, we cannot know for sure which module(s) a file is in; we could always discover an import later which changes the answer. So, here's how the AstGen workers have changed. We initially ensure that `zcu.import_table` contains the root files for all modules in this Zcu, even if we don't know any imports for them yet. Then, the AstGen workers do not need to be aware of modules. Instead, they simply ignore module imports, and only spin off more workers when they see a by-path import. During AstGen, we can't use module-root-relative paths, since we don't know which modules files are in; but we don't want to unnecessarily use absolute files either, because those are non-portable and can make `error.NameTooLong` more likely. As such, I have introduced a new abstraction, `Compilation.Path`. This type is a way of representing a filesystem path which has a *canonical form*. The path is represented relative to one of a few special directories: the lib directory, the global cache directory, or the local cache directory. As a fallback, we use absolute (or cwd-relative on WASI) paths. This is kind of similar to `std.Build.Cache.Path` with a pre-defined list of possible `std.Build.Cache.Directory`, but has stricter canonicalization rules based on path resolution to make sure deduplicating files works properly. A `Compilation.Path` can be trivially converted to a `std.Build.Cache.Path` from a `Compilation`, but is smaller, has a canonical form, and has a digest which will be consistent across different compiler processes with the same lib and cache directories (important when we serialize incremental compilation state in the future). `Zcu.File` and `Zcu.EmbedFile` both contain a `Compilation.Path`, which is used to access the file on-disk; module-relative sub paths are used quite rarely (`EmbedFile` doesn't even have one now for simplicity). After the AstGen workers all complete, we know that any file which might be imported is definitely in `import_table` and up-to-date. So, we perform a single-threaded graph traversal; similar to what `resolveReferences` plays for `AnalUnit`s, but for files instead. We figure out which files are alive, and which module each file is in. If a file turns out to be in multiple modules, we set a field on `Zcu` to indicate this error. If a file is in a different module to a prior update, we set a flag instructing `updateZirRefs` to invalidate all dependencies on the file. This traversal also discovers "import errors"; these are errors associated with a specific `@import`. With Zig's current design, there is only one possible error here: "import outside of module root". This must be identified during this traversal instead of during AstGen, because it depends on which module the file is in. I tried also representing "module not found" errors in this same way, but it turns out to be much more useful to report those in Sema, because of use cases like optional dependencies where a module import is behind a comptime-known build option. For simplicity, `failed_files` now just maps to `?[]u8`, since the source location is always the whole file. In fact, this allows removing `LazySrcLoc.Offset.entire_file` completely, slightly simplifying some error reporting logic. File-level errors are now directly built in the `std.zig.ErrorBundle.Wip`. If the payload is not `null`, it is the message for a retryable error (i.e. an error loading the source file), and will be reported with a "file imported here" note pointing to the import site discovered during the single-threaded file traversal. The last piece of fallout here is how `Builtin` works. Rather than constructing "builtin" modules when creating `Package.Module`s, they are now constructed on-the-fly by `Zcu`. The map `Zcu.builtin_modules` maps from digests to `*Package.Module`s. These digests are abstract hashes of the `Builtin` value; i.e. all of the options which are placed into "builtin.zig". During the file traversal, we populate `builtin_modules` as needed, so that when we see this imports in Sema, we just grab the relevant entry from this map. This eliminates a bunch of awkward state tracking during construction of the module graph. It's also now clearer exactly what options the builtin module has, since previously it inherited some options arbitrarily from the first-created module with that "builtin" module! The user-visible effects of this commit are: * retryable file errors are now consistently reported against the whole file, with a note pointing to a live import of that file * some theoretical bugs where imports are wrongly considered distinct (when the import path moves out of the cwd and then back in) are fixed * some consistency issues with how file-level errors are reported are fixed; these errors will now always be printed in the same order regardless of how the AstGen pass assigns file indices * incremental updates do not print retryable file errors differently between updates or depending on file structure/contents * incremental updates support files changing modules * incremental updates support files becoming unreferenced Resolves: #22696
255 lines
8.5 KiB
Zig
255 lines
8.5 KiB
Zig
const std = @import("std");
|
|
const builtin = @import("builtin");
|
|
const Cases = @import("src/Cases.zig");
|
|
|
|
pub fn addCases(ctx: *Cases, b: *std.Build) !void {
|
|
// This test is currently disabled because the leading spaces aligning non-initial lines of the
|
|
// error message don't play nice with the test runner.
|
|
if (false) {
|
|
const case = ctx.obj("multiline error message", b.graph.host);
|
|
case.addError(
|
|
\\comptime {
|
|
\\ @compileError("hello\nworld");
|
|
\\}
|
|
, &[_][]const u8{
|
|
\\:2:5: error: hello
|
|
\\ world
|
|
});
|
|
}
|
|
|
|
// This test is currently disabled because the leading spaces aligning non-initial lines of the
|
|
// error message don't play nice with the test runner.
|
|
if (false) {
|
|
const case = ctx.obj("multiline error message with trailing newline", b.graph.host);
|
|
case.addError(
|
|
\\comptime {
|
|
\\ @compileError(
|
|
\\ \\
|
|
\\ \\hello!
|
|
\\ \\I'm a multiline error message.
|
|
\\ \\I hope to be very useful!
|
|
\\ \\
|
|
\\ \\also I will leave this trailing newline here if you don't mind
|
|
\\ \\
|
|
\\ );
|
|
\\}
|
|
, &[_][]const u8{
|
|
\\:2:5: error:
|
|
\\ hello!
|
|
\\ I'm a multiline error message.
|
|
\\ I hope to be very useful!
|
|
\\
|
|
\\ also I will leave this trailing newline here if you don't mind
|
|
\\
|
|
});
|
|
}
|
|
|
|
{
|
|
const case = ctx.obj("missing semicolon at EOF", b.graph.host);
|
|
case.addError(
|
|
\\const foo = 1
|
|
, &[_][]const u8{
|
|
\\:1:14: error: expected ';' after declaration
|
|
});
|
|
}
|
|
|
|
{
|
|
const case = ctx.obj("argument causes error", b.graph.host);
|
|
|
|
case.addError(
|
|
\\pub export fn entry() void {
|
|
\\ var lib: @import("b.zig").ElfDynLib = undefined;
|
|
\\ _ = lib.lookup(fn () void);
|
|
\\}
|
|
, &[_][]const u8{
|
|
":3:12: error: unable to resolve comptime value",
|
|
":3:19: note: call to generic function instantiated with comptime-only return type '?fn () void' is evaluated at comptime",
|
|
":2:55: note: return type declared here",
|
|
":3:19: note: use '*const fn () void' for a function pointer type",
|
|
});
|
|
case.addSourceFile("b.zig",
|
|
\\pub const ElfDynLib = struct {
|
|
\\ pub fn lookup(self: *ElfDynLib, comptime T: type) ?T {
|
|
\\ _ = self;
|
|
\\ return undefined;
|
|
\\ }
|
|
\\};
|
|
);
|
|
}
|
|
|
|
{
|
|
const case = ctx.obj("astgen failure in file struct", b.graph.host);
|
|
|
|
case.addError(
|
|
\\pub export fn entry() void {
|
|
\\ _ = (@sizeOf(@import("b.zig")));
|
|
\\}
|
|
, &[_][]const u8{
|
|
":1:1: error: expected type expression, found '+'",
|
|
});
|
|
case.addSourceFile("b.zig",
|
|
\\+
|
|
);
|
|
}
|
|
|
|
{
|
|
const case = ctx.obj("invalid store to comptime field", b.graph.host);
|
|
|
|
case.addError(
|
|
\\const a = @import("a.zig");
|
|
\\
|
|
\\export fn entry() void {
|
|
\\ _ = a.S.qux(a.S{ .foo = 2, .bar = 2 });
|
|
\\}
|
|
, &[_][]const u8{
|
|
":4:23: error: value stored in comptime field does not match the default value of the field",
|
|
":2:25: note: default value set here",
|
|
});
|
|
case.addSourceFile("a.zig",
|
|
\\pub const S = struct {
|
|
\\ comptime foo: u32 = 1,
|
|
\\ bar: u32,
|
|
\\ pub fn qux(x: @This()) void {
|
|
\\ _ = x;
|
|
\\ }
|
|
\\};
|
|
);
|
|
}
|
|
|
|
{
|
|
const case = ctx.obj("file in multiple modules", b.graph.host);
|
|
case.addDepModule("foo", "foo.zig");
|
|
|
|
case.addError(
|
|
\\comptime {
|
|
\\ _ = @import("foo");
|
|
\\ _ = @import("foo.zig");
|
|
\\}
|
|
, &[_][]const u8{
|
|
":1:1: error: file exists in modules 'foo' and 'root'",
|
|
":1:1: note: files must belong to only one module",
|
|
":1:1: note: file is the root of module 'foo'",
|
|
":3:17: note: file is imported here by the root of module 'root'",
|
|
});
|
|
case.addSourceFile("foo.zig",
|
|
\\const dummy = 0;
|
|
);
|
|
}
|
|
|
|
{
|
|
const case = ctx.obj("wrong same named struct", b.graph.host);
|
|
|
|
case.addError(
|
|
\\const a = @import("a.zig");
|
|
\\const b = @import("b.zig");
|
|
\\
|
|
\\export fn entry() void {
|
|
\\ var a1: a.Foo = undefined;
|
|
\\ bar(&a1);
|
|
\\}
|
|
\\
|
|
\\fn bar(_: *b.Foo) void {}
|
|
, &[_][]const u8{
|
|
":6:9: error: expected type '*b.Foo', found '*a.Foo'",
|
|
":6:9: note: pointer type child 'a.Foo' cannot cast into pointer type child 'b.Foo'",
|
|
":1:17: note: struct declared here",
|
|
":1:17: note: struct declared here",
|
|
":9:11: note: parameter type declared here",
|
|
});
|
|
|
|
case.addSourceFile("a.zig",
|
|
\\pub const Foo = struct {
|
|
\\ x: i32,
|
|
\\};
|
|
);
|
|
|
|
case.addSourceFile("b.zig",
|
|
\\pub const Foo = struct {
|
|
\\ z: f64,
|
|
\\};
|
|
);
|
|
}
|
|
|
|
{
|
|
const case = ctx.obj("non-printable invalid character", b.graph.host);
|
|
|
|
case.addError("\xff\xfe" ++
|
|
\\export fn foo() bool {
|
|
\\ return true;
|
|
\\}
|
|
, &[_][]const u8{
|
|
":1:1: error: expected type expression, found 'invalid token'",
|
|
});
|
|
}
|
|
|
|
{
|
|
const case = ctx.obj("imported generic method call with invalid param", b.graph.host);
|
|
|
|
case.addError(
|
|
\\pub const import = @import("import.zig");
|
|
\\
|
|
\\export fn callComptimeBoolFunctionWithRuntimeBool(x: bool) void {
|
|
\\ import.comptimeBoolFunction(x);
|
|
\\}
|
|
\\
|
|
\\export fn callComptimeAnytypeFunctionWithRuntimeBool(x: bool) void {
|
|
\\ import.comptimeAnytypeFunction(x);
|
|
\\}
|
|
\\
|
|
\\export fn callAnytypeFunctionWithRuntimeComptimeOnlyType(x: u32) void {
|
|
\\ const S = struct { x: u32, y: type };
|
|
\\ import.anytypeFunction(S{ .x = x, .y = u32 });
|
|
\\}
|
|
, &[_][]const u8{
|
|
":4:33: error: unable to resolve comptime value",
|
|
":4:33: note: argument to comptime parameter must be comptime-known",
|
|
":1:29: note: parameter declared comptime here",
|
|
":8:36: error: unable to resolve comptime value",
|
|
":8:36: note: argument to comptime parameter must be comptime-known",
|
|
":2:32: note: parameter declared comptime here",
|
|
":13:32: error: unable to resolve comptime value",
|
|
":13:32: note: initializer of comptime-only struct 'tmp.callAnytypeFunctionWithRuntimeComptimeOnlyType.S' must be comptime-known",
|
|
":12:35: note: struct requires comptime because of this field",
|
|
":12:35: note: types are not available at runtime",
|
|
});
|
|
|
|
case.addSourceFile("import.zig",
|
|
\\pub fn comptimeBoolFunction(comptime _: bool) void {}
|
|
\\pub fn comptimeAnytypeFunction(comptime _: anytype) void {}
|
|
\\pub fn anytypeFunction(_: anytype) void {}
|
|
);
|
|
}
|
|
|
|
{
|
|
const case = ctx.obj("invalid byte in string", b.graph.host);
|
|
|
|
case.addError("_ = \"\x01Q\";", &[_][]const u8{
|
|
":1:6: error: string literal contains invalid byte: '\\x01'",
|
|
});
|
|
}
|
|
|
|
{
|
|
const case = ctx.obj("invalid byte in comment", b.graph.host);
|
|
|
|
case.addError("//\x01Q", &[_][]const u8{
|
|
":1:3: error: comment contains invalid byte: '\\x01'",
|
|
});
|
|
}
|
|
|
|
{
|
|
const case = ctx.obj("control character in character literal", b.graph.host);
|
|
|
|
case.addError("const c = '\x01';", &[_][]const u8{
|
|
":1:12: error: character literal contains invalid byte: '\\x01'",
|
|
});
|
|
}
|
|
|
|
{
|
|
const case = ctx.obj("invalid byte at start of token", b.graph.host);
|
|
|
|
case.addError("x = \x00Q", &[_][]const u8{
|
|
":1:5: error: expected expression, found 'invalid token'",
|
|
});
|
|
}
|
|
}
|