85 Commits

Author SHA1 Message Date
Jakub Konka
46db5e2a44 test: unroll into multiple cases, provide default parsers
Provide default parsers for obvious config options such as
`CrossTarget` or `Backend` (or any enum for that matter).

Unroll iterator loops into multiple cases - we need to create
a Cartesian product for all possibilities specified in the
test manifest.
2022-04-28 18:35:01 +02:00
Jakub Konka
bc370311cb test: add comptime struct holding default config values 2022-04-28 18:35:01 +02:00
Jakub Konka
f41dd3617e test: pass Strategy per directory of tests 2022-04-28 18:35:01 +02:00
Jakub Konka
d305a6fb55 test: parse all of manifest in TestManifest abstraction 2022-04-28 18:35:01 +02:00
Jakub Konka
d8d12d51ec test: abstract away test manifest parser into separate struct 2022-04-28 18:35:01 +02:00
Cody Tapscott
aaac8eae68 Use 0-indexing for incremental compile error tests 2022-04-13 01:09:21 -04:00
Cody Tapscott
879b562779 Add file support for incremental error tests
Compile error test cases can now be given as a sequence of files:
  - "foo.1.zig"
  - "foo.2.zig"
  - "foo.3.zig"
  - etc.

This sequence of files is tested as incremental compilation updates to a
single "foo.zig" source file.

To help avoid mistakes, we enforce strict ordering for these files.
"foo.zig" cannot co-exist with "foo.X.zig", the sequence must include
"foo.1.zig", and no numbers may be skipped.
2022-04-11 15:35:28 -07:00
Cody Tapscott
7851377e95 Improve whitespace-handling in (compile-error) test manifest parsing 2022-04-11 10:05:00 -07:00
Andrew Kelley
243afdcdf5 test harness improvements
* `-Dskip-compile-errors` is removed; `-Dskip-stage1` is added.
 * Use `std.testing.allocator` instead of a new instance of GPA.
   - Fix the memory leaks this revealed.
 * Show the file name when it is not parsed correctly such as when the
   manifest is missing.
   - Better error messages when test files are not parsed correctly.
 * Ignore unknown files such as swap files.
 * Move logic from declarative file to the test harness implementation.
 * Move stage1 tests to stage2 tests where appropriate.
2022-03-31 15:10:31 -07:00
Andrew Kelley
12e1304805 test harness: fix not honoring one_test_case_per_file
I regressed this in 9aa431cba34699ae35f7905398a0c8263b2ad453.

thanks @topolarity for pointing out the issue
2022-03-29 12:20:10 -07:00
Andrew Kelley
9aa431cba3 test harness: include case names for compile errors
in the progress nodes
2022-03-29 12:01:45 -07:00
Cody Tapscott
7f64f7c925 Add rudimentary compile error test file support
This brings two quality-of-life improvements for folks working on
compile error test cases:
 - test cases can be added/changed without re-building Zig
 - wrapping the source in a multi-line string literal is not necessary

I decided to keep things as simple as possible for this initial
implementation. The test "manifest" is a contiguous comment block at the
end of the test file:
1. The first line is the test case name
2. The second line is a blank comment
2. The following lines are expected errors

Here's an example:
```zig
const U = union(enum(u2)) {
    A: u8,
    B: u8,
    C: u8,
    D: u8,
    E: u8,
};
export fn entry() void {
    _ = U{ .E = 1 };
}

// union with too small explicit unsigned tag type
//
// tmp.zig:1:22: error: specified integer tag type cannot represent every field
// tmp.zig:1:22: note: type u2 cannot fit values in range 0...4
```

The mode of the test (obj/exe/test), as well as the target
(stage1/stage2) is determined based on the directory containing the
test.

We'll probably eventually want to support embedding this information
in the test files themselves, similar to the arocc test runner, but
that enhancement can be tackled later.
2022-03-25 12:25:43 -07:00
Andrew Kelley
69d78bdae4 stage2 test harness: ask for the backend explicitly
Follow-up to 35d6ee08c468642969b594b711dd6448bbaefa89
2022-03-17 20:10:00 -07:00
Guillaume Wenzek
f000f8a59a fix nvptx test failure #10968
allow test cases to chose wether to link libc or not.
default behavior is to not link libc, except for `exeUsingLLVMBackend`
2022-03-12 14:25:59 -05:00
Andrew Kelley
210ee1067b update more API usage of std.Progress
fixes regression introduced in 5a00e249632716b86edac088f69d19d82e307a28
2022-02-08 17:49:40 -07:00
Cody Tapscott
5065830aa0 Avoid depending on child process execution when not supported by host OS
In accordance with the requesting issue (#10750):
- `zig test` skips any tests that it cannot spawn, returning success
- `zig run` and `zig build` exit with failure, reporting the command the cannot be run
- `zig clang`, `zig ar`, etc. already punt directly to the appropriate clang/lld main(), even before this change
- Native `libc` Detection is not supported

Additionally, `exec()` and related Builder functions error at run-time, reporting the command that cannot be run
2022-02-06 22:21:46 -07:00
Sage Hane
e288148f60
fs: Use OpenMode enum instead of read/write flags. 2022-01-29 15:52:08 +02:00
Andrew Kelley
f3edff439e improve detection of how to execute binaries on the host
`getExternalExecutor` is moved from `std.zig.CrossTarget` to
`std.zig.system.NativeTargetInfo.getExternalExecutor`.

The function also now communicates a bit more information about *why*
the host is unable to execute a binary. The CLI is updated to report
this information in a useful manner.

`getExternalExecutor` is also improved to detect such patterns as:
 * x86_64 is able to execute x86 binaries
 * aarch64 is able to execute arm binaries
 * etc.

Added qemu-hexagon support to `getExternalExecutor`.

`std.Target.canExecBinaries` of is removed; callers should use the more
powerful `getExternalExecutor` instead.

Now that `zig test` tries to run the resulting binary no matter what,
this commit has a follow-up change to the build system and docgen to
utilize the `getExternalExecutor` function and pass `--test-no-exec`
in some cases to avoid getting the error.

Additionally:

 * refactor: extract NativePaths and NativeTargetInfo into their own
   files named after the structs.
 * small improvement to langref to reduce the complexity of the `callconv`
   expression in a couple examples.
2021-12-02 21:51:14 -07:00
Andrew Kelley
b24cbecdb2 zig build: promote qemu, wine, wasmtime, darling, and rosetta
from zig-specific options to generally recognized zig build options that
any project can take advantage of. See the updated usage text for more
details.
2021-12-02 15:46:22 -07:00
Jakub Konka
852841fd1f Make Rosetta availability declarative by the user
Like QEMU or Wine, you need to declare you want Rosetta
enabled for running tests/build artifacts via `-Denable-rosetta`
flag.
2021-12-02 00:22:09 +01:00
Jakub Konka
3e2f8233a8 Make Rosetta a new variant in ExternalExecutor enum
When running, check if Rosetta is available, otherwise, pass the
tests.
2021-12-02 00:22:09 +01:00
Lee Cannon
1093b09a98
allocgate: renamed getAllocator function to allocator 2021-11-30 23:32:47 +00:00
Lee Cannon
75548b50ff
allocgate: stage 1 and 2 building 2021-11-30 23:32:47 +00:00
Lee Cannon
85de022c56
allocgate: std Allocator interface refactor 2021-11-30 23:32:47 +00:00
Ryan Liptak
d31352ee85 Update all usages of mem.split/mem.tokenize for generic version 2021-08-06 02:01:47 -07:00
Andrew Kelley
7b8cb881df stage2: improvements towards zig test
* There is now a main_pkg in addition to root_pkg. They are usually the
   same. When using `zig test`, main_pkg is the user's source file and
   root_pkg has the test runner.
 * scanDecl no longer looks for test decls outside the package being
   tested. honoring `--test-filter` is still TODO.
 * test runner main function has a void return value rather than
   `anyerror!void`
 * Sema is improved to generate better AIR for for loops on slices.
 * Sema: fix incorrect capacity calculation in zirBoolBr
 * Sema: add compile errors for trying to use slice fields as an lvalue.
 * Sema: fix type coercion for error unions
 * Sema: fix analyzeVarRef generating garbage AIR
 * C codegen: fix renderValue for error unions with 0 bit payload
 * C codegen: implement function pointer calls
 * CLI: fix usage text

 Adds 4 new AIR instructions:

  * slice_len, slice_ptr: to get the ptr and len fields of a slice.
  * slice_elem_val, ptr_slice_elem_val: to get the element value of
    a slice, and a pointer to a slice.

AstGen gains a new functionality:

 * One of the unused flags of struct decls is now used to indicate
   structs that are known to have non-zero size based on the AST alone.
2021-07-23 22:42:31 -07:00
Andrew Kelley
c39c46c0d1 stage2 tests: respect -Dskip-non-native for object formats 2021-07-20 20:05:54 -07:00
Andrew Kelley
d979dd9b58 stage2: improve AstGen FileNotFound error message
Partially addresses #9203. It fixes the first case, but not the second
one mentioned in the issue.
2021-07-02 15:27:00 -07:00
Andrew Kelley
5103053977 compile errors test harness: support unknown file/line/column
This gets us 2 more passing compile error test cases.
2021-07-02 13:28:31 -07:00
Andrew Kelley
8ce880ca75 avoid calling into stage1 backend when AstGen fails
The motivation for this commit is that there exists source files which
produce ast-check errors, but crash stage1 or otherwise trigger stage1
bugs. Previously to this commit, Zig would run AstGen, collect the
compile errors, run stage1, report stage1 compile errors and exit if
any, and then report AstGen compile errors.

The main change in this commit is to report AstGen errors prior to
invoking stage1, and in fact if any AstGen errors occur, do not invoke
stage1 at all.

This caused most of the compile error tests to fail due to things such
as unused local variables and mismatched stage1/stage2 error messages.
It was taking a long time to update the test cases one-by-one, so I
took this opportunity to unify the stage1 and stage2 testing harness,
specifically with regards to compile errors. In this way we can start
keeping track of which tests pass for 1, 2, or both.
`zig build test-compile-errors` no longer works; it is now integrated
into `zig build test-stage2`.

This is one step closer to executing compile error tests in parallel; in
fact the ThreadPool object is already in scope.

There are some cases where the stage1 compile errors were actually
better; those are left failing in this commit, to be addressed in a
follow-up commit.

Other changes in this commit:

 * build.zig: improve support for -Dstage1 used with the test step.
 * AstGen: minor cosmetic changes to error messages.
 * stage2: add -fstage1 and -fno-stage1 flags. This now allows one to
   download a binary of the zig compiler and use the llvm backend of
   self-hosted. This was also needed for hooking up the test harness.
   However, I realized that stage1 calls exit() and also has memory
   leaks, so had to complicate the test harness by not using this flag
   after all and instead invoking as a child process.
   - These CLI flags will disappear once we start shipping the
     self-hosted compiler as the main compiler. Until then, they can be
     used to try out the work-in-progress stage2.
 * stage2: select the LLVM backend by default for release modes, as long
   as the target architecture is supported by LLVM.
 * test harness: support setting the optimize mode
2021-07-02 13:27:28 -07:00
Jacob G-W
9fffffb07b fix code broken from previous commit 2021-06-21 17:03:03 -07:00
Andrew Kelley
a8eac0cf07 stage2 test harness: show source file name
Also support specifying source file name in compile error test cases.

closes #8829
2021-05-19 20:59:06 -07:00
Andrew Kelley
731c35f15a stage2: get rid of NameHash
Previously, stage2 used a global decl_table for all Decl objects, keyed
by a 16-byte name hash that was hopefully unique. Now, there is a tree
of Namespace objects that own their named Decl objects.
2021-05-17 16:09:20 -07:00
Andrew Kelley
597082adf4 Merge remote-tracking branch 'origin/master' into stage2-whole-file-astgen
Conflicts:
 * build.zig
 * src/Compilation.zig
 * src/codegen/spirv/spec.zig
 * src/link/SpirV.zig
 * test/stage2/darwin.zig
   - this one might be problematic; start.zig looks for `main` in the
     root source file, not `_main`. Not sure why there is an underscore
     there in master branch.
2021-05-15 21:44:38 -07:00
Andrew Kelley
07606d12da stage2: remove SPU Mark II backend
As it stands, the backend is incomplete, and there is no active contributor,
making it dead weight.

However, anyone is free to resurrect this backend at any time.
2021-05-15 21:25:42 -07:00
Jakub Konka
402264ab0e Add experimental Darling support for cross testing macOS
* for cross testing stage2 tests, we use `darling shell` command
  since the path to the tested binary is relative to cwd
* for the `zig test` command, we simply use `darling` since the
  path to the binary is absolute
2021-05-13 20:31:33 +02:00
Andrew Kelley
6361d7a928 stage2 test harness: report multiple failures
so you can debug more than one thing at a time
2021-05-08 16:11:07 -07:00
Andrew Kelley
5619ce2406 Merge remote-tracking branch 'origin/master' into stage2-whole-file-astgen
Conflicts:
 * doc/langref.html.in
 * lib/std/enums.zig
 * lib/std/fmt.zig
 * lib/std/hash/auto_hash.zig
 * lib/std/math.zig
 * lib/std/mem.zig
 * lib/std/meta.zig
 * test/behavior/alignof.zig
 * test/behavior/bitcast.zig
 * test/behavior/bugs/1421.zig
 * test/behavior/cast.zig
 * test/behavior/ptrcast.zig
 * test/behavior/type_info.zig
 * test/behavior/vector.zig

Master branch added `try` to a bunch of testing function calls, and some
lines also had changed how to refer to the native architecture and other
`@import("builtin")` stuff.
2021-05-08 14:45:21 -07:00
Veikka Tuominen
42a95197f3 update usage of std.testing in stage2 2021-05-08 15:15:30 +03:00
Andrew Kelley
7dd33d4316 stage2: fix compile errors in test harness 2021-05-06 17:48:38 -07:00
Andrew Kelley
9088d40e83 stage2: rename zir to Zir
since it now uses top level fields
2021-04-15 19:06:39 -07:00
g-w1
c6791d87d4
stage2: delete allowing input (and output) zir from the pipeline (#8471)
Remove -femit-zir as we aren't going to need it. Also remove zir test code
This removes a TODO that asserts the file is not zir.
2021-04-09 02:11:33 -04:00
Andrew Kelley
1b702f8ddb stage2: fix the memory leaks 2021-04-08 20:52:02 -07:00
Andrew Kelley
61b868f9a5 stage2: simplify Decl src_node field
Also fix "previous definition here" error notes to be correct.
2021-04-08 20:37:19 -07:00
Veikka Tuominen
0a7be71bc2
stage2 cbe: non pointer optionals 2021-03-08 00:33:56 +02:00
Veikka Tuominen
8c6e7fb2c7
stage2: implement var args 2021-03-06 15:55:29 +02:00
Andrew Kelley
ffb2568a9f
Merge pull request #7763 from kivikakk/zig-elf-parse
std.elf: expose parsing decoupled from std.fs.File
2021-03-04 11:56:31 -08:00
Andrew Kelley
6959b177ef stage2: test harness: panic on OOM rather than assert
also add a prefix to test names for C backend
2021-02-19 20:26:48 -07:00
Tadeo Kondrak
0b5f3c2ef9
Replace @TagType uses, mostly with std.meta.Tag 2021-01-30 22:26:44 +02:00
Andrew Kelley
ecc246efa2 stage2: rework ZIR/TZIR for optionals and error unions
* fix wrong pointer const-ness when unwrapping optionals
 * allow grouped expressions and orelse as lvalues
 * ZIR for unwrapping optionals: no redundant deref
   - add notes to please don't use rlWrapPtr, this function should be
     deleted
 * catch and orelse: better ZIR for non-lvalue: no redundant deref;
   operate entirely on values. lvalue case still works properly.
   - properly propagate the result location into the target expression
 * Test harness: better output when tests fail due to compile errors.
 * TZIR: add instruction variants. These allow fewer TZIR instructions to
   be emitted from zir_sema. See the commit diff for per-instruction
   documentation.
   - is_null
   - is_non_null
   - is_null_ptr
   - is_non_null_ptr
   - is_err
   - is_err_ptr
   - optional_payload
   - optional_payload_ptr
 * TZIR: removed old naming convention instructions:
   - isnonnull
   - isnull
   - iserr
   - unwrap_optional
 * ZIR: add instruction variants. These allow fewer ZIR instructions to
   be emitted from astgen. See the commit diff for per-instruction
   documentation.
   - is_non_null
   - is_null
   - is_non_null_ptr
   - is_null_ptr
   - is_err
   - is_err_ptr
   - optional_payload_safe
   - optional_payload_unsafe
   - optional_payload_safe_ptr
   - optional_payload_unsafe_ptr
   - err_union_payload_safe
   - err_union_payload_unsafe
   - err_union_payload_safe_ptr
   - err_union_payload_unsafe_ptr
   - err_union_code
   - err_union_code_ptr
 * ZIR: removed old naming convention instructions:
   - isnonnull
   - isnull
   - iserr
   - unwrap_optional_safe
   - unwrap_optional_unsafe
   - unwrap_err_safe
   - unwrap_err_unsafe
   - unwrap_err_code
2021-01-18 19:29:18 -07:00