std.Build.addTest creates a CompileStep as before, however, this kind of
step no longer actually runs the unit tests. Instead it only compiles
it, and one must additionally create a RunStep from the CompileStep in
order to actually run the tests.
RunStep gains integration with the default test runner, which now
supports the standard --listen=- argument in order to communicate over
stdin and stdout. It also reports test statistics; how many passed,
failed, and leaked, as well as directly associating the relevant stderr
with the particular test name that failed.
This separation of CompileStep and RunStep means that
`CompileStep.Kind.test_exe` is no longer needed, and therefore has been
removed in this commit.
* build runner: show unit test statistics in build summary
* added Step.writeManifest since many steps want to treat it as a
warning and emit the same message if it fails.
* RunStep: fixed error message that prints the failed command printing
the original argv and not the adjusted argv in case an interpreter
was used.
* RunStep: fixed not passing the command line arguments to the
interpreter.
* move src/Server.zig to std.zig.Server so that the default test runner
can use it.
* the simpler test runner function which is used by work-in-progress
backends now no longer prints to stderr, which is necessary in order
for the build runner to not print the stderr as a warning message.
Instead of using `zig test` to build a special version of the compiler
that runs all the test-cases, the zig build system is now used as much
as possible - all with the basic steps found in the standard library.
For incremental compilation tests (the ones that look like foo.0.zig,
foo.1.zig, foo.2.zig, etc.), a special version of the compiler is
compiled into a utility executable called "check-case" which checks
exactly one sequence of incremental updates in an independent
subprocess. Previously, all incremental and non-incremental test cases
were done in the same test runner process.
The compile error checking code is now simpler, but also a bit
rudimentary, and so it additionally makes sure that the actual compile
errors do not include *extra* messages, and it makes sure that the
actual compile errors output in the same order as expected. It is also
based on the "ends-with" property of each line rather than the previous
logic, which frankly I didn't want to touch with a ten-meter pole. The
compile error test cases have been updated to pass in light of these
differences.
Previously, 'error' mode with 0 compile errors was used to shoehorn in a
different kind of test-case - one that only checks if a piece of code
compiles without errors. Now there is a 'compile' mode of test-cases,
and 'error' must be only used when there are greater than 0 errors.
link test cases are updated to omit the target object format argument
when calling checkObject since that is no longer needed.
The test/stage2 directory is removed; the 2 files within are moved to be
directly in the test/ directory.
