45 Commits

Author SHA1 Message Date
Andrew Kelley
9088d40e83 stage2: rename zir to Zir
since it now uses top level fields
2021-04-15 19:06:39 -07:00
g-w1
c6791d87d4
stage2: delete allowing input (and output) zir from the pipeline (#8471)
Remove -femit-zir as we aren't going to need it. Also remove zir test code
This removes a TODO that asserts the file is not zir.
2021-04-09 02:11:33 -04:00
Andrew Kelley
1b702f8ddb stage2: fix the memory leaks 2021-04-08 20:52:02 -07:00
Andrew Kelley
61b868f9a5 stage2: simplify Decl src_node field
Also fix "previous definition here" error notes to be correct.
2021-04-08 20:37:19 -07:00
Veikka Tuominen
0a7be71bc2
stage2 cbe: non pointer optionals 2021-03-08 00:33:56 +02:00
Veikka Tuominen
8c6e7fb2c7
stage2: implement var args 2021-03-06 15:55:29 +02:00
Andrew Kelley
ffb2568a9f
Merge pull request #7763 from kivikakk/zig-elf-parse
std.elf: expose parsing decoupled from std.fs.File
2021-03-04 11:56:31 -08:00
Andrew Kelley
6959b177ef stage2: test harness: panic on OOM rather than assert
also add a prefix to test names for C backend
2021-02-19 20:26:48 -07:00
Tadeo Kondrak
0b5f3c2ef9
Replace @TagType uses, mostly with std.meta.Tag 2021-01-30 22:26:44 +02:00
Andrew Kelley
ecc246efa2 stage2: rework ZIR/TZIR for optionals and error unions
* fix wrong pointer const-ness when unwrapping optionals
 * allow grouped expressions and orelse as lvalues
 * ZIR for unwrapping optionals: no redundant deref
   - add notes to please don't use rlWrapPtr, this function should be
     deleted
 * catch and orelse: better ZIR for non-lvalue: no redundant deref;
   operate entirely on values. lvalue case still works properly.
   - properly propagate the result location into the target expression
 * Test harness: better output when tests fail due to compile errors.
 * TZIR: add instruction variants. These allow fewer TZIR instructions to
   be emitted from zir_sema. See the commit diff for per-instruction
   documentation.
   - is_null
   - is_non_null
   - is_null_ptr
   - is_non_null_ptr
   - is_err
   - is_err_ptr
   - optional_payload
   - optional_payload_ptr
 * TZIR: removed old naming convention instructions:
   - isnonnull
   - isnull
   - iserr
   - unwrap_optional
 * ZIR: add instruction variants. These allow fewer ZIR instructions to
   be emitted from astgen. See the commit diff for per-instruction
   documentation.
   - is_non_null
   - is_null
   - is_non_null_ptr
   - is_null_ptr
   - is_err
   - is_err_ptr
   - optional_payload_safe
   - optional_payload_unsafe
   - optional_payload_safe_ptr
   - optional_payload_unsafe_ptr
   - err_union_payload_safe
   - err_union_payload_unsafe
   - err_union_payload_safe_ptr
   - err_union_payload_unsafe_ptr
   - err_union_code
   - err_union_code_ptr
 * ZIR: removed old naming convention instructions:
   - isnonnull
   - isnull
   - iserr
   - unwrap_optional_safe
   - unwrap_optional_unsafe
   - unwrap_err_safe
   - unwrap_err_unsafe
   - unwrap_err_code
2021-01-18 19:29:18 -07:00
Andrew Kelley
8c9ac4db97 stage2: implement error notes and regress -femit-zir
* Implement error notes
   - note: other symbol exported here
   - note: previous else prong is here
   - note: previous '_' prong is here
 * Add Compilation.CObject.ErrorMsg. This object properly converts to
   AllErrors.Message when the time comes.
 * Add Compilation.CObject.failure_retryable. Properly handles
   out-of-memory and other transient failures.
 * Introduce Module.SrcLoc which has not only a byte offset but also
   references the file which the byte offset applies to.
 * Scope.Block now contains both a pointer to the "owner" Decl and the
   "source" Decl. As an example, during inline function call, the
   "owner" will be the Decl of the caller and the "source" will be the
   Decl of the callee.
 * Module.ErrorMsg now sports a `file_scope` field so that notes can
   refer to source locations in a file other than the parent error
   message.
 * Some instances where a `*Scope` was stored, now store a
   `*Scope.Container`.
 * Some methods in the `Scope` namespace were moved to the more specific
   type, since there was only an implementation for one particular tag.
   - `removeDecl` moved to `Scope.Container`
   - `destroy` moved to `Scope.File`
 * Two kinds of Scope deleted:
   - zir_module
   - decl
 * astgen: properly use DeclVal / DeclRef. DeclVal was incorrectly
   changed to be a reference; this commit fixes it. Fewer ZIR
   instructions processed as a result.
   - declval_in_module is renamed to declval
   - previous declval ZIR instruction is deleted; it was only for .zir
     files.
 * Test harness: friendlier diagnostics when an unexpected set of errors
   is encountered.
 * zir_sema: fix analyzeInstBlockFlat by properly calling resolvingInst
   on the last zir instruction in the block.

Compile log implementation:
 * Write to a buffer rather than directly to stderr.
 * Only keep track of 1 callsite per Decl.
 * No longer mutate the ZIR Inst struct data.
 * "Compile log statement found" errors are only emitted when there are
   no other compile errors.

-femit-zir and support for .zir source files is regressed. If we wanted
to support this again, outputting .zir would need to be done as yet
another backend rather than in the haphazard way it was previously
implemented.

For parsing .zir, it was implemented previously in a way that was not
helpful for debugging. We need tighter integration with the test harness
for it to be useful; so clearly a rewrite is needed. Given that a
rewrite is needed, and it was getting in the way of progress and
organization of the rest of stage2, I regressed the feature.
2021-01-16 22:51:01 -07:00
Asherah Connor
affb57aad9 use interfaces 2021-01-15 10:34:53 +11:00
Asherah Connor
32d69d70cf expose phdr, shdr parsing 2021-01-14 15:30:28 +11:00
Andrew Kelley
5c49a137d5
Merge pull request #7725 from FireFox317/even-more-llvm
stage2: initial implementation of control flow in LLVM backend + TZIR printing
2021-01-09 12:32:10 -08:00
Jay Petacat
a9b505fa77 Reduce use of deprecated IO types
Related: #4917
2021-01-07 23:48:58 -08:00
Timon Kruiper
36950815a4 stage2 test: make sure to pass the dynamic linker to Compilation
Because this was not set, the `-dynamic-path` argument was not passed
to LLD when linking an ELF binary. This would still generate a valid glibc
binary, however when executing would result in a segfault.
2021-01-07 11:37:52 +01:00
Andrew Kelley
91e3431d4a stage2 test harness: don't try to run non-native C backend tests 2021-01-05 20:44:19 -07:00
Andrew Kelley
7b8cede61f stage2: rework the C backend
* std.ArrayList gains `moveToUnmanaged` and dead code
   `ArrayListUnmanaged.appendWrite` is deleted.
 * emit_h state is attached to Module rather than Compilation.
 * remove the implementation of emit-h because it did not properly
   integrate with incremental compilation. I will re-implement it
   in a follow-up commit.
 * Compilation: use the .codegen_failure tag rather than
   .dependency_failure tag for when `bin_file.updateDecl` fails.

C backend:
 * Use a CValue tagged union instead of strings for C values.
 * Cleanly separate state into Object and DeclGen:
   - Object is present only when generating a .c file
   - DeclGen is present for both generating a .c and .h
 * Move some functions into their respective Object/DeclGen namespace.
 * Forward decls are managed by the incremental compilation frontend; C
   backend no longer renders function signatures based on callsites.
   For simplicity, all functions always get forward decls.
 * Constants are managed by the incremental compilation frontend. C
   backend no longer has a "constants" section.
 * Participate in incremental compilation. Each Decl gets an ArrayList
   for its generated C code and it is updated when the Decl is updated.
   During flush(), all these are joined together in the output file.
 * The new CValue tagged union is used to clean up using of assigning to
   locals without an additional pointer local.
 * Fix bug with bitcast of non-pointers making the memcpy destination
   immutable.
2021-01-05 17:41:14 -07:00
Andrew Kelley
c8e44d82bd stage2: remove the Cache deadlock detection code
It's more trouble than it's worth; it didn't even catch the most recent
incident because it was across process boundaries anyway.
2021-01-03 20:34:51 -07:00
Andrew Kelley
404dc9692e stage2: fix Cache debug deadlock code memory leak 2021-01-03 20:34:51 -07:00
Timon Kruiper
0151f3b76a stage2: Add support for testing LLVM enabled builds in test-stage2
To make sure that we don't have to rebuild libc for every case, we now
have a seperate cache directory for the global cache, which remains
the same between test runs.

Also make sure to destory the Compilation before executing a child process,
otherwise the compiler deadlocks. (#7596)
2021-01-03 21:13:38 +01:00
LemonBoy
d2f6fa1608 Fix more stray uses of {} for formatting strings 2021-01-02 17:12:57 -07:00
Andrew Kelley
a46d24af1c stage2: inferred local variables
This patch introduces the following new things:

Types:
 - inferred_alloc
   - This is a special value that tracks a set of types that have been stored
     to an inferred allocation. It does not support most of the normal type queries.
     However it does respond to `isConstPtr`, `ptrSize`, `zigTypeTag`, etc.
   - The payload for this type simply points to the corresponding Value
     payload.

Values:
 - inferred_alloc
   - This is a special value that tracks a set of types that have been stored
     to an inferred allocation. It does not support any of the normal value queries.

ZIR instructions:
 - store_to_inferred_ptr,
   - Same as `store` but the type of the value being stored will be used to infer
     the pointer type.
 - resolve_inferred_alloc
   - Each `store_to_inferred_ptr` puts the type of the stored value into a set,
     and then `resolve_inferred_alloc` triggers peer type resolution on the set.
     The operand is a `alloc_inferred` or `alloc_inferred_mut` instruction, which
     is the allocation that needs to have its type inferred.

Changes to the C backend:
 * Implements the bitcast instruction. If the source and dest types
   are both pointers, uses a cast, otherwise uses memcpy.
 * Tests are run with -Wno-declaration-after-statement. Someday we can
   conform to this but not today.

In ZIR form it looks like this:

```zir
fn_body main { // unanalyzed
  %0 = dbg_stmt()
=>%1 = alloc_inferred()
  %2 = declval_in_module(Decl(add))
  %3 = deref(%2)
  %4 = param_type(%3, 0)
  %5 = const(TypedValue{ .ty = comptime_int, .val = 1})
  %6 = as(%4, %5)
  %7 = param_type(%3, 1)
  %8 = const(TypedValue{ .ty = comptime_int, .val = 2})
  %9 = as(%7, %8)
  %10 = call(%3, [%6, %9], modifier=auto)
=>%11 = store_to_inferred_ptr(%1, %10)
=>%12 = resolve_inferred_alloc(%1)
  %13 = dbg_stmt()
  %14 = ret_type()
  %15 = const(TypedValue{ .ty = comptime_int, .val = 3})
  %16 = sub(%10, %15)
  %17 = as(%14, %16)
  %18 = return(%17)
} // fn_body main
```

I have not played around with very many test cases yet. Some interesting
ones that I want to look at before merging:

```zig
var x = blk: {
  var y = foo();
  y.a = 1;
  break :blk y;
};
```

In the above test case, x and y are supposed to alias.

```zig
var x = if (bar()) blk: {
  var y = foo();
  y.a = 1;
  break :blk y;
} else blk: {
  var z = baz();
  z.b = 1;
  break :blk z;
};
```

In the above test case, x, y, and z are supposed to alias.

I also haven't tested with `var` instead of `const` yet.
2020-12-31 01:54:02 -07:00
Andrew Kelley
3d79ae2be3 stage2 tests: improve C backend testing
Add -Werror -pedantic -std=c89 to make sure our outputted C code is
squeaky clean.

Allow garbage to be printed to stderr for Execution tests because
wasmtime is printing a bogus warning, causing the CI to fail.
2020-12-29 20:01:18 -07:00
Andrew Kelley
d18b6785bb stage2: C backend improvements
* Module: improve doc comments
 * C backend: improve const-correctness
 * C backend: introduce renderTypeAndName
 * C backend: put `static` on functions when appropriate
 * C backend: fix not handling errors in genBinOp
 * C backend: handle more IR instructions
   - alloc, store, boolean comparisons, ret_ptr
 * C backend: call instruction properly stores its result
 * test harness: ensure execution tests have empty stderr
2020-12-29 17:56:30 -07:00
Andrew Kelley
1590ed9d6a stage2 tests: pass cwd to child process to fix exe path
Previous commit broke the tests for non-Windows because we were
intending to change the cwd when running the child process. However, for
Windows we don't support passing a directory handle for cwd when
spawning child processes yet. However on Linux we do. This commit
reverts the previous one but then fixes things for all systems by
passing both cwd_dir and cwd to the child process.
2020-12-29 00:33:08 -07:00
Andrew Kelley
7e1e771f02 stage2 tests: fix incorrect path used for execution binary 2020-12-28 23:06:47 -08:00
Andrew Kelley
1dd5c032bf stage2 tests: better failure reporting for failed execution 2020-12-28 22:21:25 -07:00
Andrew Kelley
7561fca435 stage2: improve test harness to support executing generated C code 2020-12-28 20:01:17 -07:00
Andrew Kelley
a54ccd8537 stage2: C backend: implement @breakpoint and clean up test harness 2020-12-28 18:43:01 -07:00
Alex Cameron
0771aac48b Don't use emit-h in test cases for other backends. 2020-12-23 01:14:35 +11:00
Noam Preil
8d6066e09c Fix a dumb 2020-12-23 01:14:35 +11:00
Noam Preil
8538053940 Add header test harness 2020-12-23 01:14:35 +11:00
Andrew Kelley
e00b6db2aa update stage2 test harness to new std.Progress API 2020-12-20 15:08:59 -07:00
Andrew Kelley
0d1cd0d482 use kprotty's ThreadPool implementation (v5) 2020-12-20 15:08:59 -07:00
Andrew Kelley
013efaf139 std: introduce a thread-local CSPRNG for general use
std.crypto.random

* cross platform, even freestanding
* can't fail. on initialization for some systems requires calling
  os.getrandom(), in which case there are rare but theoretically
  possible errors. The code panics in these cases, however the
  application may choose to override the default seed function and then
  handle the failure another way.
* thread-safe
* supports the full Random interface
* cryptographically secure
* no syscall required to initialize on Linux (AT_RANDOM)
* calls arc4random on systems that support it

`std.crypto.randomBytes` is removed in favor of `std.crypto.random.bytes`.

I moved some of the Random implementations into their own files in the
interest of organization.

stage2 no longer requires passing a RNG; instead it uses this API.

Closes #6704
2020-12-18 12:22:46 -07:00
Isaac Freund
1d8f33ca98
stage2: link musl dynamically by default if native
If targeting the native OS and the system libc is musl, link against it
dynamically by default.
2020-12-13 00:40:35 +01:00
Andrew Kelley
2ed1ed9b32 stage2: introduce Module.failed_root_source_file
Use case:

zig build-exe non_existent_file.zig

Previous behavior:

error.FileNotFound, followed by an error return trace

Behavior after this commit:

error: unable to read non_existent_file.zig: FileNotFound
(end of stderr, exit code 1)

This turns AllErrors.Message into a tagged union which now has the
capability to represent both "plain" errors as well as source-based
errors (with file, line, column, byte offset). The "no entry point found"
error has moved to be a plain error message.
2020-12-04 17:21:55 -07:00
Andrew Kelley
6d91ebce45 wire up -Dskip-non-native to stage2 tests 2020-12-02 15:52:07 -07:00
Vexu
570f610341
stage2: fix test harness tmp path handling 2020-10-30 15:58:12 +02:00
Vexu
d819da4350
stage2: support multiple files in tests 2020-09-30 17:00:57 +03:00
Andrew Kelley
30dfdfdbd0 fix a round of regressions in this branch
* Don't try to generate C header files yet since it will only cause a
   crash saying the feature is unimplemented.
 * Rename the CLI options for release modes to use the `-O` prefix to
   match C compiler precedent. Options are now `-ODebug`,
   `-OReleaseFast`, `-OReleaseSafe`, `-OReleaseSmall`. The optimization
   mode matches the enum tags of std.builtin.Mode. It is planned to, at
   some point, rename std.builtin.Mode to std.builtin.OptimizationMode
   and modify the tags to be lower case to match the style convention.
   - Update build.zig code to support this new CLI.
 * update std.zig.binNameAlloc to support an optional Version and update
   the implementation to correctly deal with dynamic library version
   suffixes.
2020-09-24 23:50:15 -07:00
Andrew Kelley
dacd36ca9b stage2: implement using the global cache dir 2020-09-22 14:08:08 -07:00
Andrew Kelley
afac5d2895 fix regressed stage2 test harness 2020-09-21 21:14:01 -07:00
Andrew Kelley
528832bd3a rename src-self-hosted/ to src/ 2020-09-21 18:38:55 -07:00