105 Commits

Author SHA1 Message Date
Andrew Kelley
2456df5f4e stage2: rename ZigModule to Module 2020-09-13 19:56:35 -07:00
Andrew Kelley
4d59f77528 stage2: rename Module to Compilation 2020-09-13 19:49:52 -07:00
Andrew Kelley
2a8fc1a18e stage2: caching system integration & Module/Compilation splitting
* update to the new cache hash API
 * std.Target defaultVersionRange moves to std.Target.Os.Tag
 * std.Target.Os gains getVersionRange which returns a tagged union
 * start the process of splitting Module into Compilation and "zig
   module".
   - The parts of Module having to do with only compiling zig code are
     extracted into ZigModule.zig.
   - Next step is to rename Module to Compilation.
   - After that rename ZigModule back to Module.
 * implement proper cache hash usage when compiling C objects, and
   properly manage the file lock of the build artifacts.
 * make versions optional to match recent changes to master branch.
 * proper cache hash integration for compiling zig code
 * proper cache hash integration for linking even when not compiling zig
   code.
 * ELF LLD linking integrates with the caching system. A comment from
   the source code:

   Here we want to determine whether we can save time by not invoking LLD when the
   output is unchanged. None of the linker options or the object files that are being
   linked are in the hash that namespaces the directory we are outputting to. Therefore,
   we must hash those now, and the resulting digest will form the "id" of the linking
   job we are about to perform.
   After a successful link, we store the id in the metadata of a symlink named "id.txt" in
   the artifact directory. So, now, we check if this symlink exists, and if it matches
   our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD.

 * implement disable_c_depfile option
 * add tracy to a few more functions
2020-09-13 19:29:07 -07:00
Andrew Kelley
03a23418ff stage2: linking with LLD and building glibc static CRT files
* implement --debug-cc and --debug-link
 * implement C source files having extra flags
   - TODO a way to pass them on the CLI
 * introduce the Directory abstraction which contains both an open file
   descriptor and a file path name. The former is preferred but the
   latter is needed when communicating paths over a command line (e.g.
   to Clang or LLD).
 * use the cache hash to choose an artifact directory
   - TODO: use separate cache hash instances for the zig module and
     each C object
 * Module: introduce the crt_files table for keeping track of built libc
   artifacts for linking.
 * Add the ability to build 4/6 of the glibc static CRT lib files.
 * The zig-cache directory is now passed as a parameter to Module.
 * Implement the CLI logic of -femit-bin and -femit-h
   - TODO: respect -fno-emit-bin
   - TODO: the emit .h feature
 * Add the -fvalgrind, -fstack-check, and --single-threaded CLI options.
 * Implement the logic for auto detecting whether to enable PIC,
   sanitize-C, stack-check, valgrind, and single-threaded.
 * Properly add PIC args (or not) to clang argv.
 * Implement renaming clang-compiled object files into their proper
   place within the cache artifact directory.
   - TODO: std lib needs a proper higher level abstraction for
     std.os.renameat.
 * Package is cleaned up to use the "Unmanaged" StringHashMap and use the
   new Directory abstraction.
 * Clean up zig lib directory detection to make proper use of directory
   handles.
 * Linker code invokes LLD.
   - TODO properly deal with the stdout and stderr that we get from it
     and expose diagnostics from the Module API that match the expected
     error message format.
 * Delete the bitrotted LLVM C ABI bindings. We'll resurrect just the
   functions we need as we introduce dependencies on them. So far it
   only has ZigLLDLink in it.
 * Remove dead timer code.
 * `zig env` now prints the path to the zig executable as well.
2020-09-12 00:51:06 -07:00
Andrew Kelley
e05ecbf165 stage2: progress towards LLD linking
* add `zig libc` command
 * add `--libc` CLI and integrate it with Module and linker code
 * implement libc detection and paths resolution
 * port LLD ELF linker line construction to stage2
 * integrate dynamic linker option into Module and linker code
 * implement default link_mode detection and error handling if
   user requests static when it cannot be fulfilled
 * integrate more linker options
 * implement detection of .so.X.Y.Z file extension as a shared object
   file. nice try, you can't fool me.
 * correct usage text for -dynamic and -static
2020-09-09 22:29:41 -07:00
Andrew Kelley
193ad413f0 stage2: compiling C objects with clang
* add target_util.zig which has ported code from src/target.cpp
 * Module gains an arena that owns memory used during initialization
   that has the same lifetime as the Module. Useful for constructing
   file paths and lists of strings that have mixed lifetimes.
   - The Module memory itself is allocated in this arena. init/deinit
     are modified to be create/destroy.
   - root_name moves to the arena and no longer needs manual free
 * implement the ability to invoke `zig clang` as a subprocess
   - there are lots of TODOs that should be solved before merging
 * Module now requires a Random object and zig_lib_dir
 * Module now requires a path to its own executable or any zig
   executable that can do `zig clang`.
 * Wire up more CLI options.
 * Module creates "zig-cache" directory and "tmp" and "o" subdirectories
   ("h" is created by the cache_hash)
 * stubbed out some of the things linker code needs to do with TODO
   prints
 * delete dead code for computing compiler id. the previous commit
   eliminated the need for it.
 * add `zig translate-c` CLI option but it's not fully hooked up yet.
   It should be possible for this to be fully wired up before merging
   this branch.
 * `zig targets` now uses canonical data for available_libcs
2020-09-09 09:28:05 -07:00
Andrew Kelley
c99e34a00e stage2: eliminate the "compiler id" concept
Instead, append a "dirty suffix" to the version string when there are
dirty git changes and use the version string as the compiler id.

This avoids a dependency on the cache hash system, and saves time on
first invocation of the compiler since it does not have to compute its
compiler id. It also saves time by not having to check the cache for a
saved compiler id.
2020-09-09 09:28:05 -07:00
Andrew Kelley
4056bb92e6 stage2: more progress moving zig cc to stage2
* std.cache_hash exposes Hasher type
 * std.cache_hash makes hasher_init a global const
 * std.cache_hash supports cloning so that clones can share the same
   open manifest dir handle as well as fork from shared hasher state
 * start to populate the cache_hash for stage2 builds
 * remove a footgun from std.cache_hash add function
 * get rid of std.Target.ObjectFormat.unknown
 * rework stage2 logic for resolving output artifact names by adding
   object_format as an optional parameter to std.zig.binNameAlloc
 * support -Denable-llvm in stage2 tests
 * Module supports the use case when there are no .zig files
 * introduce c_object_table and failed_c_objects to Module
 * propagate many new kinds of data from CLI into Module and into
   linker.Options
 * introduce -fLLVM, -fLLD, -fClang and their -fno- counterparts.
   closes #6251.
   - add logic for choosing when to use LLD or zig's self-hosted linker
 * stub code for implementing invoking Clang to build C objects
 * add -femit-h, -femit-h=foo, and -fno-emit-h CLI options
2020-09-09 09:28:05 -07:00
Andrew Kelley
4c13d020db stage2: proper split of requireRuntimeBlock and requireFunctionBlock
* improve the ZIR generated of variable decls
   - utilize the same ZIR for the type and init value when possible
   - init value gets a result location with the variable type.
     no manual coercion is required.
 * no longer use return instructions to extract values out of comptime
   blocks. Instead run the analysis and then look at the corresponding
   analyzed instruction, relying on the comptime mechanism to report
   errors when something could not be comptime evaluated.
2020-09-01 12:39:47 -07:00
Andrew Kelley
54f3b0a560 stage2: clean up SPU Mk II code
* move SPU code from std to self hosted compiler
 * change std lib comments to be descriptive rather than prescriptive
 * avoid usingnamespace
 * fix case style of error codes
 * remove duplication of producer_string
 * generalize handling of less than 64 bit arch pointers
 * clean up SPU II related test harness code
2020-08-22 13:36:08 -07:00
Noam Preil
ad9df43e49 Tests: limit SPU-II cycle count 2020-08-22 12:45:29 -07:00
Noam Preil
096c5d5e4b Tests: implement SPU-II harness 2020-08-22 12:45:29 -07:00
Noam Preil
f2fef240a1 SPU-II: Test harness skeleton 2020-08-22 12:45:29 -07:00
Andrew Kelley
89b6c47e04 stage2: decouple codegen.zig from ELF
See #6113 for an alternate way of doing this that we didn't end up
following.

Closes #6079.

I also took the opportunity here to extract C.zig and Elf.zig from
link.zig.
2020-08-21 13:25:59 -07:00
Andrew Kelley
31b58acdae stage2: minor cleanup 2020-08-18 15:11:24 -07:00
Noam Preil
4d778e630a Tests: print generated C on test failure 2020-08-12 21:58:21 -07:00
Andrew Kelley
0347df82e8 improvements & fixes for general purpose allocator integration
* std.Mutex API is improved to not have init() deinit(). This API is
   designed to support static initialization and does not require any
   resource cleanup. This also happens to work around some kind of
   stage1 behavior that wasn't letting the new allocator mutex code
   get compiled.
 * the general purpose allocator now returns a bool from deinit()
   which tells if there were any leaks. This value is used by the test
   runner to fail the tests if there are any.
 * self-hosted compiler is updated to use the general purpose allocator
   when not linking against libc.
2020-08-07 23:26:58 -07:00
Andrew Kelley
d61a9e37ae stage2 tests: fix qemu logic
I made two mistakes in the previous commit; it was not actually using
the argv that we built, and also the qemu logic was unconditionally
skipping the test.

Now I have verified that when mangling the RISC-V "hello world" test and
then using -Denable-qemu, we get a test failure.
2020-08-04 15:39:59 -07:00
Andrew Kelley
c594f8dc26 stage2 tests: support the -Denable-qemu options and friends 2020-08-04 15:32:16 -07:00
Andrew Kelley
f1c1b8c02d stage2: add DWARF info for the main compilation unit 2020-07-31 06:48:17 +00:00
pixelherodev
2c882b2e65
CBE: Make C an ObjectFormat instead of a special bool (#5849) 2020-07-12 22:56:31 -04:00
Noam Preil
d060be8804
CBE: Don't expose openCFile, always close file after an update 2020-07-08 14:10:11 -04:00
Noam Preil
173e671241
CBE: Some cleanup 2020-07-07 23:11:17 -04:00
Noam Preil
b91cf15972
CBE: Move standards determination to generated code 2020-07-07 22:57:34 -04:00
Noam Preil
cf86aa8772
Fix a dumb in tests 2020-07-07 18:43:25 -04:00
Noam Preil
6ece36a051
Working translation of empty function 2020-07-07 17:51:59 -04:00
Noam Preil
2f28ecf946
CBE: Get test more useful 2020-07-07 17:06:07 -04:00
Noam Preil
aaaebfe97f
Detect unexpected compilation errors in tests 2020-07-07 16:47:39 -04:00
Noam Preil
a17200dab1
CBE skeleton 2020-07-07 16:40:14 -04:00
Noam Preil
b4c571301b
Stage2: Refactor in preparation for C backend 2020-07-07 14:55:44 -04:00
Noam Preil
0db0258fb2
Remove old comment 2020-07-06 17:54:06 -04:00
Noam Preil
38d2c5cdf1
Rename type -> extension 2020-06-27 21:39:04 -04:00
Noam Preil
1861c25142
Improve Tranform failure output 2020-06-27 21:15:23 -04:00
Noam Preil
54148a8c88
Stage2/TestHarness: Improve progress reporting 2020-06-27 20:01:20 -04:00
Noam Preil
52787f2c9b
Fix a dumb 2020-06-26 06:52:29 -04:00
Noam Preil
6d536168b0
Stage2/Testing: Update documentation 2020-06-26 06:51:35 -04:00
Noam Preil
0e952a9f3a
Stage2/Testing: Simply incremental compilation tests 2020-06-26 05:00:53 -04:00
Noam Preil
53fead580e
Add a compiles wrapper case 2020-06-26 03:09:56 -04:00
Noam Preil
649da2df52
Stage2/Testing: Add convenience wrappers 2020-06-26 02:42:02 -04:00
Noam Preil
c88edbc46f
OOM -> catch unreachable 2020-06-24 23:34:58 -04:00
Noam Preil
5d7e981f95
Clean up test harness 2020-06-24 22:43:18 -04:00
Andrew Kelley
20b4a2cf2c self-hosted: add compare output test for new AST->ZIR code 2020-06-24 21:28:42 -04:00
Andrew Kelley
5aa3f56773 self-hosted: fix test regressions
I'm allowing incremental compilation of ZIR modules to be broken. This
is not a real use case of ZIR, and the feature requires a lot of code
duplication with incremental compilation of Zig AST (which works great).
2020-06-24 20:28:52 -04:00
Noam Preil
a99e61ebaa
Stage2/Testing: Code cleanup 2020-06-15 21:47:42 -04:00
Noam Preil
afec3e72f4
Stage2/Testing: Enable another test 2020-06-15 20:42:22 -04:00
Noam Preil
7d1c9a69cc
Stage2/Testing: Remove dead code 2020-06-15 20:33:43 -04:00
Noam Preil
adb21f1caf
Stage2/Testing: Add error tests to ZIRCase 2020-06-15 20:33:43 -04:00
Noam Preil
7ee0462f5f
Stage2/Testing: Fix transformation tests 2020-06-15 20:33:39 -04:00
Noam Preil
1e5945d0a9
Stage2/Testing: remove ZIRTransformCase 2020-06-15 20:33:39 -04:00
Noam Preil
71dca252a5
Stage2/Testing: Rename stage -> update 2020-06-15 20:33:32 -04:00