216 Commits

Author SHA1 Message Date
Andrew Kelley
03a23418ff stage2: linking with LLD and building glibc static CRT files
* implement --debug-cc and --debug-link
 * implement C source files having extra flags
   - TODO a way to pass them on the CLI
 * introduce the Directory abstraction which contains both an open file
   descriptor and a file path name. The former is preferred but the
   latter is needed when communicating paths over a command line (e.g.
   to Clang or LLD).
 * use the cache hash to choose an artifact directory
   - TODO: use separate cache hash instances for the zig module and
     each C object
 * Module: introduce the crt_files table for keeping track of built libc
   artifacts for linking.
 * Add the ability to build 4/6 of the glibc static CRT lib files.
 * The zig-cache directory is now passed as a parameter to Module.
 * Implement the CLI logic of -femit-bin and -femit-h
   - TODO: respect -fno-emit-bin
   - TODO: the emit .h feature
 * Add the -fvalgrind, -fstack-check, and --single-threaded CLI options.
 * Implement the logic for auto detecting whether to enable PIC,
   sanitize-C, stack-check, valgrind, and single-threaded.
 * Properly add PIC args (or not) to clang argv.
 * Implement renaming clang-compiled object files into their proper
   place within the cache artifact directory.
   - TODO: std lib needs a proper higher level abstraction for
     std.os.renameat.
 * Package is cleaned up to use the "Unmanaged" StringHashMap and use the
   new Directory abstraction.
 * Clean up zig lib directory detection to make proper use of directory
   handles.
 * Linker code invokes LLD.
   - TODO properly deal with the stdout and stderr that we get from it
     and expose diagnostics from the Module API that match the expected
     error message format.
 * Delete the bitrotted LLVM C ABI bindings. We'll resurrect just the
   functions we need as we introduce dependencies on them. So far it
   only has ZigLLDLink in it.
 * Remove dead timer code.
 * `zig env` now prints the path to the zig executable as well.
2020-09-12 00:51:06 -07:00
Andrew Kelley
8cf40f3445 stage2: loading glibc metadata 2020-09-10 22:24:27 -07:00
Andrew Kelley
e05ecbf165 stage2: progress towards LLD linking
* add `zig libc` command
 * add `--libc` CLI and integrate it with Module and linker code
 * implement libc detection and paths resolution
 * port LLD ELF linker line construction to stage2
 * integrate dynamic linker option into Module and linker code
 * implement default link_mode detection and error handling if
   user requests static when it cannot be fulfilled
 * integrate more linker options
 * implement detection of .so.X.Y.Z file extension as a shared object
   file. nice try, you can't fool me.
 * correct usage text for -dynamic and -static
2020-09-09 22:29:41 -07:00
Andrew Kelley
b37955f273 stage2 linker code supports opening an intermediate object file
For when linking with LLD, we always create an object rather than going
straight to the executable. Next step is putting this object on the LLD
linker line.
2020-09-09 10:54:40 -07:00
Andrew Kelley
193ad413f0 stage2: compiling C objects with clang
* add target_util.zig which has ported code from src/target.cpp
 * Module gains an arena that owns memory used during initialization
   that has the same lifetime as the Module. Useful for constructing
   file paths and lists of strings that have mixed lifetimes.
   - The Module memory itself is allocated in this arena. init/deinit
     are modified to be create/destroy.
   - root_name moves to the arena and no longer needs manual free
 * implement the ability to invoke `zig clang` as a subprocess
   - there are lots of TODOs that should be solved before merging
 * Module now requires a Random object and zig_lib_dir
 * Module now requires a path to its own executable or any zig
   executable that can do `zig clang`.
 * Wire up more CLI options.
 * Module creates "zig-cache" directory and "tmp" and "o" subdirectories
   ("h" is created by the cache_hash)
 * stubbed out some of the things linker code needs to do with TODO
   prints
 * delete dead code for computing compiler id. the previous commit
   eliminated the need for it.
 * add `zig translate-c` CLI option but it's not fully hooked up yet.
   It should be possible for this to be fully wired up before merging
   this branch.
 * `zig targets` now uses canonical data for available_libcs
2020-09-09 09:28:05 -07:00
Andrew Kelley
c99e34a00e stage2: eliminate the "compiler id" concept
Instead, append a "dirty suffix" to the version string when there are
dirty git changes and use the version string as the compiler id.

This avoids a dependency on the cache hash system, and saves time on
first invocation of the compiler since it does not have to compute its
compiler id. It also saves time by not having to check the cache for a
saved compiler id.
2020-09-09 09:28:05 -07:00
Andrew Kelley
35f334ae0f organize some TODO comments 2020-09-09 09:28:05 -07:00
Andrew Kelley
4056bb92e6 stage2: more progress moving zig cc to stage2
* std.cache_hash exposes Hasher type
 * std.cache_hash makes hasher_init a global const
 * std.cache_hash supports cloning so that clones can share the same
   open manifest dir handle as well as fork from shared hasher state
 * start to populate the cache_hash for stage2 builds
 * remove a footgun from std.cache_hash add function
 * get rid of std.Target.ObjectFormat.unknown
 * rework stage2 logic for resolving output artifact names by adding
   object_format as an optional parameter to std.zig.binNameAlloc
 * support -Denable-llvm in stage2 tests
 * Module supports the use case when there are no .zig files
 * introduce c_object_table and failed_c_objects to Module
 * propagate many new kinds of data from CLI into Module and into
   linker.Options
 * introduce -fLLVM, -fLLD, -fClang and their -fno- counterparts.
   closes #6251.
   - add logic for choosing when to use LLD or zig's self-hosted linker
 * stub code for implementing invoking Clang to build C objects
 * add -femit-h, -femit-h=foo, and -fno-emit-h CLI options
2020-09-09 09:28:05 -07:00
Andrew Kelley
472ee18486 stage2: infer --name in more ways 2020-09-09 09:28:05 -07:00
Andrew Kelley
f064f0564f stage2: improve log message format 2020-09-09 09:28:05 -07:00
Andrew Kelley
71687b30a2 work around stage1 invalid LLVM IR 2020-09-09 09:28:05 -07:00
Andrew Kelley
503ba7b27c start moving zig cc to stage2
* build.zig: repair the ability to link against llvm, clang, and lld
 * move the zig cc arg parsing logic to stage2
   - the preprocessor flag is still TODO
   - the clang arg iterator code is improved to use slices instead of
     raw pointers because it no longer has to deal with an extern
     struct.
 * clean up error printing with a `fatal` function and use log API
   for messages rather than std.debug.print
 * add support for more CLI options to stage2 & update usage text
   - hooking up most of these new options is TODO
 * clean up the way libc and libc++ are detected via command line
   options. target information is used to determine if any of the libc
   candidate names are chosen.
 * add native library directory detection
 * implement the ability to invoke clang from stage2
 * introduce a build_options.have_llvm so we can comptime branch
   on whether LLVM is linked in or not.
2020-09-09 09:28:05 -07:00
Andrew Kelley
d7268cbb24
Merge pull request #6232 from LemonBoy/fix-readall
std: Don't trust stat() size in readAllAlloc fns
2020-09-06 19:19:57 -04:00
LemonBoy
90743881cf std: Minor changes to the fs module
* Add a size_hint parameter to the read{toEnd,File}AllocOptions fns
* Rename readAllAlloc{,Options} to readToEndAlloc{,Options} as they
  don't rewind the file before reading
* Fix missing rewind in test case
2020-09-04 10:17:00 +02:00
Alexandros Naskos
e9807418e7 Added .pe ObjectFormat
MachO linker no longer collects unused dwarf debug information
2020-09-04 05:22:26 +03:00
Alexandros Naskos
e9b137f23a Completed basic PE linker for stage2
Added std.coff.MachineType
Added image characteristic and section flag valued to std.coff
Added std.Target.Cpu.Arch.toCoffMachine
Fixed stage2 --watch flag on windows
2020-09-04 05:15:03 +03:00
Lachlan Easton
bb848dbeee zig fmt: Patch rename stream to ais (auto indenting stream) & other small refactors 2020-09-02 20:16:28 +10:00
LemonBoy
73a8c9beaa std: Don't trust stat() size in readAllAlloc fns
Some files such as the ones in /proc report a st_size of zero, try to
read the file anyway if we hit that case.
2020-09-02 11:11:57 +02:00
Lachlan Easton
671b3abe5d Merge remote-tracking branch 'ziglang/master' into fmt_indentation 2020-09-01 18:41:23 +10:00
Andrew Kelley
212fe21d68 zen of zig: reword the last one a little bit 2020-08-31 14:40:08 -07:00
Lachlan Easton
a72b9d403d Refactor zig fmt indentation. Remove indent from rendering code and have a stream handle automatic indentation 2020-08-29 13:35:00 +10:00
Andrew Kelley
b1c7334355
Merge pull request #5745 from lun-4/ebadf-error
map EBADF to error values for read and write
2020-08-19 23:46:22 -04:00
Andrew Kelley
3cc1f8b624
Merge pull request #6056 from ifreund/wasm-backend
stage2: add a wasm backend
2020-08-18 00:28:05 -04:00
Andrew Kelley
502d413621 simplify zig info and rename it to zig env
also add version to it
2020-08-17 17:06:43 -07:00
Sergey Poznyak
80e70735fb add zig info command 2020-08-17 16:35:32 -07:00
Isaac Freund
3370b5f109
stage2/wasm: implement basic container generation
Thus far, we only generate the type, function, export, and code
sections. These are sufficient to generate and export simple functions.

Codegen is currently hardcoded to `i32.const 42`, the main goal of this
commit is to create infrastructure for the container format which will
work with incremental compilation.
2020-08-18 00:32:58 +02:00
Andrew Kelley
624e643872
Merge pull request #6046 from heidezomp/std-log-scoped-part2
std.log: (breaking) remove scope parameter from logging functions
2020-08-17 16:59:27 -04:00
Andrew Kelley
8d8d568854 stage2: implement zig version 2020-08-15 20:04:08 -04:00
heidezomp
16d118a8d9 update std and src-self-hosted for std.log breaking change 2020-08-13 17:14:15 +02:00
Andrew Kelley
a2a5cea286 stage2: emit DW_TAG_subprogram for function Decls
these have the virtual address range, return type, and name.
2020-08-11 19:02:21 -07:00
Andrew Kelley
20510d209b GeneralPurposeAllocator: use std.log instead of std.debug.print
`std.builtin.StackTrace` gains a `format` function.

GeneralPurposeAllocator uses `std.log.err` instead of directly printing
to stderr. Some errors are recoverable.

The test runner is modified to fail the test run if any log messages of
"err" or worse severity are encountered.

self-hosted is modified to always print log messages of "err" severity
or worse even if they have not been explicitly enabled.

This makes GeneralPurposeAllocator available on the freestanding target.
2020-08-11 02:01:32 -04:00
Andrew Kelley
fd47839064 stage2: fix crash on empty source file 2020-08-08 18:19:48 -07:00
Andrew Kelley
0347df82e8 improvements & fixes for general purpose allocator integration
* std.Mutex API is improved to not have init() deinit(). This API is
   designed to support static initialization and does not require any
   resource cleanup. This also happens to work around some kind of
   stage1 behavior that wasn't letting the new allocator mutex code
   get compiled.
 * the general purpose allocator now returns a bool from deinit()
   which tells if there were any leaks. This value is used by the test
   runner to fail the tests if there are any.
 * self-hosted compiler is updated to use the general purpose allocator
   when not linking against libc.
2020-08-07 23:26:58 -07:00
Andrew Kelley
8824491fc7 stage2: add missing newlines to cli error messages 2020-08-04 18:38:42 -07:00
Andrew Kelley
edfede575c self-hosted: add build option for log scopes
Now you can enable a set of log scopes by passing -Dlog=<scope>
2020-08-03 21:56:21 -07:00
Andrew Kelley
f1c1b8c02d stage2: add DWARF info for the main compilation unit 2020-07-31 06:48:17 +00:00
Andrew Kelley
aac6e8c418 self-hosted: AST flattening, astgen improvements, result locations, and more
* AST: flatten ControlFlowExpression into Continue, Break, and Return.
 * AST: unify identifiers and literals into the same AST type: OneToken
 * AST: ControlFlowExpression uses TrailerFlags to optimize storage
   space.
 * astgen: support `var` as well as `const` locals, and support
   explicitly typed locals. Corresponding Module and codegen code is not
   implemented yet.
 * astgen: support result locations.
 * ZIR: add the following instructions (see the corresponding doc
   comments for explanations of semantics):
   - alloc
   - alloc_inferred
   - bitcast_result_ptr
   - coerce_result_block_ptr
   - coerce_result_ptr
   - coerce_to_ptr_elem
   - ensure_result_used
   - ensure_result_non_error
   - ret_ptr
   - ret_type
   - store
   - param_type
 * the skeleton structure for result locations is set up. It's looking
   pretty clean so far.
 * add compile error for unused result and compile error for discarding
   errors.
 * astgen: split builtin calls up to implemented manually, and implement
   `@as`, `@bitCast` (and others) with respect to result locations.
 * add CLI support for hex and raw object formats. They are not
   supported by the self-hosted compiler yet, and emit errors.
 * rename `--c` CLI to `-ofmt=[objectformat]` which can be any of the
   object formats. Only ELF and C are supported so far. Also added missing
   help to the help text.
 * Remove hard tabs from C backend test cases. Shame on you Noam, you
   are grounded, you should know better, etc. Bad boy.
 * Delete C backend code and test case that relied on comptime_int
   incorrectly making it all the way to codegen.
2020-07-23 23:05:26 -07:00
pixelherodev
2c882b2e65
CBE: Make C an ObjectFormat instead of a special bool (#5849) 2020-07-12 22:56:31 -04:00
Vexu
e85fe13e44
run zig fmt on std lib and self hosted 2020-07-11 20:41:19 +03:00
Andrew Kelley
7bd0500589 Merge remote-tracking branch 'origin/master' into register-allocation 2020-07-08 20:46:06 -07:00
Noam Preil
e2aad33d4e
Stage2: facepalm. 2020-07-08 00:11:41 -04:00
Noam Preil
173e671241
CBE: Some cleanup 2020-07-07 23:11:17 -04:00
Noam Preil
b91cf15972
CBE: Move standards determination to generated code 2020-07-07 22:57:34 -04:00
Noam Preil
5461c482d0
CBE: Integrate into stage2 via --c-standard 2020-07-07 21:54:34 -04:00
Andrew Kelley
4d01385e14 fix liveness analysis and not correctly propagating link errors
We still flush the ELF file even when there are compile errors.
2020-07-07 03:48:20 +00:00
Andrew Kelley
8be8ebd698 stage2: skeleton codegen for x64 ADD
also rework Module to take advantage of the new hash map implementation.
2020-07-06 06:10:44 +00:00
Andrew Kelley
632acffcbd update std lib to new hash map API 2020-07-05 21:11:42 +00:00
Luna
f47c6d3c6b std.os: make EBADF return error for read and write 2020-06-28 18:40:43 -03:00
Andrew Kelley
20b4a2cf2c self-hosted: add compare output test for new AST->ZIR code 2020-06-24 21:28:42 -04:00
Andrew Kelley
5aa3f56773 self-hosted: fix test regressions
I'm allowing incremental compilation of ZIR modules to be broken. This
is not a real use case of ZIR, and the feature requires a lot of code
duplication with incremental compilation of Zig AST (which works great).
2020-06-24 20:28:52 -04:00