561 Commits

Author SHA1 Message Date
Linus Groh
4159add4ab std.fs.file: Rename File.Kind enum values to snake case 2023-05-25 20:17:07 +01:00
Linus Groh
0f6fa3f20b std: Move std.debug.{TTY.Config,detectTTYConfig} to std.io.tty
Also get rid of the TTY wrapper struct, which was exlusively used as a
namespace - this is done by the tty.zig root struct now.

detectTTYConfig has been renamed to just detectConfig, which is enough
given the new namespace. Additionally, a doc comment had been added.
2023-05-24 10:15:02 +01:00
Andrew Kelley
f52189834c rename omit_pkg_fetching_code to only_core_functionality
No functional changes. This renames an internal build option to better
reflect how it is used.
2023-05-17 16:00:24 -07:00
Andrew Kelley
fea6157102 zig2.c: omit non-essential subcommands
When producing zig2.c, don't waste time emitting C code for subcommands
that won't be used, such as objcopy.

This takes zig2.c down from 111M to 109M, and sidesteps some unfortunate
warnings that are currently emitted by GCC.
2023-05-17 15:43:15 -07:00
Andrew Kelley
c1add1e19e
Merge pull request #15459 from motiejus/build-id-full
stage2: implement `--build-id` styles
2023-05-17 07:19:15 -07:00
Andrew Kelley
5b06daf52b zig cc: implement -### (dry run)
closes #7170
2023-05-17 05:18:51 -07:00
Andrew Kelley
728ce2d7c1 tweaks to --build-id
* build.zig: the result of b.option() can be assigned directly in many
   cases thanks to the return type being an optional
 * std.Build: make the build system aware of the
   std.Build.Step.Compile.BuildId type when used as an option.
   - remove extraneous newlines in error logs
 * simplify caching logic
 * simplify hexstring parsing tests and use a doc test
 * simplify hashing logic. don't use an optional when the `none` tag
   already provides this meaning.
 * CLI: fix incorrect linker arg parsing
2023-05-16 20:39:01 -07:00
Motiejus Jakštys
df5085bde0 stage2: implement --build-id styles 2023-05-16 20:38:39 -07:00
Andrew Kelley
233a0d3991 CLI: remove cleanup logic for stdin temp file
In one of the happy paths, execve() is used to switch to clang in which
case any cleanup logic that exists for this temporary file will not run
and this temp file will be leaked. Oh well. It's a minor punishment for
using `-x c` which nobody should be doing. Therefore, we make no effort
to clean up. Using `-` for stdin as a source file always leaks a temp
file.

Note that the standard `zig build-exe` CLI does not support stdin as an
input file. This is only for `zig cc` C compiler compatibility.
2023-05-16 16:41:22 -07:00
Andrew Kelley
ae119a9a8d CLI: fix stdin dumping behavior
* no need to move `tmpFilePath` around
 * no need for calculating max length of `FileExt` tag name
 * provide a canonical file extension name for `FileExt` so that, e.g.
   the file will be named `stdin.S` instead of
   `stdin.assembly_with_cpp`.
 * move temp file cleanup to a function to reduce defer bloat in a large
   function.
 * fix bug caused by mixing relative and absolute paths in the cleanup
   logic.
 * remove commented out test and dead code
2023-05-16 16:41:22 -07:00
Motiejus Jakštys
2e692312f1 zig cc: support reading from non-files
echo 'some C program' | $CC -x c -

Is a common pattern to test for compiler or linker features. This patch
adds support for reading from non-regular files.

This will make at least one more Go test to pass.
2023-05-16 16:40:58 -07:00
Jacob Young
d70853ba39 main: add debug dump-zir command 2023-05-05 11:00:44 -07:00
kcbanner
440b3df702 main: parse --dynamicbase linker arg 2023-04-29 22:19:02 +02:00
zooster
bc8e1e1de4
Improvements to docs and text
* docs(std.math): elaborate on difference between absCast and absInt

* docs(std.rand.Random.weightedIndex): elaborate on likelihood

I think this makes it easier to understand.

* langref: add small reminder

* docs(std.fs.path.extension): brevity

* docs(std.bit_set.StaticBitSet): mention the specific types

* std.debug.TTY: explain what purpose this struct serves

This should also make it clearer that this struct is not supposed to provide unrelated terminal manipulation functionality such as setting the cursor position or something because terminals are complicated and we should keep this struct simple and focused on debugging.

* langref(package listing): brevity

* langref: explain what exactly `threadlocal` causes to happen

* std.array_list: link between swapRemove and orderedRemove

Maybe this can serve as a TLDR and make it easier to decide.

* PrefetchOptions.locality: clarify docs that this is a range

This confused me previously and I thought I can only use either 0 or 3.

* fix typos and more

* std.builtin.CallingConvention: document some CCs

* langref: explain possibly cryptic names

I think it helps knowing what exactly these acronyms (@clz and @ctz) and
abbreviations (@popCount) mean.

* variadic function error: add missing preposition

* std.fmt.format docs: nicely hyphenate

* help menu: say what to optimize for

I think this is slightly more specific than just calling it
"optimizations". These are speed optimizations. I used the word
"performance" here.
2023-04-23 21:06:21 +03:00
Andrew Kelley
ceff278202 fixes to the previous commit
* CompileStep: Avoid calling producesPdbFile() to determine whether the
   option should be respected. If the user asks for it, put it on the
   command line and let the Zig CLI deal with it appropriately.
 * Make the namespace of `std.dwarf.Format.dwarf32` no longer have a
   redundant "dwarf" in it.
 * Add `zig cc` integration for `-gdwarf32` and `-gdwarf64`.
 * Toss in a bonus bug fix for `-gdwarf-2`, `-gdwarf-3`, etc.
 * Avoid using default init values for struct fields unnecessarily.
 * Add missing cache hash addition for the new option.
2023-04-20 15:17:07 -07:00
David Gonzalez Martin
d026202a26 Expose an option for producing 64-bit DWARF format
This commit enables producing 64-bit DWARF format for Zig executables
that are produced through the LLVM backend. This is achieved by exposing
both command-line flags and CompileStep flags. The production of the
64-bit format only affects binaries that use the DWARF format and it is
disabled on MacOS due to it being problematic. This commit, despite
generating the interface for the Zig user to be able to tell the compile
which format is wanted, is just implemented for the LLVM backend, so
clang and the self-hosted backends will need this to be implemented in a
future commit.

This is an effort to work around #7962, since the emission of the 64-bit
format automatically produces 64-bit relocations. Further investigation
will be needed to make DWARF 32-bit format to emit bigger relocations
when needed and not make the linker angry.
2023-04-20 14:46:53 -07:00
kcbanner
40e1fca34b compilation: fix non-zig compilations not using CacheMode.whole
main: consume --debug-log argument even when logging is disabled
2023-04-17 13:46:14 -07:00
Luuk de Gram
61c08d3c7e fix zig cc linker flags for Wasm
Closes #15258
2023-04-14 15:18:18 -04:00
Andrew Kelley
c96cb98ad1 CLI: remove --enable-cache option
This use case is now handled instead by the --listen option.

closes #15025
closes #15072
2023-04-11 08:41:24 -07:00
Andrew Kelley
c22a30ac99 zig cc: handle the -r flag
This makes -r treated the same as -c which is to output an object file.
Zig's ELF linker code already handles multiple object files into an
object file with the -r flag to LLD.

closes #11683
2023-04-08 15:48:27 -04:00
Andrew Kelley
b086b7da9e zig cc: complete the -wrap flag implementation
* use a set instead of a list
* use of this flag currently requires LLD
* add documentation
* make it only a zig cc compatibility flag for now because I personally
  think this is an anti-feature.
2023-04-07 07:54:47 -07:00
Zach Cheung
1fdea551b2 add linker -wrap flag 2023-04-07 09:09:38 -04:00
Jakub Konka
a88c0b4d08 link: handle -u flag in all linkers
Also clean up parsing of linker args - reuse `ArgsIterator`.

In MachO, ensure we add every symbol marked with `-u` as undefined
before proceeding with symbol resolution. Additionally, ensure those
symbols are never garbage collected.

MachO entry_in_dylib test: pass `-u _my_main` when linking executable
so that it is not incorrectly garbage collected by the linker.
2023-04-01 14:22:44 +02:00
Jakub Konka
22e1b03360 coff: use copy in zig-cache for child process in HCS
Ideally, we would just do an atomic rename, but so far I had no
luck. I have also tried marking the file to delete-on-close but
then we cannot use it to spawn the process. So for now, let's just
put a copy in `zig-cache` and let the user decide when to recycle
the cache dir.
2023-03-30 21:08:30 +02:00
kcbanner
43ff6c14f9 main: recognize --dynamicbase 2023-03-26 13:10:36 -04:00
kcbanner
113f80bcf7 coff: change dynamicbase to default to true (to match lld), change it to pass the negation to lld, and add --no-dynamicbase
build: expose linker_dynamicbase on CompileStep and map it to emit --no-dynamicbase
2023-03-25 16:26:48 -04:00
Jakub Konka
6f15eedff1 darwin: put posix spawn constants in POSIX_SPAWN struct 2023-03-18 21:54:05 +01:00
Jakub Konka
37192bcdcb macos: HCS PoC working 2023-03-18 21:53:36 +01:00
Jakub Konka
f1e25cf43e macho: add hot-code swapping poc 2023-03-18 21:53:26 +01:00
square
887abd0f33 delete --prominent-compile-errors from help 2023-03-18 20:13:52 +02:00
Jacob Young
cfcd6698cd main: add debug option to dump unoptimized llvm ir 2023-03-17 01:57:14 -04:00
Andrew Kelley
363d4a107d add compile log output to build runner 2023-03-15 10:48:15 -07:00
Andrew Kelley
bde1293093 CLI: remove the experimental --watch flag
The compiler REPL will move to an external process that communicates
with the compiler over the binary protocol.
2023-03-15 10:48:15 -07:00
Andrew Kelley
ede5dcffea make the build runner and test runner talk to each other
std.Build.addTest creates a CompileStep as before, however, this kind of
step no longer actually runs the unit tests. Instead it only compiles
it, and one must additionally create a RunStep from the CompileStep in
order to actually run the tests.

RunStep gains integration with the default test runner, which now
supports the standard --listen=- argument in order to communicate over
stdin and stdout. It also reports test statistics; how many passed,
failed, and leaked, as well as directly associating the relevant stderr
with the particular test name that failed.

This separation of CompileStep and RunStep means that
`CompileStep.Kind.test_exe` is no longer needed, and therefore has been
removed in this commit.

 * build runner: show unit test statistics in build summary
 * added Step.writeManifest since many steps want to treat it as a
   warning and emit the same message if it fails.
 * RunStep: fixed error message that prints the failed command printing
   the original argv and not the adjusted argv in case an interpreter
   was used.
 * RunStep: fixed not passing the command line arguments to the
   interpreter.
 * move src/Server.zig to std.zig.Server so that the default test runner
   can use it.
 * the simpler test runner function which is used by work-in-progress
   backends now no longer prints to stderr, which is necessary in order
   for the build runner to not print the stderr as a warning message.
2023-03-15 10:48:14 -07:00
Andrew Kelley
a333bb91ff zig objcopy: support the compiler protocol
This commit extracts out server code into src/Server.zig and uses it
both in the main CLI as well as `zig objcopy`.

std.Build.ObjCopyStep now adds `--listen=-` to the CLI for `zig objcopy`
and observes the protocol for progress and other kinds of integrations.

This fixes the last two test failures of this branch when I run
`zig build test` locally.
2023-03-15 10:48:14 -07:00
Andrew Kelley
7cc4a6965c build runner enhancements in preparation for test-cases
* std.zig.ErrorBundle: support rendering options for whether to include
   the reference trace, whether to include the source line, and TTY
   configuration.

 * build runner: don't print progress in dumb terminals

 * std.Build.CompileStep:
   - add a way to expect compilation errors via the new `expect_errors`
     field. This is an advanced setting that can change the intent of
     the CompileStep. If this slice has nonzero length, it means that
     the CompileStep exists to check for compile errors and return
     *success* if they match, and failure otherwise.
   - remove the object format parameter from `checkObject`. The object
     format is known based on the CompileStep's target.
   - Avoid passing -L and -I flags for nonexistent directories within
     search_prefixes. This prevents a warning, that should probably be
     upgraded to an error in Zig's CLI parsing code, when the linker
     sees an -L directory that does not exist.

 * std.Build.Step:
   - When spawning the zig compiler process, takes advantage of the new
     `std.Progress.Node.setName` API to avoid ticking up a meaningless
     number at every progress update.
2023-03-15 10:48:14 -07:00
Andrew Kelley
7106a91b09 CLI: fix ast-check printing ZIR errors twice 2023-03-15 10:48:14 -07:00
Andrew Kelley
2996eb5587 std.Build.RunStep: add maxrss, duration, and cached status 2023-03-15 10:48:13 -07:00
Andrew Kelley
677a0e2941 stage2: avoid bloat when using -Donly-c 2023-03-15 10:48:13 -07:00
Andrew Kelley
d6f7766da2 stage2: avoid networking when generating zig2.c
Avoid dragging networking into zig2.c because it adds dependencies on
some linker symbols that are annoying to satisfy while bootstrapping.
2023-03-15 10:48:13 -07:00
Andrew Kelley
9bf63b0996 stage2: avoid linux-only APIs on other operating systems 2023-03-15 10:48:13 -07:00
Andrew Kelley
58edefc6d1 zig build: many enhancements related to parallel building
Rework std.Build.Step to have an `owner: *Build` field. This
simplified the implementation of installation steps, as well as provided
some much-needed common API for the new parallelized build system.

--verbose is now defined very concretely: it prints to stderr just
before spawning a child process.

Child process execution is updated to conform to the new
parallel-friendly make() function semantics.

DRY up the failWithCacheError handling code. It now integrates properly
with the step graph instead of incorrectly dumping to stderr and calling
process exit.

In the main CLI, fix `zig fmt` crash when there are no errors and stdin
is used.

Deleted steps:
 * EmulatableRunStep - this entire thing can be removed in favor of a
   flag added to std.Build.RunStep called `skip_foreign_checks`.
 * LogStep - this doesn't really fit with a multi-threaded build runner
   and is effectively superseded by the new build summary output.

build runner:
 * add -fsummary and -fno-summary to override the default behavior,
   which is to print a summary if any of the build steps fail.
 * print the dep prefix when emitting error messages for steps.

std.Build.FmtStep:
 * This step now supports exclude paths as well as a check flag.
 * The check flag decides between two modes, modify mode, and check
   mode. These can be used to update source files in place, or to fail
   the build, respectively.

Zig's own build.zig:
 * The `test-fmt` step will do all the `zig fmt` checking that we expect
   to be done. Since the `test` step depends on this one, we can simply
   remove the explicit call to `zig fmt` in the CI.
 * The new `fmt` step will actually perform `zig fmt` and update source
   files in place.

std.Build.RunStep:
 * expose max_stdio_size is a field (previously an unchangeable
   hard-coded value).
 * rework the API. Instead of configuring each stream independently,
   there is a `stdio` field where you can choose between
   `infer_from_args`, `inherit`, or `check`. These determine whether the
   RunStep is considered to have side-effects or not. The previous
   field, `condition` is gone.
 * when stdio mode is set to `check` there is a slice of any number of
   checks to make, which include things like exit code, stderr matching,
   or stdout matching.
 * remove the ill-defined `print` field.
 * when adding an output arg, it takes the opportunity to give itself a
   better name.
 * The flag `skip_foreign_checks` is added. If this is true, a RunStep
   which is configured to check the output of the executed binary will
   not fail the build if the binary cannot be executed due to being for
   a foreign binary to the host system which is running the build graph.
   Command-line arguments such as -fqemu and -fwasmtime may affect
   whether a binary is detected as foreign, as well as system
   configuration such as Rosetta (macOS) and binfmt_misc (Linux).
   - This makes EmulatableRunStep no longer needed.
 * Fix the child process handling to properly integrate with the new
   bulid API and to avoid deadlocks in stdout/stderr streams by polling
   if necessary.

std.Build.RemoveDirStep now uses the open build_root directory handle
instead of an absolute path.
2023-03-15 10:48:13 -07:00
Andrew Kelley
b5baa41077 fix zig fmt crash 2023-03-15 10:48:13 -07:00
Andrew Kelley
8b2d872020 fix std.Build.TranslateCStep 2023-03-15 10:48:13 -07:00
Andrew Kelley
0e078790fe multiplex compiler progress messages into the build runner 2023-03-15 10:48:13 -07:00
Andrew Kelley
8acbfafefb compiler: update function accepts a std.Progress.Node
This makes progress be exposed to the top-level caller of update().

I tossed in a bonus change: when the `zig build` subcommand sees exit
code 2, it omits the "following command failed" line, and the build
runner uses exit code 2 when there are compile errors. This tidies up
the output on build failure by a little bit.
2023-03-15 10:48:13 -07:00
Andrew Kelley
986a30e373 integrate the build runner and the compiler server
The compiler now provides a server protocol for an interactive session
with another process. The build runner uses this protocol to communicate
compilation errors semantically from zig compiler subprocesses to the
build runner.

The protocol is exposed via stdin/stdout, or on a network socket,
depending on whether the CLI flag `--listen=-` or e.g.
`--listen=127.0.0.1:1337` is used.

Additionally:

 * add the zig version string to the build runner cache prefix

 * remove --prominent-compile-errors CLI flag because it no longer does
   anything. Compilation errors are now unconditionally displayed at the
   bottom of the build summary output when using the terminal-based
   build runner.

 * Remove the color field from std.Build. The build steps are no longer
   supposed to interact with stderr directly. Instead they communicate
   semantically back to the build runner, which has its own logic about
   TTY configuration.

 * Use the cleanExit() pattern in the build runner.

 * Build steps can now use error.MakeFailed when they have already
   properly reported an error, or they can fail with any other error
   code in which case the build runner will create a simple message
   based on this error code.
2023-03-15 10:48:13 -07:00
Andrew Kelley
6f717b18f0 std.zig.ErrorBundle: rework binary encoding
* Separate into a "WIP" struct and a "finished" struct.
* Use a bit of indirection for error notes to simplify ergonomics of
  this data structure.
2023-03-15 10:48:13 -07:00
Andrew Kelley
572cb24d1a progress towards semantic error serialization
Introduces std.zig.ErrorBundle which is a trivially serializeable set
of compilation errors. This is in the standard library so that both
the compiler and the build runner can use it. The idea is they will
use it to communicate compilation errors over a binary protocol.

The binary encoding of ErrorBundle is a bit problematic - I got a little
too aggressive with compaction. I need to change it in a follow-up
commit to use some indirection in the error message list, otherwise
iteration is too unergonomic. In fact it's so problematic right now that
the logic getAllErrorsAlloc() actually fails to produce a viable
ErrorBundle because it puts SourceLocation data in between the root
level ErrorMessage data.

This commit has a simplification - redundant logic for rendering AST
errors to stderr has been removed in favor of moving the logic for
lowering AST errors into AstGen. So even if we get parse errors, the
errors will get lowered into ZIR before being reported. I believe this
will be useful when working on --autofix. Either way, some redundant
brittle logic was happily deleted.

In Compilation, updateSubCompilation() is improved to properly perform
error reporting when a sub-compilation object fails. It no longer dumps
directly to stderr; instead it populates an ErrorBundle object, which
gets added to the parent one during getAllErrorsAlloc().

In package fetching code, instead of dumping directly to stderr, it now
populates an ErrorBundle object, and gets properly reported at the CLI
layer of abstraction.
2023-03-15 10:48:12 -07:00
Andrew Kelley
ae8e7c8f5a stage2: hot code swapping PoC
* CLI supports --listen to accept commands on a socket
 * make it able to produce an updated executable while it is running
2023-03-15 10:48:12 -07:00