566 Commits

Author SHA1 Message Date
Andrew Kelley
629f0d23b5
Merge pull request #15579 from squeek502/mem-delimiters
Split `std.mem.split` and `tokenize` into `sequence`, `any`, and `scalar` versions
2023-06-03 13:51:02 -07:00
Evin Yulo
6c2f374556 Use the word 'base' consistently instead of 'radix' 2023-06-01 00:02:16 +03:00
Motiejus Jakštys
ac9f72d87e zig ld: handle --library :path/to/lib.so
`-l :path/to/lib.so` behavior on gcc/clang is:

- the path is recorded as-is: no paths, exact filename (`libX.so.Y`).
- no rpaths.

The previous version removed the `:` and pretended it's a positional
argument to the linker. That works in almost all cases, except in how
rules_go[1] does things (the Bazel wrapper for Go).

Test case in #15743, output:

    gcc rpath:
     0x0000000000000001 (NEEDED)     Shared library: [libversioned.so.2]
     0x000000000000001d (RUNPATH)    Library runpath: [$ORIGIN/x]
    gcc plain:
     0x0000000000000001 (NEEDED)     Shared library: [libversioned.so.2]
    zig cc rpath:
     0x0000000000000001 (NEEDED)     Shared library: [libversioned.so.2]
     0x000000000000001d (RUNPATH)    Library runpath: [$ORIGIN/x]
    zig cc plain:
     0x0000000000000001 (NEEDED)     Shared library: [libversioned.so.2]

Fixes #15743

[1]: https://github.com/bazelbuild/rules_go
2023-05-30 08:18:10 +02:00
Linus Groh
4159add4ab std.fs.file: Rename File.Kind enum values to snake case 2023-05-25 20:17:07 +01:00
Linus Groh
0f6fa3f20b std: Move std.debug.{TTY.Config,detectTTYConfig} to std.io.tty
Also get rid of the TTY wrapper struct, which was exlusively used as a
namespace - this is done by the tty.zig root struct now.

detectTTYConfig has been renamed to just detectConfig, which is enough
given the new namespace. Additionally, a doc comment had been added.
2023-05-24 10:15:02 +01:00
Andrew Kelley
f52189834c rename omit_pkg_fetching_code to only_core_functionality
No functional changes. This renames an internal build option to better
reflect how it is used.
2023-05-17 16:00:24 -07:00
Andrew Kelley
fea6157102 zig2.c: omit non-essential subcommands
When producing zig2.c, don't waste time emitting C code for subcommands
that won't be used, such as objcopy.

This takes zig2.c down from 111M to 109M, and sidesteps some unfortunate
warnings that are currently emitted by GCC.
2023-05-17 15:43:15 -07:00
Andrew Kelley
c1add1e19e
Merge pull request #15459 from motiejus/build-id-full
stage2: implement `--build-id` styles
2023-05-17 07:19:15 -07:00
Andrew Kelley
5b06daf52b zig cc: implement -### (dry run)
closes #7170
2023-05-17 05:18:51 -07:00
Andrew Kelley
728ce2d7c1 tweaks to --build-id
* build.zig: the result of b.option() can be assigned directly in many
   cases thanks to the return type being an optional
 * std.Build: make the build system aware of the
   std.Build.Step.Compile.BuildId type when used as an option.
   - remove extraneous newlines in error logs
 * simplify caching logic
 * simplify hexstring parsing tests and use a doc test
 * simplify hashing logic. don't use an optional when the `none` tag
   already provides this meaning.
 * CLI: fix incorrect linker arg parsing
2023-05-16 20:39:01 -07:00
Motiejus Jakštys
df5085bde0 stage2: implement --build-id styles 2023-05-16 20:38:39 -07:00
Andrew Kelley
233a0d3991 CLI: remove cleanup logic for stdin temp file
In one of the happy paths, execve() is used to switch to clang in which
case any cleanup logic that exists for this temporary file will not run
and this temp file will be leaked. Oh well. It's a minor punishment for
using `-x c` which nobody should be doing. Therefore, we make no effort
to clean up. Using `-` for stdin as a source file always leaks a temp
file.

Note that the standard `zig build-exe` CLI does not support stdin as an
input file. This is only for `zig cc` C compiler compatibility.
2023-05-16 16:41:22 -07:00
Andrew Kelley
ae119a9a8d CLI: fix stdin dumping behavior
* no need to move `tmpFilePath` around
 * no need for calculating max length of `FileExt` tag name
 * provide a canonical file extension name for `FileExt` so that, e.g.
   the file will be named `stdin.S` instead of
   `stdin.assembly_with_cpp`.
 * move temp file cleanup to a function to reduce defer bloat in a large
   function.
 * fix bug caused by mixing relative and absolute paths in the cleanup
   logic.
 * remove commented out test and dead code
2023-05-16 16:41:22 -07:00
Motiejus Jakštys
2e692312f1 zig cc: support reading from non-files
echo 'some C program' | $CC -x c -

Is a common pattern to test for compiler or linker features. This patch
adds support for reading from non-regular files.

This will make at least one more Go test to pass.
2023-05-16 16:40:58 -07:00
Ryan Liptak
2129f28953 Update all std.mem.split calls to their appropriate function
Everywhere that can now use `splitScalar` should get a nice little performance boost.
2023-05-13 13:45:05 -07:00
Ryan Liptak
ce9f3ec990 Fix SplitIterator and TokenIterator type instantiation 2023-05-13 13:43:50 -07:00
Jacob Young
d70853ba39 main: add debug dump-zir command 2023-05-05 11:00:44 -07:00
kcbanner
440b3df702 main: parse --dynamicbase linker arg 2023-04-29 22:19:02 +02:00
zooster
bc8e1e1de4
Improvements to docs and text
* docs(std.math): elaborate on difference between absCast and absInt

* docs(std.rand.Random.weightedIndex): elaborate on likelihood

I think this makes it easier to understand.

* langref: add small reminder

* docs(std.fs.path.extension): brevity

* docs(std.bit_set.StaticBitSet): mention the specific types

* std.debug.TTY: explain what purpose this struct serves

This should also make it clearer that this struct is not supposed to provide unrelated terminal manipulation functionality such as setting the cursor position or something because terminals are complicated and we should keep this struct simple and focused on debugging.

* langref(package listing): brevity

* langref: explain what exactly `threadlocal` causes to happen

* std.array_list: link between swapRemove and orderedRemove

Maybe this can serve as a TLDR and make it easier to decide.

* PrefetchOptions.locality: clarify docs that this is a range

This confused me previously and I thought I can only use either 0 or 3.

* fix typos and more

* std.builtin.CallingConvention: document some CCs

* langref: explain possibly cryptic names

I think it helps knowing what exactly these acronyms (@clz and @ctz) and
abbreviations (@popCount) mean.

* variadic function error: add missing preposition

* std.fmt.format docs: nicely hyphenate

* help menu: say what to optimize for

I think this is slightly more specific than just calling it
"optimizations". These are speed optimizations. I used the word
"performance" here.
2023-04-23 21:06:21 +03:00
Andrew Kelley
ceff278202 fixes to the previous commit
* CompileStep: Avoid calling producesPdbFile() to determine whether the
   option should be respected. If the user asks for it, put it on the
   command line and let the Zig CLI deal with it appropriately.
 * Make the namespace of `std.dwarf.Format.dwarf32` no longer have a
   redundant "dwarf" in it.
 * Add `zig cc` integration for `-gdwarf32` and `-gdwarf64`.
 * Toss in a bonus bug fix for `-gdwarf-2`, `-gdwarf-3`, etc.
 * Avoid using default init values for struct fields unnecessarily.
 * Add missing cache hash addition for the new option.
2023-04-20 15:17:07 -07:00
David Gonzalez Martin
d026202a26 Expose an option for producing 64-bit DWARF format
This commit enables producing 64-bit DWARF format for Zig executables
that are produced through the LLVM backend. This is achieved by exposing
both command-line flags and CompileStep flags. The production of the
64-bit format only affects binaries that use the DWARF format and it is
disabled on MacOS due to it being problematic. This commit, despite
generating the interface for the Zig user to be able to tell the compile
which format is wanted, is just implemented for the LLVM backend, so
clang and the self-hosted backends will need this to be implemented in a
future commit.

This is an effort to work around #7962, since the emission of the 64-bit
format automatically produces 64-bit relocations. Further investigation
will be needed to make DWARF 32-bit format to emit bigger relocations
when needed and not make the linker angry.
2023-04-20 14:46:53 -07:00
kcbanner
40e1fca34b compilation: fix non-zig compilations not using CacheMode.whole
main: consume --debug-log argument even when logging is disabled
2023-04-17 13:46:14 -07:00
Luuk de Gram
61c08d3c7e fix zig cc linker flags for Wasm
Closes #15258
2023-04-14 15:18:18 -04:00
Andrew Kelley
c96cb98ad1 CLI: remove --enable-cache option
This use case is now handled instead by the --listen option.

closes #15025
closes #15072
2023-04-11 08:41:24 -07:00
Andrew Kelley
c22a30ac99 zig cc: handle the -r flag
This makes -r treated the same as -c which is to output an object file.
Zig's ELF linker code already handles multiple object files into an
object file with the -r flag to LLD.

closes #11683
2023-04-08 15:48:27 -04:00
Andrew Kelley
b086b7da9e zig cc: complete the -wrap flag implementation
* use a set instead of a list
* use of this flag currently requires LLD
* add documentation
* make it only a zig cc compatibility flag for now because I personally
  think this is an anti-feature.
2023-04-07 07:54:47 -07:00
Zach Cheung
1fdea551b2 add linker -wrap flag 2023-04-07 09:09:38 -04:00
Jakub Konka
a88c0b4d08 link: handle -u flag in all linkers
Also clean up parsing of linker args - reuse `ArgsIterator`.

In MachO, ensure we add every symbol marked with `-u` as undefined
before proceeding with symbol resolution. Additionally, ensure those
symbols are never garbage collected.

MachO entry_in_dylib test: pass `-u _my_main` when linking executable
so that it is not incorrectly garbage collected by the linker.
2023-04-01 14:22:44 +02:00
Jakub Konka
22e1b03360 coff: use copy in zig-cache for child process in HCS
Ideally, we would just do an atomic rename, but so far I had no
luck. I have also tried marking the file to delete-on-close but
then we cannot use it to spawn the process. So for now, let's just
put a copy in `zig-cache` and let the user decide when to recycle
the cache dir.
2023-03-30 21:08:30 +02:00
kcbanner
43ff6c14f9 main: recognize --dynamicbase 2023-03-26 13:10:36 -04:00
kcbanner
113f80bcf7 coff: change dynamicbase to default to true (to match lld), change it to pass the negation to lld, and add --no-dynamicbase
build: expose linker_dynamicbase on CompileStep and map it to emit --no-dynamicbase
2023-03-25 16:26:48 -04:00
Jakub Konka
6f15eedff1 darwin: put posix spawn constants in POSIX_SPAWN struct 2023-03-18 21:54:05 +01:00
Jakub Konka
37192bcdcb macos: HCS PoC working 2023-03-18 21:53:36 +01:00
Jakub Konka
f1e25cf43e macho: add hot-code swapping poc 2023-03-18 21:53:26 +01:00
square
887abd0f33 delete --prominent-compile-errors from help 2023-03-18 20:13:52 +02:00
Jacob Young
cfcd6698cd main: add debug option to dump unoptimized llvm ir 2023-03-17 01:57:14 -04:00
Andrew Kelley
363d4a107d add compile log output to build runner 2023-03-15 10:48:15 -07:00
Andrew Kelley
bde1293093 CLI: remove the experimental --watch flag
The compiler REPL will move to an external process that communicates
with the compiler over the binary protocol.
2023-03-15 10:48:15 -07:00
Andrew Kelley
ede5dcffea make the build runner and test runner talk to each other
std.Build.addTest creates a CompileStep as before, however, this kind of
step no longer actually runs the unit tests. Instead it only compiles
it, and one must additionally create a RunStep from the CompileStep in
order to actually run the tests.

RunStep gains integration with the default test runner, which now
supports the standard --listen=- argument in order to communicate over
stdin and stdout. It also reports test statistics; how many passed,
failed, and leaked, as well as directly associating the relevant stderr
with the particular test name that failed.

This separation of CompileStep and RunStep means that
`CompileStep.Kind.test_exe` is no longer needed, and therefore has been
removed in this commit.

 * build runner: show unit test statistics in build summary
 * added Step.writeManifest since many steps want to treat it as a
   warning and emit the same message if it fails.
 * RunStep: fixed error message that prints the failed command printing
   the original argv and not the adjusted argv in case an interpreter
   was used.
 * RunStep: fixed not passing the command line arguments to the
   interpreter.
 * move src/Server.zig to std.zig.Server so that the default test runner
   can use it.
 * the simpler test runner function which is used by work-in-progress
   backends now no longer prints to stderr, which is necessary in order
   for the build runner to not print the stderr as a warning message.
2023-03-15 10:48:14 -07:00
Andrew Kelley
a333bb91ff zig objcopy: support the compiler protocol
This commit extracts out server code into src/Server.zig and uses it
both in the main CLI as well as `zig objcopy`.

std.Build.ObjCopyStep now adds `--listen=-` to the CLI for `zig objcopy`
and observes the protocol for progress and other kinds of integrations.

This fixes the last two test failures of this branch when I run
`zig build test` locally.
2023-03-15 10:48:14 -07:00
Andrew Kelley
7cc4a6965c build runner enhancements in preparation for test-cases
* std.zig.ErrorBundle: support rendering options for whether to include
   the reference trace, whether to include the source line, and TTY
   configuration.

 * build runner: don't print progress in dumb terminals

 * std.Build.CompileStep:
   - add a way to expect compilation errors via the new `expect_errors`
     field. This is an advanced setting that can change the intent of
     the CompileStep. If this slice has nonzero length, it means that
     the CompileStep exists to check for compile errors and return
     *success* if they match, and failure otherwise.
   - remove the object format parameter from `checkObject`. The object
     format is known based on the CompileStep's target.
   - Avoid passing -L and -I flags for nonexistent directories within
     search_prefixes. This prevents a warning, that should probably be
     upgraded to an error in Zig's CLI parsing code, when the linker
     sees an -L directory that does not exist.

 * std.Build.Step:
   - When spawning the zig compiler process, takes advantage of the new
     `std.Progress.Node.setName` API to avoid ticking up a meaningless
     number at every progress update.
2023-03-15 10:48:14 -07:00
Andrew Kelley
7106a91b09 CLI: fix ast-check printing ZIR errors twice 2023-03-15 10:48:14 -07:00
Andrew Kelley
2996eb5587 std.Build.RunStep: add maxrss, duration, and cached status 2023-03-15 10:48:13 -07:00
Andrew Kelley
677a0e2941 stage2: avoid bloat when using -Donly-c 2023-03-15 10:48:13 -07:00
Andrew Kelley
d6f7766da2 stage2: avoid networking when generating zig2.c
Avoid dragging networking into zig2.c because it adds dependencies on
some linker symbols that are annoying to satisfy while bootstrapping.
2023-03-15 10:48:13 -07:00
Andrew Kelley
9bf63b0996 stage2: avoid linux-only APIs on other operating systems 2023-03-15 10:48:13 -07:00
Andrew Kelley
58edefc6d1 zig build: many enhancements related to parallel building
Rework std.Build.Step to have an `owner: *Build` field. This
simplified the implementation of installation steps, as well as provided
some much-needed common API for the new parallelized build system.

--verbose is now defined very concretely: it prints to stderr just
before spawning a child process.

Child process execution is updated to conform to the new
parallel-friendly make() function semantics.

DRY up the failWithCacheError handling code. It now integrates properly
with the step graph instead of incorrectly dumping to stderr and calling
process exit.

In the main CLI, fix `zig fmt` crash when there are no errors and stdin
is used.

Deleted steps:
 * EmulatableRunStep - this entire thing can be removed in favor of a
   flag added to std.Build.RunStep called `skip_foreign_checks`.
 * LogStep - this doesn't really fit with a multi-threaded build runner
   and is effectively superseded by the new build summary output.

build runner:
 * add -fsummary and -fno-summary to override the default behavior,
   which is to print a summary if any of the build steps fail.
 * print the dep prefix when emitting error messages for steps.

std.Build.FmtStep:
 * This step now supports exclude paths as well as a check flag.
 * The check flag decides between two modes, modify mode, and check
   mode. These can be used to update source files in place, or to fail
   the build, respectively.

Zig's own build.zig:
 * The `test-fmt` step will do all the `zig fmt` checking that we expect
   to be done. Since the `test` step depends on this one, we can simply
   remove the explicit call to `zig fmt` in the CI.
 * The new `fmt` step will actually perform `zig fmt` and update source
   files in place.

std.Build.RunStep:
 * expose max_stdio_size is a field (previously an unchangeable
   hard-coded value).
 * rework the API. Instead of configuring each stream independently,
   there is a `stdio` field where you can choose between
   `infer_from_args`, `inherit`, or `check`. These determine whether the
   RunStep is considered to have side-effects or not. The previous
   field, `condition` is gone.
 * when stdio mode is set to `check` there is a slice of any number of
   checks to make, which include things like exit code, stderr matching,
   or stdout matching.
 * remove the ill-defined `print` field.
 * when adding an output arg, it takes the opportunity to give itself a
   better name.
 * The flag `skip_foreign_checks` is added. If this is true, a RunStep
   which is configured to check the output of the executed binary will
   not fail the build if the binary cannot be executed due to being for
   a foreign binary to the host system which is running the build graph.
   Command-line arguments such as -fqemu and -fwasmtime may affect
   whether a binary is detected as foreign, as well as system
   configuration such as Rosetta (macOS) and binfmt_misc (Linux).
   - This makes EmulatableRunStep no longer needed.
 * Fix the child process handling to properly integrate with the new
   bulid API and to avoid deadlocks in stdout/stderr streams by polling
   if necessary.

std.Build.RemoveDirStep now uses the open build_root directory handle
instead of an absolute path.
2023-03-15 10:48:13 -07:00
Andrew Kelley
b5baa41077 fix zig fmt crash 2023-03-15 10:48:13 -07:00
Andrew Kelley
8b2d872020 fix std.Build.TranslateCStep 2023-03-15 10:48:13 -07:00
Andrew Kelley
0e078790fe multiplex compiler progress messages into the build runner 2023-03-15 10:48:13 -07:00