38 Commits

Author SHA1 Message Date
Sage Hane
e288148f60
fs: Use OpenMode enum instead of read/write flags. 2022-01-29 15:52:08 +02:00
Andrew Kelley
e36718165c stage2: add @import and @embedFile to CacheHash
when using `CacheMode.whole`. Also, I verified that `addDepFilePost` is
in fact including the original C source file in addition to the files it
depends on.
2022-01-02 13:16:17 -07:00
Andrew Kelley
3e618f8432 Cache: use mutex to protect recent_problematic_timestamp
The previous commit tried to use atomics but not many CPUs support
128-bit atomics. So we use a mutex. In order to avoid contention, we
also store `recent_problematic_timestamp` locally on the `Manifest`
which is only ever accessed from a single thread at a time, and only
consult the global one if the local one is problematic.

This commit was tested by running `zig build test-behavior` in two
separate terminals at the same time.
2021-12-09 22:10:38 -07:00
Andrew Kelley
fdbb329d10 Cache: fix data race with is_problematic_timestamp
Previously `recent_problematic_timestamp` was unprotected and accessed
potentially with multiple worker threads simultaneously.

This commit protects it with atomics and also introduces a flag to
prevent multiple timestamp checks from within the same call to hit().

Unfortunately the compiler-rt function __sync_val_compare_and_swap_16 is
not yet implemented, so I will have to take a different strategy in a
follow-up commit.
2021-12-09 22:10:38 -07:00
Andrew Kelley
4da83feccb Cache: improvements to previous commit
* put `recent_problematic_timestamp` onto `Cache` so that it can be
   shared by multiple Manifest instances.
 * make `isProblematicTimestamp` return true on any filesystem error.
 * save 1 syscall by using truncate=true in createFile instead of
   calling `setEndPos`.
2021-12-09 22:10:31 -07:00
Travis Martin
72ee042ab0 Cache: fix two issues with isProblematicTimestamp
1. It was looking for trailing zero bits when it should be looking for
   trailing decimal zeros.
2. Clock timestamps had more precision than the actual file timestamps

The fix is to grab a timestamp from a 'just now changed' temp file.
This timestamp is "problematic". Any file timestamp greater than or equal
to this timestamp is considered problematic. File timestamps **prior** to
this **can** be trusted.

Downside is that it causes a disk I/O to write to and then read the
timestamp from this file ~1ms on my system. This is partially mitigated by
keeping track of the most recent problematic timestamp, and only checking
for a new problematic timestamp when checking a timestamp that is equal to
or larger than the last problematic one.

This fixes #6082.
2021-12-09 18:34:17 -07:00
Lee Cannon
85de022c56
allocgate: std Allocator interface refactor 2021-11-30 23:32:47 +00:00
Andrew Kelley
078a7ff198 Cache: add debug log statement 2021-11-24 23:08:37 -07:00
Andrew Kelley
4e5a88b288 stage2: default dynamic libraries to be linked as needed
After this change, the default for dynamic libraries (`-l` or
`--library`) is to only link them if they end up being actually used.

With the Zig CLI, the new options `-needed-l` or `--needed-library` can
be used to force link against a dynamic library.

With `zig cc`, this behavior can be overridden with `-Wl,--no-as-needed`
(and restored with `-Wl,--as-needed`).

Closes #10164
2021-11-20 17:23:44 -07:00
Andrew Kelley
6115cf2240 migrate from std.Target.current to @import("builtin").target
closes #9388
closes #9321
2021-10-04 23:48:55 -07:00
Ryan Liptak
59f5053bed Update all ensureCapacity calls to the relevant non-deprecated version 2021-09-19 13:52:56 +02:00
Ryan Liptak
d31352ee85 Update all usages of mem.split/mem.tokenize for generic version 2021-08-06 02:01:47 -07:00
Andrew Kelley
06129d7e3d std: implement a cross platform file locking abstraction
This modifies the lock semantics from using AccessMode to using
NtLockFile/NtUnlockFile.

This is a breaking change.
2021-06-29 14:25:04 -07:00
Andrew Kelley
4e61af404e stage2: Cache system handles shared objects
Fixes #9139
Fixes #9187
2021-06-29 14:25:04 -07:00
Martin Wickham
fc9430f567 Breaking hash map changes for 0.8.0
- hash/eql functions moved into a Context object
- *Context functions pass an explicit context
- *Adapted functions pass specialized keys and contexts
- new getPtr() function returns a pointer to value
- remove functions renamed to fetchRemove
- new remove functions return bool
- removeAssertDiscard deleted, use assert(remove(...)) instead
- Keys and values are stored in separate arrays
- Entry is now {*K, *V}, the new KV is {K, V}
- BufSet/BufMap functions renamed to match other set/map types
- fixed iterating-while-modifying bug in src/link/C.zig
2021-06-03 17:02:16 -05:00
Evan Haas
19aab5302c translate-c: Ensure extra_cflags are passed to clang
Additionally ensure that the Zig cache incorporates any extra cflags when
using translate-c.

Fixes the issue identified in #8662
2021-05-12 18:17:45 -04:00
Veikka Tuominen
42a95197f3 update usage of std.testing in stage2 2021-05-08 15:15:30 +03:00
LemonBoy
cd7c870bd8 std: Deprecate 'x'/'X'/'e'/'E' special cases for u8 slices
Let's follow the road paved by the removal of 'z'/'Z', the Formatter
pattern is nice enough to let us remove the remaining four special cases
and declare u8 slices free from any special casing!
2021-03-01 15:33:10 -08:00
LemonBoy
057bf1afc9 std: Add more error checking in hexToBytes
Prevent the function from turning into an endless loop that may or may
not perform OOB accesses.
2021-02-21 12:19:03 +02:00
Jay Petacat
a9b505fa77 Reduce use of deprecated IO types
Related: #4917
2021-01-07 23:48:58 -08:00
Andrew Kelley
c8e44d82bd stage2: remove the Cache deadlock detection code
It's more trouble than it's worth; it didn't even catch the most recent
incident because it was across process boundaries anyway.
2021-01-03 20:34:51 -07:00
Andrew Kelley
404dc9692e stage2: fix Cache debug deadlock code memory leak 2021-01-03 20:34:51 -07:00
Timon Kruiper
a926c91814 stage2: enable building test-stage2 with LLVM backend enabled
We can now run `zig build test-stage2 -Denable-llvm`.
2021-01-03 17:39:43 +01:00
LemonBoy
1c13ca5a05 stage2: Use {s} instead of {} when formatting strings 2021-01-02 17:12:57 -07:00
Andrew Kelley
e0a78d10cc stage2: better error message for root zig source file not found
closes #6777
closes #6893
2020-12-28 21:49:40 -07:00
Andrew Kelley
cb290ed6c9 stage2: Cache deadlock debugging only for safe build modes 2020-12-26 17:33:15 -07:00
Andrew Kelley
ed39ff202b stage2: Cache: fix resource management of the deadlock debug code 2020-12-25 19:02:15 -07:00
Andrew Kelley
c7834f274d stage2: Cache: add debug deadlock detection code 2020-12-25 18:38:49 -07:00
Andrew Kelley
64a590a311 stage2: detect redundant C/C++ source files
Cache exposes BinDigest.

Compilation gains a set of a BinDigest for every C/C++ source file. We
detect when the same source/flags have already been added and emit a
compile error. This prevents a deadlock in the caching system.

Closes #7308
2020-12-11 02:33:09 -05:00
Andrew Kelley
5b5097a22a stage2: add -femit-foo=bar args to the cache hash
Closes #6979
Closes #7036
2020-12-01 16:35:27 -07:00
Timon Kruiper
4b48fccadd When checking a cache hit, make sure to handle a (re)moved source file
When a file cannot be found in the cache, it is not a hit.

Closes #6729
2020-10-23 01:01:37 -04:00
Frank Denis
fa17447090 std/crypto: make the whole APIs more consistent
- use `PascalCase` for all types. So, AES256GCM is now Aes256Gcm.
- consistently use `_length` instead of mixing `_size` and `_length` for the
constants we expose
- Use `minimum_key_length` when it represents an actual minimum length.
Otherwise, use `key_length`.
- Require output buffers (for ciphertexts, macs, hashes) to be of the right
size, not at least of that size in some functions, and the exact size elsewhere.
- Use a `_bits` suffix instead of `_length` when a size is represented as a
number of bits to avoid confusion.
- Functions returning a constant-sized slice are now defined as a slice instead
of a pointer + a runtime assertion. This is the case for most hash functions.
- Use `camelCase` for all functions instead of `snake_case`.

No functional changes, but these are breaking API changes.
2020-10-17 18:53:08 -04:00
Andrew Kelley
9f8f446435 fixups to previous commit
* std.fs.Dir.readFile: add doc comments to explain what it means when
   the returned slice has the same length as the supplied buffer.
 * introduce readSmallFile / writeSmallFile to abstract over the
   decision to use symlink or file contents to store data.
2020-10-09 16:45:39 -07:00
Andrew Kelley
73167e80f8 stage2: fix Cache not calling ftruncate in writeManifest
this led to a corrupt cache when the number of files got smaller. it is
now fixed.
2020-09-28 22:40:50 -07:00
Andrew Kelley
91a73a177b stage2: building mingw-w64 and COFF LDD linking
still TODO is the task of creating import .lib files for DLLs on the fly
both for -lfoo and for e.g. `extern "kernel32"`
2020-09-28 00:06:06 -07:00
Andrew Kelley
054fafd7d9 stage2: implement @cImport
Also rename Cache.CacheHash to Cache.Manifest
2020-09-24 16:22:45 -07:00
Andrew Kelley
c2b1cd7c45 stage2: implement zig build
As part of this:

 * add std.process.cleanExit. closes #6395
   - use it in several places
 * adjust the alignment of text in `zig build --help` menu
 * Cache: support the concept of "unhit" so that we properly keep track
   of the cache when we find out using the secondary hash that the cache
   "hit" was actually a miss. Use this to fix false negatives of caching
   of stage1 build artifacts.
 * fix not deleting the symlink hash for stage1 build artifacts causing
   false positives.
 * implement support for Package arguments in stage1 build artifacts
 * update and add missing usage text
 * add --override-lib-dir and --enable-cache CLI options
   - `--enable-cache` takes the place of `--cache on`
 * CLI supports -femit-bin=foo combined with --enable-cache to do an
   "update file" operation. --enable-cache without that argument
   will build the output into a cache directory and then print the path
   to stdout (matching master branch behavior).
 * errors surfacing from main() now print "error: Foo" instead of
   "error: error.Foo".
2020-09-22 22:18:19 -07:00
Andrew Kelley
528832bd3a rename src-self-hosted/ to src/ 2020-09-21 18:38:55 -07:00