2543 Commits

Author SHA1 Message Date
Andrew Kelley
281fc10ec5 std.crypto siphash: fix assertion on the size of output buffer
the logic was backwards
2020-09-16 02:24:36 -07:00
Andrew Kelley
16bd0c63b4 Merge remote-tracking branch 'origin/master' into stage2-zig-cc
Pulling in the changes to libc_installation.zig
2020-09-15 18:04:37 -07:00
Rocknest
c35703825f Add an error set 2020-09-16 01:58:48 +03:00
Andrew Kelley
f82b1831f7 std: handle sharing violation when deleting a file on windows 2020-09-15 15:24:13 -07:00
Andrew Kelley
b3cc36857e
Merge pull request #6340 from Vexu/fix
Fixes
2020-09-15 14:28:16 -04:00
LemonBoy
c1c3212500 std: Fix typo in ELF section header iterator
The code accidentally used the phdr offset instead of the shdr one while
iterating over the section headers.

Fixes #6338
2020-09-15 18:14:06 +03:00
Andrew Kelley
5d8fec3d4c Merge remote-tracking branch 'origin/master' into stage2-zig-cc 2020-09-14 21:57:01 -07:00
Andrew Kelley
dffdb2844e track all TODO comments in BRANCH_TODO file
Before merging, do this for every item in the file:
 * solve the issue, or
 * convert the task to a github issue and update the comment
   to link to the issue (and remove "TODO" text from the comment).
Then delete the file.

Related: #363
2020-09-14 18:06:19 -07:00
Vexu
a3624e94f8
translate-c: determine sizeof using std.meta.sizeof 2020-09-14 23:53:38 +03:00
Vexu
29fd0c6d61
fix meta.cast behavior; add exhaustive tests 2020-09-14 23:21:26 +03:00
Andrew Kelley
778bb4bc9c move std.cache_hash from std to stage2
The API is pretty specific to the implementationt details of the
self-hosted compiler. I don't want to have to independently support
and maintain this as part of the standard library, and be obligated
to not make breaking changes to it with changes to the implementation of
stage2.
2020-09-14 11:05:51 -07:00
Veikka Tuominen
d073836894
Merge pull request #6172 from tadeokondrak/@Type(.Union)
Implement @Type for Union
2020-09-14 16:43:49 +03:00
Andrew Kelley
c49435f76b
Merge pull request #6331 from mattnite/bpf-helper-defs
BPF: helper definitions
2020-09-14 04:00:00 -04:00
Andrew Kelley
2a8fc1a18e stage2: caching system integration & Module/Compilation splitting
* update to the new cache hash API
 * std.Target defaultVersionRange moves to std.Target.Os.Tag
 * std.Target.Os gains getVersionRange which returns a tagged union
 * start the process of splitting Module into Compilation and "zig
   module".
   - The parts of Module having to do with only compiling zig code are
     extracted into ZigModule.zig.
   - Next step is to rename Module to Compilation.
   - After that rename ZigModule back to Module.
 * implement proper cache hash usage when compiling C objects, and
   properly manage the file lock of the build artifacts.
 * make versions optional to match recent changes to master branch.
 * proper cache hash integration for compiling zig code
 * proper cache hash integration for linking even when not compiling zig
   code.
 * ELF LLD linking integrates with the caching system. A comment from
   the source code:

   Here we want to determine whether we can save time by not invoking LLD when the
   output is unchanged. None of the linker options or the object files that are being
   linked are in the hash that namespaces the directory we are outputting to. Therefore,
   we must hash those now, and the resulting digest will form the "id" of the linking
   job we are about to perform.
   After a successful link, we store the id in the metadata of a symlink named "id.txt" in
   the artifact directory. So, now, we check if this symlink exists, and if it matches
   our digest. If so, we can skip linking. Otherwise, we proceed with invoking LLD.

 * implement disable_c_depfile option
 * add tracy to a few more functions
2020-09-13 19:29:07 -07:00
Andrew Kelley
1baa56a25f std.cache_hash: break up the API and improve implementation
into smaller exposed components and expose all of them. This makes it
more flexible.

`*const Cache` is now passed in with an open manifest dir handle which
the caller is responsible for managing.

Expose some of the base64 stuff.

Extract the hash helper functions into `HashHelper` and add some more
methods such as addOptional and addListOfFiles.

Add `CacheHash.toOwnedLock` so that you can deinitialize everything
except the open file handle which represents the file system lock on the
build artifacts.

Use ArrayListUnmanaged, saving space per allocated CacheHash.

Avoid 1 memory allocation in hit() with a static buffer.

hit() returns a bool; caller code is responsible for calling final() in
either case. This is a simpler and easier to use API.

writeManifest() is no longer called from deinit() with errors ignored.
2020-09-13 18:04:52 -07:00
Rocknest
988fc6f9d1 flip condition 2020-09-14 02:27:09 +03:00
Rocknest
73863cf72b fix build 2020-09-13 23:59:36 +03:00
LemonBoy
61e9e82bdc std: Make the CRC32 calculation slightly faster
Speed up a little the slicing-by-8 code path by replacing the
(load+shift+xor)*4 sequence with a single u32 load plus a xor.

Before:

```
iterative:  1018 MiB/s [000000006c3b110d]
small keys:  1075 MiB/s [0035bf3dcac00000]
```

After:

```
iterative:  1114 MiB/s [000000006c3b110d]
small keys:  1324 MiB/s [0035bf3dcac00000]
```
2020-09-13 16:32:21 -04:00
Rocknest
f6195be997 fix ref 2020-09-13 23:31:59 +03:00
LemonBoy
5e50d145d9 std: Limit the read/write size on Darwin
It turns out that the kernel won't read or write more than 0x7fffffff
bytes in a single call, failing with EINVAL when trying to do so.

Adjust the limit and curse whoever is responsible for this.

Closes #6332
2020-09-13 16:31:21 -04:00
Rocknest
d75cbb01db Reference all crypto declarations 2020-09-13 23:00:33 +03:00
Rocknest
b6385870d0 Convert asserts to errors, make sure nothing overflows 2020-09-13 22:39:54 +03:00
Rocknest
a6d947191e
Merge pull request #2 from rnapier/pbkdf2
Pbkdf2
2020-09-13 22:09:45 +03:00
Matt Knight
88dacd3b70
changed to opaque 2020-09-13 09:53:20 -07:00
Matt Knight
1afbf4fb21
removed all those kern aliases 2020-09-13 09:43:49 -07:00
Rob Napier
85366771ea pbkdf2 offset into dk should be usize, not u64. 2020-09-13 12:36:32 -04:00
Matt Knight
cb5f76bb1c
got booleans wrong 2020-09-13 09:24:56 -07:00
Rocknest
dcd229be92
Merge pull request #1 from rnapier/pbkdf2
Pbkdf2
2020-09-13 19:11:29 +03:00
Rob Napier
2f9c9662ba Use comptime to expose public method to doc system 2020-09-13 11:17:58 -04:00
Rob Napier
8a1a40276f Extract kdf.zig to provide namespace documentation 2020-09-13 11:08:06 -04:00
Rob Napier
257c5b5348 Explicitly reference std.crypto.kdf in test case 2020-09-13 10:50:46 -04:00
Rob Napier
0f85b85acb Improve doc text 2020-09-13 09:59:36 -04:00
Matt Knight
b043a31889
added helper definitions 2020-09-12 23:45:35 -07:00
Rob Napier
17156e1775 pbkdf2 "very large dklen test" should just check for crashes 2020-09-12 18:33:53 -04:00
Rob Napier
3f450b7e93 Replace Hash function with Prf. Correct offset bit-width. 2020-09-12 18:17:04 -04:00
Rob Napier
37db93e426 Review comments from pbkdf2.zig
Move block definitions inside while loop.
Use usize for offset. (This still crashes on overflow)
Remove unneeded slice syntax.
Add slow test for Very large dkLen
2020-09-12 15:03:42 -04:00
Andrew Kelley
af4cc20ce2 Merge remote-tracking branch 'origin/master' into stage2-zig-cc
Master branch added in the concept of library versioning being optional
to main.cpp. It will need to be re-added into this branch before merging
back into master.
2020-09-12 10:48:38 -07:00
Andrew Kelley
8374be1a1c std.cache_hash: bump up the base64 digest len to avoid '==' 2020-09-12 00:50:37 -07:00
LemonBoy
9fe4c89230 std: Add a gzip decoder 2020-09-11 20:02:41 -04:00
Andrew Kelley
1edf097871
Merge pull request #6323 from LemonBoy/fast-zlib
std: Make the DEFLATE decompression routine 3x faster
2020-09-11 17:25:18 -04:00
Rob Napier
c2b02d01d5 Add crypto.kdf.pbkdf2 2020-09-11 17:10:27 -04:00
Andrew Kelley
1eaf0691f9
Merge pull request #6320 from ifreund/prctl
std: add prctl and securebits definitions for linux/C
2020-09-11 16:33:57 -04:00
Isaac Freund
1078810cef
std: add prctl definition when linking libc 2020-09-11 22:32:23 +02:00
Isaac Freund
b6f4601545
std: add securebits definitions for linux 2020-09-11 22:32:23 +02:00
Isaac Freund
744b73ab46
std: add prctl wrapper to std.os 2020-09-11 22:32:23 +02:00
Isaac Freund
7b961a876b
std: add prctl definition for linux 2020-09-11 22:32:22 +02:00
Andrew Kelley
42c32dbc7b
Merge pull request #6315 from ifreund/optional-shlib-version
std, stage1: make shared library versioning optional
2020-09-11 16:07:13 -04:00
LemonBoy
4b851c7255 Re-enable a compression test
It somehow got commented out...
2020-09-11 21:04:08 +02:00
LemonBoy
41f244bd2f std: Make the DEFLATE decompression routine 3x faster
A profiler run showed that the main bottleneck was the naive decoding of
the Huffman codes, replacing it with a nice trick borrowed by Zlib gave
a substantial speedup.
Replacing a `%` with a `and (mask-1)` gave another significant
improvement (yay for low hanging fruits).

A few numbers obtained by decompressing a 22M file:

Before:
```
./decompress  2,39s user 0,00s system 99% cpu 2,400 total
```

After:
```
./decompress  0,79s user 0,00s system 99% cpu 0,798 total
````
2020-09-11 21:00:15 +02:00
Isaac Freund
0a76a9bd73
std, stage1: make shared library versioning optional
This commit changes the behavior of stage1 to emit libfoo.so instead
of libfoo.so.0.0.0 when none of the --ver-major, --ver-minor, or
--ver-patch flags are set.

It also makes it possible to create unversioned shared libraries
using the zig build system, changing the version parameter of
addSharedLibrary() to a tagged union.
2020-09-11 10:46:37 +02:00