6302 Commits

Author SHA1 Message Date
fn ⌃ ⌥
a70c86e661 Fix deprecation docs for isAlpha and isCntrl 2022-10-29 15:22:05 -04:00
Andrew Kelley
20925b2f5c
Merge pull request #13272 from topolarity/sha2-intrinsics
crypto.sha2: Use intrinsics for SHA-256 on x86-64 and AArch64
2022-10-29 03:31:42 -04:00
Andrew Kelley
c36eb4ede9
Merge pull request #13221 from topolarity/packed-mem
Introduce `std.mem.readPackedInt` and improve bitcasting of packed memory layouts
2022-10-28 21:15:16 -04:00
Zhora Trush
c66d3f6bf6 Enhance indexOfIgnoreCase with Boyer-Moore-Horspool algorithm 2022-10-28 20:29:41 -04:00
Cody Tapscott
67fa3262b1 std.crypto: Use featureSetHas to gate intrinsics
This also fixes a bug where the feature gating was not taking
effect at comptime due to https://github.com/ziglang/zig/issues/6768
2022-10-28 17:17:08 -07:00
Cody Tapscott
f9fe548e41 std.crypto: Add isComptime guard around intrinsics
Comptime code can't execute assembly code, so we need some way to
force comptime code to use the generic path. This should be replaced
with whatever is implemented for #868, when that day comes.

I am seeing that the result for the hash is incorrect in stage1 and
crashes stage2, so presumably this never worked correctly. I will follow
up on that soon.
2022-10-28 15:21:10 -07:00
Cody Tapscott
4c1f71e866 std.crypto: Optimize SHA-256 intrinsics for AMD x86-64
This gets us most of the way back to the performance I had when
I was using the LLVM intrinsics:
  - Intel Intel(R) Core(TM) i7-1068NG7 CPU @ 2.30GHz:
       190.67 MB/s (w/o intrinsics) -> 1285.08 MB/s
  - AMD EPYC 7763 (VM) @ 2.45 GHz:
       240.09 MB/s (w/o intrinsics) -> 1360.78 MB/s
  - Apple M1:
       216.96 MB/s (w/o intrinsics) -> 2133.69 MB/s

Minor changes to this source can swing performance from 400 MB/s to
1400 MB/s or... 20 MB/s, depending on how it interacts with the
optimizer. I have a sneaking suspicion that despite LLVM inheriting
GCC's extremely strict inline assembly semantics, its passes are
rather skittish around inline assembly (and almost certainly, its
instruction cost models can assume nothing)
2022-10-28 15:21:10 -07:00
Cody Tapscott
ee241c47ee std.crypto: SHA-256 Properly gate comptime conditional
This feature detection must be done at comptime so that we avoid
generating invalid ASM for the target.
2022-10-28 15:21:10 -07:00
Cody Tapscott
10edb6d352 crypto.sha2: Use intrinsics for SHA-256 on x86-64 and AArch64
There's probably plenty of room to optimize these further in the
future, but for the moment this gives ~3x improvement on Intel
x86-64 processors, ~5x on AMD, and ~10x on M1 Macs.

These extensions are very new - Most processors prior to 2020 do
not support them.

AVX-512 is a slightly older alternative that we could use on Intel
for a much bigger performance bump, but it's been fused off on
Intel's latest hybrid architectures and it relies on computing
independent SHA hashes in parallel. In contrast, these SHA intrinsics
provide the usual single-threaded, single-stream interface, and should
continue working on new processors.

AArch64 also has SHA-512 intrinsics that we could take advantage
of in the future
2022-10-28 15:21:10 -07:00
Evan Haas
c616141241 translate-c: Better support for division in macros
Perform C-style arithmetic conversions on operands to division operator
in macros

Closes #13162
2022-10-28 17:59:32 -04:00
Cody Tapscott
03ed0a59e3 std.mem: Skip read/writePackedInt test on WASM32/64 2022-10-28 08:41:04 -07:00
Cody Tapscott
3295fee911 stage2: Use mem.readPackedInt etc. for packed bitcasts
Packed memory has a well-defined layout that doesn't require
conversion from an integer to read from. Let's use it :-)

This change means that for bitcasting to/from a packed value that
is N layers deep, we no longer have to create N temporary big-ints
and perform N copies.

Other miscellaneous improvements:
  - Adds support for casting to packed enums and vectors
  - Fixes bitcasting to/from vectors outside of a packed struct
  - Adds a fast path for bitcasting <= u/i64
  - Fixes bug when bitcasting f80 which would clear following fields

This also changes the bitcast memory layout of exotic integers on
big-endian systems to match what's empirically observed on our targets.
Technically, this layout is not guaranteed by LLVM so we should probably
ban bitcasts that reveal these padding bits, but for now this is an
improvement.
2022-10-28 08:41:04 -07:00
Cody Tapscott
c639c22544 std.mem: Add readPackedInt, writePackedInt, etc.
These utility functions allow reading from (stage2) packed memory at
runtime-known offsets.
2022-10-28 08:41:04 -07:00
Frank Denis
f28e4e03ee
std.sign.ecdsa: add support for incremental signatures (#13332)
Similar to what was done for EdDSA, allow incremental creation
and verification of ECDSA signatures.

Doing so for ECDSA is trivial, and can be useful for TLS as well
as the future package manager.
2022-10-28 16:25:37 +02:00
Ali Chraghi
d6943f87f9 NativeTargetInfo: remove unused error 2022-10-28 13:58:28 +02:00
Andrew Kelley
1f4f8d34a1 compiler_rt: fix memcmp wrong for some inputs
See the new test case - this fails in the previous implementation.

See #13303
2022-10-27 22:11:20 -04:00
Andrew Kelley
2991e4a454
Merge pull request #13288 from Vexu/opt-slice
Optimize size of optional slices (+ some fixes)
2022-10-27 22:09:17 -04:00
InKryption
bc72ae5e4e Sema: Prevent coercion from tuple pointer to mutable slice.
Also fix some stdlib code affected by this.

Co-authored by: topolarity <topolarity@tapscott.me>
2022-10-27 22:00:47 -04:00
Andrew Kelley
1d1c7ae5de
Merge pull request #13257 from topolarity/ctype-fixes
stage 2: Update `CType` size/alignment
2022-10-27 21:15:22 -04:00
daurnimator
f2401a0756 Add sockaddr_vm for vSocckets on Linux 2022-10-27 20:51:35 -04:00
daurnimator
c0c8ee5ae9 Add reboot syscall
Only linux for now
2022-10-27 20:39:35 -04:00
Frank Denis
9c0d975a09
Revamp the ed25519 API (#13309) 2022-10-27 19:07:42 +02:00
Veikka Tuominen
5b79f42dc5 std.mem.Allocator: do not return undefined pointers 2022-10-27 01:31:18 +03:00
Luuk de Gram
875e98a57d
Merge pull request #13287 from Luukdegram/wasm-features
wasm-linker: feature compatibility validation
2022-10-26 14:04:16 +02:00
Jakub Konka
d42a719e8f
Merge pull request #13299 from ziglang/darwin-autumn-cleaning
Darwin autumn cleaning: x86_64-macos.13 and aarch64-macos.13 support
2022-10-26 13:56:02 +02:00
Naoki MATSUMOTO
cd4865d88c
std.crypto.sign.ecdsa: accepts unusual parameters like EcdsaP384Sha256 (#13302)
This commit accepts unusual parameters like EcdsaP384Sha256.
Some certifictes(below certs are in /etc/ssl/certs/ca-certificates.crt on Ubuntu 22.04) use EcdsaP384Sha256 to sign itself.
- Subject: C=GR, L=Athens, O=Hellenic Academic and Research Institutions Cert. Authority, CN=Hellenic Academic and Research Institutions ECC RootCA 2015
- Subject: C=US, ST=Texas, L=Houston, O=SSL Corporation, CN=SSL.com EV Root Certification Authority ECC
- Subject: C=US, ST=Texas, L=Houston, O=SSL Corporation, CN=SSL.com Root Certification Authority ECC

In verify(), hash array `h` is allocated to be larger than the scalar.encoded_length.
The array is regarded as big-endian.
Hash values are filled in the back of the array and the rest bytes in front are filled with zero.

In sign(), the hash array is allocated and filled as same as verify().
In deterministicScalar(), hash bytes are insufficient to generate `k`
To generate `k` without narrowing its value range,
this commit uses algorithm stage h. in  "Section 3.2 Generation of k" in RFC6979.
2022-10-26 13:18:06 +02:00
Frank Denis
22b71b1376 crypto/bcrypt: don't reimplement base64, just use a custom alphabet
Now that std.base64 supports everything bcrypt needs to encode its
parameters, we don't need to include another implementation.
2022-10-25 21:52:03 -07:00
Luuk de Gram
b14f605dd7
CheckObjectStep: parse and dump target_features
When an object file or binary contains the target_features section
we can now parse and then dump its contents in string format so
we can use them in our linker tests to verify the features section.
2022-10-25 20:48:08 +02:00
Jakub Konka
304eb54169 darwin: update macOS libc headers
Add aarch64-macos.13 and x86_64-macos.13 libc headers, and remove
x86_64-macos.10 headers.
2022-10-25 20:37:26 +02:00
Jakub Konka
cf2ac99f84 darwin: remove libSystem.10.tbd 2022-10-25 20:37:26 +02:00
Jakub Konka
43dbebdd8d darwin: add libSystem.13.tbd 2022-10-25 20:31:48 +02:00
Jakub Konka
2d03f38988 darwin: bump minimum supported version to 11.7.1
Drop Catalina support.
2022-10-25 20:31:48 +02:00
Jakub Konka
98285b17b3 darwin: bump max macOS version to 13.0 2022-10-25 20:31:31 +02:00
Jacob Young
55c5da1b20 fmt: fix f80 hex formatting
These ifs were missing a case for f80 which should have shifted by one,
but we can just compute the correct value instead.  Also, we want the
fractional bits to be a multiple of four, not the mantissa bits, since
the mantissa could have a leading one which we want to be separated.
2022-10-25 09:04:04 -07:00
Cody Tapscott
c50f33b111 compiler_rt: Always export "standard" symbol names
The Zig LLVM backend emits calls to softfloat methods with the "standard
compiler-rt" names. Rather than add complexity to the backend and
have to synchronize the naming scheme across all targets, the simplest
fix is just to export these symbols under both the "standard" and the
platform-specific naming convention.
2022-10-22 17:19:33 -07:00
Jan Philipp Hafer
776f7de967 update compiler_rt README and prettify it
This documents status of routines and adds the next work item
"Decimal float library routines", which are only recommended for
binary data. Complete absence of tests is also documented.
This does not document the various aliases, e.g. those for ARM.

Missing Integer library routines:
- __addvsi3
- __addvdi3
- __addvti3
- __addvdi3
- __addvti3
- __subvsi3
- __subvdi3
- __subvti3
- __subvdi3
- __subvti3
- __mulvsi3
- __mulvdi3
- __mulvti3
- __mulvdi3
- __mulvti3

Missing floating library routines:
- __powisf2
- __powidf2
- __powitf2
- __powixf2

Missing routines for symbol-level compatibility to gcc:
- __ashlsi3
- __ashrsi3
- __lshrsi3
2022-10-22 09:23:50 -07:00
Luuk de Gram
cdf7e7dba9 fix: correctly import memcmp.zig in compiler-rt
cmp.zig was accidently being referenced twice, rather than importing
memcmp.zig. This means that its symbols were also not included in
the generated compiler-rt output.
2022-10-22 16:56:33 +02:00
Andrew Kelley
09236d29b7
Merge pull request #12837 from topolarity/err-ret-trace-improvements-1923
stage2: Pop error trace frames for handled errors (#1923)
2022-10-21 20:24:37 -07:00
Cody Tapscott
f0e66ac4d0 std.Target: Remove longDoubleIs
This function is redundant with CType.sizeInBits(), and until the
previous commit they disagreed about the correct long double type
for several targets. Although they're all synced up now, it's much
simpler just to have a single source of truth.
2022-10-21 19:20:58 -07:00
Cody Tapscott
ddbdb83c86 stage 2: Update C types' size/alignment
These updates were made by testing against the `sizeof/_Alignof` reported
by Clang for all supported arch-OS-ABI combinations and correcting any
discrepancies.

This is bound to have a few errors (the recent long double fix for i386
Android is one example), but Clang is certainly not a bad place to start,
especially for our most popular targets.
2022-10-21 19:11:47 -07:00
Louis Pearson
8d4778b4f9
Support compiling for the android NDK (#13245) 2022-10-21 18:32:56 -04:00
Cody Tapscott
a4523a2d4a builtin.zig: Do not overwrite error frames when trace full
Previously, we'd overwrite the errors in a circular buffer. Now that
error return traces are intended to follow a stack discipline, we no
longer have to support the index rolling over. By treating the trace
like a saturating stack, any pop/restore code still behaves correctly
past-the-end of the trace.

As a bonus, this adds a small blurb to let the user know when the trace
saturated and x number of frames were dropped.
2022-10-21 12:40:33 -07:00
Cody Tapscott
d060cbbec7 stage2: Keep error return traces alive when storing to const
This change extends the "lifetime" of the error return trace associated
with an error to continue throughout the block of a `const` variable
that it is assigned to.

This is necessary to support patterns like this one in test_runner.zig:
```zig
const result = foo();
if (result) |_| {
    // ... success logic
} else |err| {
    // `foo()` should be included in the error trace here
    return error.TestFailed;
}
```

To make this happen, the majority of the error return trace popping logic
needed to move into Sema, since `const x = foo();` cannot be examined
syntactically to determine whether it modifies the error return trace. We
also have to make sure not to delete pertinent block information before it
makes it to Sema, so that Sema can pop/restore around blocks correctly.

* Why do this only for `const` and not `var`? *

There is room to relax things for `var`, but only a little bit. We could
do the same thing we do for const and keep the error trace alive for the
remainder of the block where the *assignment* happens. Any wider scope
would violate the stack discipline for traces, so it's not viable.

In the end, I decided the most consistent behavior for the user is just
to kill all error return traces assigned to a mutable `var`.
2022-10-21 12:40:29 -07:00
Cody Tapscott
3007fdde45 stage2: Pop error trace when storing error to var/const
In order to enforce a strict stack discipline for error return traces,
we cannot track error return traces that are stored in variables:

  ```zig
  const x = errorable(); // errorable()'s error return trace is killed here

  // v-- error trace starts here instead
  return x catch error.UnknownError;
  ```

In order to propagate error return traces, function calls need to be passed
directly to an error-handling expression (`if`, `catch`, `try` or `return`):

  ```zig
  // When passed directly to `catch`, the return trace is propagated
  return errorable() catch error.UnknownError;

  // Using a break also works
  return blk: {
      // code here
      break :blk errorable();
  } catch error.UnknownError;
  ```

Why do we need this restriction? Without it, multiple errors can co-exist
with their own error traces. Handling that situation correctly means either:
  a. Dynamically allocating trace memory and tracking lifetimes, OR
  b. Allowing the production of one error to interfere with the trace of another
     (which is the current status quo)

This is piece (3/3) of https://github.com/ziglang/zig/issues/1923#issuecomment-1218495574
2022-10-21 10:44:20 -07:00
David Gonzalez Martin
680d3cd1fc UEFI: Querying memory map size with no allocation
This makes possible to query the memory map size from EFI firmware
without making any allocation beforehand. This makes possible to be
precise about the size of the allocation which will own a copy of
the memory map from the UEFI application.
2022-10-21 12:17:41 +02:00
Andrew Kelley
0f00766661 Revert "add std.debug.todo"
This reverts commit 80ac022c4667e1995ccdf70fff90e5af26b6eb97.

I changed my mind on this one, sorry. I don't think this belongs in the
standard library.
2022-10-20 18:34:40 -07:00
Andrew Kelley
5b9c8d1d6f add m68k target CPU features 2022-10-20 09:21:06 -07:00
Matheus C. França
b41b35f578
crypto/benchmark - replace testing allocator
Fix error: Cannot use testing allocator outside of test block
2022-10-20 14:04:59 +03:00
LordMZTE
3f577f06a0 build: added unwind_tables to LibExeObjStep 2022-10-19 13:24:06 -04:00
Andrew Kelley
8a344fab39
Merge pull request #13036 from BratishkaErik/fix-installing 2022-10-19 09:34:35 -04:00