The TurboSHAKE paper just got published:
https://eprint.iacr.org/2023/342.pdf
and unlike the previous K12 paper, suggests 0x1F instead of 0x01
as the default value for "D".
* Fix SHA3 with streaming
Leftover bytes should be added to the buffer, not to the state.
(or, always to the state; we can and probably should eventually get
rid of the buffer)
Fixes#14851
* Add a test for SHA-3 with streaming
Man page for posix lists EMFILE, man page for linux ENFILE.
Also posix says "The mmap() function adds an extra reference to the file
associated with the file descriptor fildes which is not removed by a
subsequent close() on that file descriptor. This reference is removed
when there are no more mappings to the file."
It sounds counter-intuitive, that a process limit but no system limit can
be exceeeded.
As far as I understand, fildes is only used for file descriptor backed mmaps.
- Fixes the first few code units of the name being omitted (it was using `@sizeOf(FILE_NAME_INFO)` as the start of the name bytes, but that includes the length of the dummy [1]u16 field and padding; instead the start should be the offset of the dummy [1]u16 field)
- Replaces kernel32.GetFileInformationByHandleEx call with ntdll.NtQueryInformationFile
+ Contributes towards #1840
- Checks that the handle is a named pipe first before querying and checking the name, which is a much faster call than NtQueryInformationFile (this was about a 10x speedup in my probably-not-so-good/take-it-with-a-grain-of-salt benchmarking)
In Linux when interacting with the virtual file system when writing
in invalid value to a file the OS will return errno 22 (INVAL).
Instead of triggering an unreachable, this change now returns a
newly introduced error.InvalidArgument.
In Linux when writing to various files in the virtual file system,
for example /sys/fs/cgroup, if you write an invalid value to a file
you'll get errno 16.
This change allows for these specific cases to be caught instead of
being lumped together in UnexpectedError.
Previously, this API had pid, to be used on POSIX systems, and handle,
to be used on Windows.
This commit unifies the API, defining an Id type that is either the pid
or the HANDLE depending on the target OS.
This commit also prepares for the future by allowing one to import via
`std.process.Child` which is the fully qualified namespace that I intend
to migrate to in the future.
Also add `std.fs.has_executable_bit` for doing conditional compilation.
This adds the linux syscalls for chmod and fchmodat, as well as the
extern libc function declarations.
Only `fchmodat` is added to `std.os`, and it is not yet added to std.fs.
Make the Keccak permutation public, as it's useful for more than
SHA-3 (kMAC, SHAKE, TurboSHAKE, TupleHash, etc).
Our Keccak implementation was accepting f as a comptime parameter,
but always used 64-bit words and 200 byte states, so it actually
didn't work with anything besides f=1600.
That has been fixed. The ability to use reduced-round versions
was also added in order to support M14 and K12.
The state was constantly converted back and forth between bytes
and words, even though only a part of the state is actually used
for absorbing and squeezing bytes. It was changed to something
similar to the other permutations we have, so we can avoid extra
copies, and eventually add vectorized implementations.
In addition, the SHAKE extendable output function (XOF) was
added (SHAKE128, SHAKE256). It is required by newer schemes,
such as the Kyber post-quantum key exchange mechanism, whose
implementation is currently blocked by SHAKE missing from our
standard library.
Breaking change: `Keccak_256` and `Keccak_512` were renamed to
`Keccak256` and `Keccak512` for consistency with all other
hash functions.
And make it not do any installation, only objcopying. We already have
install steps for doing installation.
This commit also makes ObjCopyStep properly integrate with caching.
I believe the reason we had that check in the first place was because for both `pthread_setname_np` and `pthread_getname_np` man says:
```
CONFORMING TO
These functions are nonstandard GNU extensions; hence the suffix "_np" (nonportable) in the names.
```
However, this `is_gnu` check was never consistently applied; it was
missing in `setName` in the Linux case. It is also missing on all other call sites for other platforms (macOS, iOS, et al.).
Though, that could be because it may only apply to Linux.
I think for a best-effort approach it is okay to drop this. It's probably less non-standard than man makes it out to be.
I noticed a comment saying that the intent of a code's author was unclear.
What happened is that the author forgot to put the check for whether the
thread is the calling thread (`self.getHandle() ==
std.c.pthread_self()`) in the `if (use_pthreads)`.
If the thread is the calling thread, we use `prctl` to set or get the
thread's name and it does not take a thread id because it knows the id
of the thread we're calling `getName` or `setName` from.
I have found a source saying that using `pthread_setname_np` on either the calling thread
or any other thread by thread id would work too (so we don't need to
call `prctl`) but I was not sure if that is the case on all systems
so we keep using `pthread_setname_np` if we have a
specific thread that is not the thread we're calling from, and `prctl`
otherwise.
