Most of this migration was performed automatically with `zig fmt`. There
were a few exceptions which I had to manually fix:
* `@alignCast` and `@addrSpaceCast` cannot be automatically rewritten
* `@truncate`'s fixup is incorrect for vectors
* Test cases are not formatted, and their error locations change
* docs(std.math): elaborate on difference between absCast and absInt
* docs(std.rand.Random.weightedIndex): elaborate on likelihood
I think this makes it easier to understand.
* langref: add small reminder
* docs(std.fs.path.extension): brevity
* docs(std.bit_set.StaticBitSet): mention the specific types
* std.debug.TTY: explain what purpose this struct serves
This should also make it clearer that this struct is not supposed to provide unrelated terminal manipulation functionality such as setting the cursor position or something because terminals are complicated and we should keep this struct simple and focused on debugging.
* langref(package listing): brevity
* langref: explain what exactly `threadlocal` causes to happen
* std.array_list: link between swapRemove and orderedRemove
Maybe this can serve as a TLDR and make it easier to decide.
* PrefetchOptions.locality: clarify docs that this is a range
This confused me previously and I thought I can only use either 0 or 3.
* fix typos and more
* std.builtin.CallingConvention: document some CCs
* langref: explain possibly cryptic names
I think it helps knowing what exactly these acronyms (@clz and @ctz) and
abbreviations (@popCount) mean.
* variadic function error: add missing preposition
* std.fmt.format docs: nicely hyphenate
* help menu: say what to optimize for
I think this is slightly more specific than just calling it
"optimizations". These are speed optimizations. I used the word
"performance" here.
These are great permutations, and there's nothing wrong with them
from a practical security perspective.
However, both were competing in the NIST lightweight crypto
competition.
Gimli didn't pass the 3rd selection round, and is not much used
in the wild besides Zig and libhydrogen. It will never be
standardized and is unlikely to get more traction in the future.
Xoodyak, that Xoodoo is the permutation of, was a finalist.
It has a lot of advantages and *might* be standardized without NIST.
But this is too early to tell, and too risky to commit to it
in a standard library.
For lightweight crypto, Ascon is the one that we know NIST will
standardize and that we can safely rely on from a usage perspective.
Switch to a traditional ChaCha-based CSPRNG, with an Ascon-based one
as an option for constrained systems.
Add a RNG benchmark by the way.
Gimli and Xoodoo served us well. Their code will be maintained,
but outside the standard library.
Ascon has been selected as new standard for lightweight cryptography
in the NIST Lightweight Cryptography competition.
Ascon won over Gimli and Xoodoo.
The permutation is unlikely to change. However, NIST may tweak
the constructions (XOF, hash, authenticated encryption) before
standardizing them. For that reason, implementations of those
are better maintained outside the standard library for now.
In fact, we already had an Ascon implementation in Zig:
`std.crypto.aead.isap` is based on it. While the implementation was
here, there was no public API to access it directly.
So:
- The Ascon permutation is now available as `std.crypto.core.Ascon`,
with everything needed to use it in AEADs and other Ascon-based
constructions
- The ISAP implementation now uses std.crypto.core.Ascon instead of
keeping a private copy
- The default CSPRNG replaces Xoodoo with Ascon. And instead of an
ad-hoc construction, it's using the XOFa mode of the NIST submission.
There are still a few occurrences of "stage1" in the standard library
and self-hosted compiler source, however, these instances need a bit
more careful inspection to ensure no breakage.
and reimplement shuffle in terms of it. This allows the caller to specify an index type of a fixed bit width, allowing results to be independent usize.
Gimli was a game changer. A permutation that is large enough to be
used in sponge-like constructions, yet small enough to be compact
to implement and fast on a wide range of platforms.
And Gimli being part of the Zig standard library was awesome.
But since then, Gimli entered the NIST Lightweight Cryptography
Competition, competing againt other candidates sharing a similar set
of properties.
Unfortunately, Gimli didn't pass the 3rd round.
There are no practical attacks against Gimli when used correctly, but
NIST's decision means that Gimli is unlikely to ever get any traction.
So, maybe the time has come to move Gimli from the standard library
to another repository.
We shouldn't do it without providing an alternative, though.
And the best candidate for this is probably Xoodoo.
Xoodoo is the core function of Xoodyak, one of the finalists of the
NIST LWC competition, and the most direct competitor to Gimli. It is
also a 384-bit permutation, so it can easily be used everywhere Gimli
was used with no parameter changes.
It is the building block of Xoodyak (for actual encryption and hashing)
as well as Charm, that some Zig applications are already using.
Like Gimli that it was heavily inspired from, it is compact and
suitable for constrained environments.
This change adds the Xoodoo permutation to std.crypto.core.
The set of public functions includes everything required to later
implement existing Xoodoo-based constructions.
In order to prepare for the Gimli deprecation, the default
CSPRNG was changed to a Xoodoo-based that works exactly the same way.
- Test: Fix bucket counting. Previously, the first hit was not counted.
This off-by-one error slightly increased the mean of `*_total_variance`,
which decreased the acceptance rate for a particular random seed
from 95% to 92.6%. (Irrelevant for test failure because the seed is fixed.)
- Improve comments
* The `@bitCast` workaround is removed in favor of `@ptrCast` properly
doing element casting for slice element types. This required an
enhancement both to stage1 and stage2.
* stage1 incorrectly accepts `.{}` instead of `{}`. stage2 code that
abused this is fixed.
* Make some parameters comptime to support functions in switch
expressions (as opposed to making them function pointers).
* Avoid relying on local temporaries being mutable.
* Workarounds for when stage1 and stage2 disagree on function pointer
types.
* Workaround recursive formatting bug with a `@panic("TODO")`.
* Remove unreachable `else` prongs for some inferred error sets.
All in effort towards #89.
As suggested by @leecannon, this provides more flexibility to the
`Random` interface. For exmaple, this allows for an implementation to
provide multiple different fill functions.
These changes have been made to resolve issue #10037. The `Random`
interface was implemented in such a way that causes significant slowdown
when calling the `fill` function of the rng used.
The `Random` interface is no longer stored in a field of the rng, and is
instead returned by the child function `random()` of the rng. This
avoids the performance issues caused by the interface.
We already have a LICENSE file that covers the Zig Standard Library. We
no longer need to remind everyone that the license is MIT in every single
file.
Previously this was introduced to clarify the situation for a fork of
Zig that made Zig's LICENSE file harder to find, and replaced it with
their own license that required annual payments to their company.
However that fork now appears to be dead. So there is no need to
reinforce the copyright notice in every single file.
* add Random.enumValue()
* edits suggested by review
* applied zig fmt
* Rewrite to use std.enums.values
Implemented pfgithub's suggestion to rewrite against this function, greatly simplifying the implementation.
Co-authored-by: Justin Whear <justin@economicmodeling.com>
Xoroshiro128+ is the current default non-cryptographic random
number generator.
This algorithm was designed to generate floating-point numbers, by
only using the top 53 bits. Lower bits have a significant bias, that
contradicts the documented properties for `rand.DefaultPrng`. This
also has implications on everything using `Random.fill()`, including
the way we generate random floating-point numbers.
In addition, Xoroshiro128+ has known issues. See for example:
- https://lemire.me/blog/2017/08/22/cracking-random-number-generators-xoroshiro128/
- https://www.pcg-random.org/posts/xoroshiro-fails-truncated.html
Xoshiro256++ addresses these issues, while remaining very fast.
Conflicts:
* doc/langref.html.in
* lib/std/enums.zig
* lib/std/fmt.zig
* lib/std/hash/auto_hash.zig
* lib/std/math.zig
* lib/std/mem.zig
* lib/std/meta.zig
* test/behavior/alignof.zig
* test/behavior/bitcast.zig
* test/behavior/bugs/1421.zig
* test/behavior/cast.zig
* test/behavior/ptrcast.zig
* test/behavior/type_info.zig
* test/behavior/vector.zig
Master branch added `try` to a bunch of testing function calls, and some
lines also had changed how to refer to the native architecture and other
`@import("builtin")` stuff.
std.crypto.random
* cross platform, even freestanding
* can't fail. on initialization for some systems requires calling
os.getrandom(), in which case there are rare but theoretically
possible errors. The code panics in these cases, however the
application may choose to override the default seed function and then
handle the failure another way.
* thread-safe
* supports the full Random interface
* cryptographically secure
* no syscall required to initialize on Linux (AT_RANDOM)
* calls arc4random on systems that support it
`std.crypto.randomBytes` is removed in favor of `std.crypto.random.bytes`.
I moved some of the Random implementations into their own files in the
interest of organization.
stage2 no longer requires passing a RNG; instead it uses this API.
Closes#6704
`DefaultCsprng` is documented as a cryptographically secure RNG.
While `ISAAC` is a CSPRNG, the variant we have, `ISAAC64` is not.
A 64 bit seed is a bit small to satisfy that claim.
We also saw it being used with the current date as a seed, that
also defeats the point of a CSPRNG.
Set `DefaultCsprng` to `Gimli` instead of `ISAAC64`, rename
the parameter from `init_s` to `secret_seed` + add a comment to
clarify what kind of seed is expected here.
Instead of directly touching the internals of the Gimli implementation
(which can change/be architecture-specific), add an `init()` function
to the state.
Our Gimli-based CSPRNG was also not backtracking resistant. Gimli
is a permutation; it can be reverted. So, if the state was ever leaked,
future secrets, but also all the previously generated ones could be
recovered. Clear the rate after a squeeze in order to prevent this.
Finally, a dumb test was added just to exercise `DefaultCsprng` since
we don't use it anywhere.
Instead of having all primitives and constructions share the same namespace,
they are now organized by category and function family.
Types within the same category are expected to share the exact same API.
