11659 Commits

Author SHA1 Message Date
Andrew Kelley
0086d315f5 std.Io: add detached async 2025-10-02 16:30:59 -07:00
Andrew Kelley
5508b4c887 implement Mutex, Condition, and Queue 2025-10-02 16:30:59 -07:00
Jacob Young
b01244d225 Io: implement sleep and fix cancel bugs 2025-10-02 16:30:59 -07:00
Jacob Young
b37126bc08 EventLoop: implement thread-local queues and cancellation 2025-10-02 16:30:59 -07:00
Andrew Kelley
c278830592 std.Io: introduce cancellation 2025-10-02 16:30:59 -07:00
Andrew Kelley
79e278f6a2 better API for Io.async 2025-10-02 16:30:59 -07:00
Jacob Young
708bac1a57 EventLoop: fix futex usage
How silly of me to forget that the kernel doesn't implement its own API.
The scheduling is not great, but at least doesn't deadlock or hammer.
2025-10-02 16:30:59 -07:00
Jacob Young
507f973b5e EventLoop: get file operations working
Something is horribly wrong with scheduling, as can be seen in the
debug output, but at least it somehow manages to exit cleanly...
2025-10-02 16:30:59 -07:00
Andrew Kelley
b215ddc9fb WIP 2025-10-02 16:30:59 -07:00
Andrew Kelley
be68b28b9b start adding fs functions to std.Io 2025-10-02 16:30:59 -07:00
Andrew Kelley
5be0d0b48c free freeing wrong amount in thread pool impl 2025-10-02 16:30:59 -07:00
Jacob Young
2f2019645c EventLoop: move context after the async closure
This avoids needing to store more sizes and alignments.  Only the result
alignment needs to be stored, because `Fiber` is at a fixed zero offset.
2025-10-02 16:30:59 -07:00
Jacob Young
dfbf68e5fa EventLoop: fix incorrect alignment panic
When the previous fiber did not request to be registered as an awaiter,
it may not have actually been a full blown `Fiber`, so only create the
`Fiber` pointer when needed.
2025-10-02 16:30:59 -07:00
Andrew Kelley
3ee2399630 update threaded fibers impl to actually storing args
sorry, something still not working correctly
2025-10-02 16:30:59 -07:00
Andrew Kelley
acd02e978b fix context passing in threaded Io impl 2025-10-02 16:30:59 -07:00
Jacob Young
1e79f2c12f EventLoop: implement main idle fiber 2025-10-02 16:30:59 -07:00
Jacob Young
2c1ceb4c9c EventLoop: add threads 2025-10-02 16:30:59 -07:00
Jacob Young
19e7613a2d EventLoop: rewrite context switching 2025-10-02 16:30:59 -07:00
Jacob Young
a9723598d7 EventLoop: prepare for threading 2025-10-02 16:30:59 -07:00
Andrew Kelley
1e09d7f499 demo: single-threaded green threads implementation 2025-10-02 16:30:59 -07:00
Andrew Kelley
cfb3581fb9 make thread pool satisfy async/await interface 2025-10-02 16:30:59 -07:00
Andrew Kelley
63c554d5a7 introduce std.Io interface
which is planned to have all I/O operations in the interface, but for
now has only async and await.
2025-10-02 16:30:59 -07:00
Alex Rønne Petersen
a4f95b1e61
std.debug.Dwarf.Unwind: deal with invalid def_cfa_reg by GNU toolchains 2025-10-02 15:27:35 +02:00
Alex Rønne Petersen
97de46dc16
std.debug: add riscv32-linux and riscv64-linux unwind support 2025-10-01 23:47:47 +02:00
Alex Rønne Petersen
8520e9312e
std.debug: add loongarch64-linux unwind support 2025-10-01 23:47:47 +02:00
Alex Rønne Petersen
b46867848e
std.debug: some adjustments to target handling
* driverkit handling missing in a few places.
* x86-solaris is a dead target.
* aarch64_be does not exist on Darwin, FreeBSD, Windows.
2025-10-01 23:47:47 +02:00
Alex Rønne Petersen
771410cbf2
std.debug.SelfInfo: rename Darwin to MachO 2025-10-01 23:47:47 +02:00
Alex Rønne Petersen
e1fb662f60
std.debug: don't use SelfInfo.Windows for UEFI
It is, in fact, Windows-only.
2025-10-01 23:47:47 +02:00
Alex Rønne Petersen
59633e54a2
std.debug: select SelfInfo using ObjectFormat.default() 2025-10-01 23:47:47 +02:00
Ryan Liptak
dcfc851349 ArrayHashMapWithAllocator: add sortUnstable fn alongside sort 2025-09-30 19:33:03 -07:00
Kendall Condon
f50c647977 add deflate compression, simplify decompression
Implements deflate compression from scratch. A history window is kept in
the writer's buffer for matching and a chained hash table is used to
find matches. Tokens are accumulated until a threshold is reached and
then outputted as a block. Flush is used to indicate end of stream.

Additionally, two other deflate writers are provided:
* `Raw` writes only in store blocks (the uncompressed bytes). It
  utilizes data vectors to efficiently send block headers and data.
* `Huffman` only performs Huffman compression on data and no matching.

The above are also able to take advantage of writer semantics since they
do not need to keep a history.

Literal and distance code parameters in `token` have also been reworked.
Their parameters are now derived mathematically, however the more
expensive ones are still obtained through a lookup table (expect on
ReleaseSmall).

Decompression bit reading has been greatly simplified, taking advantage
of the ability to peek on the underlying reader. Additionally, a few
bugs with limit handling have been fixed.
2025-09-30 18:28:47 -07:00
mlugg
1120546f72
std.debug.SelfInfo: remove shared logic
There were only a few dozen lines of common logic, and they frankly
introduced more complexity than they eliminated. Instead, let's accept
that the implementations of `SelfInfo` are all pretty different and want
to track different state. This probably fixes some synchronization and
memory bugs by simplifying a bunch of stuff. It also improves the DWARF
unwind cache, making it around twice as fast in a debug build with the
self-hosted x86_64 backend, because we no longer have to redundantly go
through the hashmap lookup logic to find the module. Unwinding on
Windows will also see a slight performance boost from this change,
because `RtlVirtualUnwind` does not need to know the module whatsoever,
so the old `SelfInfo` implementation was doing redundant work. Lastly,
this makes it even easier to implement `SelfInfo` on freestanding
targets; there is no longer a need to emulate a real module system,
since the user controls the whole implementation!

There are various other small refactors here in the `SelfInfo`
implementations as well as in the DWARF unwinding logic. This change
turned out to make a lot of stuff simpler!
2025-09-30 14:18:26 +01:00
mlugg
12ceb896fa
Dwarf.Unwind: fix typo 2025-09-30 13:44:56 +01:00
mlugg
a90eb50c80
typo 2025-09-30 13:44:56 +01:00
mlugg
8950831d3c
Dwarf.Unwind: handle macOS deviation from standard
Apparently the `__eh_frame` in Mach-O binaries doesn't include the
terminator entry, but in all other respects it acts like `.eh_frame`
rather than `.debug_frame`. I have no idea.
2025-09-30 13:44:56 +01:00
mlugg
156cd8f678
std.debug: significantly speed up capturing stack traces
By my estimation, these changes speed up DWARF unwinding when using the
self-hosted x86_64 backend by around 7x. There are two very significant
enhancements: we no longer iterate frames which don't fit in the stack
trace buffer, and we cache register rules (in a fixed buffer) to avoid
re-parsing and evaluating CFI instructions in most cases. Alongside this
are a bunch of smaller enhancements, such as pre-caching the result of
evaluating the CIE's initial instructions, avoiding re-parsing of CIEs,
and big simplifications to the `Dwarf.Unwind.VirtualMachine` logic.
2025-09-30 13:44:56 +01:00
mlugg
dbda011ae6
std.debug.SelfInfo: mark ARM unwinding as unsupported
We need to parse the `.ARM.exidx` section to be able to reliably unwind
the stack on ARM.
2025-09-30 13:44:56 +01:00
mlugg
950a9d2a10
typo 2025-09-30 13:44:56 +01:00
mlugg
f7e0ff8a5f
std: clarify cpu_context register order rationale 2025-09-30 13:44:56 +01:00
mlugg
b0f222777c
std.debug: cap total stack trace frames
...just in case there is broken debug info and/or bad values on the
stack, either of which could cause stack unwinding to potentially loop
forever.
2025-09-30 13:44:56 +01:00
mlugg
c41bf99684
std.debug: don't assume return address register is defined if not specified
This logic was causing some occasional infinite looping on ARM, where
the `.debug_frame` section is often incomplete since the `.exidx`
section is used for unwind information. But the information we're
getting from the compiler is totally *valid*: it's leaving the rule as
the default, which is (as with most architectures) equivalent to
`.undefined`!
2025-09-30 13:44:55 +01:00
mlugg
099a950410
std.debug.SelfInfo: thread safety
This has been a TODO for ages, but in the past it didn't really matter
because stack traces are typically printed to stderr for which a mutex
is held so in practice there was a mutex guarding usage of `SelfInfo`.

However, now that `SelfInfo` is also used for simply capturing traces,
thread safety is needed. Instead of just a single mutex, though, there
are a couple of different mutexes involved; this helps make critical
sections smaller, particularly when unwinding the stack as `unwindFrame`
doesn't typically need to hold any lock at all.
2025-09-30 13:44:55 +01:00
mlugg
9c1821d3bf
ElfModule: fix assertion failure 2025-09-30 13:44:55 +01:00
mlugg
084e92879a
std: don't get CPU context when using CBE targeting MSVC
Calling `current` here causes compilation failures as the C backend
currently does not emit valid MSVC inline assembly. This change means
that when building for MSVC with the self-hosted C backend, only FP
unwinding can be used.
2025-09-30 13:44:55 +01:00
mlugg
dae703d3c0
std.posix.abort: only trigger breakpoint on Windows if being debugged
Processes should reasonably be able to expect their children to abort
with typical exit codes, rather than a debugger breakpoint signal. This
flag in the PEB is what would be checked by `IsDebuggerPresent` in
kernel32, which is the function you would typically use for this
purpose.

This fixes `test-stack-trace` failures on Windows, as these tests were
expecting exit code 3 to indicate abort.
2025-09-30 13:44:55 +01:00
mlugg
2ab650b481
std.debug: go back to storing return addresses instead of call addresses
...and just deal with signal handlers by adding 1 to create a fake
"return address". The system I tried out where the addresses returned by
`StackIterator` were pre-subtracted didn't play nicely with error
traces, which in hindsight, makes perfect sense. This definition also
removes some ugly off-by-one issues in matching `first_address`, so I do
think this is a better approach.
2025-09-30 13:44:55 +01:00
mlugg
9434bab313
std: work around crash parsing LLVM PDB
This crash exists on master, and seems to have existed since 2019; I
think it's just very rare and depends on the exact binary generated. In
theory, a stream block should always be a "data" block rather than a FPM
block; the FPMs use blocks `1, 4097, 8193, ...` and `2, 4097, 8194, ...`
respectively. However, I have observed LLVM emitting an otherwise valid
PDB which maps FPM blocks into streams. This is not a bug in
`std.debug.Pdb`, because `llvm-pdbutil` agrees with our stream indices.
I think this is arguably an LLVM bug; however, we don't really lose
anything from just weakening this check. To be fair, MSF doesn't have an
explicit specification, and LLVM's documentation (which is the closest
thing we have) does not explicitly state that FPM blocks cannot be
mapped into streams, so perhaps this is actually valid.

In the rare case that LLVM emits this, previously, stack traces would
have been completely useless; now, stack traces will work okay.
2025-09-30 13:44:55 +01:00
mlugg
23d6381e8b
std.debug: fix typo 2025-09-30 13:44:55 +01:00
mlugg
0c24b8ec66
update to new std.debug changes 2025-09-30 13:44:55 +01:00
mlugg
3a9c680ad7
std: allow disabling stack tracing
This option disables both capturing and printing stack traces. The
default is to disable if debug info is stripped.
2025-09-30 13:44:55 +01:00