Start implementing https://github.com/ziglang/zig/issues/4917 which is to rename instream/outstream to reader/writer. This first change allows code to use Writer/writer instead of OutStream/outStream, but still maintains the old outstream names with "Deprecated" comments.
To prevent cache misses, token ids go in their own array, and the
start/end offsets go in a different one.
perf measurement before:
2,667,914 cache-misses:u
2,139,139,935 instructions:u
894,167,331 cycles:u
perf measurement after:
1,757,723 cache-misses:u
2,069,932,298 instructions:u
858,105,570 cycles:u
The DocComment AST node now only points to the first doc comment token.
API users are expected to iterate over the following tokens directly.
After this commit there are no more linked lists in use in the
self-hosted AST API.
Performance impact is negligible. Memory usage slightly reduced.
* Extract Call ast node tag out of SuffixOp; parameters go in memory
after Call.
* Demote AsmInput and AsmOutput from AST nodes to structs inside the
Asm node.
* The following ast nodes get their sub-node lists directly following
them in memory:
- ErrorSetDecl
- Switch
- BuiltinCall
* ast.Node.Asm gets slices for inputs, outputs, clobbers instead of
singly linked lists
Performance changes:
throughput: 72.7 MiB/s => 74.0 MiB/s
maxrss: 72 KB => 69 KB (nice)
block statements are now directly following the Block AST node rather
than a singly linked list. This had negligible impact on performance:
throughput: 72.3 MiB/s => 72.7 MiB/s
however it greatly improves the API since the statements are laid out in
a flat array in memory.
These SuffixOp nodes have their own ast.Node tags now:
* ArrayInitializer
* ArrayInitializerDot
* StructInitializer
* StructInitializerDot
Their sub-expression lists are general-purpose-allocator allocated
and then copied into the arena after completion of parsing.
throughput: 72.9 MiB/s => 74.4 MiB/s
maxrss: 68 KB => 72 KB
The API is also nicer since the sub expression lists are now flat arrays
instead of singly linked lists.
Instead of being its own node, it's a struct inside FnProto.
Instead of FnProto having a SinglyLinkedList of ParamDecl nodes,
ParamDecls are appended directly in memory after the FnProto.
throughput: 72.2 MiB/s => 72.9 MiB/s
maxrss: 70 KB => 68 KB
Importantly, the API is improved as well since the data is arranged
linearly in memory.
This makes fields and decl ast nodes part of the Root and ContainerDecl
AST nodes.
Surprisingly, it's a performance regression from using a singly-linked
list for these nodes:
throughput: 76.5 MiB/s => 69.4 MiB/s
However it has much better memory usage:
maxrss: 392 KB => 77 KB
It's also better API for consumers of the parser, since it is a flat
list in memory.
std.ast uses a singly linked list for lists of things. This is a
breaking change to the self-hosted parser API.
std.ast.Tree has been separated into a private "Parser" type which
represents in-progress parsing, and std.ast.Tree which has only
"output" data. This means cleaner, but breaking, API for parse results.
Specifically, `tokens` and `errors` are no longer SegmentedList but a
slice.
The way to iterate over AST nodes has necessarily changed since lists of
nodes are now singly linked lists rather than SegmentedList.
From these changes, I observe the following on the
self-hosted-parser benchmark from ziglang/gotta-go-fast:
throughput: 45.6 MiB/s => 55.6 MiB/s
maxrss: 359 KB => 342 KB
This commit breaks the build; more updates are necessary to fix API
usage of the self-hosted parser.
The main goal here is to make the function pointers comptime, so that we
don't have to do the crazy stuff with async function frames.
Since InStream, OutStream, and SeekableStream are already generic
across error sets, it's not really worse to make them generic across the
vtable as well.
See #764 for the open issue acknowledging that using generics for these
abstractions is a design flaw.
See #130 for the efforts to make these abstractions non-generic.
This commit also changes the OutStream API so that `write` returns
number of bytes written, and `writeAll` is the one that loops until the
whole buffer is written.
* rework os.sendfile and add macosx support, and a fallback
implementation for any OS.
* fix sendto compile error
* std.os write functions support partial writes. closes#3443.
* std.os pread / pwrite functions can now return `error.Unseekable`.
* std.fs.File read/write functions now have readAll/writeAll variants
which loop to complete operations even when partial reads/writes
happen.
* Audit std.os read/write functions with respect to Linux returning
EINVAL for lengths greater than 0x7fff0000.
* std.os read/write shim functions do not unnecessarily loop. Since
partial reads/writes are part of the API, the caller will be forced
to loop anyway, and so that would just be code bloat.
* Improve doc comments
* Add a non-trivial test for std.os.sendfile
* Fix std.os.pread on 32 bit Linux
* Add missing SYS_sendfile bit on aarch64
