282 Commits

Author SHA1 Message Date
Andrew Kelley
8252c8b9d6 stage2 parser: different multiline string literal parsing strategy
and using flat memory rather than singly linked list

roughly equivalent performance, slightly reduced memory usage, better
API.
2020-05-21 23:25:15 -04:00
Andrew Kelley
19de259936 stage2 parser: arrays and structs directly in memory after the node
Slightly reduced memory usage. Roughly the same API and perf.
2020-05-21 22:52:45 -04:00
Andrew Kelley
9377af934f stage2 parser: SwitchCase uses intrusive array instead of linkedlist
no perf impact, but the API is better
2020-05-21 22:28:30 -04:00
Andrew Kelley
d37b81d43b stage2 parser performance/API improvements
* Extract Call ast node tag out of SuffixOp; parameters go in memory
   after Call.
 * Demote AsmInput and AsmOutput from AST nodes to structs inside the
   Asm node.
 * The following ast nodes get their sub-node lists directly following
   them in memory:
   - ErrorSetDecl
   - Switch
   - BuiltinCall
 * ast.Node.Asm gets slices for inputs, outputs, clobbers instead of
   singly linked lists

Performance changes:

throughput: 72.7 MiB/s => 74.0 MiB/s
maxrss: 72 KB => 69 KB (nice)
2020-05-21 22:01:16 -04:00
Andrew Kelley
32ecb416f3 fix compile errors when setting NodeIndex/TokenIndex to u32 2020-05-21 00:30:08 -04:00
Andrew Kelley
d57d9448aa stage2 parsing: rework block statements AST memory layout
block statements are now directly following the Block AST node rather
than a singly linked list. This had negligible impact on performance:

throughput: 72.3 MiB/s => 72.7 MiB/s

however it greatly improves the API since the statements are laid out in
a flat array in memory.
2020-05-20 23:47:04 -04:00
Andrew Kelley
688aa114e4 Revert "stage2 parser: elide memcpy of large initialization lists"
This reverts commit 84df1d4f3d0312553f5a3857ed67042319c20846.

Not worth the complexity! Always memcpy initialization lists into the
arena.
2020-05-20 22:42:43 -04:00
Andrew Kelley
84df1d4f3d stage2 parser: elide memcpy of large initialization lists
throughput: 71.4 MiB/s => 72.9 MiB/s
2020-05-20 22:41:18 -04:00
Andrew Kelley
897f23f20f stage2 parser: split off some SuffixOp AST nodes into separate tags
These SuffixOp nodes have their own ast.Node tags now:
 * ArrayInitializer
 * ArrayInitializerDot
 * StructInitializer
 * StructInitializerDot

Their sub-expression lists are general-purpose-allocator allocated
and then copied into the arena after completion of parsing.

throughput: 72.9 MiB/s => 74.4 MiB/s
maxrss: 68 KB => 72 KB

The API is also nicer since the sub expression lists are now flat arrays
instead of singly linked lists.
2020-05-20 19:18:14 -04:00
Andrew Kelley
7c2c0e36f8 stage2 parser: different memory layout of ParamDecl
Instead of being its own node, it's a struct inside FnProto.
Instead of FnProto having a SinglyLinkedList of ParamDecl nodes,
ParamDecls are appended directly in memory after the FnProto.

throughput: 72.2 MiB/s => 72.9 MiB/s
maxrss: 70 KB => 68 KB

Importantly, the API is improved as well since the data is arranged
linearly in memory.
2020-05-20 17:39:54 -04:00
Andrew Kelley
64c149ca19 fields and decls: ArrayList appending, memcpy to ast arena
This makes fields and decl ast nodes part of the Root and ContainerDecl
AST nodes.

Surprisingly, it's a performance regression from using a singly-linked
list for these nodes:

throughput: 76.5 MiB/s => 69.4 MiB/s

However it has much better memory usage:

maxrss: 392 KB => 77 KB

It's also better API for consumers of the parser, since it is a flat
list in memory.
2020-05-20 16:13:02 -04:00
Andrew Kelley
8c10178a1e golly jeepers it's taking a long time to update translate-c 2020-05-20 13:53:53 -04:00
Andrew Kelley
4617c5907a fix zig fmt test regressions 2020-05-20 00:19:00 -04:00
Andrew Kelley
82225f6568 zig fmt parser tests are compiling again 2020-05-19 23:08:48 -04:00
Andrew Kelley
93384f7428 use singly linked lists for std.zig.parse
std.ast uses a singly linked list for lists of things. This is a
breaking change to the self-hosted parser API.

std.ast.Tree has been separated into a private "Parser" type which
represents in-progress parsing, and std.ast.Tree which has only
"output" data. This means cleaner, but breaking, API for parse results.
Specifically, `tokens` and `errors` are no longer SegmentedList but a
slice.

The way to iterate over AST nodes has necessarily changed since lists of
nodes are now singly linked lists rather than SegmentedList.

From these changes, I observe the following on the
self-hosted-parser benchmark from ziglang/gotta-go-fast:

throughput: 45.6 MiB/s => 55.6 MiB/s
maxrss: 359 KB => 342 KB

This commit breaks the build; more updates are necessary to fix API
usage of the self-hosted parser.
2020-05-19 21:22:52 -04:00
Jakub Konka
3d267bab71 Re-enable refAllDecls gen and check in std.zig 2020-05-18 21:05:29 +02:00
Vexu
28d449b38d
fix zig fmt regression 2020-05-17 15:13:19 +03:00
Vexu
eda03354dc
Merge pull request #5358 from Vexu/parser
Fix infinite loop with invalid comptime
2020-05-16 20:49:37 +03:00
Vexu
081ffe24cf
fix infinite loop with invalid comptime 2020-05-16 19:23:59 +03:00
Vexu
74f7d710bb
Merge pull request #5032 from LakeByTheWoods/redo_translate_c
Translate C: Redo Add comment containing c source location for failed decls.
2020-05-16 17:37:19 +03:00
Vexu
a05011d403
move ParamType to Node.ParamDecl 2020-05-16 12:54:51 +03:00
Vexu
ed62081d38
recover from missing semicolon after if stmt 2020-05-16 12:29:01 +03:00
Vexu
6ca0def499
recover from invalid global error set access 2020-05-16 12:09:34 +03:00
Vexu
b2f16d4484
fix infinite loop with mismatced bracket 2020-05-15 23:50:47 +03:00
Vexu
a00fd6e254
properly handle extra closing braces at top level 2020-05-15 14:30:49 +03:00
Vexu
440189a04a
cleanup* remove unecessary error* properly handle ReturnTypeInvalid in ast.zig functions* assert that the tree is clean in render.zig* simplify parser recovery with top level decls 2020-05-15 11:35:37 +03:00
Vexu
c77fee0344
fix infinite loop
findToken wasn't as generic as I thought it was
2020-05-14 19:56:55 +03:00
Vexu
c4552ee8ed
store rparen in ReturnType.Invalid
This is useful for getting a partial function signature
2020-05-14 17:18:14 +03:00
Vexu
a32e240540
improve recovery from invalid container members
Instead of trying to find the end of the block or the next comma/semicolon
we no try to find the next token that can start a container member.
2020-05-14 12:09:40 +03:00
Vexu
89f2923a8a
recover from missing semicolon 2020-05-14 11:19:50 +03:00
Vexu
ac319b2734
remove useless nosuspend parsing
nosuspend cannot be used in a type expression and
all other use casesare covered by PrimaryExpr
2020-05-14 11:19:14 +03:00
Vexu
c3b76d0913
recover from invalid builtin/async call 2020-05-14 00:16:56 +03:00
Vexu
1f81887a78
recover after invalid inline/extern 2020-05-13 23:35:58 +03:00
Vexu
ad71d959d7
correctly recover from invalid top level declarations 2020-05-13 23:28:04 +03:00
Vexu
2296906e2a
modernize std.zig.tokenizer 2020-05-13 23:08:42 +03:00
Vexu
23c5ff94e9
improve recovery on top level declarations 2020-05-13 20:42:18 +03:00
Vexu
cefc04348e
continue parsing on invalid and token 2020-05-13 17:36:06 +03:00
Vexu
be392777b7
continue parsing after missing commas and invalid statements 2020-05-13 17:21:27 +03:00
Vexu
91358f3092
continue parsing on extra qualifier errors 2020-05-13 16:51:23 +03:00
Vexu
df22c7dfef
std.zig attempt to continue parsing on error 2020-05-12 22:37:39 +03:00
Vexu
fa57463bb9
make parser testError take a list of expected errors 2020-05-12 21:44:08 +03:00
Vexu
f2d3266075
Merge pull request #4932 from Qix-/fix-private-access
Fix private access
2020-05-08 18:21:15 +03:00
Josh Junon
b6dc7fc9ff
publicize member functions affected by #4909 2020-05-08 14:26:28 +03:00
Tadeo Kondrak
84a0a9688c
update docs/tests for async/extern fn removal 2020-05-05 10:31:32 -06:00
Tadeo Kondrak
d0e996405b
add zig fmt fix for async/extern fn 2020-05-05 09:37:59 -06:00
Tadeo Kondrak
7ada59f873
remove nakedcc/stdcallcc/async fn/extern fn fnproto 2020-05-05 09:37:28 -06:00
Andrew Kelley
e6955688ac
Merge pull request #5272 from tadeokondrak/noasync-to-nosuspend
Noasync to nosuspend
2020-05-05 11:21:02 -04:00
Tadeo Kondrak
af00afed98
zig fmt 2020-05-05 05:55:25 -06:00
Tadeo Kondrak
8d5636ebe4
Rename noasync to nosuspend in self-hosted, add rewriter 2020-05-05 05:17:33 -06:00
Jonathan Marler
75b699b2c6 os.zig: add ETIMEDOUT error case to read function
According to documentation ETIMEDOUT (110) is a valid error code for the read function.  I just had my long-running  (been running for about 7 weeks) network program crash because it did not handle the ETIMEDOUT error code from "read".
2020-05-04 13:48:34 -04:00