337 Commits

Author SHA1 Message Date
Vexu
010c58e303
fix zig fmt out of bounds on empty file 2020-05-30 23:07:51 +03:00
Jakub Konka
e61e8c94be Reenable zig parser tests disabled targeting Wasm
I'm not sure why I disabled them when landing extended Wasm/WASI
support, but they pass the parser tests just fine now, so I'm gonna
go ahead and re-enable them.
2020-05-26 21:01:54 -04:00
Vexu
e07b467c7c
fix missing compile error on while/for missing block 2020-05-25 23:25:06 +03:00
Andrew Kelley
8df0841d6a stage2 parser: token ids in their own array
To prevent cache misses, token ids go in their own array, and the
start/end offsets go in a different one.

perf measurement before:
         2,667,914      cache-misses:u
     2,139,139,935      instructions:u
       894,167,331      cycles:u

perf measurement after:
         1,757,723      cache-misses:u
     2,069,932,298      instructions:u
       858,105,570      cycles:u
2020-05-22 12:34:12 -04:00
Andrew Kelley
93384f7428 use singly linked lists for std.zig.parse
std.ast uses a singly linked list for lists of things. This is a
breaking change to the self-hosted parser API.

std.ast.Tree has been separated into a private "Parser" type which
represents in-progress parsing, and std.ast.Tree which has only
"output" data. This means cleaner, but breaking, API for parse results.
Specifically, `tokens` and `errors` are no longer SegmentedList but a
slice.

The way to iterate over AST nodes has necessarily changed since lists of
nodes are now singly linked lists rather than SegmentedList.

From these changes, I observe the following on the
self-hosted-parser benchmark from ziglang/gotta-go-fast:

throughput: 45.6 MiB/s => 55.6 MiB/s
maxrss: 359 KB => 342 KB

This commit breaks the build; more updates are necessary to fix API
usage of the self-hosted parser.
2020-05-19 21:22:52 -04:00
Jakub Konka
3d267bab71 Re-enable refAllDecls gen and check in std.zig 2020-05-18 21:05:29 +02:00
Vexu
28d449b38d
fix zig fmt regression 2020-05-17 15:13:19 +03:00
Vexu
081ffe24cf
fix infinite loop with invalid comptime 2020-05-16 19:23:59 +03:00
Vexu
ed62081d38
recover from missing semicolon after if stmt 2020-05-16 12:29:01 +03:00
Vexu
6ca0def499
recover from invalid global error set access 2020-05-16 12:09:34 +03:00
Vexu
b2f16d4484
fix infinite loop with mismatced bracket 2020-05-15 23:50:47 +03:00
Vexu
a00fd6e254
properly handle extra closing braces at top level 2020-05-15 14:30:49 +03:00
Vexu
440189a04a
cleanup* remove unecessary error* properly handle ReturnTypeInvalid in ast.zig functions* assert that the tree is clean in render.zig* simplify parser recovery with top level decls 2020-05-15 11:35:37 +03:00
Vexu
c77fee0344
fix infinite loop
findToken wasn't as generic as I thought it was
2020-05-14 19:56:55 +03:00
Vexu
a32e240540
improve recovery from invalid container members
Instead of trying to find the end of the block or the next comma/semicolon
we no try to find the next token that can start a container member.
2020-05-14 12:09:40 +03:00
Vexu
89f2923a8a
recover from missing semicolon 2020-05-14 11:19:50 +03:00
Vexu
c3b76d0913
recover from invalid builtin/async call 2020-05-14 00:16:56 +03:00
Vexu
1f81887a78
recover after invalid inline/extern 2020-05-13 23:35:58 +03:00
Vexu
ad71d959d7
correctly recover from invalid top level declarations 2020-05-13 23:28:04 +03:00
Vexu
23c5ff94e9
improve recovery on top level declarations 2020-05-13 20:42:18 +03:00
Vexu
cefc04348e
continue parsing on invalid and token 2020-05-13 17:36:06 +03:00
Vexu
be392777b7
continue parsing after missing commas and invalid statements 2020-05-13 17:21:27 +03:00
Vexu
df22c7dfef
std.zig attempt to continue parsing on error 2020-05-12 22:37:39 +03:00
Vexu
fa57463bb9
make parser testError take a list of expected errors 2020-05-12 21:44:08 +03:00
Tadeo Kondrak
84a0a9688c
update docs/tests for async/extern fn removal 2020-05-05 10:31:32 -06:00
Tadeo Kondrak
d0e996405b
add zig fmt fix for async/extern fn 2020-05-05 09:37:59 -06:00
Tadeo Kondrak
7ada59f873
remove nakedcc/stdcallcc/async fn/extern fn fnproto 2020-05-05 09:37:28 -06:00
Tadeo Kondrak
8d5636ebe4
Rename noasync to nosuspend in self-hosted, add rewriter 2020-05-05 05:17:33 -06:00
Andrew Kelley
b7914d901c add test coverage for top level fields
closes #2022
2020-05-02 14:53:20 -04:00
Vexu
e72f45475d
Merge pull request #4683 from LakeByTheWoods/parser_test
Add visible newlines to parser_test output when there's a failure.
2020-04-30 12:04:23 +03:00
Vexu
87c9696121
move printWithVisibleNewlines to testing.expectEqualStrings 2020-04-30 10:34:18 +03:00
Vexu
2d06e731ec
rename diffIndex to indexOfDiff 2020-04-30 10:33:50 +03:00
Robin Voetter
32e5248820 Remove old-style @typeOf compatibility 2020-04-20 22:38:29 -04:00
Vexu
b6fe839248
update std lib to decls being disallowed between fields 2020-04-18 23:56:05 +03:00
Vexu
ca3bf6e6ad
translate-c cleanup and zig fmt 2020-04-15 15:15:32 +03:00
Lachlan Easton
daff072af2 Add visible newlines to parser_test output when there's a failure.
Also print first line that differs between expected and result.
2020-04-10 10:38:36 +10:00
daurnimator
e535057364
std: use std.ArrayList(u8).OutStream instead of std.Buffer.OutStream 2020-04-01 10:36:38 -04:00
Michael Raymond
de9933761c std.zig.render: fix newlines before DocComments 2020-03-29 22:41:39 -04:00
Andrew Kelley
13d04f9963
Merge pull request #4741 from momumi/master
allow `_` separators in number literals (stage 1)
2020-03-23 00:54:54 -04:00
momumi
2d18178c27 minor fixes and more tests for _ separators
* Make the tokenizer spit out an Invalid token on the first invalid
  character found in the number literal.
* More parsing and tokenizer tests for number literals
* fix invalid switch statement in ir.zig
2020-03-23 09:21:34 +10:00
LemonBoy
28dbc58837 Address review comments 2020-03-21 09:54:49 +01:00
LemonBoy
128e70ff3a ir: Allow errdefer with payload
Closes #1265
2020-03-21 09:54:49 +01:00
LemonBoy
11a4ce42c1
zig fmt: Respect trailing commas in error set declarations
The logic is not perfect as it doesn't take into account the presence of
doc comments, but it's an improvement over the status quo.
2020-03-18 11:15:22 -04:00
Andrew Kelley
18f1fef142
update standard library to new I/O streams API 2020-03-10 18:44:30 -04:00
Vexu
1ad831a0ef fix zig fmt on noasync block 2020-03-10 15:27:11 -04:00
Vexu
45da72c5b6
remove usages of @typeId, @memberCount, @memberName and @memberType 2020-02-24 23:09:01 +02:00
LemonBoy
95a71e29f8
zig fmt: Fix callconv rewriting for extern + string
Closes #4473
2020-02-18 18:18:15 -05:00
Andrew Kelley
cb3a818699
zig fmt: support noasync await 2020-02-16 01:44:52 -05:00
Benjamin Feng
699c50a375 Switch a bunch of FBA to use testing.allocator 2020-02-12 17:17:56 -06:00
LemonBoy
1658becb62 fmt: Fix one more edge case in container formatting 2020-02-03 16:03:48 -05:00