* Scan from line start when finding tag in tokenizer
This resolves a crash that can occur for invalid bytes like carriage
returns that are valid characters when not parsed from within literals.
There are potentially other edge cases this could resolve as well, as
the calling code for this function didn't account for any potential
'pending_invalid_tokens' that could be queued up by the tokenizer from
within another state.
* Fix carriage return crash in multiline string
Follow the guidance of #38:
> However CR directly before NL is interpreted as only a newline and not part of the multiline string. zig fmt will delete the CR.
Zig fmt already had code for deleting carriage returns, but would still
crash - now it no longer does so. Carriage returns encountered before
line-feeds are now appropriately removed on program compilation as well.
* Only accept carriage returns before line feeds
Previous commit was much less strict about this, this more closely
matches the desired spec of only allow CR characters in a CRLF pair, but
not otherwise.
* Fix CR being rejected when used as whitespace
Missed this comment from ziglang/zig-spec#83:
> CR used as whitespace, whether directly preceding NL or stray, is still unambiguously whitespace. It is accepted by the grammar and replaced by the canonical whitespace by zig fmt.
* Add tests for carriage return handling
* improve error message when build manifest file is missing
* update std.zig.Ast to support ZON
* Compilation.AllErrors.Message: make the notes field a const slice
* move build manifest parsing logic into src/Manifest.zig and add more
checks, and make the checks integrate into the standard error
reporting code so that reported errors look sexy
closes#14290
* std.zig.parse is moved to std.zig.Ast.parse
* the new function has an additional parameter that requires passing
Mode.zig or Mode.zon
* moved parser.zig code to Parse.zig
* added parseZon function next to parseRoot function
* allow file level `union {}` to parse as tuple field
this was found while fuzzing zls.
* before this patch the input `union {}` crashed the parser. after
this, it parses correctly just like `struct {}`.
* adds behavior tests for both inputs `struct {}` and `union {}`,
checking that each becomes a file level tuple field.
these were found while fuzzing zls.
this patch prevents overflow for the following file contents and adds
tests for them.
* `enum(u32)` - causes overflow in std.zig.Ast.fullContainerDecl()
* `*x` - causes overflow in std.zig.Ast.fullPtrType()
* `**x` - causes overflow in std.zig.Ast.firstToken()
There are still a few occurrences of "stage1" in the standard library
and self-hosted compiler source, however, these instances need a bit
more careful inspection to ensure no breakage.
If a '(' is found where the continue expression was expected and it is
on the same line as the previous token issue an error about missing
colon before the continue expression.
For some errors if the found token is not on the same line as
the previous token, point to the end of the previous token.
This usually results in more helpful errors.
* AstGen: use Ast.zig helper methods to avoid copy pasting token counting logic
- take advantage of the `first_doc_comment` field we already have for
param AST nodes
* Add missing ZIR docs
* Remove the builtins `@addWithSaturation`, `@subWithSaturation`,
`@mulWithSaturation`, and `@shlWithSaturation` now that we have
first-class syntax for saturating arithmetic.
* langref: Clarify the behavior of `@shlExact`.
* Ast: rename `bit_shift_left` to `shl` and `bit_shift_right` to `shr`
for consistency.
* Air: rename to include underscore separator with consistency with
the rest of the ops.
* Air: add shl_exact instruction
* Use non-extended tags for saturating arithmetic, to keep it
simple so that all the arithmetic operations can be done the same
way.
- Sema: unify analyzeArithmetic with analyzeSatArithmetic
- implement comptime `+|`, `-|`, and `*|`
- allow float operands to saturating arithmetic
* `<<|` allows any integer type for the RHS.
* C backend: fix rebase conflicts
* LLVM backend: reduce the amount of branching for arithmetic ops
* zig.h: fix magic number not matching actual size of C integer types
- adds initial support for the operators +|, -|, *|, <<|, +|=, -|=, *|=, <<|=
- uses operators in addition to builtins in behavior test
- adds binOpExt() and assignBinOpExt() to AstGen.zig. these need to be audited
