9465 Commits

Author SHA1 Message Date
Andrew Kelley
2e12b45d8b introduce tool for dumping coverage file
with debug info resolved.

begin efforts of providing `std.debug.Info`, a cross-platform
abstraction for loading debug information into an in-memory format that
supports queries such as "what is the source location of this virtual
memory address?"

Unlike `std.debug.SelfInfo`, this API does not assume the debug
information in question happens to match the host CPU architecture, OS,
or other target properties.
2024-08-07 00:48:32 -07:00
Andrew Kelley
107b272766 fuzzer: share zig to html rendering with autodocs 2024-08-07 00:48:32 -07:00
Andrew Kelley
e0ffac4e3c introduce a web interface for fuzzing
* new .zig-cache subdirectory: 'v'
  - stores coverage information with filename of hash of PCs that want
    coverage. This hash is a hex encoding of the 64-bit coverage ID.
* build runner
  * fixed bug in file system inputs when a compile step has an
    overridden zig_lib_dir field set.
  * set some std lib options optimized for the build runner
    - no side channel mitigations
    - no Transport Layer Security
    - no crypto fork safety
  * add a --port CLI arg for choosing the port the fuzzing web interface
    listens on. it defaults to choosing a random open port.
  * introduce a web server, and serve a basic single page application
    - shares wasm code with autodocs
    - assets are created live on request, for convenient development
      experience. main.wasm is properly cached if nothing changes.
    - sources.tar comes from file system inputs (introduced with the
      `--watch` feature)
  * receives coverage ID from test runner and sends it on a thread-safe
    queue to the WebServer.
* test runner
  - takes a zig cache directory argument now, for where to put coverage
    information.
  - sends coverage ID to parent process
* fuzzer
  - puts its logs (in debug mode) in .zig-cache/tmp/libfuzzer.log
  - computes coverage_id and makes it available with
    `fuzzer_coverage_id` exported function.
  - the memory-mapped coverage file is now namespaced by the coverage id
    in hex encoding, in `.zig-cache/v`
* tokenizer
  - add a fuzz test to check that several properties are upheld
2024-08-07 00:48:32 -07:00
Alex Rønne Petersen
ef502daafe
std.Target: Handle mesa3d in c_type_bit_size(). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
23b5a6c71e
std.Target: Treat spirv as identical to spirv64 for ABI size/alignment purposes.
This is arbitrary since spirv (as opposed to spirv32/spirv64) refers to the
version with logical memory layout, i.e. no 'real' pointers. This change at
least matches what clang does.
2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
231f322a65
std.Target: Fix C long double size for ve (16, not 8). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
7b47ebe576
std.Target: Fix C long double size for sparc32 (8, not 16). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
eef499812c
std.Target: Fix C long double size for amdhsa, amdpal, and mesa3d (8, not 16). 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
a1d3e56763
std.Target: Fix C long double size for opencl (8, not 16).
This value was correct for the old SPIR, but not for SPIR-V.
2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
606d011acf
std.Target: Fix C long long size for opencl (8, not 16).
This value was correct for the old SPIR, but not for SPIR-V.
2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
5dd66cd964
std.Target: Fix C type alignment calculation for ve. 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
e678862520
std.Target: Fix C type alignment calculation for sparc64. 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
98153c8d81
std.Target: Fix C type alignment calculation for loongarch64. 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
29321ca4a2
std.Target: Fix C type alignment calculation for spirv32. 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
e67388c2e5
std.Target: Fix C type alignment calculation for dxil. 2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
db8f00e277
std.Target: Fix ptrBitWidth_cpu_abi() for sparc32.
CPU feature set has nothing to do with ABI choice; the pointer bit width is
determined only by looking at the choice of sparc vs sparc64.
2024-08-06 20:17:03 +02:00
Alex Rønne Petersen
ab154be7f0
std.Target: Fix ptrBitWidth_cpu_abi() for dxil (64-bit, not 32-bit).
The DXIL documentation claims 32-bit pointers:

https://github.com/microsoft/DirectXShaderCompiler/blob/main/docs/DXIL.rst#memory-accesses

Despite this, Clang considers pointers 64-bit when targeting it.
2024-08-06 20:17:03 +02:00
Andrew Kelley
fab5df4028
Merge pull request #20934 from alexrp/sparc32-leon3
`std.Target`: Bump baseline CPU for sparc32 to v9; add `sparcv9-linux-gnu` to `process_headers.zig`
2024-08-05 09:52:52 -07:00
Alex Rønne Petersen
388248ad33 std.zig.target: Set minimum glibc version for csky to 2.29.
https://lists.gnu.org/archive/html/info-gnu/2019-01/msg00018.html
2024-08-05 09:49:47 -07:00
Alex Rønne Petersen
27775f1a9e
std.Target: Use v9 as the baseline CPU for sparc32.
It is impossible to even build projects like glibc when targeting a generic
SPARC v8 CPU; LEON3 is effectively considered the baseline for `sparc-linux-gnu`
now, particularly due to it supporting a CASA instruction similar to the one in
SPARC v9. However, it's slightly incompatible with SPARC v9 due to having a
different ASI tag, so resulting binaries would not be portable to regular SPARC
CPUs. So, as the least bad option, make v9 the baseline for sparc32.
2024-08-05 12:20:41 +02:00
Fri3dNstuff
a655c15c40
std.sort: Remove key argument from binary-search-like functions (#20927)
closes #20110
2024-08-04 22:02:15 +00:00
Linus Groh
a18293214d std.Target: Specify dynamic linker and C type sizes for serenity 2024-08-04 12:03:43 -07:00
Andrew Kelley
6d606cc38b reintroduce std.Dwarf.abi.supportsUnwinding
There are two concepts here: one for whether dwarf supports unwinding on
that target, and another for whether the Zig standard library
implements it yet.
2024-08-02 14:15:03 -07:00
Andrew Kelley
975c185b92 fix compilation on powerpc GNU systems
...which have a ucontext_t but not a PC register. The current stack
unwinding implementation does not yet support this architecture.

Also fix name of `std.debug.SelfInfo.openSelf` to remove redundancy.

Also removed this hook into root providing an "openSelfDebugInfo"
function. Sorry, this debugging code is not of sufficient quality to
offer a plugin API right now.
2024-08-02 14:14:59 -07:00
Andrew Kelley
48d584e3a3 std.debug: reorg and clarify API goals
After this commit:

`std.debug.SelfInfo` is a cross-platform abstraction for the current
executable's own debug information, with a goal of minimal code bloat
and compilation speed penalty.

`std.debug.Dwarf` does not assume the current executable is itself the
thing being debugged, however, it does assume the debug info has the
same CPU architecture and OS as the current executable. It is planned to
remove this limitation.
2024-08-01 23:11:59 -07:00
Andrew Kelley
290966c249 std.debug: rename Info to SelfInfo
This code has the hard-coded goal of supporting the executable's own
debug information and makes design choices along that goal, such as
memory-mapping the inputs, using dl_iterate_phdr, and doing conditional
compilation on the host target.

A more general-purpose implementation of debug information may be able
to share code with this, but there are some fundamental
incompatibilities. For example, the "SelfInfo" implementation wants to
avoid bloating the binary with PDB on POSIX systems, and likewise DWARF
on Windows systems, while a general-purpose implementation needs to
support both PDB and DWARF from the same binary. It might, for example,
inspect the debug information from a cross-compiled binary.

`SourceLocation` now lives at `std.debug.SourceLocation` and is
documented.

Deprecate `std.debug.runtime_safety` because it returns the optimization
mode of the standard library, when the caller probably wants to use the
optimization mode of their own module.

`std.pdb.Pdb` is moved to `std.debug.Pdb`, mirroring the recent
extraction of `std.debug.Dwarf` from `std.dwarf`.

I have no idea why we have both Module (with a Windows-specific
definition) and WindowsModule. I left some passive aggressive doc
comments to express my frustration.
2024-08-01 22:11:35 -07:00
Andrew Kelley
ab0253f662 std.debug.Info: rename ModuleDebugInfo to Module 2024-08-01 22:11:29 -07:00
Andrew Kelley
1ba6b56c81 std.debug.Info: extract to separate file 2024-08-01 22:11:23 -07:00
Andrew Kelley
e5b46eab3b std: dwarf namespace reorg
std.debug.Dwarf is the parsing/decoding logic. std.dwarf remains the
unopinionated types and bits alone.

If you look at this diff you can see a lot less redundancy in
namespaces.
2024-08-01 13:56:12 -07:00
Andrew Kelley
377274ee9a std.debug.DebugInfo: rename to std.debug.Info
avoiding redundancy in the fully qualified namespace
2024-08-01 13:47:42 -07:00
Andrew Kelley
12d0c9a2fc add std.fs.Dir.Handle
mirrors std.fs.File.Handle
2024-08-01 13:47:23 -07:00
Alex Rønne Petersen
624fa8523a std.os.linux: Unbreak the build
Happened because I wrote #20869 before #20870.
2024-08-01 13:05:26 +02:00
Andrew Kelley
7a0da805a8
Merge pull request #20797 from alexrp/start-more-arches
`start`: Add POSIX arc, csky, and hexagon support
2024-08-01 01:34:33 -07:00
Andrew Kelley
8f7cbaa4c0
Merge pull request #20870 from alexrp/target-cleanup-3
`std.Target`: Remove more dead OS/architecture tags
2024-08-01 01:32:32 -07:00
Andrew Kelley
f17f73b4fa
Merge pull request #20869 from alexrp/linux-syscalls
`std.os.linux`: Add syscall enums for all remaining architectures
2024-08-01 01:21:10 -07:00
Andrew Kelley
91163b44dd
Merge pull request #20857 from alexrp/tls-porting
`std.os.linux.tls`: Refactor, improve documentation, fix a bug, and port to more architectures
2024-08-01 01:15:17 -07:00
Andrew Kelley
16dde6d260
Merge pull request #20772 from alexrp/linux-audit
`std.os.linux.AUDIT`: Rewrite ARCH in terms of std.elf.EM.
2024-08-01 01:09:22 -07:00
Alex Rønne Petersen
8d5eaadb05 std.Target: Handle loongarch in Os.Tag.archName(). 2024-08-01 01:08:20 -07:00
Alex Rønne Petersen
f03d54f069 std.atomic: Don't lie to the compiler about memory clobbers in spinLoopHint(). 2024-08-01 01:07:56 -07:00
Alex Rønne Petersen
604e87a958 std.Target: Use hexagonv60 as the baseline CPU model for hexagon. 2024-08-01 01:01:24 -07:00
Andrew Kelley
9c22a6b3ad
Merge pull request #20823 from alexrp/start-pie-mips-ppc
`std.os.linux.start_pie`: Add mips and powerpc support
2024-08-01 00:57:10 -07:00
Andrew Kelley
63aa85e7af
Merge pull request #20872 from alexrp/riscv-gp
start: Initialize `gp` to `__global_pointer$` on riscv.
2024-08-01 00:47:05 -07:00
Jakub Konka
8ea323822b
Merge pull request #20884 from Rexicon226/riscv 2024-08-01 07:17:40 +02:00
Andrew Kelley
eb1a199dff
Merge pull request #20885 from ziglang/simplify-tokenizer
std.zig.tokenizer: simplification and spec conformance
2024-07-31 19:52:34 -07:00
Andrew Kelley
059856acfc
Merge pull request #20878 from tiehuis/std-math-complex-fixes
std.math.complex fixes
2024-07-31 19:19:27 -07:00
Andrew Kelley
c2b8afcac9 tokenizer: tabs and carriage returns spec conformance 2024-07-31 16:57:42 -07:00
Andrew Kelley
a7029496d1 remove hard tabs from source code
these are illegal according to the spec
2024-07-31 16:57:42 -07:00
Andrew Kelley
377e8579f9 std.zig.tokenizer: simplify
I pointed a fuzzer at the tokenizer and it crashed immediately. Upon
inspection, I was dissatisfied with the implementation. This commit
removes several mechanisms:
* Removes the "invalid byte" compile error note.
* Dramatically simplifies tokenizer recovery by making recovery always
  occur at newlines, and never otherwise.
* Removes UTF-8 validation.
* Moves some character validation logic to `std.zig.parseCharLiteral`.

Removing UTF-8 validation is a regression of #663, however, the existing
implementation was already buggy. When adding this functionality back,
it must be fuzz-tested while checking the property that it matches an
independent Unicode validation implementation on the same file. While
we're at it, fuzzing should check the other properties of that proposal,
such as no ASCII control characters existing inside the source code.

Other changes included in this commit:

* Deprecate `std.unicode.utf8Decode` and its WTF-8 counterpart. This
  function has an awkward API that is too easy to misuse.
* Make `utf8Decode2` and friends use arrays as parameters, eliminating a
  runtime assertion in favor of using the type system.

After this commit, the crash found by fuzzing, which was
"\x07\xd5\x80\xc3=o\xda|a\xfc{\x9a\xec\x91\xdf\x0f\\\x1a^\xbe;\x8c\xbf\xee\xea"
no longer causes a crash. However, I did not feel the need to add this
test case because the simplified logic eradicates most crashes of this
nature.
2024-07-31 16:57:42 -07:00
David Rubin
2b8a71489a
start: remove riscv condition 2024-07-31 13:40:00 -07:00
Alex Rønne Petersen
653eb75355
start: Disable the gp initialization code for the self-hosted riscv64 backend. 2024-07-31 02:06:57 +02:00