8932 Commits

Author SHA1 Message Date
Andrew Kelley
f6549a956d std.ArrayList: add initBuffer to the unmanaged array list
This is useful when you want to have an array list backed by a fixed
slice of memory and no Allocator will be used.

It's an alternative to BoundedArray as you will see in the following
commit.
2023-11-22 11:32:14 -07:00
Andrew Kelley
d5e21a4f1a std: remove meta.trait
In general, I don't like the idea of std.meta.trait, and so I am
providing some guidance by deleting the entire namespace from the
standard library and compiler codebase.

My main criticism is that it's overcomplicated machinery that bloats
compile times and is ultimately unnecessary given the existence of Zig's
strong type system and reference traces.

Users who want this can create a third party package that provides this
functionality.

closes #18051
2023-11-22 13:24:27 -05:00
Hong Shick Pak
994e191643
std.Uri: fix parsing edge case panic 2023-11-22 07:40:30 +00:00
Mikko Kaihlavirta
ea4a07701e add missing timeval struct 2023-11-22 01:15:11 +02:00
Andrew Kelley
be6f76655f
Merge pull request #18055 from ziglang/zig-init 2023-11-21 11:59:24 -05:00
Jan Philipp Hafer
27b34a5b77 std.net: enable forcing non-blocking mode for accept
Justification: It is common for non-CPU bound short routines to do
non-blocking accept to eliminate unnecessary delays before subscribing
to data, for example in hardware integration tests.
2023-11-21 18:04:33 +02:00
Michael Pfaff
478c89b46f
std.heap: Use @alignOf(T) rather than 0 if not manually overridden for alignment of MemoryPool items 2023-11-21 13:23:53 +00:00
salo-dea
a58ecf7b09 Do not assume that FILE_BOTH_DIR_INFORMATION is correctly aligned 2023-11-21 15:21:41 +02:00
Wooster
5d241a1478 std.debug: detect general protection faults on x86_64-linux
```zig
const std = @import("std");

pub fn main() !void {
	var addr: *u8 = @ptrFromInt(0xaaaaaaaaaaaaaaaa);
	addr.* = 1;
}
```

On x86_64-linux:

Before:
```
$ zig run x.zig
Segmentation fault at address 0x0
/home/wooster/Desktop/zig/x.zig:5:5: 0x21d887 in main (x)
    addr.* = 1;
    ^
/home/wooster/Desktop/zig-linux-x86_64/lib/std/start.zig:583:37: 0x21d847 in posixCallMainAndExit (x)
            const result = root.main() catch |err| {
                                    ^
/home/wooster/Desktop/zig-linux-x86_64/lib/std/start.zig:251:5: 0x21d371 in _start (x)
    asm volatile (switch (native_arch) {
    ^
???:?:?: 0x0 in ??? (???)
Aborted (core dumped)
```

After:
```
$ zig run x.zig --zig-lib-dir lib
General protection exception
/home/wooster/Desktop/zig/x.zig:5:5: 0x21d907 in main (x)
    addr.* = 1;
    ^
/home/wooster/Desktop/zig/lib/std/start.zig:583:37: 0x21d8c7 in posixCallMainAndExit (x)
            const result = root.main() catch |err| {
                                    ^
/home/wooster/Desktop/zig/lib/std/start.zig:251:5: 0x21d3f1 in _start (x)
    asm volatile (switch (native_arch) {
    ^
???:?:?: 0x0 in ??? (???)
Aborted (core dumped)
```

As @IntegratedQuantum pointed out in <https://github.com/ziglang/zig/issues/17745#issuecomment-1783815386>,
it seems that if `code` of the `siginfo_t` instance is a certain value (128), you are able to distinguish between
a general protection exception and a segmentation fault.

This does not seem to be documented on `man sigaction`:
```
The following values can be placed in si_code for a SIGSEGV signal:

           SEGV_MAPERR
                  Address not mapped to object.

           SEGV_ACCERR
                  Invalid permissions for mapped object.

           SEGV_BNDERR (since Linux 3.19)
                  Failed address bound checks.

           SEGV_PKUERR (since Linux 4.6)
                  Access was denied by memory protection keys.  See pkeys(7).  The protection key which applied to this access is available via si_pkey.
```
(those constants are 1, 2, 3, and 4; none of them are the 128)

I can't find a lot of documentation about this but it seems to work consistently for me on x86_64-linux.
Here is a gist which provides additional evidence that this is a reliable way of checking for a general protection fault:
https://gist.github.com/ytoshima/5682393 (read comment in first line)

See also: https://stackoverflow.com/questions/64309366/why-is-the-segfault-address-null-when-accessing-memory-that-has-any-of-the-16-mo

This only seems to affect x86_64 and on 32-bit x86 this does not seem to be a problem.

Helps with #17745 but doesn't close it because the issue still exists on Windows and other POSIX OSs.

I also limited this to x86_64-linux for now because that's the only platform where I tested it. Might work on more POSIX OSs.
2023-11-21 14:22:11 +02:00
expikr
40bd93e2a2 replace qpf and qpc
Update windows.zig

Update windows.zig

Update windows.zig

Update windows.zig

Update windows.zig

Update windows.zig

Update windows.zig

Update ntdll.zig

Update windows.zig

Update ntdll.zig

Update kernel32.zig
2023-11-21 14:10:25 +02:00
Naboris
c25f2fa561 std.RingBuffer.zig: Fix typo in readLastAssumeLength 2023-11-21 13:55:09 +02:00
Ryan Liptak
15a6b27957 std.unicode: Disable utf8 -> utf16 ASCII fast path on mips
Fixes a compile error when the target is mips, since std.simd.interlace does not work correctly on mips and raises a compile error if it is used.
2023-11-21 13:51:03 +02:00
Andrew Kelley
f645022d16 merge zig init-exe and zig init-lib into zig init
Instead of `zig init-lib` and `zig init-exe`, now there is only
`zig init`, which initializes any of the template files that do not
already exist, and makes a package that contains both an executable and
a static library. The idea is that the user can delete whatever they
don't want. In fact, I think even more things should be added to the
build.zig template.
2023-11-20 23:01:45 -07:00
Maciej 'vesim' Kuliński
f64f3423e4 std.tar: trim also spaces from the beginning of file size 2023-11-21 00:55:22 -05:00
Tw
8ca4a5240e linux/bpf: update helper definitions
Signed-off-by: Tw <tw19881113@gmail.com>
2023-11-19 19:01:18 +02:00
mlugg
766306793a
std: correct faulty test 2023-11-19 11:11:50 +00:00
mlugg
51595d6b75
lib: correct unnecessary uses of 'var' 2023-11-19 09:55:07 +00:00
dweiller
325e0f5f0e test: check compile errors when compilation has no errors 2023-11-19 00:12:43 +02:00
Eric Joldasov
4e212f1650 std.enums: allow non-exhaustive enums in EnumIndexer and make count comptime_int instead of usize
Seems like this restriction was actual when Ziglang had extern enums,
but now it's not neccessary and can be lifted. It was present since
original PR which introduced std.enums, https://www.github.com/ziglang/zig/pull/8171.
See also: https://ziggit.dev/t/catching-invalid-enum-value-errors/2206/11

* Make `count` comptime_int instead of usize

With previous type, creating EnumIndexer for enum(usize) and enum(isize)
would cause compile error since `count` could not store maxInt(usize) + 1.
Now it can store it and reflects len field from std.builtin.Type.Array
(most common use case of count field inside std.enums functions is creating arrays).

Signed-off-by: Eric Joldasov <bratishkaerik@getgoogleoff.me>
2023-11-18 16:25:55 +02:00
Lucas Santos
d44f822821 Faster implementation of intToEnum. 2023-11-18 11:24:13 +02:00
Igor Anić
be4a77d9aa io_uring: fix ci failing test
Getting this error in ci:
run test std-arm-linux-none-generic-Debug: error: 'test.accept/connect/send_zc/recv' failed: /home/ci/actions-runner1/_work/zig/zig/lib/std/os/linux/io_uring.zig:60:23: 0x70b06b in init_params (test)
            .NOSYS => return error.SystemOutdated,
                      ^
/home/ci/actions-runner1/_work/zig/zig/lib/std/os/linux/io_uring.zig:27:16: 0x70b6b7 in init (test)
        return try IO_Uring.init_params(entries, &params);
               ^
/home/ci/actions-runner1/_work/zig/zig/lib/std/os/linux/io_uring.zig:3807:16: 0x72405b in test.accept/connect/send_zc/recv (test)
    var ring = try IO_Uring.init(16, 0);

https://github.com/ziglang/zig/actions/runs/6909813408/job/18801841015?pr=18025
2023-11-18 05:40:24 +01:00
Igor Anić
7eee0d3353 io_uring: try to pass windows/macos tests 2023-11-17 23:39:29 +01:00
Igor Anić
ccf5a6cc5c io_uring: make Linux version check runtime instead od comptime
Reverting previous change.
I'm building test bin and then running it in virtual machines with different
kernels. So Linux kernel checks has to be runtime instead of comptime.
2023-11-17 22:17:00 +01:00
Jakub Konka
91329ce944 std.macho: fix LoadCommandIterator to work with underaligned data 2023-11-17 20:49:32 +01:00
Igor Anić
a7001b86f1 io_uring: include review comments
Thanks @rootbeer for review.
This adds description to send_zc behavior.
Cleans up tests.
2023-11-17 19:58:23 +01:00
Igor Anić
257b8131ec io_uring: use kernel version test in send_zc test 2023-11-17 17:21:54 +01:00
Igor Anić
87dd8d5669 io_uring: simplify tests by checking kernel version
So far we relied on getting EINVAL in CQE for operations that kernel don't
support. The problem with that approach is that there are many other reasons
(like wrong params) to get EINVAL. The other problem is when we have an
operation that existed before and gets new behavior via different attributes,
like accept and accept_direct. Older kernels can fall back to non direct
operation although we set attributes for direct operation. Operation completes
successfully in both cases but with different results.

This commit introduces kernel version check at the start of the test. Making
body of the test free of checking for various kernel version differences.

Feature availability references:
  * https://manpages.debian.org/unstable/liburing-dev/io_uring_enter.2.en.html
  * https://kernel.dk/axboe-kr2022.pdf
  * 5acf7969bc/lib/std/os/linux.zig (L3727)
  * 5acf7969bc/lib/std/os/linux.zig (L3993)
2023-11-17 16:31:57 +01:00
Igor Anić
256384a2ec io_uring: add direct operations
Add operation on direct file descriptors. Also referred to as fixed or
registered files.

References:
https://kernel.dk/axboe-kr2022.pdf
https://lwn.net/Articles/863071/

Added functions:

IO_Uring
  accept_direct
  accept_multishot_direct
  openat_direct
  close_direct
  socket
  socket_direct
  socket_direct_alloc

Raw prepare operations:
  io_uring_prep_accept_direct
  io_uring_prep_multishot_accept_direct
  io_uring_prep_openat_direct
  io_uring_prep_close_direct
  io_uring_prep_socket
  io_uring_prep_socket_direct
  io_uring_prep_socket_direct_alloc

Tested on this kernels:

5.4.0-164-generic
2559 passed; 70 skipped; 0 failed.

5.8.0-63-generic
2573 passed; 56 skipped; 0 failed.

5.11.0-49-generic
2576 passed; 53 skipped; 0 failed.

5.13.0-52-generic
2576 passed; 53 skipped; 0 failed.

5.15.0-87-generic
2579 passed; 50 skipped; 0 failed.

5.19.0-46-geneic
2584 passed; 45 skipped; 0 failed.

6.2.0-35-generic.log
2585 passed; 44 skipped; 0 failed.

6.5.0-9-generic
2585 passed; 44 skipped; 0 failed.
2023-11-17 13:48:17 +01:00
David
941090d94f
Move duplicate field detection for struct init expressions into AstGen
Partially addresses #17916.
2023-11-16 14:38:16 +00:00
Jacob Young
0c6cb8d8c8 x86_64: actually track state during @divFloor of i128
Closes #17998
2023-11-14 22:50:05 -05:00
Jakub Konka
6fd1c64f23
Merge pull request #17978 from ziglang/elf-x86-tls
x86_64+elf: TLS support
2023-11-14 22:09:15 +01:00
Jakub Konka
ea3f1d27e0 std: disable failing test 2023-11-14 16:51:38 +01:00
xdBronch
5de0636577 expose to build system 2023-11-13 14:57:51 +02:00
Jacob Young
2eeb735822 Dwarf: improve x86_64 backend debug info
Closes #17811
2023-11-12 03:21:52 -05:00
dweiller
138a35df8f zstandard: fix division by zero when using RingBuffer
This change fixes some division-by-zero bugs introduced by the optimized
ring buffer read/write functions in d8c067966.

There are edge cases where decompression can use a length zero ring
buffer as the size of the ring buffer used is exactly the the window
size specified by a Zstandard frame, and this can be zero. Switching
away from loops to mem copies means that we need to ensure ring buffers
do not have length zero ring when attempting to read/write from them.
2023-11-10 15:18:16 -05:00
Igor Anić
087ee497d4 io_uring: add zero-copy send operation
`send_zc` tries to avoid making intermediate copies of data. Zerocopy execution
is not guaranteed and may fall back to copying.

The flags field of the first struct io_uring_cqe may likely contain
IORING_CQE_F_MORE , which means that there will be a second completion event /
notification for the request, with the user_data field set to the same value.
The user must not modify the data buffer until the notification is posted. The
first cqe follows the usual rules and so its res field will contain the number
of bytes sent or a negative error code. The notification's res field will be set
to zero and the flags field will contain IORING_CQE_F_NOTIF. The two step model
is needed because the kernel may hold on to buffers for a long time, e.g.
waiting for a TCP ACK, and having a separate cqe for request completions allows
userspace to push more data without extra delays. Note, notifications are only
responsible for controlling the lifetime of the buffers, and as such don't mean
anything about whether the data has atually been sent out or received by the
other end. Even errored requests may generate a notification, and the user must
check for IORING_CQE_F_MORE rather than relying on the result.

Available since kernel 6.0.

References:
https://man7.org/linux/man-pages/man3/io_uring_prep_send_zc.3.html
https://man7.org/linux/man-pages/man2/io_uring_enter.2.html
2023-11-10 16:44:18 +01:00
Jakub Konka
c550eb3e8a
Merge pull request #17933 from ziglang/elf-r-mode
elf: the dreaded `-r` mode
2023-11-10 07:57:52 +01:00
Jakub Konka
acd7cbf0b5 elf: init output COMDAT group sections 2023-11-09 17:41:14 +01:00
Bogdan Romanyuk
ee8da40769 replace deprecated std.math.absCast 2023-11-09 17:39:27 +02:00
Linus Groh
e72049bc61 std.math: Add isPositiveZero() and isNegativeZero() 2023-11-09 00:55:43 -07:00
Andrew Kelley
b2ed2c4d4f
Merge pull request #17888 from AdamGoertz/zig-reduce
zig-reduce: Add reductions for `if` and `while`
2023-11-09 00:52:38 -07:00
frmdstryr
f258a391da
Speed up ast.tokenLocation 2023-11-09 01:45:25 +00:00
Veikka Tuominen
10a28bc4c4
Merge pull request #17935 from ianic/fix_io_uring_test_kernel_5_4
Fix io_uring tests on kernel 5.4
2023-11-09 00:49:22 +02:00
Igor Anić
715e5f757f fix io_uring tests on kernel 5.4
Unsupported tests are now skipped on kernel 5.4:
```
uname -a
Linux d20 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:31:37 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
zig test lib/std/std.zig --zig-lib-dir lib --main-mod-path lib/std 2>&1 | cat
709/2616 test.nop... OK
710/2616 test.readv... OK
711/2616 test.writev/fsync/readv... OK
712/2616 test.write/read... SKIP
713/2616 test.splice/read... SKIP
714/2616 test.write_fixed/read_fixed... OK
715/2616 test.openat... SKIP
716/2616 test.close... SKIP
717/2616 test.accept/connect/send/recv... SKIP
718/2616 test.sendmsg/recvmsg... OK
719/2616 test.timeout (after a relative time)... SKIP
720/2616 test.timeout (after a number of completions)... OK
721/2616 test.timeout_remove... SKIP
722/2616 test.accept/connect/recv/link_timeout... SKIP
723/2616 test.fallocate... SKIP
724/2616 test.statx... SKIP
725/2616 test.accept/connect/recv/cancel... SKIP
726/2616 test.register_files_update... SKIP
727/2616 test.shutdown... SKIP
728/2616 test.renameat... SKIP
729/2616 test.unlinkat... SKIP
730/2616 test.mkdirat... SKIP
731/2616 test.symlinkat... SKIP
732/2616 test.linkat... SKIP
733/2616 test.provide_buffers: read... SKIP
734/2616 test.remove_buffers... SKIP
735/2616 test.provide_buffers: accept/connect/send/recv... SKIP
736/2616 test.accept multishot... SKIP
```
2023-11-08 18:52:03 +01:00
Igor Anić
7a1dbbd4f9 fix io_uring timeout_remove test on kernel 5.4
There is no grantee that `copy_cqes` will return exactly wait_nr number of cqes.
If there are ready cqes it can return > 0 but < wait_nr number of cqes.
2023-11-08 18:25:29 +01:00
Frank Denis
a70d8d29d5
Curve25519.fromEdwards25519(): don't assume normalized coordinates (#17920)
The low-level `Curve25519.fromEdwards25519()` function assumed
that the X/Y coordinates were not scaled (Z=1).

But this is not guaranteed to be the case.

In most real-world applications, the coordinates are freshly decoded,
either directly or via the `X25519.fromEd25519()` function, so this
is not an issue.

However, since we offer the ability to do that conversion after
arbitrary computations, the assertion was not correct.
2023-11-08 11:56:56 +01:00
tison
ee47643b6e
std.math.big: fix sqrt with bits > limb_bits
Signed-off-by: tison <wander4096@gmail.com>
2023-11-08 08:39:08 +02:00
David Rubin
03abac3824 fix sqrt(0) 2023-11-07 14:37:43 +02:00
Jakub Konka
bf0387b6bb
Merge pull request #17873 from ziglang/elf-archive
elf: implement archiving input object files
2023-11-07 03:22:14 +01:00
Jakub Konka
261db02018 CheckObject: support parsing and dumping archive symtab for ELF 2023-11-06 21:18:26 +01:00