1322 Commits

Author SHA1 Message Date
Stephen Gregoratto
3200fae9c5 Linux: Add syscall bindings, enhance documentation.
- Add syscall bindings/structures for the `futex2` family.
  The documentation is taken from the syscall definitions.
- Add documnentation for the `cachestat` bindings and structures.
  Taken from work I did in Cosmopolitian libc.
- Add binding for `map_shadow_stack`.
  No documentation for this one, since the kernel devs didn't bother to
  do it ¯\_(ツ)_/¯.
2024-01-15 20:05:03 +11:00
Stephen Gregoratto
cca021c211 Linux: Update syscalls for the 6.7 release
This release adds the rest of the futex2[1] syscalls, along with
shadow stack[2] support for more architectures.

[1]: https://lwn.net/Articles/940944/
[2]: https://lwn.net/Articles/926649/
2024-01-15 20:05:03 +11:00
Andrew Kelley
a5934edd48 std.os.linux.io_uring: fix compilation errors
Commit f0c42fbba15a31113dc989ab65c054047423afa3 was created before the
additional compile error for var used instead of const.
2024-01-14 19:20:41 -07:00
Andrew Kelley
f0c42fbba1
Merge pull request #18025 from ianic/io_uring_send_zc
io_uring: add send_zc and *_direct operations
2024-01-14 18:10:51 -08:00
Stephen Gregoratto
01cc904fc7 Rework fchmodat tests
Based on the Linux kernel fchmodat2 test
2024-01-13 23:52:01 -07:00
Stephen Gregoratto
cf6751ae55 Add fchmodat2 bits to os/linux.zig 2024-01-13 23:52:01 -07:00
Stephen Gregoratto
26db31f6f6 Add fchmodat2 to the Linux syscall list
This syscall was added to simplify the the libc implementations of
fchmodat, as the original syscall does not take a `flags` argument.

Another syscall, `map_shadow_stack`, was also added for x86_64.
2024-01-13 23:52:01 -07:00
Matthew Wozniak
aaf1e0b25b
add ability to open dlls with platform-specific flags (#18370) 2024-01-09 20:11:22 -05:00
David Rubin
2f8e4347b1
Optimized std.mem.eql with SIMD (#18389)
* optimized memeql
* add `sched_setaffinity` to `std.os.linux`

Co-authored-by: Protty <45520026+kprotty@users.noreply.github.com>
Co-authored-by: Ryan Liptak <squeek502@hotmail.com>
2024-01-09 02:50:22 -05:00
Karl Böhlmark
9a56228c2b
doc: Fix typos in bpf docs 2024-01-03 12:34:10 +02:00
Andrew Kelley
96b4eeab55 std: disable failing test on macos
See tracking issue https://github.com/ziglang/zig/issues/18395
2024-01-01 19:49:08 -07:00
Veikka Tuominen
94c63f31f2
Merge pull request #18328 from ExpidusOS/feat/uefi-time
std: add support for getting the time in UEFI
2023-12-26 01:03:10 +02:00
Carl Åstholm
60982ea5bd Correct CreateProcessW parameter types 2023-12-22 12:23:27 +02:00
Tristan Ross
97fd100471
os.uefi: add ns to epoch 2023-12-20 20:05:16 -08:00
Tristan Ross
a86cd91389
os.uefi: add time to epoch conversion 2023-12-19 23:42:47 -08:00
Ryan Liptak
11a398af3e File.stat: Support detection of Kind.sym_link on Windows
Requires an extra NtQueryInformationFile call when FILE_ATTRIBUTE_REPARSE_POINT is set to determine if it's actually a symlink or some other kind of reparse point (https://learn.microsoft.com/en-us/windows/win32/fileio/reparse-point-tags). This is something that `File.Metadata.kind` was already doing, so the same technique is used in `stat`.

Also, replace the std.os.windows.DeviceIoControl call in `metadata` with NtQueryInformationFile (NtQueryInformationFile is what gets called during kernel32.GetFileInformationByHandleEx with FileAttributeTagInfo, verified using NtTrace).
2023-12-19 23:36:23 -08:00
loris
37458cc886 allow sqe entries up to 32768 2023-12-12 06:49:19 +02:00
xdBronch
7a46c20a79 switch to mem.span
Co-authored-by: erikarvstedt <36110478+erikarvstedt@users.noreply.github.com>
2023-12-08 15:58:08 +02:00
xdBronch
e9bd10cfda add getName helper to inotify_event 2023-12-08 15:58:08 +02:00
David Rubin
1e42a3de89
Remove all usages of std.mem.copy and remove std.mem.set (#18143) 2023-11-29 16:03:02 -05:00
Ryan Liptak
0bc4ee1792 Remove std.os.windows.ole32/shell32 2023-11-23 03:06:47 -08:00
Ryan Liptak
2c326d4390 std.os.windows: Add HKEY_CLASSES_ROOT 2023-11-23 01:11:53 -08:00
Andrew Kelley
6fc20b3b09 std.os.windows: remove volatile from inline asm
This inline assembly has no side effects.
2023-11-23 00:52:35 -05:00
Mikko Kaihlavirta
ea4a07701e add missing timeval struct 2023-11-22 01:15:11 +02:00
expikr
40bd93e2a2 replace qpf and qpc
Update windows.zig

Update windows.zig

Update windows.zig

Update windows.zig

Update windows.zig

Update windows.zig

Update windows.zig

Update ntdll.zig

Update windows.zig

Update ntdll.zig

Update kernel32.zig
2023-11-21 14:10:25 +02:00
Tw
8ca4a5240e linux/bpf: update helper definitions
Signed-off-by: Tw <tw19881113@gmail.com>
2023-11-19 19:01:18 +02:00
mlugg
51595d6b75
lib: correct unnecessary uses of 'var' 2023-11-19 09:55:07 +00:00
Igor Anić
be4a77d9aa io_uring: fix ci failing test
Getting this error in ci:
run test std-arm-linux-none-generic-Debug: error: 'test.accept/connect/send_zc/recv' failed: /home/ci/actions-runner1/_work/zig/zig/lib/std/os/linux/io_uring.zig:60:23: 0x70b06b in init_params (test)
            .NOSYS => return error.SystemOutdated,
                      ^
/home/ci/actions-runner1/_work/zig/zig/lib/std/os/linux/io_uring.zig:27:16: 0x70b6b7 in init (test)
        return try IO_Uring.init_params(entries, &params);
               ^
/home/ci/actions-runner1/_work/zig/zig/lib/std/os/linux/io_uring.zig:3807:16: 0x72405b in test.accept/connect/send_zc/recv (test)
    var ring = try IO_Uring.init(16, 0);

https://github.com/ziglang/zig/actions/runs/6909813408/job/18801841015?pr=18025
2023-11-18 05:40:24 +01:00
Igor Anić
7eee0d3353 io_uring: try to pass windows/macos tests 2023-11-17 23:39:29 +01:00
Igor Anić
ccf5a6cc5c io_uring: make Linux version check runtime instead od comptime
Reverting previous change.
I'm building test bin and then running it in virtual machines with different
kernels. So Linux kernel checks has to be runtime instead of comptime.
2023-11-17 22:17:00 +01:00
Igor Anić
a7001b86f1 io_uring: include review comments
Thanks @rootbeer for review.
This adds description to send_zc behavior.
Cleans up tests.
2023-11-17 19:58:23 +01:00
Igor Anić
257b8131ec io_uring: use kernel version test in send_zc test 2023-11-17 17:21:54 +01:00
Igor Anić
87dd8d5669 io_uring: simplify tests by checking kernel version
So far we relied on getting EINVAL in CQE for operations that kernel don't
support. The problem with that approach is that there are many other reasons
(like wrong params) to get EINVAL. The other problem is when we have an
operation that existed before and gets new behavior via different attributes,
like accept and accept_direct. Older kernels can fall back to non direct
operation although we set attributes for direct operation. Operation completes
successfully in both cases but with different results.

This commit introduces kernel version check at the start of the test. Making
body of the test free of checking for various kernel version differences.

Feature availability references:
  * https://manpages.debian.org/unstable/liburing-dev/io_uring_enter.2.en.html
  * https://kernel.dk/axboe-kr2022.pdf
  * 5acf7969bc/lib/std/os/linux.zig (L3727)
  * 5acf7969bc/lib/std/os/linux.zig (L3993)
2023-11-17 16:31:57 +01:00
Igor Anić
256384a2ec io_uring: add direct operations
Add operation on direct file descriptors. Also referred to as fixed or
registered files.

References:
https://kernel.dk/axboe-kr2022.pdf
https://lwn.net/Articles/863071/

Added functions:

IO_Uring
  accept_direct
  accept_multishot_direct
  openat_direct
  close_direct
  socket
  socket_direct
  socket_direct_alloc

Raw prepare operations:
  io_uring_prep_accept_direct
  io_uring_prep_multishot_accept_direct
  io_uring_prep_openat_direct
  io_uring_prep_close_direct
  io_uring_prep_socket
  io_uring_prep_socket_direct
  io_uring_prep_socket_direct_alloc

Tested on this kernels:

5.4.0-164-generic
2559 passed; 70 skipped; 0 failed.

5.8.0-63-generic
2573 passed; 56 skipped; 0 failed.

5.11.0-49-generic
2576 passed; 53 skipped; 0 failed.

5.13.0-52-generic
2576 passed; 53 skipped; 0 failed.

5.15.0-87-generic
2579 passed; 50 skipped; 0 failed.

5.19.0-46-geneic
2584 passed; 45 skipped; 0 failed.

6.2.0-35-generic.log
2585 passed; 44 skipped; 0 failed.

6.5.0-9-generic
2585 passed; 44 skipped; 0 failed.
2023-11-17 13:48:17 +01:00
David
941090d94f
Move duplicate field detection for struct init expressions into AstGen
Partially addresses #17916.
2023-11-16 14:38:16 +00:00
Igor Anić
087ee497d4 io_uring: add zero-copy send operation
`send_zc` tries to avoid making intermediate copies of data. Zerocopy execution
is not guaranteed and may fall back to copying.

The flags field of the first struct io_uring_cqe may likely contain
IORING_CQE_F_MORE , which means that there will be a second completion event /
notification for the request, with the user_data field set to the same value.
The user must not modify the data buffer until the notification is posted. The
first cqe follows the usual rules and so its res field will contain the number
of bytes sent or a negative error code. The notification's res field will be set
to zero and the flags field will contain IORING_CQE_F_NOTIF. The two step model
is needed because the kernel may hold on to buffers for a long time, e.g.
waiting for a TCP ACK, and having a separate cqe for request completions allows
userspace to push more data without extra delays. Note, notifications are only
responsible for controlling the lifetime of the buffers, and as such don't mean
anything about whether the data has atually been sent out or received by the
other end. Even errored requests may generate a notification, and the user must
check for IORING_CQE_F_MORE rather than relying on the result.

Available since kernel 6.0.

References:
https://man7.org/linux/man-pages/man3/io_uring_prep_send_zc.3.html
https://man7.org/linux/man-pages/man2/io_uring_enter.2.html
2023-11-10 16:44:18 +01:00
Bogdan Romanyuk
ee8da40769 replace deprecated std.math.absCast 2023-11-09 17:39:27 +02:00
Igor Anić
715e5f757f fix io_uring tests on kernel 5.4
Unsupported tests are now skipped on kernel 5.4:
```
uname -a
Linux d20 5.4.0-166-generic #183-Ubuntu SMP Mon Oct 2 11:31:37 UTC 2023 aarch64 aarch64 aarch64 GNU/Linux
zig test lib/std/std.zig --zig-lib-dir lib --main-mod-path lib/std 2>&1 | cat
709/2616 test.nop... OK
710/2616 test.readv... OK
711/2616 test.writev/fsync/readv... OK
712/2616 test.write/read... SKIP
713/2616 test.splice/read... SKIP
714/2616 test.write_fixed/read_fixed... OK
715/2616 test.openat... SKIP
716/2616 test.close... SKIP
717/2616 test.accept/connect/send/recv... SKIP
718/2616 test.sendmsg/recvmsg... OK
719/2616 test.timeout (after a relative time)... SKIP
720/2616 test.timeout (after a number of completions)... OK
721/2616 test.timeout_remove... SKIP
722/2616 test.accept/connect/recv/link_timeout... SKIP
723/2616 test.fallocate... SKIP
724/2616 test.statx... SKIP
725/2616 test.accept/connect/recv/cancel... SKIP
726/2616 test.register_files_update... SKIP
727/2616 test.shutdown... SKIP
728/2616 test.renameat... SKIP
729/2616 test.unlinkat... SKIP
730/2616 test.mkdirat... SKIP
731/2616 test.symlinkat... SKIP
732/2616 test.linkat... SKIP
733/2616 test.provide_buffers: read... SKIP
734/2616 test.remove_buffers... SKIP
735/2616 test.provide_buffers: accept/connect/send/recv... SKIP
736/2616 test.accept multishot... SKIP
```
2023-11-08 18:52:03 +01:00
Igor Anić
7a1dbbd4f9 fix io_uring timeout_remove test on kernel 5.4
There is no grantee that `copy_cqes` will return exactly wait_nr number of cqes.
If there are ready cqes it can return > 0 but < wait_nr number of cqes.
2023-11-08 18:25:29 +01:00
Igor Anić
bec36aa7c0 io_uring: add multishot accept operation
Server networking application typically accept multiple connections. Multishot
accept simplifies handling these situations. Applications submits once and
receives CQE whenever a new connection request comes in.

Multishot is active until it is canceled or experience error. While active, and
further notification are expected CQE completion will have IORING_CQE_F_MORE set
in the flags. If this flag isn't set, the application must re-arm this request
by submitting a new one.

Reference: [io_uring and networking in 2023](https://github.com/axboe/liburing/wiki/io_uring-and-networking-in-2023#multi-shot)
2023-11-06 00:12:39 +02:00
Jacob Young
509be7cf1f x86_64: fix std test failures 2023-11-03 23:18:21 -04:00
Ryan Liptak
ae6df9e967 start.zig: Replace kernel32 usage with ntdll
Co-authored-by: e4m2 <git@e4m2.com>
2023-11-03 19:25:46 +02:00
XXIV
c2cf47ab28
std.os.windows: remove unnecessary @ptrCast (#17816) 2023-11-01 18:45:51 -04:00
Kai Jellinghaus
7ee7c90274 Fix regressing #17290 from #17734 2023-11-01 18:30:56 +02:00
Andrew Kelley
3fc6fc6812 std.builtin.Endian: make the tags lower case
Let's take this breaking change opportunity to fix the style of this
enum.
2023-10-31 21:37:35 -04:00
Jacob Young
d890e81761 mem: fix ub in writeInt
Use inline to vastly simplify the exposed API.  This allows a
comptime-known endian parameter to be propogated, making extra functions
for a specific endianness completely unnecessary.
2023-10-31 21:37:35 -04:00
Kamil T
37295696ec std.os.windows additions and fixes 2023-10-29 11:01:54 +02:00
Andrew Kelley
e7ead9c5ea std: delete psapi, gdi32, and winmm
The only remaining Windows APIs now are the ones that the standard
library itself depends on for its cross-platform abstractions.

closes #4426
2023-10-29 00:58:00 -04:00
Andrew Kelley
d817a3c517 delete std.os.windows.user32
This is progress towards #4426
Closes #17417
2023-10-28 13:16:47 -04:00
Jacob Young
3b0dce8ebd
Merge pull request #17716 from jacobly0/x86_64
x86_64: pass more tests
2023-10-27 08:29:24 -04:00