336 Commits

Author SHA1 Message Date
Alex Rønne Petersen
2d4fc1bb7a
std.os.linux.tls: Fix setThreadPointer() for armeb and thumbeb. 2024-07-23 21:17:29 +02:00
Alex Rønne Petersen
0473457b68
std.os.linux.tls: Set some constants correctly for powerpcle. 2024-07-23 21:14:19 +02:00
Alex Rønne Petersen
c631110f5c
std.os.linux.tls: Set tls_tcb_size correctly for thumbeb. 2024-07-23 21:14:19 +02:00
Alex Rønne Petersen
d694ddd279
std.os.linux.tls: Set tls_variant correctly for thumbeb. 2024-07-23 21:14:19 +02:00
Andrew Kelley
33c7984183 add std.testing.random_seed
closes #17609
2024-07-23 11:43:12 -07:00
Andrew Kelley
bde8c4a46a update start code to use @disableInstrumentation
This prevents it from trying to access thread local storage before it
has set up thread local storage, particularly when code coverage
instrumentation is enabled.
2024-07-22 14:26:17 -07:00
Andrew Kelley
01337e2093 fix regression of flock being called on wasi targets
* common symbols are now public from std.c even if they live in
  std.posix
* LOCK is now one of the common symbols since it is the same on 100% of
  operating systems.
* flock is now void value on wasi and windows
* std.fs.Dir now uses flock being void as feature detection, avoiding
  trying to call it on wasi and windows
2024-07-19 11:35:22 -07:00
Andrew Kelley
e8c4e79499 std.c reorganization
It is now composed of these main sections:
* Declarations that are shared among all operating systems.
* Declarations that have the same name, but different type signatures
  depending on the operating system. Often multiple operating systems
  share the same type signatures however.
* Declarations that are specific to a single operating system.
  - These are imported one per line so you can see where they come from,
    protected by a comptime block to prevent accessing the wrong one.

Closes #19352 by changing the convention to making types `void` and
functions `{}`, so that it becomes possible to update `@hasDecl` sites
to use `@TypeOf(f) != void` or `T != void`. Happily, this ended up
removing some duplicate logic and update some bitrotted feature
detection checks.

A handful of types have been modified to gain namespacing and type
safety. This is a breaking change.

Oh, and the last usage of `usingnamespace` site is eliminated.
2024-07-19 00:30:32 -07:00
Jora Troosh
13070448f5
std: fix typos (#20560) 2024-07-09 14:25:42 -07:00
Tw
49f2cca872 bpf: sync map/prog/attach type with latest linux kernel
Note that the original `cgroup_storage` MapType has been deprecated,
so renamed to `cgroup_storage_deprecated`.

Signed-off-by: Tw <tw19881113@gmail.com>
2024-07-09 17:16:51 -04:00
Igor Anić
b8890f8ee1 io_uring: don't assume completions order (2)
In my first [try](https://github.com/ziglang/zig/pull/20224) to fix
20212 I didn't reproduce bug on required kernel (6.9.2) and wrongly
concluded that first two completions have different order on newer
kernel.
On my current kernel (6.5.0) order of completions is: send1, recv,
send2. On 6.9.2 order is send1, send2, recv. This fix allows second two
completions to arrive in any order.
Tested on both kernels.

Fixes: #20212
2024-06-20 18:03:28 -04:00
Ryan Liptak
76fb2b685b std: Convert deprecated aliases to compile errors and fix usages
Deprecated aliases that are now compile errors:

- `std.fs.MAX_PATH_BYTES` (renamed to `std.fs.max_path_bytes`)
- `std.mem.tokenize` (split into `tokenizeAny`, `tokenizeSequence`, `tokenizeScalar`)
- `std.mem.split` (split into `splitSequence`, `splitAny`, `splitScalar`)
- `std.mem.splitBackwards` (split into `splitBackwardsSequence`, `splitBackwardsAny`, `splitBackwardsScalar`)
- `std.unicode`
  + `utf16leToUtf8Alloc`, `utf16leToUtf8AllocZ`, `utf16leToUtf8`, `fmtUtf16le` (all renamed to have capitalized `Le`)
  + `utf8ToUtf16LeWithNull` (renamed to `utf8ToUtf16LeAllocZ`)
- `std.zig.CrossTarget` (moved to `std.Target.Query`)

Deprecated `lib/std/std.zig` decls were deleted instead of made a `@compileError` because the `refAllDecls` in the test block would trigger the `@compileError`. The deleted top-level `std` namespaces are:

- `std.rand` (renamed to `std.Random`)
- `std.TailQueue` (renamed to `std.DoublyLinkedList`)
- `std.ChildProcess` (renamed/moved to `std.process.Child`)

This is not exhaustive. Deprecated aliases that I didn't touch:
  + `std.io.*`
  + `std.Build.*`
  + `std.builtin.Mode`
  + `std.zig.c_translation.CIntLiteralRadix`
  + anything in `src/`
2024-06-13 10:18:59 -04:00
Shane Peelar
ec337051a9 Fix slight deviation from spec in handling Elf*_Rela relative relocations
`Elf*_Rela` relocations store their argument in `r_addend`, including for `R_*_RELATIVE` relocations.  Unlike `Elf*_Rel` relocations, they are not applied as a delta to the destination virtual address.  Instead, they are computed from `base_address + r_addend` directly.
2024-06-08 16:02:47 -04:00
Igor Anić
45b62c4529 io_uring: don't assume completions order
We are posting two submission (zero copy send and receive) and then
reading two completions. There is no guarantee that those completions
will be in the order of submissions.
This test was expecting fist send completion then receive.
Fix is allowing them to come other way too.
2024-06-08 15:35:41 -04:00
Andrew Kelley
63754916c5 disable failing IoUring test
tracked by #20212
2024-06-06 11:13:22 -07:00
YANG Xudong
87150468fc generate loongarch64 Linux syscalls 2024-06-05 18:54:14 -04:00
Karl Bohlmark
6eb17a034a fix integer overflow in IoUring buffer ring size calculation 2024-05-09 16:00:01 -07:00
Nameless
aecd9cc6d1 std.posix.iovec: use .base and .len instead of .iov_base and .iov_len 2024-04-28 00:20:30 -07:00
Jared Baur
c352845e88 Fix usage of unexpectedErrno
`unexpectedErrno` comes from `std.posix`, not `std.os`.
2024-04-20 19:37:41 -07:00
Igor Anić
254c05a9e1 io_uring: simplify copy_cqe logic
First copy as much as we can in this cycle. If there is more needed
wrap and start from the buffer 0 position.
2024-03-31 23:57:16 +02:00
Igor Anić
704660c81a io_uring: fix copy_cqes logic 2024-03-31 23:52:46 +02:00
Igor Anić
631c28c9e3 io_uring: add failing test
For issue: #19451
2024-03-31 23:51:40 +02:00
Andrew Kelley
8c94950c24 fix compilation failures found by CI 2024-03-19 16:18:18 -07:00
Andrew Kelley
cd62005f19 extract std.posix from std.os
closes #5019
2024-03-19 11:45:09 -07:00
Igor Anić
3ea1276eeb
io_uring: ring mapped buffers (#17806)
* io_uring: ring mapped buffers

Ring mapped buffers are newer implementation of ring provided buffers, supported
since kernel 5.19. Best described in Jens Axboe [post](https://github.com/axboe/liburing/wiki/io_uring-and-networking-in-2023#provided-buffers)

This commit implements low level io_uring_*_buf_ring_* functions as mostly
direct translation from liburing. It also adds BufferGroup abstraction over those
low level functions.

* io_uring: add multishot recv to BufferGroup

Once we have ring mapped provided buffers functionality it is possible to use
multishot recv operation. Multishot receive is submitted once, and completions
are posted whenever data arrives on the socket. Received data are placed in a
new buffer from buffer group.

Reference: [io_uring and networking in 2023](https://github.com/axboe/liburing/wiki/io_uring-and-networking-in-2023#multi-shot)

Getting NOENT for cancel completion result, meaning:
  -ENOENT
    The request identified by user_data could not be located.
    This could be because it completed before the cancelation
    request was issued, or if an invalid identifier is used.

https://man7.org/linux/man-pages/man3/io_uring_prep_cancel.3.html
https://github.com/ziglang/zig/actions/runs/6801394000/job/18492139893?pr=17806

Result in cancel/recv cqes are different depending on the kernel.
on older kernel (tested with v6.0.16, v6.1.57, v6.2.12, v6.4.16)
  cqe_cancel.err() == .NOENT
  cqe_crecv.err() == .NOBUFS
on kernel (tested with v6.5.0, v6.5.7)
  cqe_cancel.err() == .SUCCESS
  cqe_crecv.err() == .CANCELED
2024-03-15 22:34:31 -04:00
Tristan Ross
6067d39522
std.builtin: make atomic order fields lowercase 2024-03-11 07:09:10 -07:00
mlugg
265f42d472
std.os.linux: rework io_uring support
* `linux.IO_Uring` -> `linux.IoUring` to align with naming conventions.
* All functions `io_uring_prep_foo` are now methods `prep_foo` on `io_uring_sqe`, which is in a file of its own.
* `SubmissionQueue` and `CompletionQueue` are namespaced under `IoUring`.

This is a breaking change.

The new file and namespace layouts are more idiomatic, and allow us to
eliminate one more usage of `usingnamespace` from the standard library.
2 remain.
2024-03-08 08:02:45 +00:00
Andrew Kelley
6129ecd4fe std.net, std.http: simplify 2024-02-23 02:37:11 -07:00
CPestka
0c725a354a Replaced loop with memcpys 2024-02-12 12:58:33 -08:00
Andrew Kelley
fad5e7a997
Merge pull request #18898 from psnszsn/iouring_waitid
io_uring: add waitid operation
2024-02-12 12:20:12 -08:00
Andrew Kelley
7680c5330c some API work on std.c, std.os, std.os.wasi
* std.c: consolidate some definitions, making them share code. For
  example, freebsd, dragonfly, and openbsd can all share the same
  `pthread_mutex_t` definition.
* add type safety to std.c.O
  - this caught a bug where mode flags were incorrectly passed as the
    open flags.
* 3 fewer uses of usingnamespace keyword
* as per convention, remove purposeless field prefixes from struct field
  names even if they have those prefixes in the corresponding C code.
* fix incorrect wasi libc Stat definition
* remove C definitions from incorrectly being in std.os.wasi
* make std.os.wasi definitions type safe
* go through wasi native APIs even when linking libc because the libc
  APIs are problematic and wasteful
* don't expose WASI definitions in std.posix
* remove std.os.wasi.rights_t.ALL: this is a footgun. should it be all
  future rights too? or only all current rights known? both are
  the wrong answer.
2024-02-11 13:38:55 -07:00
Vlad Pănăzan
d2789908ed io_uring: add waitid operation
This is the equivalent of a waitid(2) syscall and can be used to
be notified about child process state changes.

Available since kernel 6.7
2024-02-11 15:47:03 +01:00
Andrew Kelley
9f3165540e std.os.linux.MAP: use a packed struct
Introduces type safety to this constant. Eliminates one use of
`usingnamespace`.
2024-02-06 21:12:11 -07:00
Constantin Pestka
e460ba66a5
std: add docs about return value of submit() (#18721)
being misleading if used alongside with IORING_SETUP_SQPOLL
2024-01-29 21:26:52 -05:00
Stephen Gregoratto
cca021c211 Linux: Update syscalls for the 6.7 release
This release adds the rest of the futex2[1] syscalls, along with
shadow stack[2] support for more architectures.

[1]: https://lwn.net/Articles/940944/
[2]: https://lwn.net/Articles/926649/
2024-01-15 20:05:03 +11:00
Andrew Kelley
a5934edd48 std.os.linux.io_uring: fix compilation errors
Commit f0c42fbba15a31113dc989ab65c054047423afa3 was created before the
additional compile error for var used instead of const.
2024-01-14 19:20:41 -07:00
Andrew Kelley
f0c42fbba1
Merge pull request #18025 from ianic/io_uring_send_zc
io_uring: add send_zc and *_direct operations
2024-01-14 18:10:51 -08:00
Stephen Gregoratto
26db31f6f6 Add fchmodat2 to the Linux syscall list
This syscall was added to simplify the the libc implementations of
fchmodat, as the original syscall does not take a `flags` argument.

Another syscall, `map_shadow_stack`, was also added for x86_64.
2024-01-13 23:52:01 -07:00
Karl Böhlmark
9a56228c2b
doc: Fix typos in bpf docs 2024-01-03 12:34:10 +02:00
loris
37458cc886 allow sqe entries up to 32768 2023-12-12 06:49:19 +02:00
Tw
8ca4a5240e linux/bpf: update helper definitions
Signed-off-by: Tw <tw19881113@gmail.com>
2023-11-19 19:01:18 +02:00
mlugg
51595d6b75
lib: correct unnecessary uses of 'var' 2023-11-19 09:55:07 +00:00
Igor Anić
be4a77d9aa io_uring: fix ci failing test
Getting this error in ci:
run test std-arm-linux-none-generic-Debug: error: 'test.accept/connect/send_zc/recv' failed: /home/ci/actions-runner1/_work/zig/zig/lib/std/os/linux/io_uring.zig:60:23: 0x70b06b in init_params (test)
            .NOSYS => return error.SystemOutdated,
                      ^
/home/ci/actions-runner1/_work/zig/zig/lib/std/os/linux/io_uring.zig:27:16: 0x70b6b7 in init (test)
        return try IO_Uring.init_params(entries, &params);
               ^
/home/ci/actions-runner1/_work/zig/zig/lib/std/os/linux/io_uring.zig:3807:16: 0x72405b in test.accept/connect/send_zc/recv (test)
    var ring = try IO_Uring.init(16, 0);

https://github.com/ziglang/zig/actions/runs/6909813408/job/18801841015?pr=18025
2023-11-18 05:40:24 +01:00
Igor Anić
7eee0d3353 io_uring: try to pass windows/macos tests 2023-11-17 23:39:29 +01:00
Igor Anić
ccf5a6cc5c io_uring: make Linux version check runtime instead od comptime
Reverting previous change.
I'm building test bin and then running it in virtual machines with different
kernels. So Linux kernel checks has to be runtime instead of comptime.
2023-11-17 22:17:00 +01:00
Igor Anić
a7001b86f1 io_uring: include review comments
Thanks @rootbeer for review.
This adds description to send_zc behavior.
Cleans up tests.
2023-11-17 19:58:23 +01:00
Igor Anić
257b8131ec io_uring: use kernel version test in send_zc test 2023-11-17 17:21:54 +01:00
Igor Anić
87dd8d5669 io_uring: simplify tests by checking kernel version
So far we relied on getting EINVAL in CQE for operations that kernel don't
support. The problem with that approach is that there are many other reasons
(like wrong params) to get EINVAL. The other problem is when we have an
operation that existed before and gets new behavior via different attributes,
like accept and accept_direct. Older kernels can fall back to non direct
operation although we set attributes for direct operation. Operation completes
successfully in both cases but with different results.

This commit introduces kernel version check at the start of the test. Making
body of the test free of checking for various kernel version differences.

Feature availability references:
  * https://manpages.debian.org/unstable/liburing-dev/io_uring_enter.2.en.html
  * https://kernel.dk/axboe-kr2022.pdf
  * 5acf7969bc/lib/std/os/linux.zig (L3727)
  * 5acf7969bc/lib/std/os/linux.zig (L3993)
2023-11-17 16:31:57 +01:00
Igor Anić
256384a2ec io_uring: add direct operations
Add operation on direct file descriptors. Also referred to as fixed or
registered files.

References:
https://kernel.dk/axboe-kr2022.pdf
https://lwn.net/Articles/863071/

Added functions:

IO_Uring
  accept_direct
  accept_multishot_direct
  openat_direct
  close_direct
  socket
  socket_direct
  socket_direct_alloc

Raw prepare operations:
  io_uring_prep_accept_direct
  io_uring_prep_multishot_accept_direct
  io_uring_prep_openat_direct
  io_uring_prep_close_direct
  io_uring_prep_socket
  io_uring_prep_socket_direct
  io_uring_prep_socket_direct_alloc

Tested on this kernels:

5.4.0-164-generic
2559 passed; 70 skipped; 0 failed.

5.8.0-63-generic
2573 passed; 56 skipped; 0 failed.

5.11.0-49-generic
2576 passed; 53 skipped; 0 failed.

5.13.0-52-generic
2576 passed; 53 skipped; 0 failed.

5.15.0-87-generic
2579 passed; 50 skipped; 0 failed.

5.19.0-46-geneic
2584 passed; 45 skipped; 0 failed.

6.2.0-35-generic.log
2585 passed; 44 skipped; 0 failed.

6.5.0-9-generic
2585 passed; 44 skipped; 0 failed.
2023-11-17 13:48:17 +01:00
David
941090d94f
Move duplicate field detection for struct init expressions into AstGen
Partially addresses #17916.
2023-11-16 14:38:16 +00:00