436 Commits

Author SHA1 Message Date
Alex Rønne Petersen
ab69482a5d std.fs: Disable file operations on directories test on WASI.
https://github.com/ziglang/zig/issues/20747
2024-08-23 22:43:03 -07:00
Ian Johnson
0a70455095 Fix handling of empty XDG environment variables
Closes #21132

According to the XDG Base Directory specification
(https://specifications.freedesktop.org/basedir-spec/latest/#variables),
empty values for these environment variables should be treated the same
as if they are unset. Specifically, for the instances changed in this
commit,

> $XDG_DATA_HOME defines the base directory relative to which
> user-specific data files should be stored. If $XDG_DATA_HOME is either
> not set **or empty**, a default equal to $HOME/.local/share should be
> used.

and

> $XDG_CACHE_HOME defines the base directory relative to which
> user-specific non-essential data files should be stored. If
> $XDG_CACHE_HOME is either not set **or empty**, a default equal to
> $HOME/.cache should be used.

(emphasis mine)

In addition to the case mentioned in the linked issue, all other uses of
XDG environment variables were corrected.
2024-08-19 23:30:14 -07:00
Andrew Kelley
12d0c9a2fc add std.fs.Dir.Handle
mirrors std.fs.File.Handle
2024-08-01 13:47:23 -07:00
Alex Rønne Petersen
13945548fc
std.fs: Rework to always use statx() instead of fstat()/fstatat() on Linux.
statx() is strictly superior to stat() and friends. We can do this because the
standard library declares Linux 4.19 to be the minimum version supported in
std.Target. This is also necessary on riscv32 where there is only statx().

While here, I improved std.fs.File.metadata() to gather as much information as
possible when calling statx() since that is the expectation from this particular
API.
2024-07-29 09:50:41 +02:00
Alex Rønne Petersen
4e5068c35c
std: Stop supporting Linux/glibc versions older than declared in std.Target. 2024-07-29 09:50:41 +02:00
Andrew Kelley
08e83fee57
Merge pull request #20297 from sno2/wtf8-conversion-buffer-overflows
std: fix buffer overflows from improper WTF encoding
2024-07-28 20:24:31 -07:00
Carter Snook
219acaa1d6 std.fs.Dir: Refactor atomicSymLink from std.fs
Deprecates std.fs.atomicSymLink and removes the allocator requirement
from the new std.fs.Dir.atomicSymLink. Replaces the two usages of this
within std.

I did not include the TODOs from the original code that were based
off of `switch (err) { ..., else => return err }` not having correct
inference that cases handled in `...` are impossible in the error
union return type because these are not specified in many places but
I can add them back if wanted.

Thank you @squeek502 for help with fixing buffer overflows!
2024-07-28 20:08:52 -07:00
Andrew Kelley
01337e2093 fix regression of flock being called on wasi targets
* common symbols are now public from std.c even if they live in
  std.posix
* LOCK is now one of the common symbols since it is the same on 100% of
  operating systems.
* flock is now void value on wasi and windows
* std.fs.Dir now uses flock being void as feature detection, avoiding
  trying to call it on wasi and windows
2024-07-19 11:35:22 -07:00
Andrew Kelley
e8c4e79499 std.c reorganization
It is now composed of these main sections:
* Declarations that are shared among all operating systems.
* Declarations that have the same name, but different type signatures
  depending on the operating system. Often multiple operating systems
  share the same type signatures however.
* Declarations that are specific to a single operating system.
  - These are imported one per line so you can see where they come from,
    protected by a comptime block to prevent accessing the wrong one.

Closes #19352 by changing the convention to making types `void` and
functions `{}`, so that it becomes possible to update `@hasDecl` sites
to use `@TypeOf(f) != void` or `T != void`. Happily, this ended up
removing some duplicate logic and update some bitrotted feature
detection checks.

A handful of types have been modified to gain namespacing and type
safety. This is a breaking change.

Oh, and the last usage of `usingnamespace` site is eliminated.
2024-07-19 00:30:32 -07:00
Krzysztof Wolicki
0cc42d090f
std.fs.Dir: Rename OpenDirOptions to OpenOptions (#20542)
* std.fs.Dir: Rename OpenDirOptions to OpenOptions
https://ziglang.org/documentation/master/#Avoid-Redundant-Names-in-Fully-Qualified-Namespaces

* std.fs.Dir: Add deprecated alias `OpenDirOptions`
2024-07-09 13:36:38 -07:00
Michael Bradshaw
21cad3e09f Rename MAX_NAME_BYTES to max_name_bytes 2024-07-03 17:42:09 +01:00
Carter Snook
0b3508073c std: fix buffer overflows from improper WTF encoding
Closes #20288
2024-06-14 15:42:17 -05:00
Ryan Liptak
76fb2b685b std: Convert deprecated aliases to compile errors and fix usages
Deprecated aliases that are now compile errors:

- `std.fs.MAX_PATH_BYTES` (renamed to `std.fs.max_path_bytes`)
- `std.mem.tokenize` (split into `tokenizeAny`, `tokenizeSequence`, `tokenizeScalar`)
- `std.mem.split` (split into `splitSequence`, `splitAny`, `splitScalar`)
- `std.mem.splitBackwards` (split into `splitBackwardsSequence`, `splitBackwardsAny`, `splitBackwardsScalar`)
- `std.unicode`
  + `utf16leToUtf8Alloc`, `utf16leToUtf8AllocZ`, `utf16leToUtf8`, `fmtUtf16le` (all renamed to have capitalized `Le`)
  + `utf8ToUtf16LeWithNull` (renamed to `utf8ToUtf16LeAllocZ`)
- `std.zig.CrossTarget` (moved to `std.Target.Query`)

Deprecated `lib/std/std.zig` decls were deleted instead of made a `@compileError` because the `refAllDecls` in the test block would trigger the `@compileError`. The deleted top-level `std` namespaces are:

- `std.rand` (renamed to `std.Random`)
- `std.TailQueue` (renamed to `std.DoublyLinkedList`)
- `std.ChildProcess` (renamed/moved to `std.process.Child`)

This is not exhaustive. Deprecated aliases that I didn't touch:
  + `std.io.*`
  + `std.Build.*`
  + `std.builtin.Mode`
  + `std.zig.c_translation.CIntLiteralRadix`
  + anything in `src/`
2024-06-13 10:18:59 -04:00
Ryan Liptak
b32aa99b87 File.getOrEnableAnsiEscapeSupport: Do not attempt to set DISABLE_NEWLINE_AUTO_RETURN
Follow up to #20172. Fixes #20188
2024-06-05 18:06:29 -04:00
Ryan Liptak
016e87e7ef fs tests: Fix leaking a directory handle in "makepath existing directories" test 2024-06-04 19:02:16 -07:00
Ryan Liptak
5490021ab7 Dir.makeOpenPathAccessMaskW: Fix leaking intermediate directory handles
Fixes a regression introduced in 67455c5e70e86dbb7805ff9a415f1b13b14f36da. The `errdefer` cannot run since its not possible for an error to occur, and we don't want it to run on the last handle, so we move the closing back down to where it was before 67455c5e70e86dbb7805ff9a415f1b13b14f36da.
2024-06-04 18:45:25 -07:00
Ryan Liptak
337f09e932 Add File.getOrEnableAnsiEscapeSupport and use it
On Windows, the console mode flag `ENABLE_VIRTUAL_TERMINAL_PROCESSING` determines whether or not ANSI escape codes are parsed/acted on. On the newer Windows Terminal, this flag is set by default, but on the older Windows Console, it is not set by default, but *can* be enabled (since Windows 10 RS1 from June 2016).

The new `File.getOrEnableAnsiEscapeSupport` function will get the current status of ANSI escape code support, but will also attempt to enable `ENABLE_VIRTUAL_TERMINAL_PROCESSING` on Windows if necessary which will provide better/more consistent results for things like `std.Progress` and `std.io.tty`.

This type of change was not done previously due to a mistaken assumption (on my part) that the console mode would persist after the run of a program. However, it turns out that the console mode is always reset to the default for each program run in a console session.
2024-06-02 16:46:21 -07:00
Garfield Lee
17dc939346 lib/std/fs/File: enable VT seq support for Windows Console
* Newer versions of Windows added VT seq support not only in Windows Terminal, but also in the old-fashioned Windows Console (standalone conhost.exe), though not enabled by default.
* Try setting the newer console mode flags provides better experience for Windows Console users.

Co-authored-by: Kexy Biscuit <kexybiscuit@biscuitt.in>
2024-06-02 15:37:38 -07:00
Andrew Kelley
3b77f1ed7e rename zig-cache to .zig-cache
closes #20077
2024-05-29 10:20:15 -07:00
Georgijs
084c2cd90f
windows: use permissive file share flags everywhere (#19505)
* use permissive file share flags everywhere

* remove file_share_delete from createnamedpipefile
2024-05-11 23:34:40 +02:00
Jacob Young
67455c5e70 fs: handle OBJECT_NAME_COLLISION in makeOpenPath
This fixes a race condition when two threads/processes try to
`makeOpenPath` the same path simultaneously.
2024-05-07 05:23:29 -04:00
Ryan Liptak
b86c4bde64 Rename Dir.writeFile2 -> Dir.writeFile and update all callsites
writeFile was deprecated in favor of writeFile2 in f645022d16361865e24582d28f1e62312fbc73bb. This commit renames writeFile2 to writeFile and makes writeFile2 a compile error.
2024-05-03 13:29:22 -07:00
Nameless
aecd9cc6d1 std.posix.iovec: use .base and .len instead of .iov_base and .iov_len 2024-04-28 00:20:30 -07:00
Travis Staloch
8af59d1f98 ComptimeStringMap: return a regular struct and optimize
this patch renames ComptimeStringMap to StaticStringMap, makes it
accept only a single type parameter, and return a known struct type
instead of an anonymous struct.  initial motivation for these changes
was to reduce the 'very long type names' issue described here
https://github.com/ziglang/zig/pull/19682.

this breaks the previous API.  users will now need to write:
`const map = std.StaticStringMap(T).initComptime(kvs_list);`

* move `kvs_list` param from type param to an `initComptime()` param
* new public methods
  * `keys()`, `values()` helpers
  * `init(allocator)`, `deinit(allocator)` for runtime data
  * `getLongestPrefix(str)`, `getLongestPrefixIndex(str)` - i'm not sure
     these belong but have left in for now incase they are deemed useful
* performance notes:
  * i posted some benchmarking results here:
    https://github.com/travisstaloch/comptime-string-map-revised/issues/1
  * i noticed a speedup reducing the size of the struct from 48 to 32
    bytes and thus use u32s instead of usize for all length fields
  * i noticed speedup storing KVs as a struct of arrays
  * latest benchmark shows these wall_time improvements for
    debug/safe/small/fast builds: -6.6% / -10.2% / -19.1% / -8.9%. full
    output in link above.
2024-04-22 15:31:41 -07:00
Linus Groh
7789e87230 std.fs.Dir.openDir: use wasi libc API when -lc
Same as #19680 but for directories.
2024-04-20 15:42:44 -07:00
Jacob Young
c7ffdbcd41 Revert "disable flaky std.fs test"
This reverts commit d080622cc3f5db1ae3cd628c21c1ffaeb2b899d5.

Workaround applied to CI.
2024-04-20 06:08:26 -04:00
Andrew Kelley
d080622cc3 disable flaky std.fs test
Tracked by #17134
2024-04-19 19:01:13 -07:00
Andrew Kelley
d06e5b4349 std.fs.Dir.openFile: use wasi libc API when -lc
Also removes the LOCK namespace from std.c.wasi because wasi libc does
not have flock.

closes #19336
related to #19352

Co-authored-by: Ryan Liptak <squeek502@hotmail.com>
2024-04-18 17:49:05 -07:00
Andrew Kelley
36060a774e fix namespacing of std.fs.Dir.Walker.Entry 2024-04-14 18:08:42 -07:00
Andrew Kelley
10106660e9 std.fs.Dir.Walker: maintain a null byte in path names 2024-04-14 17:59:43 -07:00
Jacob Young
0c83fa2fd0 haiku: fix directory iteration 2024-04-08 13:20:14 -04:00
Jacob Young
2dd74cd312 haiku: debitrot 2024-03-23 18:11:32 +01:00
Andrew Kelley
12191c8a22 std: promote tests to doctests
Now these show up as "example usage" in generated documentation.
2024-03-21 14:11:46 -07:00
Andrew Kelley
8c94950c24 fix compilation failures found by CI 2024-03-19 16:18:18 -07:00
Andrew Kelley
cd62005f19 extract std.posix from std.os
closes #5019
2024-03-19 11:45:09 -07:00
Michael Dusan
d7cf25f5ca
bsd: debitrot std.c
- follow-up to f4bf061d8a8
- updated std.fs.Dir to use properly named symbols
2024-03-15 02:28:50 -04:00
Ryan Liptak
f1dd1ee5ed fs/test: Make testWithAllSupportedPathTypes also test all supported path separators
Now, all the tests that use `testWithAllSupportedPathTypes` will also run each test with both `/` and `\` as the path separator on Windows.

Also, removes the now-redundant "Dir.symLink with relative target that has a / path separator" since the same thing is now tested in the "Dir.readLink" test
2024-02-29 17:55:27 -08:00
Ryan Liptak
e233971e4f Fix symLink's handling of / path separators on Windows
Symlink targets require canonicalized path separators on Windows
2024-02-29 16:12:24 -08:00
Ryan Liptak
ae7f3fc360 Eliminate error.InvalidHandle from OpenError and RealPathError
InvalidHandle in OpenError is no longer a possible error on any platform. In the past it was able to be returned in `openOptionsFromFlagsWasi`, but the implementation was changed in 7680c5330cbc9141b9a5444e30c512b6068ab50d to make it no longer possible.

InvalidHandle in RealPathError was a holdover from before d5312d53a066092ba9efd687e25b29a87eb6290c, which made realpath a compile error on WASI. However, InvalidHandle was also a possible error in the FreeBSD fallback implementation added in 537624734c4db9e0cdbdc0ebce57375d17172a70. This commit changes the FreeBSD fallback implementation to return FileNotFound instead of InvalidHandle which matches how EBADF is handled in all the other `realpath` implementations (including the FreeBSD non-fallback implementation).

Closes #19084
2024-02-27 15:59:50 -08:00
Ryan Liptak
9fec608b3b Add std.fs.path.fmtAsUtf8Lossy/fmtWtf16LeAsUtf8Lossy 2024-02-24 14:05:24 -08:00
Ryan Liptak
68b87918df Fix handling of Windows (WTF-16) and WASI (UTF-8) paths
Windows paths now use WTF-16 <-> WTF-8 conversion everywhere, which is lossless. Previously, conversion of ill-formed UTF-16 paths would either fail or invoke illegal behavior.

WASI paths must be valid UTF-8, and the relevant function calls have been updated to handle the possibility of failure due to paths not being encoded/encodable as valid UTF-8.

Closes #18694
Closes #1774
Closes #2565
2024-02-24 14:05:24 -08:00
Ryan Liptak
80508b98c2 Update deprecated std.unicode function usages 2024-02-24 14:04:59 -08:00
Michael Dusan
7e83e7d9a8 bsd: debitrot type-safe std.c.O
Minor changes as per 7680c5330cb mostly about pipe2() flags.

closes #18927
2024-02-14 19:26:12 -08:00
Andrew Kelley
07c1dd3d1d std.os.windows.OpenFile: add missing error
Encountered in a recent CI run on an aarch64-windows dev kit.

Pretty sure I disabled the virus scanner but it looks like it turned
itself back on with a Windows Update.

Rather than marking the new error code as unreachable in the places
where it is unexpected, this commit makes it return `error.Unexpected`.
2024-02-14 11:14:43 -08:00
Andrew Kelley
7680c5330c some API work on std.c, std.os, std.os.wasi
* std.c: consolidate some definitions, making them share code. For
  example, freebsd, dragonfly, and openbsd can all share the same
  `pthread_mutex_t` definition.
* add type safety to std.c.O
  - this caught a bug where mode flags were incorrectly passed as the
    open flags.
* 3 fewer uses of usingnamespace keyword
* as per convention, remove purposeless field prefixes from struct field
  names even if they have those prefixes in the corresponding C code.
* fix incorrect wasi libc Stat definition
* remove C definitions from incorrectly being in std.os.wasi
* make std.os.wasi definitions type safe
* go through wasi native APIs even when linking libc because the libc
  APIs are problematic and wasteful
* don't expose WASI definitions in std.posix
* remove std.os.wasi.rights_t.ALL: this is a footgun. should it be all
  future rights too? or only all current rights known? both are
  the wrong answer.
2024-02-11 13:38:55 -07:00
Andrew Kelley
54bbc73f85
Merge pull request #18712 from Vexu/std.options
std: make options a struct instance instead of a namespace
2024-02-09 13:38:42 -08:00
Ryan Liptak
f2e249e920 fs tests: Make path type show up in stack traces of testWithAllSupportedPathTypes
This allows for easier debugging without the need to print out the path type to stderr.

Context: 5a3ae38f3b79a69cb6f4ad28934a51165cae2ef1
2024-02-02 18:03:17 -08:00
Andrew Kelley
5a3ae38f3b std: remove garbage being printed to stderr during unit tests 2024-02-01 18:05:14 -07:00
Veikka Tuominen
a4f27e8987 remove std.io.Mode 2024-02-01 15:22:36 +02:00
Veikka Tuominen
b0bea72588 remove std.event 2024-02-01 15:22:36 +02:00