80 Commits

Author SHA1 Message Date
Veikka Tuominen
2b67f56c35 std.fs: split Dir into IterableDir
Also adds safety check for attempting to iterate directory not opened with `iterate = true`.
2022-07-15 13:04:21 +03:00
protty
18f3034629
std.Thread: ResetEvent improvements (#11523)
* std: start removing redundant ResetEvents

* src: fix other uses of std.Thread.ResetEvent

* src: add builtin.sanitize_thread for tsan detection

* atomic: add Atomic.fence for proper fencing with tsan

* Thread: remove the other ResetEvent's and rewrite the current one

* Thread: ResetEvent docs

* zig fmt + WaitGroup.reset() fix

* src: fix build issues for ResetEvent + tsan

* Thread: ResetEvent tests

* Thread: ResetEvent module doc

* Atomic: replace llvm *p memory constraint with *m

* panicking: handle spurious wakeups in futex.wait() when waiting for abort()

* zig fmt
2022-04-26 16:48:56 -05:00
Cody Tapscott
7b090df668 stdlib std.os: Improve wasi-libc parity for WASI CWD emulation
Two major changes here:
  1. We store the CWD as a simple `[]const u8` and lookup Preopens for
     every absolute or CWD-referenced file operation, based on the
     Preopen with the longest match (i.e. most specific path)
  2. Preorders are normalized to POSIX absolute paths at init time.
     Behavior depends on the "cwd_root" parameter of `initPreopensWasi`:

	`cwd_root` is used for any Preopens that start with "."

	  For example:
            "./foo/bar" - inits to -> "{cwd_root}/foo/bar"
            "foo/bar"   - inits to -> "/foo/bar"
	    "/foo/bar"  - inits to -> "/foo/bar"

        `cwd_root` must be an absolute path.

	Using "/" as `cwd_root` gives behavior similar to wasi-libc.
2022-04-16 18:08:05 +02:00
Evan Haas
618398b7d3 std.fs: prevent possible integer overflow in Dir.makePath
The call to `makeDir` for the top-level component of `sub_path`
can return `error.FileNotFound` if the directory represented by
`self` has been deleted.

Fixes #11397
2022-04-15 11:19:23 +03:00
Ryan Liptak
9c509f1526 Enable passing 'Dir.rename directories' fs test on Windows
Looks like d3f87f8ac01039722197a13a12342fc747a90567 fixed the standard cases of dir renaming, but the edge cases (renaming onto an existing empty/non-empty directory) are still behaving differently than on non-Windows.
2022-04-13 19:55:16 +02:00
Ryan Liptak
17daba1806 std/fs/test.zig: Add test for renaming a dir onto an empty dir
Also split the Dir.rename on directories test into 3 tests:
- General rename of a directory
- Rename of a directory onto an existing empty directory
- Rename of a directory onto an existing non-empty directory

The only new case is the rename onto an existing empty directory, but splitting the tests this way made them much more understandable.
2022-04-12 06:28:16 -04:00
Cody Tapscott
ade2d0c6a2 stdlib WASI: Add realpath() support for non-absolute Preopens 2022-03-03 14:31:49 -07:00
Cody Tapscott
58f961f4cb stdlib: Add emulated CWD to std.os for WASI targets
This adds a special CWD file descriptor, AT.FDCWD (-2), to refer to the
current working directory. The `*at(...)` functions look for this and
resolve relative paths against the stored CWD. Absolute paths are
dynamically matched against the stored Preopens.

"os.initPreopensWasi()" must be called before std.os functions will
resolve relative or absolute paths correctly. This is asserted at
runtime.

Support has been added for: `open`, `rename`, `mkdir`, `rmdir`, `chdir`,
`fchdir`, `link`, `symlink`, `unlink`, `readlink`, `fstatat`, `access`,
and `faccessat`.

This also includes limited support for `getcwd()` and `realpath()`.
These return an error if the CWD does not correspond to a Preopen with
an absolute path. They also do not currently expand symlinks.
2022-03-03 14:31:49 -07:00
ominitay
11b4cc589c
std.fs: Implement cross-platform metadata API
Implements a cross-platform metadata API, aiming to reduce unnecessary Unix-dependence of the `std.fs` api. Presently, all OSes beside Windows are treated as Unix; this is likely the best way to treat things by default, instead of explicitly listing each Unix-like OS.

Platform-specific operations are not provided by `File.Metadata`, and instead are to be accessed from `File.Metadata.inner`.

Adds:

- File.setPermissions() : Sets permission of a file according to a `Permissions` struct (not available on WASI)

- File.Permissions : A cross-platform representation of file permissions
  - Permissions.readOnly() : Returns whether the file is read-only
  - Permissions.setReadOnly() : Sets whether the file is read-only
  - Permissions.unixSet() : Sets permissions for a class (UNIX-only)
  - Permissions.unixGet() : Checks a permission for a class (UNIX-only)
  - Permissions.unixNew() : Returns a new Permissions struct to represent the passed mode (UNIX-only)

- File.Metadata : A cross-platform representation of file metadata
  - Metadata.size() : Returns the size of a file
  - Metadata.permissions() : Returns a `Permissions` struct, representing permissions on the file
  - Metadata.kind() : Returns the `Kind` of the file
  - Metadata.accessed() : Returns the time the file was last accessed
  - Metadata.modified() : Returns the time the file was last modified
  - Metadata.created() : Returns the time the file was created (this is an optional, as the underlying filesystem, or OS may not support this)

Methods of `File.Metadata` are also available for the below, so I won't repeat myself
The below may be used for platform-specific functionality

- File.MetadataUnix : The internal implementation of `File.Metadata` on Unices

- File.MetadataLinux : The internal implementation of `File.Metadata` on Linux

- File.MetadataWindows : The implementation of `File.Metadata` on Windows
2022-02-13 20:56:06 +00:00
Jakub Konka
dd7309bde4
Merge pull request #10404 from ominitay/iterator
std: Fix using `fs.Dir.Iterator` twice
2022-01-30 14:25:50 +01:00
Sage Hane
e288148f60
fs: Use OpenMode enum instead of read/write flags. 2022-01-29 15:52:08 +02:00
ominitay
dc11fe4047
std: Fix using fs.Dir.Iterator twice
This fixes the use of multiple `Iterator`s in a row on a directory.
Previously, on many platforms, using an `Iterator` on an
already-iterated directory would give no entries.

Fixing this involved seeking to the beginning of the directory on the
first call of `next()`.
2022-01-28 16:12:05 +00:00
ominitay
c3ef85aa76
Add test for using fs.Dir.Iterator twice 2022-01-27 20:54:48 +00:00
Lee Cannon
1093b09a98
allocgate: renamed getAllocator function to allocator 2021-11-30 23:32:47 +00:00
Lee Cannon
85de022c56
allocgate: std Allocator interface refactor 2021-11-30 23:32:47 +00:00
Ominitay
796687f156 Add chmod and chown 2021-11-15 20:04:55 -05:00
Andrew Kelley
6115cf2240 migrate from std.Target.current to @import("builtin").target
closes #9388
closes #9321
2021-10-04 23:48:55 -07:00
Stephen Gregoratto
87fd502fb6 Initial bringup of the Solaris/Illumos port 2021-09-24 14:06:16 -04:00
Andrew Kelley
d29871977f remove redundant license headers from zig standard library
We already have a LICENSE file that covers the Zig Standard Library. We
no longer need to remind everyone that the license is MIT in every single
file.

Previously this was introduced to clarify the situation for a fork of
Zig that made Zig's LICENSE file harder to find, and replaced it with
their own license that required annual payments to their company.
However that fork now appears to be dead. So there is no need to
reinforce the copyright notice in every single file.
2021-08-24 12:25:09 -07:00
Ryan Liptak
2f6dbaa0ea fs.Dir.walk: Do not close the initial dir during/after walking it
Closing the initial directory was unexpected to me, and does not mesh very well with how the rest of the Dir API works.

Fixes #9556
2021-08-20 10:02:54 +03:00
Ryan Liptak
f6bb56f8c7 Improve fs.Walker test
- Take into account that iteration order is undefined by checking against a map instead of relying on numerically sorted iteration order
- Check both path and basename for each entry instead of just path
2021-08-13 16:43:52 -07:00
Ominitay
c1285a1bbe Move fs.Walker to fs.Dir.Walker
fs.Walker has been replaced with fs.Dir.Walker. Paths of entries
returned are relative to the Dir.
2021-07-28 21:39:36 +03:00
kprotty
0a1def7833 changes to accomodate std.Thread update 2021-06-30 21:48:59 -05:00
Jacob G-W
9fffffb07b fix code broken from previous commit 2021-06-21 17:03:03 -07:00
Michael Dusan
bfe3558efe netbsd: add more std.os.bits 2021-06-16 14:46:25 -04:00
Michael Dusan
75cd37b8f7 dragonfly: pass zig build test 2021-05-24 10:24:41 -04:00
Ryan Liptak
78019452f7 Add updateFile to . and .. fs tests 2021-05-23 16:35:31 -07:00
Ryan Liptak
1bc0df7250 Add . and .. tests for std.fs functions
These tests check that . and .. are able to be handled by the std.fs functions. Before #7664, these tests would have failed on Windows.

Closes #4659
2021-05-23 16:22:27 -07:00
Jonathan Marler
b0116afd8a handle relative paths with too many ".." 2021-05-21 01:29:40 -06:00
Jonathan Marler
59de5d0350 add the openDir cwd parent test 2021-05-20 14:00:41 -06:00
Jonathan Marler
666584067a implement nt path conversion for windows 2021-05-20 14:00:40 -06:00
LemonBoy
c065e12dbe std: Add more tests, fix broken code
It turns out that nothing in the test suite was exercising
preadv/pwritev and so the previous commits silently broke them.

Adding tests revealed readvAll and preadvAll were also broken and not
covered by any test.
2021-05-11 12:33:48 +02:00
Veikka Tuominen
fd77f2cfed std: update usage of std.testing 2021-05-08 15:15:30 +03:00
LemonBoy
566adc2510 std: Swap arguments in Thread.spawn
Beside the new order being consistent with the ThreadPool API and making
more sense, this shuffling allows to write the context argument type in
terms of the startFn arguments, reducing the use of anytype (eg. less
explicit casts when using comptime_int parameters, yay).

Sorry for the breakage.

Closes #8082
2021-02-28 14:03:19 +01:00
Andrew Kelley
a9667b5a85 organize std lib concurrency primitives and add RwLock
* move concurrency primitives that always operate on kernel threads to
   the std.Thread namespace
 * remove std.SpinLock. Nobody should use this in a non-freestanding
   environment; the other primitives are always preferable. In
   freestanding, it will be necessary to put custom spin logic in there,
   so there are no use cases for a std lib version.
 * move some std lib files to the top level fields convention
 * add std.Thread.spinLoopHint
 * add std.Thread.Condition
 * add std.Thread.Semaphore
 * new implementation of std.Thread.Mutex for Windows and non-pthreads Linux
 * add std.Thread.RwLock

Implementations provided by @kprotty
2021-01-14 20:41:37 -07:00
Andrew Kelley
99203e4177
Merge pull request #7578 from vrischmann/walker
fix segfault in std.fs.Walker
2021-01-01 13:28:01 -08:00
Frank Denis
6c2e0c2046 Year++ 2020-12-31 15:45:24 -08:00
Vincent Rischmann
399c428cb0 fs: add a test for the walker
Written like this it triggers the segfault reported in #7560
2020-12-28 18:14:27 +01:00
Andrew Kelley
c2b1c88953 std: fix compile errors introduced in previous commit 2020-12-23 20:49:38 -07:00
Andrew Kelley
177377b6e3 rework std.ResetEvent, improve std lib Darwin integration
* split std.ResetEvent into:
   - ResetEvent - requires init() at runtime and it can fail. Also
     requires deinit().
   - StaticResetEvent - can be statically initialized and requires no
     deinitialization. Initialization cannot fail.
 * the POSIX sem_t implementation can in fact fail on initialization
   because it is allowed to be implemented as a file descriptor.
 * Completely define, clarify, and explain in detail the semantics of
   these APIs. Remove the `isSet` function.
 * `ResetEvent.timedWait` returns an enum instead of a possible error.
 * `ResetEvent.init` takes a pointer to the ResetEvent instead of
   returning a copy.
 * On Darwin, `ResetEvent` is implemented using Grand Central Dispatch,
   which is exposed by libSystem.

stage2 changes:
 * ThreadPool: use a single, pre-initialized `ResetEvent` per worker.
 * WaitGroup: now requires init() and deinit() and init() can fail.
   - Add a `reset` function.
   - Compilation initializes one for the work queue in creation and
     re-uses it for every update.
   - Rename `stop` to `finish`.
   - Simplify the implementation based on the usage pattern.
2020-12-23 16:57:18 -08:00
LemonBoy
ac443b941d std: Restore file locking test on Windows
Make it even more robust wrt timing problems.

Closes #7010
2020-12-04 12:24:50 -08:00
LemonBoy
f8ddc3d873 std: Fix file locking logic for BSD targets 2020-11-23 18:00:05 +01:00
g-w1
a0226ab05b
std: openDirAbsolute and accessAbsolute (#7082)
* add more abosolutes

* added wrong files

* adding 2 tests and changing the function signatures because of lazy analysis not checking them

* fix a bug that got uncovered by lazy eval

* Add compile error when using WASI with openDirAbsolute and accessAbsolute

* typo
2020-11-18 08:42:35 +01:00
Sébastien Marie
ab4b34f75f openbsd: skip tests using Dir.realpath 2020-11-13 12:30:14 -05:00
Andrew Kelley
ceaa569bfa disable flaky file locking test on Windows
See #7010
2020-11-06 17:00:51 -07:00
LemonBoy
b9391c9564 std: Make file locking tests less fragile 2020-11-06 14:02:11 -05:00
Andrew Kelley
27b04d5905 disable the failing std lib freebsd tests
enable std lib freebsd tests on the CI

See #1759
2020-10-12 20:08:23 -07:00
Vignesh Rajagopalan
2ab0c7391a Rename .macosx to .macos 2020-10-12 18:56:25 -04:00
Ryan Liptak
bd449b184a Add deleteDir test with exception for not-empty directory on Windows
Re-adds the test that was added and then reverted in https://github.com/ziglang/zig/pull/6397, but with the test for https://github.com/ziglang/zig/issues/5537 skipped for now since that issue is no longer fixed.
2020-09-30 02:35:17 -04:00
Jakub Konka
bb636cb3bf Revert "Fix for Windows: std.os.windows.DeleteFile()" 2020-09-29 19:35:44 -04:00