50 Commits

Author SHA1 Message Date
Igor Anić
b84301c8e5 std.tar don't overwrite existing file
Fail with error if file already exists. File is not silently overwritten
but an error is raised.

Fixes: #18089
2024-02-24 23:37:55 +01:00
Igor Anić
0a86b117bf std.tar fix integer overflow in header size parse
Found by fuzzing. Fixing code and adding test.
2024-02-23 21:57:40 +01:00
Igor Anić
f67aa8b9b3 std.tar fix parsing mode field in tar header
Found by fuzzing. Previous numeric function assumed that is is getting
buffer of size 12, mode is size 8. Fuzzing found overflow.
Fixing and adding test cases.
2024-02-23 21:57:15 +01:00
Andrew Kelley
256c5934bf std.tar: remove abuse of inline fn
In general, any `inline fn` should document why it is using `inline`
because the rule of thumb is: don't use inline.
2024-02-23 01:16:44 -08:00
Igor Anić
30f15e3afe fix crash in tar found by fuzzing
Running fuzzing tar test with [zig std lib
fuzzing](https://github.com/squeek502/zig-std-lib-fuzzing) reached and
assert in tar implementation. Assert (in std lib) should not be
reachable by external input, so I'm fixing this to return error.
2024-02-22 18:20:05 -08:00
cipharius
f25c499d12 Permits tar directory path without trailing slash 2024-01-15 18:31:32 +02:00
Igor Anić
7d3a31872e tar: improve diagnostic reporting
Using Python testtar file (mentioned in #14310) to test diagnostic
reporting.
Added computing checksum by using both unsigned and signed header bytes
values.
Added skipping gnu exteneded sparse headers while reporting unsupported
header in diagnostic.

Note on testing:

wget https://github.com/python/cpython/raw/3.11/Lib/test/testtar.tar -O
/tmp/testtar.tar

```
test "Python testtar.tar file" {
    const file_name = "testtar.tar";

    var file = try std.fs.cwd().openFile("/tmp/" ++ file_name, .{});
    defer file.close();

    var diag = Options.Diagnostics{ .allocator = std.testing.allocator };
    defer diag.deinit();

    var iter = iterator(file.reader(), &diag);
    while (try iter.next()) |f| {
        std.debug.print("supported: {} {s} {d}\n", .{ f.kind, f.name, f.size });
        try f.skip();
    }
    for (diag.errors.items) |e| {
        switch (e) {
            .unsupported_file_type => |u| {
                std.debug.print("unsupported: {} {s}\n", .{ u.file_type, u.file_name });
            },
            else => unreachable,
        }
    }
}
```
2024-01-13 19:37:33 -07:00
Igor Anić
7923a53996 tar: rename reader to iterator
Itarator has `next` function, iterates over tar files. When using from
outside of module with `tar.` prefix makes more sense.

var iter = tar.iterator(reader, null);
while (try iter.next()) |file| {
...
}
2024-01-13 19:37:33 -07:00
Igor Anić
76fe1f53d5 tar: fix tests on 32-bit platforms 2024-01-13 19:37:33 -07:00
Igor Anić
a75fd4ff15 tar: move test cases to std/tar/testdata
Create std/tar/test.zig for test which uses cases from testdata.
2024-01-13 19:37:33 -07:00
Igor Anić
f8e42d6b30 tar: add Go test case files to the project 2024-01-13 19:37:33 -07:00
Igor Anić
c07527abac tar: reorganize file, functions before tests 2024-01-13 19:37:33 -07:00
Igor Anić
c76abe0e18 tar: use file word in less places 2024-01-13 19:37:33 -07:00
Igor Anić
4a6d67ab1a tar: remove stratch from tar reader
Use explicit buffers for name, link_name instead.
It is cleaner that way.
2024-01-13 19:37:33 -07:00
Igor Anić
9f7dd32308 tar: refactor pax attribute
Make it little readable.
2024-01-13 19:37:33 -07:00
Igor Anić
dbab45cfc6 tar: replace custom buffered reader with std.io 2024-01-13 19:37:33 -07:00
Igor Anić
58e0e509c6 tar: add module comment and references 2024-01-13 19:37:33 -07:00
Igor Anić
a3cf8ec71e tar: add pax file reader tests 2024-01-13 19:37:33 -07:00
Igor Anić
7b0bbc680f tar: add file mode to result of tarbal iteration
So we have information to set executable bit on write to file system.
2024-01-13 19:37:33 -07:00
Igor Anić
2a432d3008 tar: prefix test cases with 'tar'
To make it little easier to filter from all stdlib tests.
2024-01-13 19:37:33 -07:00
Igor Anić
2ed9a276a7 tar: use Go test cases path from env variable
Skip tests if env is not set.
2024-01-13 19:37:33 -07:00
Igor Anić
6bfa7bf197 tar: use scratch buffer for file names
That makes names strings stable during the iteration. Otherwise string
buffers can be overwritten while reading file content.
2024-01-13 19:37:33 -07:00
Igor Anić
6e7a39c935 tar: refactor reading pax attributes 2024-01-13 19:37:33 -07:00
Igor Anić
c761dfc176 tar: add gnu path and link extensions handling 2024-01-13 19:37:33 -07:00
Igor Anić
48b160c1bf tar: handle pax null attrs and pax attr ending 2024-01-13 19:37:33 -07:00
Igor Anić
16c40fc471 tar: add header chksum checking 2024-01-13 19:37:33 -07:00
Igor Anić
169f28d3e6 tar: fix import path 2024-01-13 19:37:33 -07:00
Igor Anić
e1424b84b8 tar: add parsing size in gnu extended format
Reference:
https://www.gnu.org/software/tar/manual/html_node/Extensions.html#Extensions

If the leading byte is 0x80 (128), the non-leading bytes of the field
are concatenated in big-endian order, with the result being a positive
number expressed in binary form.
2024-01-13 19:37:33 -07:00
Igor Anić
6d5283e835 tar: refactor reader and iterator
Make it more readable.
2024-01-13 19:37:33 -07:00
Igor Anić
be5d04ab79 tar: add pax linkpath attribute parsing
Name of symbolic link can be also found in pax attribute.
2024-01-13 19:37:33 -07:00
Igor Anić
1817063375 tar: add initial test cases
Just adding tests, without changing functionality.
2024-01-13 19:37:33 -07:00
Igor Anić
4381241237 tar: refactor Buffer
Move reader into Buffer and make it BufferedReader. This doesn't
introduce any new functionality just grouping similar things.
2024-01-13 19:37:33 -07:00
Igor Anić
ff8544daa5 tar: refactor code to be more testable
Split reading/parsing tar file and writing results to the disk in two
separate steps. So we can later test parsing part without need to write
everyting to the disk.
2024-01-13 19:37:33 -07:00
Maciej 'vesim' Kuliński
f64f3423e4 std.tar: trim also spaces from the beginning of file size 2023-11-21 00:55:22 -05:00
mlugg
51595d6b75
lib: correct unnecessary uses of 'var' 2023-11-19 09:55:07 +00:00
Andrew Kelley
7b25d050e6 std.tar: fix creation of symlinks with omit_empty_directories 2023-10-08 16:54:31 -07:00
Andrew Kelley
51e15a9650 std.tar: add option for omitting empty directories 2023-10-08 16:54:30 -07:00
Andrew Kelley
21181181bf zig fetch: enhanced error reporting
* Package: use std.tar diagnostics to give detailed error messages
* std.tar: add diagnostic for unsupported file type
2023-10-02 17:02:25 -07:00
Andrew Kelley
ef9966c985 introduce the 'zig fetch' command + symlink support
zig fetch [options] <url>
zig fetch [options] <path>

Fetches a package which is found at <url> or <path> into the global
cache directory, printing the package hash to stdout.

Closes #16972
Related to #14280

Additionally, this commit:

* Adds uncompressed .tar support to package fetching
* Introduces symlink support to package fetching
2023-10-02 17:02:25 -07:00
Andrew Kelley
a5144d19b7 std.tar: support symlinks
closes #16678
2023-10-02 17:02:24 -07:00
none
39a98dc426 std.tar: add support for file path in pax attributes
Handles .extended_header type to parse PAX attributes and check if they override
the path of the next file. Increases file path limit to std.fs.MAX_PATH_BYTES.

Fixes #15342
2023-09-08 21:55:14 +03:00
mlugg
f26dda2117 all: migrate code to new cast builtin syntax
Most of this migration was performed automatically with `zig fmt`. There
were a few exceptions which I had to manually fix:

* `@alignCast` and `@addrSpaceCast` cannot be automatically rewritten
* `@truncate`'s fixup is incorrect for vectors
* Test cases are not formatted, and their error locations change
2023-06-24 16:56:39 -07:00
Eric Joldasov
50339f595a all: zig fmt and rename "@XToY" to "@YFromX"
Signed-off-by: Eric Joldasov <bratishkaerik@getgoogleoff.me>
2023-06-19 12:34:42 -07:00
Motiejus Jakštys
d41111d7ef mem: rename align*Generic to mem.align*
Anecdote 1: The generic version is way more popular than the non-generic
one in Zig codebase:

     git grep -w alignForward | wc -l
    56
     git grep -w alignForwardGeneric | wc -l
    149

     git grep -w alignBackward | wc -l
    6
     git grep -w alignBackwardGeneric | wc -l
    15

Anecdote 2: In my project (turbonss) that does much arithmetic and
alignment I exclusively use the Generic functions.

Anecdote 3: we used only the Generic versions in the Macho Man's linker
workshop.
2023-06-17 12:49:13 -07:00
Andrew Kelley
125221cce9 std: update to use @memcpy directly 2023-04-28 13:24:43 -07:00
Travis Staloch
40029d6875 std.tar: make sub dirs + trim spaces
closes #15222. these changes allow the .tgz from this issue to
decompress and the test code to succeed.
2023-04-23 20:39:28 +03:00
Andrew Kelley
ea6e0e33a7 zig build: add executable bit and file path to package hash
Unfortunately, due to the Windows equivalent of executable permissions
being a bit tricky, there is follow-up work to be done.

What is done in this commit is the hash modifications. At the fetch
layer, executable bits inside packages are ignored. In the hash
computation layer, executable bit is implemented for POSIX but not yet
for Windows. This means that the hash will not break again in the future
for packages that do not have any executable files, but it will break
for packages that do.

This is a hash-breaking change.

Closes #14308
2023-02-01 18:42:29 -07:00
Andrew Kelley
476cbe871a fix build failures on 32-bit arm due to u64/usize coercion 2023-01-11 15:39:49 -08:00
Andrew Kelley
d4e829d0a0 std.tar: add strip_components option 2023-01-11 15:39:49 -08:00
Andrew Kelley
585b9970ef add std.tar for tar file unpacking 2023-01-11 15:39:48 -08:00