* read directly into the ArrayList buffers.
* respect max_output_bytes
* std.ArrayList:
- make `allocatedSlice` public.
- add `unusedCapacitySlice`.
I removed the Windows implementation of this stuff; I am doing a partial
merge of LemonBoy's patch with the understanding that a later patch can
add the Windows implementation after it is vetted.
Keep polling until there are enough open handles, if the child process
terminates closing the handles or explicitly closes them we just quit
polling and wait for the process handle to signal the termination
condition.
Reading stdin&stderr at different times may lead to nasty deadlocks (eg.
when stdout is read before stderr and the child process doesn't write
anything onto stdout).
Implement a polling mechanism to make sure this won't happen: we read
data from stderr/stdout as it becomes ready and then it's copied into an
ArrayList provided by the user, avoiding any kind of blocking read.
We were violating the POSIX standard which resulted in a deadlock on
musl v1.1.24 on aarch64 alpine linux, uncovered with the new ThreadPool
usage in the stage2 compiler.
std.os execv functions that accept an Allocator parameter are removed
because they are footguns. The POSIX standard does not allow calls to
malloc() between fork() and execv() and since it is common to both
(1) call execv() after fork() and (2) use std.heap.c_allocator,
Programmers are encouraged to go through the `std.process` API
instead, causing some dissonance when combined with `std.os` APIs.
I also slapped a big warning message on all the relevant doc comments.
Comment reproduced here:
If we're linking libc, some naughty applications may have
registered atexit handlers which we really do not want to
run in the fork child. I caught LLVM doing this and it
caused a deadlock instead of doing an exit syscall. In
the words of Avril Lavigne, "Why'd you have to go and
make things so complicated?"
Before merging, do this for every item in the file:
* solve the issue, or
* convert the task to a github issue and update the comment
to link to the issue (and remove "TODO" text from the comment).
Then delete the file.
Related: #363
- correct uid_t from i32 to u32 on linux
- define uid_t and gid_t for OSes missing definitions
- use uid_t/gid_t instead of plain u32s throughout std.os
Replace them with `std.os.windows.OpenFile` instead. To allow
creation/opening of directories, `std.os.windows.OpenFileOptions`
now features a `.expect_dir: bool` member which is meant to emualate
POSIX's `O_DIRECTORY` flag.
This new name (and the fact that it is a function returning a type) will
make it more clear which use cases are better suited for ArrayList and
which are better suited for ArrayListSentineled.
Also for consistency with ArrayList,
* `append` => `appendSlice`
* `appendByte` => `append`
Thanks daurnimator for pointing out the confusion of std.Buffer.
The main goal here is to make the function pointers comptime, so that we
don't have to do the crazy stuff with async function frames.
Since InStream, OutStream, and SeekableStream are already generic
across error sets, it's not really worse to make them generic across the
vtable as well.
See #764 for the open issue acknowledging that using generics for these
abstractions is a design flaw.
See #130 for the efforts to make these abstractions non-generic.
This commit also changes the OutStream API so that `write` returns
number of bytes written, and `writeAll` is the one that loops until the
whole buffer is written.
* re-introduce `std.build.Target` which is distinct from `std.Target`.
`std.build.Target` wraps `std.Target` so that it can be annotated as
"the native target" or an explicitly specified target.
* `std.Target.Os` is moved to `std.Target.Os.Tag`. The former is now a
struct which has the tag as well as version range information.
* `std.elf` gains some more ELF header constants.
* `std.Target.parse` gains the ability to parse operating system
version ranges as well as glibc version.
* Added `std.Target.isGnuLibC()`.
* self-hosted dynamic linker detection and glibc version detection.
This also adds the improved logic using `/usr/bin/env` rather than
invoking the system C compiler to find the dynamic linker when zig
is statically linked. Related: #2084
Note: this `/usr/bin/env` code is work-in-progress.
* `-target-glibc` CLI option is removed in favor of the new `-target`
syntax. Example: `-target x86_64-linux-gnu.2.27`
closes#1907
Some C compilers, such as Clang, are known to rely on
argv[0] to find the path to their own executable,
without even bothering to resolve PATH. This results
in the message:
error: unable to execute command: Executable "" doesn't exist!
So we tell ChildProcess to expand argv[0] to the absolute path
to give them a helping hand.
* libc_installation.cpp is deleted.
src-self-hosted/libc_installation.zig is now used for both stage1 and
stage2 compilers.
* (breaking) move `std.fs.File.access` to `std.fs.Dir.access`. The API
now encourages use with an open directory handle.
* Add `std.os.faccessat` and related functions.
* Deprecate the "C" suffix naming convention for null-terminated
parameters. "C" should be used when it is related to libc. However
null-terminated parameters often have to do with the native system
ABI rather than libc. "Z" suffix is the new convention. For example,
`std.os.openC` is deprecated in favor of `std.os.openZ`.
* Add `std.mem.dupeZ` for using an allocator to copy memory and add a
null terminator.
* Remove dead struct field `std.ChildProcess.llnode`.
* Introduce `std.event.Batch`. This API allows expressing concurrency
without forcing code to be async. It requires no Allocator and does
not introduce any failure conditions. However it is not thread-safe.
* There is now an ongoing experiment to transition away from
`std.event.Group` in favor of `std.event.Batch`.
* `std.os.execvpeC` calls `getenvZ` rather than `getenv`. This is
slightly more efficient on most systems, and works around a
limitation of `getenv` lack of integration with libc.
* (breaking) `std.os.AccessError` gains `FileBusy`, `SymLinkLoop`, and
`ReadOnlyFileSystem`. Previously these error codes were all reported
as `PermissionDenied`.
* Add `std.Target.isDragonFlyBSD`.
* stage2: access to the windows_sdk functions is done with a manually
maintained .zig binding file instead of `@cImport`.
* Update src-self-hosted/libc_installation.zig with all the
improvements that stage1 has seen to src/libc_installation.cpp until
now. In addition, it now takes advantage of Batch so that evented I/O
mode takes advantage of concurrency, but it still works in blocking
I/O mode, which is how it is used in stage1.
* `zig test` gainst `--test-evented-io` parameter and gains the ability
to seamlessly run async tests.
* `std.ChildProcess` opens its child process pipe with O_NONBLOCK when
using evented I/O
* `std.io.getStdErr()` gives a File that is blocking even in evented
I/O mode.
* Delete `std.event.fs`. The functionality is now merged into `std.fs`
and async file system access (using a dedicated thread) is
automatically handled.
* `std.fs.File` can be configured to specify whether its handle is
expected to block, and whether that is OK to block even when in
async I/O mode. This makes async I/O work correctly for e.g. the
file system as well as network.
* `std.fs.File` has some deprecated functions removed.
* Missing readv,writev,pread,pwrite,preadv,pwritev functions are added
to `std.os` and `std.fs.File`. They are all integrated with async
I/O.
* `std.fs.Watch` is still bit rotted and needs to be audited in light
of the new async/await syntax.
* `std.io.OutStream` integrates with async I/O
* linked list nodes in the std lib have default `null` values for
`prev` and `next`.
* Windows async I/O integration is enabled for reading/writing file
handles.
* Added `std.os.mode_t`. Integer sizes need to be audited.
* Fixed#4403 which was causing compiler to crash.
This is working towards:
./zig test ../test/stage1/behavior.zig --test-evented-io
Which does not successfully build yet. I'd like to enable behavioral
tests and std lib tests with --test-evented-io in the test matrix in the
future, to prevent regressions.
