This finishes LemonBoy's Draft PR ziglang#6750. It updates ChildProcess to collect the output from stdout/stderr asynchronously using Overlapped IO and named pipes.
Keep polling until there are enough open handles, if the child process
terminates closing the handles or explicitly closes them we just quit
polling and wait for the process handle to signal the termination
condition.
Reading stdin&stderr at different times may lead to nasty deadlocks (eg.
when stdout is read before stderr and the child process doesn't write
anything onto stdout).
Implement a polling mechanism to make sure this won't happen: we read
data from stderr/stdout as it becomes ready and then it's copied into an
ArrayList provided by the user, avoiding any kind of blocking read.
This patch adjusts the exit code for a child process to be a u8. Since
the WEXITSTATUS macro returns the lower eight bits, it's safe to assume
that we can truncate the returned u32.
* std: Better handing of POLLHUP in ChildProcess
Upon hitting the EOF condition there are two main differences between
how Linux and the *BSD-derived systems behave: the former sets POLLHUP
and POLLIN and, after reading any residual data, only POLLHUP remains
set. The latter signal the EOF condition by setting both flags thus
requiring some extra checks to determine if the stream is "done".
DragonFly workaround/hack for POLLHUP is no longer required.
Closes#8969
- hash/eql functions moved into a Context object
- *Context functions pass an explicit context
- *Adapted functions pass specialized keys and contexts
- new getPtr() function returns a pointer to value
- remove functions renamed to fetchRemove
- new remove functions return bool
- removeAssertDiscard deleted, use assert(remove(...)) instead
- Keys and values are stored in separate arrays
- Entry is now {*K, *V}, the new KV is {K, V}
- BufSet/BufMap functions renamed to match other set/map types
- fixed iterating-while-modifying bug in src/link/C.zig
- more support for linux, android, freebsd, netbsd, openbsd, dragonfly
- centralize musl utils; musl logic is no longer intertwined with csu
- fix musl compilation to build crti/crtn for full archs list
- fix openbsd to support `zig build-lib -dynamic`
- initial dragonfly linking success (with a warning)
ancillary:
- fix emutls (openbsd) tests to use `try`
Conflicts:
* doc/langref.html.in
* lib/std/enums.zig
* lib/std/fmt.zig
* lib/std/hash/auto_hash.zig
* lib/std/math.zig
* lib/std/mem.zig
* lib/std/meta.zig
* test/behavior/alignof.zig
* test/behavior/bitcast.zig
* test/behavior/bugs/1421.zig
* test/behavior/cast.zig
* test/behavior/ptrcast.zig
* test/behavior/type_info.zig
* test/behavior/vector.zig
Master branch added `try` to a bunch of testing function calls, and some
lines also had changed how to refer to the native architecture and other
`@import("builtin")` stuff.
* read directly into the ArrayList buffers.
* respect max_output_bytes
* std.ArrayList:
- make `allocatedSlice` public.
- add `unusedCapacitySlice`.
I removed the Windows implementation of this stuff; I am doing a partial
merge of LemonBoy's patch with the understanding that a later patch can
add the Windows implementation after it is vetted.
Keep polling until there are enough open handles, if the child process
terminates closing the handles or explicitly closes them we just quit
polling and wait for the process handle to signal the termination
condition.
Reading stdin&stderr at different times may lead to nasty deadlocks (eg.
when stdout is read before stderr and the child process doesn't write
anything onto stdout).
Implement a polling mechanism to make sure this won't happen: we read
data from stderr/stdout as it becomes ready and then it's copied into an
ArrayList provided by the user, avoiding any kind of blocking read.
We were violating the POSIX standard which resulted in a deadlock on
musl v1.1.24 on aarch64 alpine linux, uncovered with the new ThreadPool
usage in the stage2 compiler.
std.os execv functions that accept an Allocator parameter are removed
because they are footguns. The POSIX standard does not allow calls to
malloc() between fork() and execv() and since it is common to both
(1) call execv() after fork() and (2) use std.heap.c_allocator,
Programmers are encouraged to go through the `std.process` API
instead, causing some dissonance when combined with `std.os` APIs.
I also slapped a big warning message on all the relevant doc comments.
Comment reproduced here:
If we're linking libc, some naughty applications may have
registered atexit handlers which we really do not want to
run in the fork child. I caught LLVM doing this and it
caused a deadlock instead of doing an exit syscall. In
the words of Avril Lavigne, "Why'd you have to go and
make things so complicated?"
Before merging, do this for every item in the file:
* solve the issue, or
* convert the task to a github issue and update the comment
to link to the issue (and remove "TODO" text from the comment).
Then delete the file.
Related: #363
- correct uid_t from i32 to u32 on linux
- define uid_t and gid_t for OSes missing definitions
- use uid_t/gid_t instead of plain u32s throughout std.os
Replace them with `std.os.windows.OpenFile` instead. To allow
creation/opening of directories, `std.os.windows.OpenFileOptions`
now features a `.expect_dir: bool` member which is meant to emualate
POSIX's `O_DIRECTORY` flag.
