In https://github.com/ziglang/zig/pull/19641, this binding changed from `[*]u16` to `LPWSTR` which made it a sentinel-terminated pointer. This introduced a compiler error in the `std.os.windows.GetModuleFileNameW` wrapper since it takes a `[*]u16` pointer. This commit changes the binding back to what it was before instead of introducing a breaking change to `std.os.windows.GetModuleFileNameW`
Related: https://github.com/ziglang/zig/issues/20858
To facilitate #1840, this commit slims `std.windows.kernel32` to only
have the functions needed by the standard library. Since this will break
projects that relied on these, I offer two solutions:
- Make an argument as to why certain functions should be added back in.
Note that they may just be wrappers around `ntdll` APIs, which would
go against #1840.
If necessary I'll add them back in *and* make wrappers in
`std.windows` for it.
- Maintain your own list of APIs. This is the option taken by bun[1],
where they wrap functions with tracing.
- Use `zigwin32`.
I've also added TODO comments that specify which functions can be
reimplemented using `ntdll` APIs in the future.
Other changes:
- Group functions into groups (I/O, process management etc.).
- Synchronize definitions against Microsoft documentation to use the
proper parameter types/names.
- Break all functions with parameters over multiple lines.
- From lib/libc/include/any-windows-any/wincon.h#L235
- See also https://learn.microsoft.com/en-us/windows/console/setconsolemode
- Also add DISABLE_NEWLINE_AUTO_RETURN constant which will be used by SetConsoleMode in lib/std/os/windows.
Co-authored-by: Kexy Biscuit <kexybiscuit@biscuitt.in>
> Note: This first part is mostly a rephrasing of https://flatt.tech/research/posts/batbadbut-you-cant-securely-execute-commands-on-windows/
> See that article for more details
On Windows, it is possible to execute `.bat`/`.cmd` scripts via CreateProcessW. When this happens, `CreateProcessW` will (under-the-hood) spawn a `cmd.exe` process with the path to the script and the args like so:
cmd.exe /c script.bat arg1 arg2
This is a problem because:
- `cmd.exe` has its own, separate, parsing/escaping rules for arguments
- Environment variables in arguments will be expanded before the `cmd.exe` parsing rules are applied
Together, this means that (1) maliciously constructed arguments can lead to arbitrary command execution via the APIs in `std.process.Child` and (2) escaping according to the rules of `cmd.exe` is not enough on its own.
A basic example argv field that reproduces the vulnerability (this will erroneously spawn `calc.exe`):
.argv = &.{ "test.bat", "\"&calc.exe" },
And one that takes advantage of environment variable expansion to still spawn calc.exe even if the args are properly escaped for `cmd.exe`:
.argv = &.{ "test.bat", "%CMDCMDLINE:~-1%&calc.exe" },
(note: if these spawned e.g. `test.exe` instead of `test.bat`, they wouldn't be vulnerable; it's only `.bat`/`.cmd` scripts that are vulnerable since they go through `cmd.exe`)
Zig allows passing `.bat`/`.cmd` scripts as `argv[0]` via `std.process.Child`, so the Zig API is affected by this vulnerability. Note also that Zig will search `PATH` for `.bat`/`.cmd` scripts, so spawning something like `foo` may end up executing `foo.bat` somewhere in the PATH (the PATH searching of Zig matches the behavior of cmd.exe).
> Side note to keep in mind: On Windows, the extension is significant in terms of how Windows will try to execute the command. If the extension is not `.bat`/`.cmd`, we know that it will not attempt to be executed as a `.bat`/`.cmd` script (and vice versa). This means that we can just look at the extension to know if we are trying to execute a `.bat`/`.cmd` script.
---
This general class of problem has been documented before in 2011 here:
https://learn.microsoft.com/en-us/archive/blogs/twistylittlepassagesallalike/everyone-quotes-command-line-arguments-the-wrong-way
and the course of action it suggests for escaping when executing .bat/.cmd files is:
- Escape first using the non-cmd.exe rules
- Then escape all cmd.exe 'metacharacters' (`(`, `)`, `%`, `!`, `^`, `"`, `<`, `>`, `&`, and `|`) with `^`
However, escaping with ^ on its own is insufficient because it does not stop cmd.exe from expanding environment variables. For example:
```
args.bat %PATH%
```
escaped with ^ (and wrapped in quotes that are also escaped), it *will* stop cmd.exe from expanding `%PATH%`:
```
> args.bat ^"^%PATH^%^"
"%PATH%"
```
but it will still try to expand `%PATH^%`:
```
set PATH^^=123
> args.bat ^"^%PATH^%^"
"123"
```
The goal is to stop *all* environment variable expansion, so this won't work.
Another problem with the ^ approach is that it does not seem to allow all possible command lines to round trip through cmd.exe (as far as I can tell at least).
One known example:
```
args.bat ^"\^"key^=value\^"^"
```
where args.bat is:
```
@echo %1 %2 %3 %4 %5 %6 %7 %8 %9
```
will print
```
"\"key value\""
```
(it will turn the `=` into a space for an unknown reason; other minor variations do roundtrip, e.g. `\^"key^=value\^"`, `^"key^=value^"`, so it's unclear what's going on)
It may actually be possible to escape with ^ such that every possible command line round trips correctly, but it's probably not worth the effort to figure it out, since the suggested mitigation for BatBadBut has better roundtripping and leads to less garbled command lines overall.
---
Ultimately, the mitigation used here is the same as the one suggested in:
https://flatt.tech/research/posts/batbadbut-you-cant-securely-execute-commands-on-windows/
The mitigation steps are reproduced here, noted with one deviation that Zig makes (following Rust's lead):
1. Replace percent sign (%) with %%cd:~,%.
2. Replace the backslash (\) in front of the double quote (") with two backslashes (\\).
3. Replace the double quote (") with two double quotes ("").
4. ~~Remove newline characters (\n).~~
- Instead, `\n`, `\r`, and NUL are disallowed and will trigger `error.InvalidBatchScriptArg` if they are found in `argv`. These three characters do not roundtrip through a `.bat` file and therefore are of dubious/no use. It's unclear to me if `\n` in particular is relevant to the BatBadBut vulnerability (I wasn't able to find a reproduction with \n and the post doesn't mention anything about it except in the suggested mitigation steps); it just seems to act as a 'end of arguments' marker and therefore anything after the `\n` is lost (and same with NUL). `\r` seems to be stripped from the command line arguments when passed through a `.bat`/`.cmd`, so that is also disallowed to ensure that `argv` can always fully roundtrip through `.bat`/`.cmd`.
5. Enclose the argument with double quotes (").
The escaped command line is then run as something like:
cmd.exe /d /e:ON /v:OFF /c "foo.bat arg1 arg2"
Note: Previously, we would pass `foo.bat arg1 arg2` as the command line and the path to `foo.bat` as the app name and let CreateProcessW handle the `cmd.exe` spawning for us, but because we need to pass `/e:ON` and `/v:OFF` to cmd.exe to ensure the mitigation is effective, that is no longer tenable. Instead, we now get the full path to `cmd.exe` and use that as the app name when executing `.bat`/`.cmd` files.
---
A standalone test has also been added that tests two things:
1. Known reproductions of the vulnerability are tested to ensure that they do not reproduce the vulnerability
2. Randomly generated command line arguments roundtrip when passed to a `.bat` file and then are passed from the `.bat` file to a `.exe`. This fuzz test is as thorough as possible--it tests that things like arbitrary Unicode codepoints and unpaired surrogates roundtrip successfully.
Note: In order for the `CreateProcessW` -> `.bat` -> `.exe` roundtripping to succeed, the .exe must split the arguments using the post-2008 C runtime argv splitting implementation, see https://github.com/ziglang/zig/pull/19655 for details on when that change was made in Zig.
* Fix GetFileInformationByHandle compile error
The wrapper function was mistakenly referencing ntdll.zig when the actual function is declared in kernel32.zig.
* delete GetFileInformationByHandle since it's not used by the stdlib
This fixes a bug in std.net caused during the introduction of
meta.assumeSentinel due to the unfortunate semantics of mem.span()
This leaves only 3 remaining uses of meta.assumeSentinel() in the
standard library, each of which could be a simple @ptrCast([*:0]T, foo)
instead. I think this function should likely be removed.
windows: add RtlCaptureContext, RtlLookupFunctionEntry, RtlVirtualUnwind and supporting types
windows: fix alignment of CONTEXT structs to match winnt.h as required by RtlCaptureContext (fxsave instr)
windows aarch64: fix __chkstk being defined twice if libc is not linked on msvc
Co-authored-by: Jakub Konka <kubkon@jakubkonka.com>
We already have a LICENSE file that covers the Zig Standard Library. We
no longer need to remind everyone that the license is MIT in every single
file.
Previously this was introduced to clarify the situation for a fork of
Zig that made Zig's LICENSE file harder to find, and replaced it with
their own license that required annual payments to their company.
However that fork now appears to be dead. So there is no need to
reinforce the copyright notice in every single file.
This finishes LemonBoy's Draft PR ziglang#6750. It updates ChildProcess to collect the output from stdout/stderr asynchronously using Overlapped IO and named pipes.
Reading stdin&stderr at different times may lead to nasty deadlocks (eg.
when stdout is read before stderr and the child process doesn't write
anything onto stdout).
Implement a polling mechanism to make sure this won't happen: we read
data from stderr/stdout as it becomes ready and then it's copied into an
ArrayList provided by the user, avoiding any kind of blocking read.
Socket I/O methods such as read, readv, write, writev, send, recv,
sendmsg, recvmsg have been generalized to read(buf, flags), write(buf,
flags), readVectorized(vectors, flags), and writeVectorized(vectors,
flags). There is still some work left to be done abstracting both
readVectorized and writeVectorized properly across platforms, which is
work to be done in a future PR.
Support for setting the linger timeout of a socket, querying the remote
address of a socket, setting whether or not keep-alive messages are to
be sent through a connection-oriented socket periodically depending on
host operating system settings has been added.
`std.io.Reader` and `std.io.Writer` wrappers around `Socket` has been
implemented, which wrap around Socket.read(buf, flags) and
Socket.write(buf, flags). Both wrappers may be provided flags which are
passed to Socket.read / Socket.write accordingly.
Cross-platform support for `getpeername()` has been implemented.
Windows support for the new `std.x.os.Socket` has been implemented. To
accomplish this, a full refactor of `std.os.windows.ws2_32` has been
done to supply any missing definitions and constants based on
auto-generated Windows syscall bindings by @marler8997.
`std.x.net.TCP.Listener.setQuickACK` has been moved to
`std.x.net.TCP.Client.setQuickACK`.
Windows support for resolving the scope ID of an interface name
specified in an IPv6 address has been provided.
`sockaddr_storage` definitions have been provided for Windows, Linux,
and Darwin. `sockaddr_storage` is used to allocate space before any
socket addresses are queried via. calls such as accept(), getsockname(),
and getpeername().
Zig-friendly wrappers for GetQueuedCompletionStatusEx(), getpeername(),
SetConsoleCtrlHandler(), SetFileCompletionNotificationModes() syscalls
on Windows have been provided.
Socket.setOption() was provided to set the value of a socket option in
place of os.setsockopt. Socket.getOption() will be provided in a future
PR.
There is still further work to be done regarding querying socket option
values on Windows, which is to be done in a subsequent PR.
* move concurrency primitives that always operate on kernel threads to
the std.Thread namespace
* remove std.SpinLock. Nobody should use this in a non-freestanding
environment; the other primitives are always preferable. In
freestanding, it will be necessary to put custom spin logic in there,
so there are no use cases for a std lib version.
* move some std lib files to the top level fields convention
* add std.Thread.spinLoopHint
* add std.Thread.Condition
* add std.Thread.Semaphore
* new implementation of std.Thread.Mutex for Windows and non-pthreads Linux
* add std.Thread.RwLock
Implementations provided by @kprotty
* remove GetModuleHandleA from kernel32.zig. use of A functions
considered harmful.
* make it a compile error to expose WinMain instead of wWinMain. same
thing.
* start code declares wWinMainCRTStartup instead of WinMainCRTStartup
when it has the choice.
