I had accidentally regressed support for -gdwarf in 461fb499f3cff9038a427eae120fb34defc9ab38 when I changed the logic to
use the already-mapped exe/dll image instead of loading it from disk. The string table is mapped as all zeroes by the loader,
so if a section header's name is longer than 8 bytes (like the ones generated by -gdwarf), then the name can't be read.
Now, if any section headers require the string table, the file is mapped from disk.
windows: Add NtCreateSection/NtMapViewOfSection/NtUnmapViewOfSection
This allows doing Windows-style case insensitive comparisons from any target, but means that it will need to include its own copy of the uppercase data table (5,088 bytes) to do so.
When targeting Windows, the ntdll functions are used instead to avoid including a redundant copy of the uppercase data in the resulting binary.
Most of this migration was performed automatically with `zig fmt`. There
were a few exceptions which I had to manually fix:
* `@alignCast` and `@addrSpaceCast` cannot be automatically rewritten
* `@truncate`'s fixup is incorrect for vectors
* Test cases are not formatted, and their error locations change
There are many different types of Windows paths, and there are a few different possible namespaces on top of that. Before this commit, NT namespaced paths were somewhat supported, and for Win32 paths (those without a namespace prefix), only relative and drive absolute paths were supported. After this commit, all of the following are supported:
- Device namespaced paths (`\\.\`)
- Verbatim paths (`\\?\`)
- NT-namespaced paths (`\??\`)
- Relative paths (`foo`)
- Drive-absolute paths (`C:\foo`)
- Drive-relative paths (`C:foo`)
- Rooted paths (`\foo`)
- UNC absolute paths (`\\server\share\foo`)
- Root local device paths (`\\.` or `\\?` exactly)
Plus:
- Any of the path types and namespace types can be mixed and matched together as appropriate.
- All of the `std.os.windows.*ToPrefixedFileW` functions will accept any path type, prefixed or not, and do the appropriate thing to convert them to an NT-prefixed path if necessary.
This is achieved by making the `std.os.windows.*ToPrefixedFileW` functions behave like `ntdll.RtlDosPathNameToNtPathName_U`, but with a few differences:
- Does not allocate on the heap (this is why we can't use `ntdll.RtlDosPathNameToNtPathName_U` directly, it does internal heap allocation).
- Relative paths are kept as relative unless they contain too many .. components, in which case they are treated as 'drive relative' and resolved against the CWD (this is how it behaved before this commit as well).
- Special case device names like COM1, NUL, etc are not handled specially (TODO)
- `.` and space are not stripped from the end of relative paths (potential TODO)
Most of the non-trivial conversion of non-relative paths is done via `ntdll.RtlGetFullPathName_U`, which AFAIK is used internally by `ntdll.RtlDosPathNameToNtPathName_U`.
Some relevant reading on Windows paths:
- https://googleprojectzero.blogspot.com/2016/02/the-definitive-guide-on-win32-to-nt.html
- https://chrisdenton.github.io/omnipath/Overview.htmlCloses#8205
Might close (untested) #12729
Note:
- This removes checking for illegal characters in `std.os.windows.sliceToPrefixedFileW`, since the previous solution (iterate the whole string and error if any illegal characters were found) was naive and won't work for all path types. This is further complicated by things like file streams (where `:` is used as a delimiter, e.g. `file.ext:stream_name:$DATA`) and things in the device namespace (where a path like `\\.\GLOBALROOT\??\UNC\localhost\C$\foo` is valid despite the `?`s in the path and is effectively equivalent to `C:\foo`). Truly validating paths is complicated and would need to be tailored to each path type. The illegal character checking being removed may open up users to more instances of hitting `OBJECT_NAME_INVALID => unreachable` when using `fs` APIs.
+ This is related to https://github.com/ziglang/zig/issues/15607
ccf670c made using `return` from within a comptime block in a non-inline
function illegal, since it is a use of runtime control flow in a
comptime block. It is allowed if the function in question is `inline`,
since no actual control flow occurs in this case. A few functions from
std (notably `std.fmt.comptimePrint`) needed to be marked `inline` to
support this change.
`GetPhysicallyInstalledSystemMemory` uses SMBios to grab the physical
memory size which can lead to unecessary allocation and inacurate
representation of the total memory. Using `System_Basic_Information`
help to retrieve the physical memory which is not reserved for the
kernel/tables. This aligns better with the linux side as `/proc/meminfo`
does the same thing.
`GetProcessMemoryInfo` is implemented using `NtQueryInformationProcess`
with `ProcessVmCounters` to obtain `VM_COUNTERS`. The structs, enum
definitions are found in `winternl.h` or `ntddk.h` in the latest WDK.
This should give the same results as using `K32GetProcessMemoryInfo`
* Fix GetFileInformationByHandle compile error
The wrapper function was mistakenly referencing ntdll.zig when the actual function is declared in kernel32.zig.
* delete GetFileInformationByHandle since it's not used by the stdlib
- Fixes the first few code units of the name being omitted (it was using `@sizeOf(FILE_NAME_INFO)` as the start of the name bytes, but that includes the length of the dummy [1]u16 field and padding; instead the start should be the offset of the dummy [1]u16 field)
- Replaces kernel32.GetFileInformationByHandleEx call with ntdll.NtQueryInformationFile
+ Contributes towards #1840
- Checks that the handle is a named pipe first before querying and checking the name, which is a much faster call than NtQueryInformationFile (this was about a 10x speedup in my probably-not-so-good/take-it-with-a-grain-of-salt benchmarking)
This fixes a bug in std.net caused during the introduction of
meta.assumeSentinel due to the unfortunate semantics of mem.span()
This leaves only 3 remaining uses of meta.assumeSentinel() in the
standard library, each of which could be a simple @ptrCast([*:0]T, foo)
instead. I think this function should likely be removed.
windows: add RtlCaptureContext, RtlLookupFunctionEntry, RtlVirtualUnwind and supporting types
windows: fix alignment of CONTEXT structs to match winnt.h as required by RtlCaptureContext (fxsave instr)
windows aarch64: fix __chkstk being defined twice if libc is not linked on msvc
Co-authored-by: Jakub Konka <kubkon@jakubkonka.com>
There are still a few occurrences of "stage1" in the standard library
and self-hosted compiler source, however, these instances need a bit
more careful inspection to ensure no breakage.
alongside the typical msghdr struct, Zig has added a msghdr_const
type that can be used with sendmsg which allows const data to
be provided. I believe that data pointed to by the iov and control
fields in msghdr are also left unmodified, in which case they can
be marked const as well.
EnvMap provides the same API as the previously used BufMap (besides `putMove` and `getPtr`), so usage sites of `getEnvMap` can usually remain unchanged.
For non-Windows, EnvMap is a wrapper around BufMap. On Windows, it uses a new EnvMapWindows to handle some Windows-specific behavior:
- Lookups use Unicode-aware case insensitivity (but `get` cannot return an error because EnvMapWindows has an internal buffer to use for lookup conversions)
- Canonical names are returned when iterating the EnvMap
Fixes#10561, closes#4603
Windows does Unicode-aware case-insensitivity comparisons for environment variable names. Before, os.getenvW was only doing ASCII case-insensitivity. We can take advantage of RtlEqualUnicodeString in NtDll to get the proper Unicode case insensitivity.
