717 Commits

Author SHA1 Message Date
Ryan Liptak
5f15acc463 Add preliminary support for Windows .manifest files
An embedded manifest file is really just XML data embedded as a RT_MANIFEST resource (ID = 24). Typically, the Windows-only 'Manifest Tool' (`mt.exe`) is used to embed manifest files, and `mt.exe` also seems to perform some transformation of the manifest data before embedding, but in testing it doesn't seem like the transformations are necessary to get the intended result.

So, to handle embedding manifest files, Zig now takes the following approach:

- Generate a .rc file with the contents `1 24 "path-to-manifest.manifest"`
- Compile that generated .rc file into a .res file
- Link the .res file into the final binary

This effectively achieves the same thing as `mt.exe` minus the validation/transformations of the XML data that it performs.

How this is used:

On the command line:
```
zig build-exe main.zig main.manifest
```
(on the command line, specifying a .manifest file when the target object format is not COFF is an error)

or in build.zig:

```
const exe = b.addExecutable(.{
    .name = "manifest-test",
    .root_source_file = .{ .path = "main.zig" },
    .target = target,
    .optimize = optimize,
    .win32_manifest = .{ .path = "main.manifest" },
});
```
(in build.zig, the manifest file is ignored if the target object format is not COFF)

Note: Currently, only one manifest file can be specified per compilation. This is because the ID of the manifest resource is currently always 1. Specifying multiple manifests could be supported if a way for the user to specify an ID for each manifest is added (manifest IDs must be a u16).

Closes #17406

options
2023-10-15 13:33:16 -07:00
Ryan Liptak
2769215b90 Add zig rc subcommand, a drop-in replacement for rc.exe
Uses resinator under-the-hood (see https://github.com/ziglang/zig/pull/17069)

Closes #9564
2023-10-12 16:19:34 -07:00
Andrew Kelley
da7e4fb31a revert compiler_rt: no need to put it in a static library
This mostly reverts 6e0904504155d3cba80955c108116170fd739aec however it
leaves intact the linker supporting both obj and lib files, and the
frontend choosing which one to create.
2023-10-10 11:23:39 -07:00
Andrew Kelley
8679c7a607 Compilation: default to self-hosted backend when not using libllvm
When not using libllvm, it means the compiler is not capable of
producing an object file or executable, making the self-hosted backends
be a better default.
2023-10-10 11:17:29 -07:00
Andrew Kelley
6e09045041 compiler_rt: no need to put it in a static library
It's simpler to link against compiler_rt.o directly.
2023-10-10 11:17:29 -07:00
Andrew Kelley
5eb5d523b5 give modules friendly names for error reporting 2023-10-08 20:58:04 -07:00
Andrew Kelley
1b372f1872 fix using the wrong allocator for modules 2023-10-08 18:23:38 -07:00
Andrew Kelley
f48ec4b8ee use long-lived arena for @cImport-generated Module 2023-10-08 16:54:31 -07:00
Andrew Kelley
f708c5fafc CLI: finish updating module API usage
Finish the work started in 4c4fb839972f66f55aa44fc0aca5f80b0608c731.
Now the compiler compiles again.

Wire up dependency tree fetching code in the CLI for `zig build`.
Everything is hooked up except for `createDependenciesModule` is not yet
implemented.
2023-10-08 16:54:31 -07:00
Andrew Kelley
d0bcc390e8 get zig fetch working with the new system
* start renaming "package" to "module" (see #14307)
  - build system gains `main_mod_path` and `main_pkg_path` is still
    there but it is deprecated.
* eliminate the object-oriented memory management style of what was
  previously `*Package`. Now it is `*Package.Module` and all pointers
  point to externally managed memory.
* fixes to get the new Fetch.zig code working. The previous commit was
  work-in-progress. There are still two commented out code paths, the
  one that leads to `Compilation.create` and the one for `zig build`
  that fetches the entire dependency tree and creates the required
  modules for the build runner.
2023-10-08 16:54:31 -07:00
Jakub Konka
8b4e3b6aee comp: add support for -fdata-sections 2023-10-04 11:21:56 -07:00
Andrew Kelley
9d069d98e3 C backend: start handling anonymous decls
Start keeping track of dependencies on anon decls for dependency
ordering during flush()

Currently this causes use of undefined symbols because these
dependencies need to get rendered into the output.
2023-10-03 12:12:51 -07:00
Andrew Kelley
53775b0999 CLI: fix -fno-clang
Aro/Clang detection logic treated `-fno-clang` the same as `-fclang`.
2023-10-01 21:37:02 -07:00
Veikka Tuominen
fc4d53e2ea
Merge pull request #17221 from Vexu/aro-translate-c
Aro translate-c
2023-10-02 07:08:53 +03:00
Veikka Tuominen
31ecf75311 aro-translate-c: translate enums 2023-10-01 23:51:54 +03:00
Veikka Tuominen
fef94da958 add compiler flag for selecting C frontend 2023-10-01 23:51:54 +03:00
Jacob Young
3bd1b9e15f x86_64: implement and test unary float builtins 2023-10-01 15:09:52 -04:00
Andrew Kelley
7c605ba62c C backend: remove ?*Decl from DeclGen
Another simplification. DeclGen already has `decl_index` which can be
used to retrieve the `*Decl` if needed.
2023-09-29 19:14:17 -07:00
Andrew Kelley
8f2f12f940 Compilation: introduce saveState()
This commit introduces `--debug-incremental` so that we can start
playing around with incremental compilation while it is still being
developed, and before it is enabled by default.

Currently it saves InternPool data, and has TODO comments for the
remaining things. Deserialization is not implemented yet, which will
require some post-processing such as to build a string map out of
null-terminated string table bytes.

The saved compiler state is stored in a file called <root-name>.zcs
alongside <root-name>.o, <root-name>.pdb, <root-name>.exe, etc. In case
of using the zig build system, these files are all in a zig-cache
directory.

For the self-hosted compiler, here is one data point on the performance
penalty of saving this data:

```
Benchmark 1 (3 runs): zig build-exe ...
  measurement          mean ± σ            min … max           outliers         delta
  wall_time          51.1s  ±  354ms    50.7s  … 51.4s           0 ( 0%)        0%
  peak_rss           3.91GB ±  354KB    3.91GB … 3.91GB          0 ( 0%)        0%
  cpu_cycles          212G  ± 3.17G      210G  …  216G           0 ( 0%)        0%
  instructions        274G  ± 57.5M      274G  …  275G           0 ( 0%)        0%
  cache_references   13.1G  ± 97.6M     13.0G  … 13.2G           0 ( 0%)        0%
  cache_misses       1.12G  ± 24.6M     1.10G  … 1.15G           0 ( 0%)        0%
  branch_misses      1.53G  ± 1.46M     1.53G  … 1.53G           0 ( 0%)        0%
Benchmark 2 (3 runs): zig build-exe ... --debug-incremental
  measurement          mean ± σ            min … max           outliers         delta
  wall_time          51.8s  ±  271ms    51.5s  … 52.1s           0 ( 0%)          +  1.3% ±  1.4%
  peak_rss           3.91GB ±  317KB    3.91GB … 3.91GB          0 ( 0%)          -  0.0% ±  0.0%
  cpu_cycles          213G  ±  398M      212G  …  213G           0 ( 0%)          +  0.3% ±  2.4%
  instructions        275G  ± 79.1M      275G  …  275G           0 ( 0%)          +  0.1% ±  0.1%
  cache_references   13.1G  ± 26.9M     13.0G  … 13.1G           0 ( 0%)          -  0.1% ±  1.2%
  cache_misses       1.12G  ± 5.66M     1.11G  … 1.12G           0 ( 0%)          -  0.6% ±  3.6%
  branch_misses      1.53G  ± 1.75M     1.53G  … 1.54G           0 ( 0%)          +  0.2% ±  0.2%
```

At the end of each compilation with `--debug-incremental`, we end up
with a 43 MiB `zig.zcs` file that contains all of the InternPool data
serialized.

Of course, it will necessarily be more expensive to save the state than
to not save the state. However, this data point shows just how cheap the
save state operation is, with all of the groundwork laid for using a
serialization-friendly in-memory data layout.
2023-09-28 16:28:41 -07:00
Andrew Kelley
5d907171e2
Merge pull request #17152 from mikdusan/macos-sdk
macos SDK updates and enhancements
2023-09-26 13:16:07 -07:00
Phil Richards
15ce965252
define _WIN32_WINNT for windows compilations based on target minver (#17224) 2023-09-26 15:25:08 -04:00
Michael Dusan
ebd0776b28
kubkon review changes: 3
- make vendored settings failure unreachable
- rename field `darwinSdkLayout` → `darwin_sdk_layout`
- make `darwin_sdk_layout` optional
2023-09-25 17:07:41 -04:00
Michael Dusan
9357973912
kubkon review changes: 1
general:
- rename `DarwinSdkLayout` → `DarwinSdkLayout`
- drop `DarwinSdkLayout.installation` (not needed for darwin)
- document struct

inferSdkVersion:
- use explicit allocator
- avoid trying to infer SDK ver from vendored path
2023-09-25 15:53:05 -04:00
Michael Dusan
5d6521d281
macos: better SDK version detection
SDK version detection:
- read SDKSettings.json before inferral from SDK path
- vendored libc: add SDKSettings.json for SDK version info

resolveLibSystem:
- adjust search order to { search_dirs, { sysroot or vendored }}
- previous search order was { sysroot, search_dirs, vendored }
2023-09-25 15:53:05 -04:00
Michael Dusan
15fd7cd154
macos: vendored libc: combine headers: part 2
- update include dirs to use combined dir
- use one libSystem.tbd (drop use of libSystem.VERSION.tbd)
- update canBuildLibC to check for minimum os version only
2023-09-25 15:53:05 -04:00
Techatrix
2adb932ad6 translate-c: convert clang errors messages into std.zig.ErrorBundle 2023-09-25 18:10:44 +03:00
mlugg
fb6fff2561 resinator: do not include in only_core_functionality builds
This prevents resinator from being included in zig1 and zig2.
2023-09-24 06:57:11 +01:00
Ryan Liptak
865b2e259b Fix reportRetryableWin32ResourceError
Follow up to https://github.com/ziglang/zig/pull/17069.

This TODO being left in was a complete oversight.

Before, any 'retryable' error would hit:

error: thread 2920 panic: access of union field 'success' while field 'failure_retryable' is active

Now, it will be reported/handled properly:

C:\Users\Ryan\Programming\Zig\zig\test\standalone\windows_resources\res\zig.rc:1:1: error: FileNotFound
2023-09-23 11:01:02 -07:00
Andrew Kelley
3fc7413574
Merge pull request #17069 from squeek502/resinator
Add a `.rc` -> `.res` compiler to the Zig compiler
2023-09-22 12:18:50 -07:00
Ryan Liptak
572956ce24 classifyFileExt: Use case-insensitive matching for .rc and .res 2023-09-21 21:30:21 -07:00
Andrew Kelley
66312c0b51 clang: -fno-sanitize=function workaround
It is very common, and well-defined, for a pointer on one side of a C ABI
to have a different but compatible element type. Examples include:

- `char*` vs `uint8_t*` on a system with 8-bit bytes
- `const char*` vs `char*`
- `char*` vs `unsigned char*`

Without this flag, Clang would invoke UBSAN when such an extern
function was called.

Might be nice to file an upstream issue and find out if there is a more
precise way to disable the problematic check.
`-fsanitize-cfi-icall-generalize-pointers` looks promising according to
the documentation, but empirically it does not work.
2023-09-19 09:37:53 -07:00
Andrew Kelley
5d4439cc3e libcxx: update to LLVM 17
release/17.x branch, commit 8f4dd44097c9ae25dd203d5ac87f3b48f854bba8

This adds the flag `-D_LIBCPP_PSTL_CPU_BACKEND_SERIAL`. A future
enhancement could possibly pass something different if there is a
compelling parallel implementation. That libdispatch one might be worth
looking into.
2023-09-19 09:37:31 -07:00
Ryan Liptak
471f279cd6 Fix rc preprocessing when using the MinGW includes and targeting the GNU abi
Also update the standalone test so that this failure would have been detected on any host system.
2023-09-17 13:09:16 -07:00
Ryan Liptak
0168ed7bf1 rc compilation: Use MSVC includes if present, fallback to mingw
The include directories used when preprocessing .rc files are now separate from the target, and by default will use the system MSVC include paths if the MSVC + Windows SDK are present, otherwise it will fall back to the MinGW includes distributed with Zig. This default behavior can be overridden by the `-rcincludes` option (possible values: any (the default), msvc, gnu, or none).

This behavior is useful because Windows resource files may `#include` files that only exist with in the MSVC include dirs (e.g. in `<MSVC install directory>/atlmfc/include` which can contain other .rc files, images, icons, cursors, etc). So, by defaulting to the `any` behavior (MSVC if present, MinGW fallback), users will by default get behavior that is most-likely-to-work.

It also should be okay that the include directories used when compiling .rc files differ from the include directories used when compiling the main binary, since the .res format is not dependent on anything ABI-related. The only relevant differences would be things like `#define` constants being different values in the MinGW headers vs the MSVC headers, but any such differences would likely be a MinGW bug.
2023-09-17 03:09:58 -07:00
Ryan Liptak
4fac7a5263 Only populate rc_include_dirs if there are .rc files in the compilation 2023-09-17 03:09:45 -07:00
Ryan Liptak
28f6559947 Add the ATLMFC include directory to the libc include dir list
https://learn.microsoft.com/en-us/cpp/mfc/mfc-and-atl

Note that this include directory gets added to %INCLUDE% by vcvarsall.bat, and is especially crucial when working with resource files (many .rc files within the https://github.com/microsoft/Windows-classic-samples/ set reference files from the ATLMFC include directory).
2023-09-17 03:09:45 -07:00
Ryan Liptak
2a56fe1175 Add a .rc -> .res compiler to the Zig compiler 2023-09-17 03:09:45 -07:00
Ryan Liptak
8e35be0640 ErrorBundle: rename addBundle to addBundleAsNotes, add addBundleAsRoots 2023-09-15 23:36:44 -07:00
Ryan Liptak
30e1883834 Add -includes option to zig libc
Prints the detected libc include paths for the target and exits
2023-09-14 11:04:34 -07:00
Michael Dusan
64d03faae7 Compilation: support --verbose_llvm_cpu_features 2023-09-04 22:31:39 +02:00
Jakub Konka
f4c9e19bc3
Merge pull request #17020 from ziglang/macho-versions
macho: big-ish refactor and report errors to the user using Zig's API
2023-08-31 07:50:29 +02:00
Jakub Konka
05c9d6c00b macho: add simple error reporting for misc errors 2023-08-29 11:39:34 +02:00
Jacob Young
7a251c4cb8 Sema: revert reference trace changes that are no longer needed 2023-08-28 17:43:37 -04:00
Jakub Konka
b73ef34289 frontend: directly pass resolved frameworks container to the linker
We can infer the framework name from the included resolved framework
path.

Fix hash implementation, and bump linker hash value from 9 to 10.
2023-08-21 08:07:44 +02:00
Jakub Konka
4793dafa04 frontend: move framework path resolution from the linker to frontend 2023-08-20 10:43:20 +02:00
Jakub Konka
01836c7bbe comp: make --verbose-cc actually thread-safe 2023-08-16 18:04:10 -07:00
Andrew Kelley
b820d5df79
Merge pull request #16747 from jacobly0/llvm-wo-libllvm
llvm: enable the backend even when not linked to llvm
2023-08-10 12:02:57 -07:00
Zachary Raineri
0461a64a93
change uses of std.builtin.Mode to OptimizeMode (#16745)
std.builtin.Mode is deprecated.
2023-08-09 14:39:34 -04:00
Jacob Young
c4848694d2 llvm: enable even without libllvm linked 2023-08-09 05:47:13 -04:00
Andrew Kelley
4923e64199 Compilation: detect sysroot from libc installation 2023-08-03 09:52:15 -07:00