The original impetus for making a change here was a typo in --add-header
causing the script to fail. However, upon inspection, I was alarmed that
we were making a --recursive upload to the *root directory* of
ziglang.org. This could result in garbage files being uploaded to the
website, or important files being overwritten. As I addressed this concern,
I decided to take on file compression as well.
Removed compression prior to sending to S3. I am vetoing pre-compressing
objects for the following reasons:
* It prevents clients from working which do not support gzip encoding.
* It breaks a premise that objects on S3 are stored 1-to-1 with what is
on disk.
* It prevents Cloudflare from using a more efficient encoding, such as
brotli, which they have started doing recently.
These systems such as Cloudflare or Fastly already do compression on
the fly, and we should interop with these systems instead of fighting them.
Cloudfront has an arbitrary limit of 9.5 MiB for auto-compression. I looked
and did not see a way to increase this limit. The data.js file is currently
16 MiB. In order to fix this problem, we need to do one of the following things:
* Reduce the size of data.js to less than 9.5 MiB.
* Figure out how to adjust the Cloudfront settings to increase the max size
for auto-compressed objects.
* Migrate to Fastly. Fastly appears to not have this limitation. Note
that we already plan to migrate to Fastly for the website.
NetBSD CI is disabled because it is not yet supported in
zig-bootstrap. Once NetBSD has proper zig-bootstrap support, it can be
re-enabled.
Windows is not solved here yet; will be pushing a separate commit for
that.
* remove unused download page html. It's now handled in the
www.ziglang.org website repo.
* add netbsd to the downloads index.json file that we send to
the www.ziglang.org website repo.
* shallow clone the website repo to avoid downloading old copies of
data.js unnecessarily.
* no longer depend on apt.llvm.org, instead we rely on a zig-bootstrap
tarball with pre-built Zig, LLVM, LLD, and Clang. Similar to the
Windows Dev Kit but for Linux. This also makes the script no longer
depend on Docker, libxml2, or GCC.
* remove bash retry logic; it was only needed for flaky apt.llvm.org
and isn't needed for the other resources we download.
* build and upload the experimental std lib automatically generated
docs. langref.html is moved to docs/langref.html because the std lib
docs are in docs/std/.
* the superfluous "zig" directory in $prefix/lib/zig/std/std.zig is
removed from the tarball.
* update_download_page makes a commit in the www.ziglang.org repository
updating data/releases.json so that repository can manage deploys.
This reverts commit c0c9303bd63468afe146ad729cf963ee06a1777f.
Thanks to commit c08444b33ff49981cd0a39cced479e2197dc4fd7 we can now
re-enable FreeBSD in the CI.
Each of the 3 CI services now trigger a sr.ht build via
the on_master_success script. The sr.ht build checks if all builds have
completed successfully by trying to download the JSON file for the
particular version. If all builds have completed successfully then this
sr.ht job will update the download page.
