Skip to content

Commit

Permalink
Add support for building with Zig
Browse files Browse the repository at this point in the history
This adds a build.zig that allows building the garbage collector using
the Zig build system. It implements a subset of the configuration
options offered by CMake and following the naming convention in CMake
for all those options.

There is a new GitHub Actions configuration file that runs a job that
uses zig to compile the collector and targetting different targets using
cross-compilation. It doesn't run any actual tests (and it should!).

The README is slightly updated, not just adding an explanation of zig
but restructuring the whole section on how to build the GC somewhat.

The build.zig follows the format used in zig v0.12. The final 0.12
version is not out yet, so users will have to use a build from the zig
master branch for now. This is not ideal but the whole zig build system
is so new that there have been backwards incompatible changes from 0.11
which is why we opt to use 0.12 that is likely going to be closer to the
final form.
  • Loading branch information
plajjan committed Dec 15, 2023
1 parent 14ebfe7 commit 4934489
Show file tree
Hide file tree
Showing 3 changed files with 290 additions and 36 deletions.
29 changes: 29 additions & 0 deletions .github/workflows/zig-build.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
# This workflow uses Zig and its excellent cross-compilation support to test
# compiling for multiple platforms. No tests are actually run since it would
# require emulation. We should add a test for the native platform though.
name: zig build

on: [push, pull_request]

jobs:
build:
name: ${{ matrix.ttriple }}
runs-on: ubuntu-latest

strategy:
fail-fast: false
matrix:
ttriple: [ aarch64-linux-musl, wasm32-wasi, x86_64-linux-gnu.2.27, x86_64-linux-musl, x86_64-windows-gnu ]

steps:
- uses: actions/checkout@v4
- name: "Install zig"
run: |
curl -o zig.tar.xz https://ziglang.org/builds/zig-linux-x86_64-0.12.0-dev.1814+5c0d58b71.tar.xz
mkdir -p zig
cd zig
tar Jx --strip-components=1 -f ../zig.tar.xz
cd ..
- name: Build
run: |
zig/zig build -Dtarget=${{ matrix.ttriple }}
122 changes: 86 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -172,7 +172,29 @@ stored on the thread's stack for the duration of their lifetime.
(This is arguably a longstanding bug, but it hasn't been fixed yet.)


## Installation and Portability
## Building and Installing

There are multiple ways to build the collector:

- CMake
- GNU autoconf
- Zig
- somewhat experimental support for using the Zig build system
- Zig is excellent at cross-compilation
- Makefile
- a static Makefile that offers a subset of configurable options
- Manual C compilation

### Configurable Macros

The library can be configured more precisely during the build by defining
the macros listed in [README.macros](docs/README.macros) file.

The library is built with threads support enabled (i.e. for thread-safe
operation) by default, unless explicitly disabled:
- `--disable-threads` is passed to `./configure`
- `-Denable_threads=OFF` is passed to `cmake`
- `-Denable_threads=false` is passed to `zig build`

The collector operates silently in the default configuration.
In the event of issues, this can usually be changed by defining the
Expand All @@ -183,53 +205,56 @@ Things don't appear to add up for a variety of reasons, most notably
fragmentation losses. These are probably much more significant for the
contrived program `gctest` than for your application.)

On most Unix-like platforms, the collector can be built either using a
GNU autoconf-based build infrastructure (type `./configure; make` in the
simplest case), or using CMake (see the sample below), or with a classic
makefile by itself (type `make -f Makefile.direct`).
### GNU Autoconf

Please note that the collector source repository does not contain configure
Please note that the collector source repository does not contain `configure`
and similar auto-generated files, thus the full procedure of autoconf-based
build of `master` branch of the collector could look like:

git clone https://github.com/ivmai/bdwgc
cd bdwgc
git clone https://github.com/ivmai/libatomic_ops
./autogen.sh
./configure
make -j
make check
``` sh
git clone https://github.com/ivmai/bdwgc
cd bdwgc
./autogen.sh
./configure
make -j
make check
```

Cloning of `libatomic_ops` is now optional provided the compiler supports
atomic intrinsics. See [README.autoconf](docs/README.autoconf) for details.

As noted above, alternatively, the collector could be built with CMake, like
this:
### CMake

mkdir out
cd out
cmake -Dbuild_tests=ON ..
cmake --build .
ctest
```sh
mkdir out && cd out
cmake -Dbuild_tests=ON ..
cmake --build .
ctest
```

See [README.cmake](docs/README.cmake) for details.

Finally, on most targets, the collector could be built and tested directly
with a single compiler invocation, like this:
### Zig

gcc -I include -o gctest tests/gctest.c extra/gc.c && ./gctest
Building using zig is in its simplest form straight forward:

On Windows, CMake could be used to build the library as described above or
by typing `nmake -f NT_MAKEFILE`, this assumes you have Microsoft command-line
tools installed and suitably configured. See
[README.win32](docs/platforms/README.win32) for details.
```sh
zig build
```

The library is built with threads support on (i.e. for thread-safe operation)
by default, unless `--disable-threads` is passed to `./configure` (or
`-Denable_threads=OFF` is passed to `cmake` tool).
It is possible to configure the build through the use of variables, for example
`zig build -Denable_redirect_malloc -Denable_threads=false`. Zig offers
excellent cross-compilation functionality, for example to compile the collector
for MacOS on Apple Silicon (M1 / M2 / M3):

The library could be configured more precisely during the build by defining
the macros listed in [README.macros](docs/README.macros) file.
```sh
zig build -Dtarget=aarch64-macos-gnu
```

Currently, a nightly version of zig 0.12 is required, which can be downloaded
from https://ziglang.org/download/

### Makefile

Below we focus on the collector build using classic makefile. For the
Makefile.direct-based process, typing `make check` instead of `make` will
Expand All @@ -243,19 +268,44 @@ desktops. It may use up to about 30 MB of memory. (The multi-threaded
version will use more. 64-bit versions may use more.) `make check` will also,
as its last step, attempt to build and test the "cord" string library.)

Makefile.direct will generate a library libgc.a which you should link against.
Typing `make -f Makefile.direct cords` will build the cord library (libcord.a)
as well.

The GNU style build process understands the usual targets. `make check`
runs a number of tests. `make install` installs at least libgc, and libcord.
Try `./configure --help` to see the configuration options. It is currently
not possible to exercise all combinations of build options this way.

Makefile.direct will generate a library libgc.a which you should link against.
Typing `make -f Makefile.direct cords` will build the cord library (libcord.a)
as well.


### Manual C compilation

Finally, on most targets, the collector could be built and tested directly
with a single compiler invocation, like this:

``` sh
cc -I include -o gctest tests/gctest.c extra/gc.c && ./gctest
```

### Windows nmake

On Windows, CMake could be used to build the library as described above or
by typing `nmake -f NT_MAKEFILE`, this assumes you have Microsoft command-line
tools installed and suitably configured. See
[README.win32](docs/platforms/README.win32) for details.

All include files that need to be used by clients will be put in the
include subdirectory. (Normally this is just gc.h. `make cords` adds
"cord.h" and "ec.h".)

### Atomic ops

The GC requires atomic ops. Most modern compilers offer builtin atomics. In case
your compiler does not, you can download and use `libatomic_ops` from
https://github.com/ivmai/libatomic_ops

## Portability

The collector currently is designed to run essentially unmodified on
machines that use a flat 32-bit or 64-bit address space.
That includes the vast majority of Workstations and x86 (i386 or later) PCs.
Expand Down
175 changes: 175 additions & 0 deletions build.zig
Original file line number Diff line number Diff line change
@@ -0,0 +1,175 @@
// THIS MATERIAL IS PROVIDED AS IS, WITH ABSOLUTELY NO WARRANTY EXPRESSED
// OR IMPLIED. ANY USE IS AT YOUR OWN RISK.
//
// Permission is hereby granted to use or copy this program
// for any purpose, provided the above notices are retained on all copies.
// Permission to modify the code and to distribute modified code is granted,
// provided the above notices are retained, and a notice that the code was
// modified is included with the above copyright notice.

const std = @import("std");
const print = @import("std").debug.print;

pub fn build(b: *std.build.Builder) void {
const optimize = b.standardOptimizeOption(.{});
const target = b.standardTargetOptions(.{});
const t = target.toTarget();
var default_enable_threads = true;
if (t.isWasm()) { // matches both emscripten and wasi
default_enable_threads = false;
}
const enable_threads = b.option(bool, "enable_threads", "Support threads") orelse default_enable_threads;
const enable_parallel_mark = b.option(bool, "enable_parallel_mark", "Parallelize marking and free list construction") orelse true;
const enable_thread_local_alloc = b.option(bool, "enable_thread_local_alloc", "Turn on thread-local allocation optimization") orelse true;
const enable_gcj_support = b.option(bool, "enable_gcj_support", "Support for gcj") orelse true;
const enable_java_finalization = b.option(bool, "enable_java_finalization", "Support for java finalization") orelse true;
const enable_atomic_uncollectable = b.option(bool, "enable_atomic_uncollectable", "Support for atomic uncollectible allocation") orelse true;
const enable_redirect_malloc = b.option(bool, "enable_redirect_malloc", "Redirect malloc and friend to GC routines") orelse false;
const enable_disclaim = b.option(bool, "enable_disclaim", "Support alternative finalization interface") orelse true;
const enable_large_config = b.option(bool, "enable_large_config", "Optimize for large heap or root set") orelse true;
const enable_mmap = b.option(bool, "enable_mmap", "Use mmap instead of sbrk to expand the heap") orelse true;
const enable_munmap = b.option(bool, "enable_munmap", "Return page to the OS if empty for N collections") orelse true;

const lib = b.addStaticLibrary(.{
.name = "gc",
.target = target,
.optimize = optimize,
});

var flags = std.ArrayList([]const u8).init(b.allocator);
defer flags.deinit();

if (t.abi.isMusl()) {
std.log.debug("Musl detected", .{});
flags.appendSlice(&.{"-DNO_GETCONTEXT"}) catch unreachable;
}

flags.appendSlice(&.{
"-fno-sanitize=undefined",
// Always enabled
"-DALL_INTERIOR_POINTERS",
"-DNO_EXECUTE_PERMISSION",
// Zig comes with clang that supports atomics so we can hardcode this
"-DGC_BUILTIN_ATOMIC",
// TODO: Should this be configurable?
"-DNO_PROC_FOR_LIBRARIES",
}) catch unreachable;

if (enable_threads) {
flags.appendSlice(&.{"-DGC_THREADS"}) catch unreachable;
if (enable_parallel_mark)
flags.appendSlice(&.{"-DPARALLEL_MARK"}) catch unreachable;
if (enable_thread_local_alloc) {
lib.addCSourceFiles(.{
.files = &.{
"specific.c",
"thread_local_alloc.c",
},
.flags = flags.items,
});
}
if (t.os.tag == .windows) {
lib.addCSourceFiles(.{
.files = &.{"win32_threads.c"},
.flags = flags.items,
});
} else {
lib.addCSourceFiles(.{
.files = &.{
"gc_dlopen.c",
"pthread_start.c",
"pthread_support.c"
},
.flags = flags.items,
});
if (t.os.tag == .macos) {
lib.addCSourceFiles(.{
.files = &.{"darwin_stop_world.c"},
.flags = flags.items,
});
} else {
lib.addCSourceFiles(.{
.files = &.{"pthread_stop_world.c"},
.flags = flags.items,
});
}
}
}

if (enable_gcj_support) {
flags.appendSlice(&.{"-DGC_GCJ_SUPPORT"}) catch unreachable;
lib.addCSourceFiles(.{
.files = &.{"gcj_mlc.c"},
.flags = flags.items,
});
}

if (enable_disclaim) {
flags.appendSlice(&.{"-DENABLE_DISCLAIM"}) catch unreachable;
lib.addCSourceFiles(.{
.files = &.{"fnlz_mlc.c"},
.flags = flags.items,
});
}

if (enable_java_finalization)
flags.appendSlice(&.{"-DJAVA_FINALIZATION"}) catch unreachable;

if (enable_atomic_uncollectable)
flags.appendSlice(&.{"-DGC_ATOMIC_UNCOLLECTABLE"}) catch unreachable;

if (enable_redirect_malloc) {
flags.appendSlice(&.{
"-DREDIRECT_MALLOC=GC_malloc",
"-DIGNORE_FREE",
}) catch unreachable;
if (t.os.tag == .windows) {
flags.appendSlice(&.{"-DREDIRECT_MALLOC_IN_HEADER"}) catch unreachable;
} else {
flags.appendSlice(&.{"-DGC_USE_DLOPEN_WRAP"}) catch unreachable;
}
}

if (enable_munmap) {
flags.appendSlice(&.{"-DUSE_MMAP"}) catch unreachable;
flags.appendSlice(&.{"-DUSE_MUNMAP"}) catch unreachable;
} else if (enable_mmap) {
flags.appendSlice(&.{"-DUSE_MMAP"}) catch unreachable;
}

if (enable_large_config)
flags.appendSlice(&.{"-DLARGE_CONFIG"}) catch unreachable;

const source_files = [_][]const u8{
"alloc.c",
"reclaim.c",
"allchblk.c",
"misc.c",
"mach_dep.c",
"os_dep.c",
"mark_rts.c",
"headers.c",
"mark.c",
"obj_map.c",
"blacklst.c",
"finalize.c",
"new_hblk.c",
"dbg_mlc.c",
"malloc.c",
"dyn_load.c",
"typd_mlc.c",
"ptr_chck.c",
"mallocx.c",
};

lib.addCSourceFiles(.{
.files = &source_files,
.flags = flags.items,
});
lib.addIncludePath(.{ .path = "include" });
lib.linkLibC();
lib.installHeader("include/gc.h", "gc.h");
lib.installHeadersDirectory("include/gc", "gc");
b.installArtifact(lib);

}

0 comments on commit 4934489

Please sign in to comment.