Compiling C on Unix: build.sh

What do I want from a build system?

I want a build system to be:

  • Easy to maintain
  • Minimal dependencies
  • Configurable

And optionally:

  • multi-threaded / incremental

The unity build

Unity builds are not incremental. They work by #include-ing all source code into one file and compiling it as a single translation unit.

// unity.c
#include "utils.h"
#include "utils.c"
#include "fs.h"
#include "fs.c"
#include "main.c"

#!/bin/bash

CC="clang"
output=(./bin/App)
sources=(./src/unity.c)
compiler_flags=(-Wall -pedantic -std=c99 -Isrc/)
linker_flags=(-fuse-ld=mold)

${CC} ${compiler_flags[*]} ${sources[*]}  ${linker_flags[*]} -o ${output}

If all your exposure is from “real build systems,” you might find this baffling. My advice is to form your own opinions—don’t accept the gospel of others without a second thought.

Unity builds have numerous benefits and should be the go-to for shipping. Builds of this kind are easy to port and give the compiler more opportunities for optimization. It is also by far the simplest method.

Structuring your program with the knowledge that it will use a unity build lets us do some tricks. Source files aren’t compiled separately, and we control their inclusion order. Exploiting this removes messy #include statements. You can even disregard header guards if you dare.

The benefit is less clutter in your header and source files. Less includes also means less work for the compiler. In some cases, unity builds have shorter clean-build times. Doing this will make LSPs or editor linting less effective, but that’s not a big deal. The only real problem is symbol collisions inside the single translation unit, which may or may not be trivial.

The incremental build

Although unity builds provide many benefits, you may want to do an incremental build if your project is “too complex.”

Before we start, if your project is too complex, your first attempt should be to fix the core problem. Incremental builds are not an excuse to spread source code across hundreds of files and to create cyclic dependencies.

With that out of the way, here is a build.sh for multiple source files built in parallel.

#!/bin/bash
set -e

CC="ccache clang"
output=(./bin/App)
sources=(src/main.c src/fs.c)
compiler_flags=(-Wall -Wextra -pedantic -std=c99 -Isrc/)
linker_flags=(-fuse-ld=mold)
objects=()

for s in "${sources[@]}"; do
    mkdir -p $(dirname ".obj/${s}.o")
    ${CC} -c ${compiler_flags[*]} ${s} -o .obj/${s}.o &
    objects+=".obj/${s}.o "
done

wait $BACK_PID # wait for compile to finish
${CC} ${linker_flags[*]}  ${objects[*]} -o ${output}

On top of sometimes being faster due to multi-threading, incremental builds let us leverage ccache to prevent recompiling unchanged code. Since we still have to re-link, using mold as our linker will further increase speeds.

Adding complexity

Let’s build upon the incremental build by adding the following:

  • Ensure ./bin/ is created
  • Automatically find source files
  • More compiler flags
  • User-specific compiler flags
  • Metaprogram or custom pre-processor
  • Echo state of the program
#!/bin/bash
set -e

### find and filter sources ###

mapfile -t sources < <(find -name *.c | sed\
    -e "\|./game/config.*|d" \
    -e "\|./gs/.*|d" \
    -e "\|.*impl.*|d" \
    -e "\|.*third_party.*|d" \
    -e "\|./codegen/.*|d" \
)

### flags ###

CC="ccache clang"
output=(./bin/App)
compiler_flags=(
    -O0 -march=native -ffast-math
    -I./game/source
    -Wall -Wno-missing-braces -Wno-unused-function -Wno-unused-variable -Wno-unused-but-set-variable -Wno-initializer-overrides -Wfatal-errors
    -g
)
linker_flags=(
    -fuse-ld=mold -ldl -lX11 -lXi -lm -lpthread -lavformat -lavcodec -lswscale -lavutil -ludev -lomp -lz -lcurl -luv -lssl -lcrypto
)
common_flags=(
    -pthread -fopenmp
)
if [ $(whoami) = "halvard" ]; then
    common_flags+=(
        -fsanitize=address,float-divide-by-zero,float-cast-overflow -fno-sanitize=null,alignment -fno-sanitize-recover=all -fno-omit-frame-pointer
    )
else
    # ...
fi

mkdir -p $(dirname "${output}")

### Compile and run metaprogram ###

if [ ! -f "./bin/codegen" ] || [ "./codegen/codegen.c" -nt "./bin/codegen" ]; then
   ${CC} -g ./codegen/codegen.c -fuse-ld=mold -o ./bin/codegen
fi

./bin/codegen

### compile program ###

objects=()

for i in "${!sources[@]}"; do
    #echo "adding source file ${sources[i]}..."
    mkdir -p $(dirname ".obj/${sources[i]}.o")
    ${CC} -c ${compiler_flags[*]} ${common_flags[*]} ${sources[i]} -o .obj/${sources[i]}.o &
    objects+=".obj/${sources[i]}.o "
done

echo "Compiling..."
wait $BACK_PID # wait for compile to finish

### link program ###

echo "Linking..."
${CC} ${linker_flags[*]} ${common_flags[*]} ${objects[*]} -o ${output}
echo "Done!"
· misc, programming, C