Building Docker images on sourcehut's build service, builds.sr.ht

April 6, 2021 on Michael Serajnik’s blog

I’ve recently migrated my public projects from GitHub to sourcehut. sourcehut is a free and open source software development platform by Drew Devault, the creator of projects such as Sway and aerc.

Up until the move, I had been using Docker Hub’s automated builds feature to build Docker images whenever I made a new release for one of my projects. However, it’s currently not possible to connect sourcehut Git repositories for this purpose, which meant that I had to look for an alternative.

Thankfully, sourcehut features its own build service in the form of builds.sr.ht. builds.sr.ht uses so-called “build manifests” to trigger jobs and provides several ways to submit them. One of these ways is having a build manifest stored as .build.yml in the Git repository, which will then produce a new job each time you push.

I couldn’t find much specific information about building Docker images on builds.sr.ht, so I had to figure this out on my own. This is the manifest I came up with for one of my projects, synupkeep:

 1image: alpine/latest
 2packages:
 3  - docker
 4sources:
 5  - https://git.sr.ht/~mser/synupkeep
 6environment:
 7  project: synupkeep
 8  docker_image: mserajnik/synupkeep
 9  docker_username: mserajnik
10secrets:
11  - 2be455df-9ff9-4803-bf5e-2c65d9d986a5
12tasks:
13  - skip_if_not_release: |
14      cd $project
15      git describe --exact-match HEAD || complete-build      
16  - setup: |
17      curl -s https://api.github.com/repos/tomwright/dasel/releases/latest | \
18        grep browser_download_url | \
19        grep linux_amd64 | \
20        cut -d '"' -f 4 | \
21        xargs -n1 curl -Lso dasel && \
22        chmod +x dasel && \
23        sudo mv dasel /usr/local/bin/dasel
24      sudo addgroup $(whoami) docker
25      sudo service docker start      
26  - build: |
27      cd $project
28
29      version=$(/usr/local/bin/dasel -f pyproject.toml -s "tool.poetry.version")
30
31      set +e
32
33      docker_ps_return=1
34      until [ ${docker_ps_return} -eq 0 ]; do
35        docker ps >/dev/null 2>&1
36        docker_ps_return=$?
37        sleep 1
38      done
39
40      set -e
41
42      docker build . -t $docker_image:latest -t $docker_image:$version      
43  - publish: |
44      cat ~/.docker-hub-personal-access-token | docker login --username $docker_username --password-stdin
45      docker push $docker_image --all-tags      

Most of the configuration should be pretty self-explanatory, but I wanted to point out a few things that might be useful for your own projects.

Skipping builds that are not releases

Since synupkeep uses semantic versioning, it makes sense to have each release available under a separate Docker image tag (e.g., 1.0.0, 1.1.0 etc.). However, since I didn’t want or need to build images for unreleased commits on master (or other branches), I started looking for a method to achieve that and found inspiration in build.sr.ht’s own build configuration:

1tasks:
2  - skip_if_not_release: |
3      cd $project
4      git describe --exact-match HEAD || complete-build      

The complete-build command instructs the build service to abort the job at this point if no tag can be found (without flagging it as failed).

Determining the Docker image version tag

As mentioned previously, I wanted to have each release available under a separate Docker image tag. I couldn’t find a way to extract that information from the build environment, so I decided to use a neat tool called dasel to query the version number from my project’s pyproject.toml (a file that belongs to Poetry, a package manager for Python).

dasel isn’t available as an apk package on Alpine Linux, but thankfully provides binary releases, so it’s simple enough to install:

1tasks:
2  - setup: |
3      curl -s https://api.github.com/repos/tomwright/dasel/releases/latest | \
4        grep browser_download_url | \
5        grep linux_amd64 | \
6        cut -d '"' -f 4 | \
7        xargs -n1 curl -Lso dasel && \
8        chmod +x dasel && \
9        sudo mv dasel /usr/local/bin/dasel      

Using dasel is straightforward thanks to its intuitive API. I store the version into an environment variable so I can use it when executing docker build:

1tasks:
2  - build: |
3      version=$(/usr/local/bin/dasel -f pyproject.toml -s "tool.poetry.version")
4
5      docker build . -t $docker_image:latest -t $docker_image:$version      

Waiting for the Docker daemon to become ready

I found that after starting the Docker daemon via sudo service docker start, it might take a while before it becomes ready, which initially resulted in some failed builds when docker build was executed just a bit too early.

To work around that, I decided to utilize the docker ps command, which exits with code 1 if the daemon isn’t available. I simply put it into an until loop and execute docker ps every second until it no longer errors:

 1tasks:
 2  - build: |
 3      set +e
 4
 5      docker_ps_return=1
 6      until [ ${docker_ps_return} -eq 0 ]; do
 7        docker ps >/dev/null 2>&1
 8        docker_ps_return=$?
 9        sleep 1
10      done
11
12      set -e      

set +e is necessary to prevent the build from failing when docker ps exits with code 1. Afterwards, I restore the original behavior with set -e (or otherwise the job also wouldn’t fail if, e.g., docker build errored).


I hope this example illustrates how easy it is to get started with builds.sr.ht. If you haven’t, you should definitely check out sourcehut. It’s an awesome alternative to GitHub and similar platforms; see here for some of the reasons you might care about it.

Do you have a comment or question? Feel free to send it to my public inbox (please review the mailing list etiquette before doing so).