CI pipelines with Woodpecker

I’ve been managing my git repositories including some Mastodon bots over at codeberg.org for some time now. Up until now these were built as docker images, exported to tar balls and copied over to the server running them. An easy enough approach, but I wanted to set up a proper CI and a docker registry – just for fun and learning purposes, really.

Woodpecker

Codeberg provides Woodpecker as a CI platform. I’s a fork of Drone CI that’s available under the Apache License. While you can use their instance, you can also host Woodpecker yourself and connect it to codeberg (or other git forges). Woodpecker is a breeze to install. I’m running my services using docker-compose, so that’s what I did here, as well. The woodpecker docs provide a good starting point, here’s my setup. It makes Woodpecker available through a reverse proxy.

Writing pipelines is easy at first, here’s an example build pipeline:

I won’t go into the details, because there’s documentation for that. I will write something about secrets management further down, because it wasn’t clear to me, though.

Docker registry

Automated builds are one thing, but I also wanted to introduce a private docker registry to store my images. There’s an official image for that and again I’ve set this up using docker-compose. The docker registry image comes headless without a UI, so I integrated a simple UI with it. Again I’m making it available through my reverse proxy and because it’s my private registry I’ve also enabled basic auth authentication. You can create the required password file using this command – replace the username, of course:

htpasswd -Bc registry.password <username>

Pushing images to the registry

I don’t want every commit to create a new image, so I decided to work with git tags. They can be used as a pipeline trigger in Woodpecker. Here’s a sample pipeline:

	when:
	- event: tag
	steps:
	- name: push-image
	image: eclipse-temurin:17-jdk-alpine
	environment:
	REGISTRY_USERNAME:
	from_secret: registry_username
	REGISTRY_PASSWORD:
	from_secret: registry_password
	commands:
	- ./gradlew jib -Papp.version=${CI_COMMIT_TAG} -Pdocker.repo.username=$${REGISTRY_USERNAME} -Pdocker.repo.password=$${REGISTRY_PASSWORD}

view raw push-image.yml hosted with ❤ by GitHub

As you can see, it gets triggered when a new tag is pushed. The project uses Gradle as its build tool and images are built by the Jib gradle plugin. The registry to be used is defined in the image name prefix (it’s redacted in the example). Since it needs authentication, there’s some secrets to be managed. Woodpecker can store these for you and pass them to your pipelines. It’s taken me quite a while to get it right, because the documentation seems to be a bit broken here: the section on secrets in commands doesn’t work for me. You can however expose secrets as environment variables, but you need to escape those variables to work in the build container (hence the $${…}) in the pipeline.

Here’s the relevant Jib configuration from the gradle build file:

	jib {
	container {
	creationTime.set(DateTimeFormatter.ISO_DATE_TIME.format(Instant.now().atZone(ZoneId.of("UTC"))))
	}
	to {
	image = "<redacted_registry_url>/<redacted_repository_name>/<redacted_image_name>"
	tags = if (project.hasProperty("app.version")) {
	setOf("v" + project.property("app.version").toString())
	} else {
	setOf("dev-snapshot")
	}
	auth {
	username = if (project.hasProperty("docker.repo.username"))
	project.property("docker.repo.username").toString() else ""
	password = if (project.hasProperty("docker.repo.password"))
	project.property("docker.repo.password").toString() else ""
	}
	}
	}

view raw build.gradle.kts hosted with ❤ by GitHub

Pulling images

With that out of the way, images are now pushed to my registry and I can use it in my other services to pull those images. Again, since the registry is private I need to login to the registry first. But that’s really it.

I now have a number of projects to migrate to this.