balena published the write-up on 30 January 2025. balenaOS is the embedded Linux they ship for containers on single-board computers. They support more than 100 public device types and more than 40 private ones.
Each pull request builds a draft OS for every device type they support. That is more than 100 draft releases per merge. Tests cover the OS, balena HUP over-the-air updates, and balenaCloud. Many of those runs sit on hardware-in-the-loop autokits. The OS team works about 10 pull requests a day. That is more than 1,000 OS releases and more than 3,000 test jobs, including retries.
If you are looking at the same route, start with the Jenkins → GitHub Actions playbook. balena’s post is a reusable BTD workflow rolled across device repos, not a one-repo Jenkinsfile rewrite.
Jenkins at that scale
Jenkins had been there from the start. Yocto builds, base images, end-to-end tests, Leviathan. They piled on more than 50 plugins for GitHub checks, artifacts, secrets, and templates. Custom balena-yocto-scripts tried to keep complexity out of the pipeline. The scripts became their own debt.
After hardware-in-the-loop testing landed in 2021, the cracks were specific. Jenkins plugins and GitHub Checks fought. Statuses raced. Pull requests merged without passing tests. Results were hard to attach to a PR. Retries were tribal knowledge. External contributors could not see why a run failed, because Jenkins sat behind SSO.
They already ran Flowzone, a reusable Actions workflow, on GitHub. Native YAML, secrets, and hosted or self-hosted runners were the draw. Jenkins agent / stages / steps mapped close enough to runs-on / jobs / steps that the team could read the new files.
One reusable workflow, then a caller in each device repo
They audited Jenkins jobs first and ran a proof of concept: one Actions job that could build a balenaOS release. Weeks went into runner layout, dependencies, and caches. Inputs started large and shrank. The job now infers most of what it needs from the repo and the Actions environment.
They tried one reusable workflow per stage. Re-init of a Yocto environment per job was too expensive. The BTD pipeline is one reusable workflow with build, test, and deploy inside it. Each device repo gets a small caller file for triggers and overrides, including workflow_dispatch.
Leviathan had grown around Jenkins. They taught it to read the Actions environment. Tests run as a matrix: suite, environment, worker type, runner. Failed jobs retry without rebuilding the OS. Artifacts live in GitHub Artifact Storage for a week so the test job can see the build. Test runs now take one third of the time. That is balena’s number.
First deploy was a staging fork of balena-raspberrypi. Public and private device types, secure boot, ESR images, alternate environments. Then larger device repos. A secrets and policy bot pushed permissions, required checks, and env settings. Four weeks later they shut down the self-hosted Jenkins.
What customers got
Jenkins logs were behind SSO. Actions logs follow repo visibility. Customers can see a failing check, debug a build, and send a patch without a balenaOS engineer in the middle.
They do not publish a dollar figure. Reliability, maintenance, and cost are listed as reasons to leave. The concrete gains they write down are the one-third test time, retries without a full rebuild, and public logs.
Observability across hundreds of workflows is the next job they name. They had not shipped that when the post went up.
What I would copy
Prove one device repo in a fork before you roll a caller to a hundred repositories. Their raspberrypi staging fork is that gate.
Keep build and test in one workflow if setup is huge. A clean “one job per stage” split failed them on Yocto. A matrix of tests is where they won time.
Do not trust Jenkins-to-GitHub check plugins to be the merge gate. balena had PRs merge on a raced status. Required Actions checks on the repo are the replacement.
Leave Jenkins up until a known PR shows the same OS, HUP, and cloud results on Actions, and an external fork can read the log. That visibility was the customer-facing reason to finish.
The Jenkins → GitHub Actions playbook is the controller checklist. Use the balena post for the reusable caller pattern and the hardware-in-the-loop matrix.