Should we move our CI/CD from GitHub Actions to self-hosted runners for a 50-developer team spending $8K/month on Actions minutes with 400 builds per day?
Deploy a hybrid CI/CD model migrating heavy workflows to self-hosted runners on Kubernetes while retaining GitHub...
Decision
72%
Execution
—
Uncertainty
—
This verdict was re-examined after censor review.
Decision
Implement a hybrid CI/CD model: migrate heavy workflows (compilation, integration tests, Docker image builds) to self-hosted runners orchestrated via Kubernetes (ARC on EKS/GKE), while retaining GitHub-hosted runners for lightweight and low-frequency tasks. Target 40-60% cost reduction from $8K to $4-5K/month. Self-hosted infrastructure must reliably handle at least 200 of the 400 daily builds.
Critical failure mode: Runner image drift. GitHub-hosted runners update base images weekly with ~200 pre-installed tools. Self-hosted runners diverge within 2-3 weeks, breaking builds that worked on hosted runners. This is the primary reason self-hosted migrations get reverted. Mitigate with automated weekly image rebuilds matching GitHub's runner image releases.
Second failure mode: Spot interruptions affecting 5-10% of instances. Use mixed instance types and maintain 3+ on-demand baseline runners that never scale to zero. Set termination grace periods to allow in-flight builds to complete.
Critical threshold: DevOps staffing. This infrastructure requires ~0.5 FTE dedicated DevOps capacity. If your team lacks this, the TCO advantage collapses — a $75K+ annual staffing cost against ~$36-48K annual savings makes this marginal. Only proceed if existing DevOps capacity can absorb the load.
Next actions
Run a 2-week build profiling analysis: instrument all 400 daily builds to measure per-workflow GitHub Actions minutes, categorize by type (compile/test/lint/deploy), and identify top 10 costliest workflows
infra · immediate
Measure existing team Kubernetes expertise — survey DevOps/platform engineers on ARC familiarity and estimate available FTE capacity for runner infrastructure maintenance
infra · immediate
Deploy a proof-of-concept ARC cluster with 3 on-demand nodes, migrate the single costliest workflow, and measure cost/reliability over 2 weeks before broader rollout
infra · before_launch
Set up automated weekly runner image rebuilds that track GitHub's runner-images repository releases to prevent image drift
infra · before_launch
Create a CI/CD cost and reliability dashboard tracking: monthly spend, build success rate, queue wait times, and spot interruption frequency — with alerts if build failure rate exceeds 5% or queue time exceeds 5 minutes
infra · ongoing
This verdict stops being true when
Build profiling reveals that workflow optimization and aggressive caching alone can reduce GitHub Actions spend by 40%+ (to $4.8K or below) without infrastructure changes → Optimize existing GitHub-hosted workflows (caching, parallelization, deduplication) instead of migrating to self-hosted runners
Team has zero Kubernetes expertise and hiring/contracting a DevOps engineer would cost $150K+/year, making TCO savings negative → Stay on GitHub-hosted runners and focus purely on workflow optimization, or evaluate turnkey managed runner services (e.g., Buildjet, Namespace, Actuated) that provide cost savings without self-management
GitHub significantly reduces Actions pricing or introduces a volume tier that brings the 400 builds/day cost below $5K/month → Remain on GitHub-hosted runners — the operational complexity of self-hosted infrastructure is not justified for marginal savings
Full council reasoning, attack grid, and flip conditions included with Pro
Council notes
Socrates
Instead of focusing on cost reduction, first conduct a comprehensive CI/CD strategy review to determine if the 400 bu...
Vulcan
Evaluate a hybrid CI/CD model where heavy workflows are migrated to self-hosted runners (hosted in a cloud environmen...
Daedalus
**Recommendation: Deploy Actions Runner Controller (ARC) on a Kubernetes cluster (EKS or GKE) with spot/preemptible i...
Loki
Self-hosted spot instances for 400 builds/day? Spot interruptions could kill 10-20% of builds mid-flight, forcing ret...
Assumptions
The $8K/month spend is primarily driven by a subset of heavy workflows that can be isolated and migrated independently
The team has or can allocate ~0.5 FTE of DevOps/platform engineering capacity for runner infrastructure maintenance
Build workflows can be cleanly categorized into 'heavy' (suitable for self-hosted) and 'light' (retain on GitHub-hosted) without significant cross-dependencies
The team operates in a cloud environment (AWS/GCP/Azure) where Kubernetes infrastructure can be provisioned, and has existing cloud accounts and networking in place
Security and compliance requirements do not prohibit running CI/CD workloads on self-managed infrastructure
Operational signals to watch
reversal — Build profiling reveals that workflow optimization and aggressive caching alone can reduce GitHub Actions spend by 40%+ (to $4.8K or below) without infrastructure changes
reversal — Team has zero Kubernetes expertise and hiring/contracting a DevOps engineer would cost $150K+/year, making TCO savings negative
reversal — GitHub significantly reduces Actions pricing or introduces a volume tier that brings the 400 builds/day cost below $5K/month
Unresolved uncertainty
Actual build profile distribution is unknown — the 200/200 split between heavy and light workflows is assumed, not measured. If 350+ builds are heavy, the hybrid approach saves less because more infrastructure is needed
Team's existing DevOps capacity and Kubernetes expertise is unspecified — if no current Kubernetes competency exists, ramp-up time and staffing costs could eliminate the cost advantage for 6-12 months
b004's core point remains unaddressed: whether 400 builds/day is optimal or includes redundant/wasteful builds. Build caching and workflow optimization alone might reduce spend by 20-30% with zero infrastructure changes
Killed branch b003 had the most specific architecture (ARC, exact instance types, capacity math showing 16 concurrent runners needed at peak) but was eliminated for underestimating DevOps staffing costs — its technical specifics may still be the right implementation details
Security and compliance implications of self-hosted runners (secrets management, network isolation, audit logging) are unaddressed by any surviving branch