Proposal ยท Dev box on Docker
A Docker + GitHub Actions dev box
A plan to stand up a new dev box that auto-deploys on every merge to develop: GitHub Actions runs the tests, builds Docker images, and pushes them to AWS ECR; Ansible then rolls them out on the box with zero downtime. It replaces today’s manual Fabric deploy and moves CI from CircleCI to GitHub Actions โ a pilot for one standardized pipeline.
developRead this first
What this page is
A proposal โ a direction we’re considering for a brand-new dev box, not something already running. Any code shown is a starting template.
A greenfield dev box on Docker, as a pilot. The plan moves CI from CircleCI to GitHub Actions and adds an automated build + deploy. Smaller open items are flagged to confirm; the bigger calls are in Points to discuss.
The starting point
How things work today
Today the app runs directly on the VM โ systemd services in a Python venv. No Docker, and the deploy is manual.
| Concern | Today |
|---|---|
| CI | CircleCI โ runs tests + mypy; never deploys. This proposal moves CI to GitHub Actions. |
| Deploy | Fabric โ “scripted SSH”: git pull, pip install, migrate, restart systemd. Manual. |
| Provisioning | Ansible (server/) โ builds a fresh box into a diviac server. Run rarely. |
| Runtime | Runs on the VM as systemd services in a venv โ no Docker. |
| Docker today | Local dev + a CI image only. Dockerfiles exist for backend + every frontend. |
| Dev DB | Postgres local on the box (db_host: localhost). |
| Secrets | AWS Secrets Manager โ already used in our AWS setup. |
The choice
The decision: Docker
Keep building the dev box the current way (app on the VM, manual Fabric deploy) โ or containerize it. We recommend Docker: GitHub Actions builds images โ pushes to ECR โ Ansible rolls them out with zero downtime. We already have Dockerfiles and a working docker-compose.yml, so a greenfield box is the lowest-risk place to validate it.
| Dimension | Today (VM + systemd) | Proposed (Docker) pick |
|---|---|---|
| New-box setup | Replay the whole large Ansible role | ~“install Docker + Ansible” |
| Reproducibility | Depends on the box’s state | The exact tested image runs |
| CI + deploy | CircleCI + manual Fabric | GitHub Actions + Ansible |
| Rollback | Forward-only (git pull) | Redeploy a previous ECR tag |
| Downtime | In-place restart | Zero-downtime switch (~1 min) |
The mechanism
How it works
build โ store โ ship โ switch. GitHub Actions is the trigger (and now runs CI); Ansible is the executor on the box (replacing Fabric). The box needs only Docker + Ansible + the compose file + access to ECR and Secrets Manager.
Build
On merge to
develop, GitHub Actions runs the tests, then (path-filtered) builds only the affected images โ one backend image (web, celery, beat, flower) + one per frontend, tagged with the commit SHA.Store
Push to AWS ECR; a lifecycle policy keeps the last 3–5 images for rollback.
Ship
An Ansible playbook logs in to ECR, renders
.env.devfrom Secrets Manager, pulls images, runs migrations.Switch
Bring up the new version alongside the running one, wait for its health-check, flip the reverse-proxy to it, then stop the old one. Overlap ~1 min, no downtime. Rollback = re-run pinned to a previous ECR tag.
| Service | Zero-downtime switch? |
|---|---|
| Backend web, frontends (stateless) | Yes โ new and old run together behind the proxy; switch when healthy |
| Celery workers / beat | Overlap-and-replace (tasks must be idempotent) |
| Postgres / Elasticsearch / Redis (stateful) | No โ stay put; never two writers |
During the switch, the old and new versions briefly share one database. Use expand/contract (additive migration โ deploy โ later cleanup); a destructive migration breaks the old version mid-switch. Lower-stakes in dev, but worth the habit.
Settled choices
Key decisions
| Topic | Decision |
|---|---|
| Registry | AWS ECR โ OIDC auth (no static keys); lifecycle keeps last 3–5 images for rollback. (GHCR considered, rejected.) |
| CI + builds | GitHub Actions, path-filtered โ run tests, then rebuild only the services that changed. |
| Dev data services | Postgres, Redis, Elasticsearch as containers โ cheap and disposable for dev. |
| Persistence | Volumes for what you can’t rebuild. Postgres = source of truth โ volume + nightly pg_dumpโS3. ES rebuildable from Postgres; Redis disposable. |
| Secrets | AWS Secrets Manager. Ansible renders .env.dev on the box (mode 600); nothing sensitive committed. |
| Images | Multi-stage, slim bases, layer caching โ faster builds/pulls (smaller = faster switchover). |
down -v in routine ops
It deletes the named volumes (pg_data, redis_data, esdata) and the data with them. Use plain down to restart.
Be aware
Risks & guardrails
| Risk | Guardrail |
|---|---|
| Broken code auto-deploys | The box deploys whatever lands on develop โ require the GitHub Actions checks to pass via branch protection. |
| Safe migrations | Enforce backward-compatible (expand/contract) migrations. |
| Rollback | Keep the last 3–5 ECR tags; re-run the deploy pinned to a prior tag. |
| SSH reachability | If the box firewalls SSH โ use a self-hosted runner. |
| Volume deletion | Never down -v in routine ops. |
To decide together
Points to discuss
SSO for dev TBD
How access to the dev environment is authenticated โ single sign-on for the dev site/box. Agree on the approach.
Error tracing โ Sentry or Datadog TBD
Which tool to use for error tracing: Sentry or Datadog. Decide based on volume.
What it takes
What building it involves
Three files to add, plus some one-time AWS/GitHub setup. These are starting templates; the full versions and the step-by-step checklist live in the repo doc.
| File | What it does |
|---|---|
docker-compose.dev.yml | Deploy-oriented stack โ pre-built ECR images, named volumes, health checks, env file from Secrets Manager. |
server/deploy_docker.yml | Ansible deploy โ ECR login โ secrets โ pull โ migrate โ zero-downtime switch โ prune. Replaces the Fabric deploy. |
.github/workflows/deploy-dev.yml | On merge: run tests โ path-filter โ build affected images โ push to ECR (OIDC) โ trigger Ansible. |
One-time setup: an AWS VM with an IAM instance role (ECR pull + read the dev secret), ECR repos + lifecycle policy, a GitHub OIDC role, the diviac/dev/env secret, and SSH deploy keys. No static AWS keys anywhere. Full checklist + templates: documents/dev-box-docker-github-actions.md.
Go deeper
References
Full proposal (un-abridged compose, Ansible playbook, workflow, lifecycle policy, checklist): documents/dev-box-docker-github-actions.md. In-repo: CI .circleci/ ยท current deploy fabfile.py, Makefile ยท provisioning server/ ยท Docker docker-compose.yml, compose/.
How code ships today is on the Deployments page.