Proposal ยท Dev box on Docker

A Docker + GitHub Actions dev box

A plan to stand up a new dev box that auto-deploys on every merge to develop: GitHub Actions runs the tests, builds Docker images, and pushes them to AWS ECR; Ansible then rolls them out on the box with zero downtime. It replaces today’s manual Fabric deploy and moves CI from CircleCI to GitHub Actions โ€” a pilot for one standardized pipeline.

DockerRecommended runtime for the dev box
on mergeAuto build + deploy on every push to develop
ECRImage registry โ€” keeps last 3–5 for rollback
zero downtimeNew version up, health-check, switch โ€” ~1 min overlap

Read this first

What this page is

A proposal โ€” a direction we’re considering for a brand-new dev box, not something already running. Any code shown is a starting template.

Status: proposal not live yet

A greenfield dev box on Docker, as a pilot. The plan moves CI from CircleCI to GitHub Actions and adds an automated build + deploy. Smaller open items are flagged to confirm; the bigger calls are in Points to discuss.

The starting point

How things work today

Today the app runs directly on the VM โ€” systemd services in a Python venv. No Docker, and the deploy is manual.

ConcernToday
CICircleCI โ€” runs tests + mypy; never deploys. This proposal moves CI to GitHub Actions.
DeployFabric โ€” “scripted SSH”: git pull, pip install, migrate, restart systemd. Manual.
ProvisioningAnsible (server/) โ€” builds a fresh box into a diviac server. Run rarely.
RuntimeRuns on the VM as systemd services in a venv โ€” no Docker.
Docker todayLocal dev + a CI image only. Dockerfiles exist for backend + every frontend.
Dev DBPostgres local on the box (db_host: localhost).
SecretsAWS Secrets Manager โ€” already used in our AWS setup.

The choice

The decision: Docker

Keep building the dev box the current way (app on the VM, manual Fabric deploy) โ€” or containerize it. We recommend Docker: GitHub Actions builds images โ†’ pushes to ECR โ†’ Ansible rolls them out with zero downtime. We already have Dockerfiles and a working docker-compose.yml, so a greenfield box is the lowest-risk place to validate it.

DimensionToday (VM + systemd)Proposed (Docker) pick
New-box setupReplay the whole large Ansible role~“install Docker + Ansible”
ReproducibilityDepends on the box’s stateThe exact tested image runs
CI + deployCircleCI + manual FabricGitHub Actions + Ansible
RollbackForward-only (git pull)Redeploy a previous ECR tag
DowntimeIn-place restartZero-downtime switch (~1 min)

The mechanism

How it works

build โ†’ store โ†’ ship โ†’ switch. GitHub Actions is the trigger (and now runs CI); Ansible is the executor on the box (replacing Fabric). The box needs only Docker + Ansible + the compose file + access to ECR and Secrets Manager.

  1. Build

    On merge to develop, GitHub Actions runs the tests, then (path-filtered) builds only the affected images โ€” one backend image (web, celery, beat, flower) + one per frontend, tagged with the commit SHA.

  2. Store

    Push to AWS ECR; a lifecycle policy keeps the last 3–5 images for rollback.

  3. Ship

    An Ansible playbook logs in to ECR, renders .env.dev from Secrets Manager, pulls images, runs migrations.

  4. Switch

    Bring up the new version alongside the running one, wait for its health-check, flip the reverse-proxy to it, then stop the old one. Overlap ~1 min, no downtime. Rollback = re-run pinned to a previous ECR tag.

100%Zoom, or open full screen, then scroll to explore. Vector diagram โ€” stays sharp at any size.
flowchart LR dev["Merge to develop"] --> gha["GitHub Actions"] gha --> test["Run tests<br/>pytest + mypy"] test -->|"pass"| build["Build changed images<br/>(path-filtered)"] subgraph STORE["store"] build -->|"push :sha"| ecr["AWS ECR<br/>keeps last 3–5"] end subgraph SHIP["ship & switch โ€” Ansible on the dev box"] build -.->|"ansible-playbook"| box["Dev box (Docker)"] ecr -->|"docker compose pull"| box box -->|"migrate (one-off)"| up["start new version, health-check"] up -->|"switch traffic, stop old"| live["new version live"] end live -.->|"rollback = re-deploy previous ECR tag"| ecr
ServiceZero-downtime switch?
Backend web, frontends (stateless)Yes โ€” new and old run together behind the proxy; switch when healthy
Celery workers / beatOverlap-and-replace (tasks must be idempotent)
Postgres / Elasticsearch / Redis (stateful)No โ€” stay put; never two writers
Critical โ€” migrations must be backward-compatible

During the switch, the old and new versions briefly share one database. Use expand/contract (additive migration โ†’ deploy โ†’ later cleanup); a destructive migration breaks the old version mid-switch. Lower-stakes in dev, but worth the habit.

Settled choices

Key decisions

TopicDecision
RegistryAWS ECR โ€” OIDC auth (no static keys); lifecycle keeps last 3–5 images for rollback. (GHCR considered, rejected.)
CI + buildsGitHub Actions, path-filtered โ€” run tests, then rebuild only the services that changed.
Dev data servicesPostgres, Redis, Elasticsearch as containers โ€” cheap and disposable for dev.
PersistenceVolumes for what you can’t rebuild. Postgres = source of truth โ†’ volume + nightly pg_dumpโ†’S3. ES rebuildable from Postgres; Redis disposable.
SecretsAWS Secrets Manager. Ansible renders .env.dev on the box (mode 600); nothing sensitive committed.
ImagesMulti-stage, slim bases, layer caching โ€” faster builds/pulls (smaller = faster switchover).
Never down -v in routine ops

It deletes the named volumes (pg_data, redis_data, esdata) and the data with them. Use plain down to restart.

Be aware

Risks & guardrails

RiskGuardrail
Broken code auto-deploysThe box deploys whatever lands on develop โ€” require the GitHub Actions checks to pass via branch protection.
Safe migrationsEnforce backward-compatible (expand/contract) migrations.
RollbackKeep the last 3–5 ECR tags; re-run the deploy pinned to a prior tag.
SSH reachabilityIf the box firewalls SSH โ†’ use a self-hosted runner.
Volume deletionNever down -v in routine ops.

To decide together

Points to discuss

SSO for dev TBD

How access to the dev environment is authenticated โ€” single sign-on for the dev site/box. Agree on the approach.

Error tracing โ€” Sentry or Datadog TBD

Which tool to use for error tracing: Sentry or Datadog. Decide based on volume.

What it takes

What building it involves

Three files to add, plus some one-time AWS/GitHub setup. These are starting templates; the full versions and the step-by-step checklist live in the repo doc.

FileWhat it does
docker-compose.dev.ymlDeploy-oriented stack โ€” pre-built ECR images, named volumes, health checks, env file from Secrets Manager.
server/deploy_docker.ymlAnsible deploy โ€” ECR login โ†’ secrets โ†’ pull โ†’ migrate โ†’ zero-downtime switch โ†’ prune. Replaces the Fabric deploy.
.github/workflows/deploy-dev.ymlOn merge: run tests โ†’ path-filter โ†’ build affected images โ†’ push to ECR (OIDC) โ†’ trigger Ansible.

One-time setup: an AWS VM with an IAM instance role (ECR pull + read the dev secret), ECR repos + lifecycle policy, a GitHub OIDC role, the diviac/dev/env secret, and SSH deploy keys. No static AWS keys anywhere. Full checklist + templates: documents/dev-box-docker-github-actions.md.

Go deeper

References

Full proposal (un-abridged compose, Ansible playbook, workflow, lifecycle policy, checklist): documents/dev-box-docker-github-actions.md. In-repo: CI .circleci/ ยท current deploy fabfile.py, Makefile ยท provisioning server/ ยท Docker docker-compose.yml, compose/.

How code ships today is on the Deployments page.