Docker Compose Deep Dive

Docker Compose lets us declare a multi-container application in one YAML file and bring it up with a single command. In this guide we'll cover the modern Compose Specification as implemented by Docker Compose v2 — meaning we use the docker compose command (space, not hyphen) and we do not put a version: key at the top of the file. The version field is obsolete and silently ignored by Compose v2.

Top-Level Keys

A Compose file is structured around four main top-level keys:

• services — the containers that make up your application.
• networks — user-defined networks the services attach to.
• volumes — named volumes for persistent data.
• secrets — secret values mounted into services (file-based outside of Swarm).

A minimal file looks like this:

services:
  web:
    image: nginx:1.27-alpine
    ports:
      - "8080:80"

Filename note: Compose v2 looks for compose.yaml, compose.yml, docker-compose.yaml, or docker-compose.yml in the current directory, in that order. The modern preferred name is compose.yaml.

build vs image

A service can either pull a pre-built image (image:) or build one from a Dockerfile (build:). You can use both: build tells Compose how to construct the image, and image names the resulting tag.

services:
  api:
    # Build from ./api/Dockerfile and tag as myorg/api:dev
    build:
      context: ./api
      dockerfile: Dockerfile
      args:
        APP_ENV: development
      target: dev          # multi-stage: stop at the `dev` stage
    image: myorg/api:dev   # name for the built image (useful for `docker push`)

  cache:
    # Pure pull, no build
    image: redis:7-alpine

ports vs expose (and the security implication)

This trips people up constantly. They are not the same.

• ports: publishes a container port on the host. Anyone who can reach the host on that port can reach the container.
• expose: only declares that a port is available to other containers on the same Compose network. It is not published on the host.

By default, ports: "5432:5432" binds to all interfaces (0.0.0.0) on the host. That means your database is reachable from the network — including from the internet if the host has a public IP and no firewall. Always bind to localhost for anything that should not leave the host:

services:
  db:
    image: postgres:16
    ports:
      - "127.0.0.1:5432:5432"   # localhost-only — safe for dev
    # NOT: "5432:5432"  ← publishes on 0.0.0.0, reachable from LAN

If the database only needs to be reached by other services in the Compose file, drop ports entirely and use expose (or rely on the default — services on the same network can reach each other on any port without expose; it's mostly documentation).

environment vs env_file

Two ways to inject environment variables. Both work; the difference is where the values live.

services:
  api:
    image: myorg/api:dev
    environment:
      LOG_LEVEL: info
      DATABASE_URL: postgres://app:app@db:5432/app
      # Pull from host env if it's set, otherwise empty:
      SENTRY_DSN: ${SENTRY_DSN:-}

  worker:
    image: myorg/api:dev
    env_file:
      - ./.env.worker        # KEY=value per line, no quotes needed

Compose also automatically loads variables from a top-level .env file for substitution into the Compose file itself (e.g. ${IMAGE_TAG}). That is different from env_file, which injects variables into the container.

depends_on with healthchecks

The classic mistake: declaring depends_on: [db] and assuming the database is ready when the API starts. It isn't — depends_on by itself only waits for the container to start, not for the service inside to be healthy.

The fix is to declare a healthcheck on the dependency and use the long form of depends_on:

services:
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: app
      POSTGRES_PASSWORD: app
      POSTGRES_DB: app
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d app"]
      interval: 5s
      timeout: 3s
      retries: 10
      start_period: 10s

  api:
    image: myorg/api:dev
    depends_on:
      db:
        condition: service_healthy

Healthcheck fields:

• test — command run inside the container. CMD-SHELL runs it through /bin/sh -c.
• interval — how often to run the check after it has started.
• timeout — how long a single check may take before counting as a failure.
• retries — consecutive failures before the container is marked unhealthy.
• start_period — grace period at startup during which failures don't count toward retries. Critical for slow-starting services like databases or JVM apps.

Restart Policies

Tell Docker what to do when a container exits.

services:
  api:
    image: myorg/api:dev
    restart: unless-stopped   # restart on failure or daemon restart; honor manual stop
    # other values:
    #   no            — never restart (default)
    #   always        — restart even if you `docker compose stop`-ed it (rare for dev)
    #   on-failure    — restart only on non-zero exit; supports max-retries

For typical long-running services, unless-stopped is the safest default: it survives daemon restarts but still respects an explicit docker compose stop.

Volumes — Named vs Bind Mounts

Two distinct mount types. Use the right one for the job.

• Named volume (my_volume:/var/lib/postgres) — managed by Docker, lives under /var/lib/docker/volumes/, survives docker compose down (but not down -v). Use for persistent data (databases, uploads).
• Bind mount (./src:/app/src) — a host path mounted into the container. Use for source code in development so edits are reflected live.

services:
  api:
    image: myorg/api:dev
    volumes:
      - ./src:/app/src                 # bind mount for live reload in dev
      - api_node_modules:/app/node_modules  # named volume, opaque to the host

  db:
    image: postgres:16
    volumes:
      - db_data:/var/lib/postgresql/data  # named volume — persistent

volumes:
  api_node_modules:
  db_data:

⚠️ Destructive: docker compose down -v deletes named volumes too. Reach for it only when you want a clean slate.

Multiple Networks for Isolation

By default, Compose creates one network and attaches every service to it. For better isolation (especially in production-shaped setups), declare multiple networks and attach services only where they need to talk.

services:
  proxy:
    image: nginx:1.27-alpine
    ports:
      - "443:443"
    networks: [frontend]

  api:
    image: myorg/api:dev
    networks: [frontend, backend]   # proxy can reach api; api can reach db

  db:
    image: postgres:16
    networks: [backend]             # NOT on frontend — proxy can't reach it

networks:
  frontend:
  backend:

Containers on the same network resolve each other by service name (e.g. the api can connect to postgres://db:5432). Containers on different networks cannot reach each other at all — exactly what we want for a database.

Profiles — Optional Services

Profiles let you mark services as optional and start them only when explicitly requested. Great for things like a debug shell, a one-off seed job, or a heavy monitoring stack.

services:
  api:
    image: myorg/api:dev
    # no profile — always started

  jaeger:
    image: jaegertracing/all-in-one:1.57
    profiles: ["tracing"]   # only started when 'tracing' profile is active
    ports:
      - "16686:16686"

# Start the default set only:
#   docker compose up
#
# Start including tracing:
#   docker compose --profile tracing up

Secrets (file-based)

Compose secrets in non-Swarm mode are just files mounted read-only into the container at /run/secrets/<name>. They're cleaner than putting passwords in environment (which leaks into docker inspect and process listings).

services:
  db:
    image: postgres:16
    environment:
      POSTGRES_USER: app
      POSTGRES_DB: app
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password

secrets:
  db_password:
    file: ./secrets/db_password.txt   # keep this out of git: add to .gitignore

Note: the full secrets feature (encrypted at rest, distributed across nodes) only applies to Swarm. In plain Compose this is essentially a tidy way to mount a file.

Override Files

Compose automatically merges compose.yaml with compose.override.yaml (or docker-compose.override.yml) if present. This is the canonical pattern for "same base, different environment."

The classic pattern: the base file is production-shaped, and the override turns on dev conveniences (bind mounts, exposed debug ports, looser env vars).

services:
  api:
    image: myorg/api:1.4.2
    restart: unless-stopped
    environment:
      LOG_LEVEL: info

services:
  api:
    build:
      context: ./api
      target: dev
    image: myorg/api:dev
    ports:
      - "127.0.0.1:3000:3000"
    volumes:
      - ./api:/app
    environment:
      LOG_LEVEL: debug
      DEBUG: "*"

For non-default override files (e.g. CI), pass them explicitly:

docker compose -f compose.yaml -f compose.ci.yaml up --abort-on-container-exit

Worked Example 1: nginx + app + postgres

A realistic small stack: an Nginx reverse proxy in front of a Node-style app, talking to a Postgres database. Both networks are isolated; the database is only reachable from the backend network and has a healthcheck.

services:
  proxy:
    image: nginx:1.27-alpine
    ports:
      - "127.0.0.1:8080:80"
    volumes:
      - ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
    depends_on:
      app:
        condition: service_healthy
    networks: [frontend]
    restart: unless-stopped

  app:
    build:
      context: ./app
    image: myorg/app:dev
    environment:
      DATABASE_URL: postgres://app:app@db:5432/app
      PORT: "3000"
    healthcheck:
      test: ["CMD", "wget", "-qO-", "http://localhost:3000/healthz"]
      interval: 10s
      timeout: 3s
      retries: 5
      start_period: 15s
    depends_on:
      db:
        condition: service_healthy
    networks: [frontend, backend]
    restart: unless-stopped

  db:
    image: postgres:16
    environment:
      POSTGRES_USER: app
      POSTGRES_DB: app
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password
    secrets:
      - db_password
    volumes:
      - db_data:/var/lib/postgresql/data
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d app"]
      interval: 5s
      timeout: 3s
      retries: 10
      start_period: 10s
    networks: [backend]
    restart: unless-stopped

networks:
  frontend:
  backend:

volumes:
  db_data:

secrets:
  db_password:
    file: ./secrets/db_password.txt

Bring it up and watch the health states transition:

docker compose up -d
docker compose ps
docker compose logs -f app
docker compose down            # stop and remove containers/networks
docker compose down -v         # also remove the named volume (DESTROYS db_data)

Worked Example 2: Minimal Dev Override

This is the smallest useful override pattern: the base file is what you'd deploy, and the override enables a local bind mount and a debugger port.

services:
  api:
    image: myorg/api:1.4.2
    restart: unless-stopped
    environment:
      NODE_ENV: production

services:
  api:
    build:
      context: ./api
      target: dev
    image: myorg/api:dev
    command: npm run dev
    volumes:
      - ./api:/app
      - /app/node_modules    # anonymous volume so host's node_modules doesn't clobber container's
    ports:
      - "127.0.0.1:3000:3000"
      - "127.0.0.1:9229:9229"   # Node inspector
    environment:
      NODE_ENV: development

To run the production-shaped base without the dev override (e.g. to test the production image locally):

docker compose -f compose.yaml up

Useful Day-to-Day Commands

# Bring everything up in the background
docker compose up -d

# Rebuild images and restart
docker compose up -d --build

# See what's running and the health status
docker compose ps

# Stream logs (all services or a single one)
docker compose logs -f
docker compose logs -f api

# Open a shell in a running container
docker compose exec api sh

# Run a one-off command (new container, removed afterwards)
docker compose run --rm api npm test

# Validate & render the merged config (great for debugging overrides)
docker compose config

# Pull updated images for all services
docker compose pull

# Stop everything cleanly
docker compose down

# Nuke containers, networks, AND named volumes (DESTRUCTIVE)
docker compose down -v

Summary

• ✓ Modern Compose: no version: key, use docker compose (v2).
• ✓ Top-level keys: services, networks, volumes, secrets.
• ✓ ports publishes on the host (default 0.0.0.0 — bind to 127.0.0.1 for dev DBs); expose is documentation only.
• ✓ Use healthcheck + depends_on.condition: service_healthy to wait for real readiness.
• ✓ Named volumes for persistent data, bind mounts for source code.
• ✓ Multiple networks isolate sensitive services from the public-facing ones.
• ✓ Profiles for optional services; overrides for environment-specific tweaks.
• ✓ Prefer file-based secrets over env vars for passwords.

Compose is the right tool for local dev environments, integration tests, and small single-host deployments. For multi-host orchestration, graduate to Kubernetes or Swarm — but the mental model you build here transfers cleanly.