Skip to content

Docker Orchestration

Docker provides several building blocks or layers of abstraction for orchestrating containers: docker compose, docker service, docker stack, and docker swarm. They are all related to running and managing Docker containers, but they operate at different scopes and with different capabilities.

Here’s a breakdown.

0. TL;DR

docker --help
[...]
Swarm Commands:
  config      Manage Swarm configs
  node        Manage Swarm nodes
  secret      Manage Swarm secrets
  service     Manage Swarm services
  stack       Manage Swarm stacks
  swarm       Manage Swarm

1. docker swarm

  • What it is: This is Docker’s native clustering and orchestration system.
  • Purpose: To turn a pool of Docker hosts (machines) into a single, virtual Docker host. It allows you to deploy, manage, and scale applications across multiple machines.
  • Key Action: docker swarm init (on a manager node) and docker swarm join (on worker/manager nodes).
  • Manages: Services (which are made up of tasks/containers), nodes, networks, secrets, configs within the cluster.
  • Scope: Multi-host cluster.
  • Analogy: The entire orchestra pit and the conductor. It’s the environment where services will run and be managed.
  • Relationship to others:
    • You create a docker swarm first.
    • docker service commands are used to create and manage individual services within that swarm.
    • docker stack commands are used to deploy and manage groups of related services (defined in a Compose file) onto that swarm.

2. docker service

  • What it is: A command-line interface (CLI) command and a Swarm-level abstraction for defining and managing a set of replicated tasks (containers) that should be running within a Docker Swarm.
  • Purpose: To define the desired state of a specific application component within the swarm (e.g., “I want 3 replicas of my web frontend image, exposing port 80”). Swarm will then work to maintain this state.
  • Key Actions: docker service create ..., docker service ls, docker service scale ..., docker service update ..., docker service rm ....
  • Manages: The lifecycle of a single type of containerized application component across the swarm, including:
    • Number of replicas (desired instances).
    • Image to use.
    • Port mappings.
    • Network connections.
    • Update policies.
    • Placement constraints.
  • Scope: Single application component within a multi-host Swarm cluster.
  • Analogy: The instructions for a single section of the orchestra (e.g., “the violins should play this part, and there should be 3 of them”).
  • Relationship to others:
    • docker service commands are the direct way to interact with and manage individual application components running on a docker swarm.
    • docker stack uses the concepts of services internally; a stack is a collection of services.

3. docker compose

  • What it is: A tool (and a file format - docker-compose.yml) for defining and running multi-container Docker applications on a single Docker host.
  • Purpose: To simplify the development, testing, and deployment of applications that consist of multiple interconnected containers (e.g., a web server, a database, a caching service) on one machine.
  • Key Actions: docker-compose up, docker-compose down, docker-compose ps, docker-compose logs.
  • Manages: Multiple containers, their networks, and volumes as defined in a docker-compose.yml file, but only on the local Docker engine where the command is run.
  • Scope: Multi-container application on a single host.
  • Analogy: A blueprint for assembling a complex Lego model (your multi-container app) on your desk (a single Docker host).
  • Relationship to others:
    • The docker-compose.yml file format (specifically version 3 and above) is also used by docker stack. This allows you to use the same definition for local development (docker-compose) and for deploying to a docker swarm (with docker stack).
    • docker compose itself does not directly interact with docker swarm or docker service in a clustering sense. It’s a standalone tool for single-host scenarios.

4. docker stack

  • What it is: A CLI command for managing a collection of related services (an “application stack”) on a Docker Swarm. It uses a docker-compose.yml file (version 3+) as its input.
  • Purpose: To deploy, update, and manage an entire multi-service application as a single unit on a Swarm cluster. It’s a higher-level abstraction over docker service.
  • Key Actions: docker stack deploy -c <compose_file> <stack_name>, docker stack ls, docker stack services <stack_name>, docker stack rm <stack_name>.
  • Manages: A group of Swarm services, networks, and (sometimes) secrets/configs defined in a Compose file, across the Swarm cluster.
  • Scope: Multi-service application deployed across a multi-host Swarm cluster.
  • Analogy: The entire musical score for the symphony (your multi-service app defined in docker-compose.yml), which the conductor (docker swarm manager) uses to instruct all the orchestra sections (docker services).
  • Relationship to others:
    • docker stack deploy takes a docker-compose.yml file and translates its service definitions into docker service creations/updates on the docker swarm.
    • It essentially automates the creation and management of multiple related docker services.

Summary

Here’s a table summarizing the key distinctions:

Feature docker swarm (Swarm mode) docker service docker compose (CLI tool) docker stack
Primary Goal Cluster Management & Orchestration Manage single service in Swarm Manage multi-container app on 1 host Manage multi-service app in Swarm
Scope Multi-host cluster Single service (across cluster) Single host Multi-service app (across cluster)
Input init, join commands CLI flags, parameters docker-compose.yml docker-compose.yml (v3+)
Creates/Manages Nodes, Services, Networks in cluster Replicated Tasks (Containers) for one app component Containers, Networks, Volumes (local) Swarm Services, Networks (cluster-wide)
Analogy Orchestra Pit & Conductor Instructions for one orchestra section Blueprint for local Lego assembly Full musical score for the symphony
When to Use To set up a Docker cluster To manage individual components in Swarm Local dev, single-host deployment Deploying full apps to Swarm

How they flow together in a Swarm deployment:

  1. Initialize the Swarm:
    bash # On manager node 1 docker swarm init # On other manager/worker nodes docker swarm join --token <TOKEN> <MANAGER_IP>:<PORT>
    You now have a docker swarm.

  2. Define your multi-service application:
    Create a docker-compose.yml file (e.g., myapp-stack.yml):
    yaml version: '3.8' services: web: image: myapp/web:latest ports: - "80:80" deploy: replicas: 3 api: image: myapp/api:v1 deploy: replicas: 2 # ... other services, networks, etc.
    This file is compatible with both docker compose (for local testing) and docker stack.

  3. Deploy the application stack to the Swarm:
    bash docker stack deploy -c myapp-stack.yml myapp
    docker stack reads the myapp-stack.yml file. For each service defined (e.g., web, api), it internally creates or updates a docker service on the Swarm.

  4. Manage individual services (if needed, though often managed via stack updates):
    bash docker service ls docker service scale myapp_web=5 # Scales the 'web' service within the 'myapp' stack docker service logs myapp_api

So, docker swarm is the environment. docker stack is the preferred way to deploy applications to that environment using a docker-compose.yml file, and docker stack itself uses docker service primitives under the hood. docker compose is the tool you’d use with the same docker-compose.yml file for local development on a single machine.

Page last modified: 2025-05-23 15:34:32