Skip to content

docker.md

Latest commit

 

History

History
301 lines (211 loc) · 8.1 KB

docker.md

File metadata and controls

301 lines (211 loc) · 8.1 KB
Raw

Working with Docker

The sections below detail how to use Docker to build and run reproducible, portable environments for data science and software development projects.

Install required software

Note

Docker Desktop is also listed in the install sections of vscode.md, python_uv.md, and r_renv.md. You only need to install it once.

Core concepts

Term What it means
Image A read-only snapshot of a filesystem and runtime environment. Think of it as a template. Built from a Dockerfile.
Container A running instance of an image. Isolated from the host system, but can share files via volumes.
Dockerfile A text file with step-by-step instructions for building an image (install dependencies, copy code, set entrypoint, etc.).
docker-compose A tool for defining and running multi-container applications using a docker-compose.yml file. Also useful for single-container projects to avoid long docker run commands.
Volume A mechanism to mount a folder from the host machine into the container. Used to give containers access to local data without baking it into the image.

Essential commands

Building and running

# Build an image from the Dockerfile in the current directory, tag it as "image-name"
docker build -t image-name .

# Run a container interactively; --rm removes it automatically when it exits
docker run --rm -it image-name

# Run a container and mount a local data folder into /app/data inside the container
docker run --rm -v $(pwd)/data:/app/data image-name

# On Windows CMD, use %cd% instead of $(pwd):
docker run --rm -v %cd%/data:/app/data image-name

Managing containers

# List running containers
docker ps

# List all containers (including stopped ones)
docker ps -a

# Stop a running container
docker stop <container-id-or-name>

# Remove a stopped container
docker rm <container-id-or-name>

# View logs from a container (useful for debugging)
docker logs <container-id-or-name>

Managing images

# List all local images
docker images

# Remove an image
docker rmi image-name

Dockerfile for Python (uv-based)

This example uses a multi-stage build to keep the final image lean. It uses uv for dependency management.

# syntax=docker/dockerfile:1

# --- Stage 1: install dependencies ---
FROM ghcr.io/astral-sh/uv:python3.12-bookworm-slim AS builder

WORKDIR /app

# Copy only the files needed to resolve dependencies first (improves layer caching)
COPY pyproject.toml uv.lock ./

# Install production dependencies only (no dev dependencies) into a fixed location
RUN uv sync --no-dev --frozen

# --- Stage 2: final runtime image ---
FROM python:3.12-slim-bookworm

WORKDIR /app

# Copy the installed virtual environment from the builder stage
COPY --from=builder /app/.venv /app/.venv

# Copy application source code
COPY src/ ./src/

# Ensure the venv's Python is used
ENV PATH="/app/.venv/bin:$PATH"

ENTRYPOINT ["python", "src/main.py"]

Note

--frozen ensures the lockfile is not modified during the build - it will fail if uv.lock is out of date, which is the correct behaviour for reproducible builds.

Dockerfile for R

This example uses a rocker base image and restores dependencies from renv.lock.

FROM rocker/r-ver:4.4.0

WORKDIR /app

# Install system libraries commonly required by R packages
RUN apt-get update && apt-get install -y \
    libcurl4-openssl-dev \
    libssl-dev \
    libxml2-dev \
    && rm -rf /var/lib/apt/lists/*

# Install renv
RUN Rscript -e "install.packages('renv', repos='https://cloud.r-project.org')"

# Copy only the lockfile first to allow Docker layer caching of the restore step
COPY renv.lock ./

# Restore the exact package versions from the lockfile
RUN Rscript -e "renv::restore()"

# Copy the rest of the project
COPY . .

ENTRYPOINT ["Rscript", "src/main.R"]

Tip

Pin the rocker/r-ver tag to an explicit R version (e.g. 4.4.0) rather than using latest so builds remain reproducible over time.

docker-compose

docker-compose makes it easy to manage containers without memorising long docker run flags. Create a docker-compose.yml at the project root:

services:
  app:
    build: .
    image: my-project
    volumes:
      # Mount local data folder into the container (read-only)
      - ./data:/app/data:ro
      # Mount output folder so results are written back to the host
      - ./outputs:/app/outputs
    env_file:
      - .env

Common commands:

# Build the image (or rebuild after Dockerfile changes)
docker compose build

# Start the container
docker compose up

# Start in the background (detached mode)
docker compose up -d

# Stop and remove containers
docker compose down

Environment variables in Docker

Docker can read your .env file directly, tying into the pattern described in env.md.

With docker run:

docker run --rm --env-file .env image-name

With docker-compose (already shown above):

env_file:
  - .env

Warning

Never copy the .env file into the image with a COPY .env . instruction in the Dockerfile. The .env file contains secrets and local paths that should only be injected at runtime, not baked into the image. Add .env to .dockerignore to prevent it from being included accidentally.

.dockerignore

A .dockerignore file at the project root tells Docker which files to exclude when building the image. This speeds up builds and prevents secrets or large data files from being copied into the image.

Create .dockerignore with at minimum:

# Python
.venv/
__pycache__/
*.pyc
*.pyo

# R
renv/library/
.Rhistory
.Rproj.user/

# Environment and secrets
.env

# Data and outputs (provide these at runtime via volumes)
data/
outputs/

# Version control
.git/
.gitignore

# Editor
.vscode/

Dev Containers in VSCode

The Dev Containers extension lets VSCode open your project inside a Docker container, giving every team member an identical development environment with no local setup beyond Docker Desktop.

Prerequisites

Install the Dev Containers extension in VSCode (listed in vscode.md).

Setup

  1. Create a .devcontainer/ folder at the project root.

  2. Add a devcontainer.json file. A minimal example for a Python/uv project:

    {
    "name": "My Project",
    "build": {
        "dockerfile": "../Dockerfile"
    },
    "mounts": [
        "source=${localWorkspaceFolder}/data,target=/app/data,type=bind,consistency=cached"
    ],
    "customizations": {
        "vscode": {
            "extensions": [
                "ms-python.python",
                "ms-python.pylance",
                "charliermarsh.ruff"
            ]
        }
    },
    "postCreateCommand": "uv sync"
}

  3. Open VSCode and press Ctrl+Shift+P → Dev Containers: Reopen in Container. VSCode will build the image (or pull it) and reopen the workspace inside the container.

Note

Any VSCode extensions listed under customizations.vscode.extensions are installed inside the container automatically. Extensions installed on the host are not available inside the container unless listed here.