Skip to content

Latest commit

 

History

History
659 lines (495 loc) · 16 KB

MULTIARCH.md

File metadata and controls

659 lines (495 loc) · 16 KB

Multi-Architecture Support Guide

Lightstack supports both ARM64 (Apple Silicon, AWS Graviton, Raspberry Pi 4+) and AMD64 (Intel/AMD) architectures with native performance and automatic platform detection.


Table of Contents


Overview

Supported Platforms

Platform Architecture Status Use Case
linux/arm64 aarch64/ARM64 ✅ Tested Apple Silicon, AWS Graviton, Raspberry Pi 4+
linux/amd64 x86_64 ✅ Tested Intel/AMD CPUs, most cloud VMs

Key Features

  • ✅ Zero code changes - Pure Go cross-compiles automatically
  • ✅ Automatic detection - Docker selects correct architecture
  • ✅ Native performance - No emulation overhead when using correct arch
  • ✅ Single Dockerfile - Multi-arch support built-in
  • ✅ Docker Compose compatible - Works seamlessly

Quick Start

Build for Current Platform

The simplest approach - automatically detects your architecture:

cd /path/to/lightstack

# Using convenience script
./deployments/docker/build-local.sh

# Or with Docker Compose
docker compose build lightstack

# Or manually
docker build -f deployments/docker/Dockerfile -t lightstack:latest .

Use with Docker Compose

Docker Compose automatically uses the correct architecture:

docker compose up -d

No configuration needed! Docker detects your platform and uses the appropriate image.


Architecture Detection

How It Works

The Dockerfile uses Docker's built-in multi-architecture support:

# Builder stage - runs on native architecture
FROM --platform=$BUILDPLATFORM golang:1-alpine3.24 AS builder

# Build arguments automatically set by Docker
ARG TARGETOS      # Target OS (linux)
ARG TARGETARCH    # Target arch (arm64 or amd64)

# Cross-compile for target architecture
RUN CGO_ENABLED=0 GOOS=${TARGETOS} GOARCH=${TARGETARCH} \
    go build -ldflags="-w -s" -o lightstack ./cmd/lightstack

# Runtime stage - uses target architecture
FROM alpine:3.24
COPY --from=builder /build/lightstack /app/lightstack

Build Flow

┌─────────────────────────────────────────┐
│  Docker Build Command                   │
│  (with or without --platform flag)      │
└──────────────┬──────────────────────────┘
               │
               ▼
┌─────────────────────────────────────────┐
│  Builder Stage                          │
│  - Runs on: $BUILDPLATFORM (native)    │
│  - Compiles for: $TARGETARCH            │
│  - Fast (uses native Go toolchain)      │
└──────────────┬──────────────────────────┘
               │
               ▼
┌─────────────────────────────────────────┐
│  Runtime Stage                          │
│  - Runs on: $TARGETARCH                 │
│  - Contains compiled binary             │
│  - Small                                │
└─────────────────────────────────────────┘

Benefits:

  1. Builder runs natively → Fast compilation
  2. Cross-compilation is pure Go → No CGO dependencies
  3. Final image matches target → Native runtime performance

Building Images

Option 1: Single Platform Build (Fast)

Build only for your current platform:

Using script:

./deployments/docker/build-local.sh

Manual:

# Automatic platform detection
docker build -f deployments/docker/Dockerfile -t lightstack:latest .

# Explicit platform
docker build \
  --platform linux/arm64 \
  -f deployments/docker/Dockerfile \
  -t lightstack:arm64 \
  .

Build time: ~10 seconds (native compilation)

Option 2: Multi-Platform Build (Push to Registry)

Build for both architectures and push to Docker Hub:

Setup (one-time):

docker buildx create --name multiarch --use
docker buildx inspect --bootstrap

Build and push:

./deployments/docker/build-multiarch.sh yourusername/lightstack latest true

Or manually:

docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -t yourusername/lightstack:latest \
  --push \
  -f deployments/docker/Dockerfile \
  .

Result: Single manifest that works on both architectures!

# Pull on any platform - gets correct architecture automatically
docker pull yourusername/lightstack:latest

# On ARM Mac → gets ARM64 image
# On Intel/AMD → gets AMD64 image

Build time: ~60 seconds (parallel cross-compilation)

Option 3: Multi-Platform Build (Local)

Build for both architectures but keep locally:

docker buildx build \
  --platform linux/amd64,linux/arm64 \
  -f deployments/docker/Dockerfile \
  -t lightstack:multiarch \
  .

Note: To use locally, load for your platform:

docker buildx build \
  --platform linux/$(uname -m | sed 's/x86_64/amd64/;s/aarch64/arm64/') \
  --load \
  -t lightstack:latest \
  -f deployments/docker/Dockerfile \
  .

Verification

Check Image Architecture

# Inspect built image
docker inspect lightstack:latest | jq '.[0].Architecture'

# Output examples:
# "arm64"  → ARM64 image
# "amd64"  → AMD64 image

Check Running Container

# Check container architecture
docker exec lightstack uname -m

# Output examples:
# "aarch64" → ARM64 (running natively on ARM)
# "x86_64"  → AMD64 (running natively on Intel/AMD)

Verify Binary Architecture

# Run quick test
docker run --rm lightstack:latest uname -m

# Or inside running container
docker exec lightstack /bin/sh -c "uname -m"

Check Multi-Arch Manifest

For images pushed to registry:

docker manifest inspect yourusername/lightstack:latest

Expected output:

{
  "manifests": [
    {
      "platform": {
        "architecture": "amd64",
        "os": "linux"
      }
    },
    {
      "platform": {
        "architecture": "arm64",
        "os": "linux"
      }
    }
  ]
}

Cross-Compilation

Build ARM64 on AMD64 Machine

docker build \
  --platform linux/arm64 \
  -f deployments/docker/Dockerfile \
  -t lightstack:arm64 \
  .

Note: Uses QEMU emulation (slower, ~45 seconds)

Build AMD64 on ARM64 Machine (Mac M1/M2/M3)

docker build \
  --platform linux/amd64 \
  -f deployments/docker/Dockerfile \
  -t lightstack:amd64 \
  .

Note: Uses QEMU emulation (slower, ~45 seconds)

Test Cross-Compiled Image

# Build for different platform
docker build --platform linux/amd64 -t lightstack:amd64 -f deployments/docker/Dockerfile .

# Run with platform override (uses emulation)
docker run --rm --platform linux/amd64 lightstack:amd64 uname -m
# Output: x86_64 (even on ARM Mac!)

# Verify it works
docker run --rm --platform linux/amd64 lightstack:amd64 /app/lightstack --help

CI/CD Integration

GitHub Actions

name: Build Multi-Arch Docker Image

on:
  push:
    branches: [main]
    tags: ['v*']

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v2

      - name: Login to Docker Hub
        uses: docker/login-action@v2
        with:
          username: ${{ secrets.DOCKERHUB_USERNAME }}
          password: ${{ secrets.DOCKERHUB_TOKEN }}

      - name: Extract metadata
        id: meta
        uses: docker/metadata-action@v4
        with:
          images: yourusername/lightstack
          tags: |
            type=ref,event=branch
            type=ref,event=pr
            type=semver,pattern={{version}}
            type=semver,pattern={{major}}.{{minor}}

      - name: Build and push
        uses: docker/build-push-action@v4
        with:
          context: .
          file: deployments/docker/Dockerfile
          platforms: linux/amd64,linux/arm64
          push: true
          tags: ${{ steps.meta.outputs.tags }}
          labels: ${{ steps.meta.outputs.labels }}
          cache-from: type=gha
          cache-to: type=gha,mode=max

GitLab CI

build:
  image: docker:latest
  services:
    - docker:dind
  before_script:
    - docker login -u $CI_REGISTRY_USER -p $CI_REGISTRY_PASSWORD $CI_REGISTRY
    - docker buildx create --use
  script:
    - docker buildx build
      --platform linux/amd64,linux/arm64
      --push
      -t $CI_REGISTRY_IMAGE:$CI_COMMIT_SHA
      -t $CI_REGISTRY_IMAGE:latest
      -f deployments/docker/Dockerfile
      .
  only:
    - main

Jenkins

pipeline {
    agent any
    stages {
        stage('Build Multi-Arch') {
            steps {
                sh '''
                    docker buildx create --name multiarch --use || true
                    docker buildx build \
                        --platform linux/amd64,linux/arm64 \
                        --push \
                        -t yourusername/lightstack:${BUILD_NUMBER} \
                        -t yourusername/lightstack:latest \
                        -f deployments/docker/Dockerfile \
                        .
                '''
            }
        }
    }
}

Performance

Build Times (Apple M2 Mac)

Target Platform Build Time Method
ARM64 (native) ~10s Native Go compilation
AMD64 (cross) ~45s QEMU emulation
Both (buildx) ~60s Parallel cross-compilation

Runtime Performance

Platform CPU Type Performance Notes
ARM64 native Apple M2 100% Full native performance
AMD64 native Intel/AMD 100% Full native performance
ARM64 emulated Intel/AMD ~40% QEMU overhead (not recommended)
AMD64 emulated Apple M2 ~40% QEMU overhead (not recommended)

Recommendation: Always deploy native architecture images for production.

Why Native Builds Matter

Native ARM64 on Apple M2:
- API request: <5ms
- VM creation: <1s
- Zero emulation overhead

Emulated AMD64 on Apple M2:
- API request: ~15ms (3x slower)
- VM creation: ~3s (3x slower)
- QEMU translation overhead

Troubleshooting

Error: "exec format error"

Cause: Running an image built for a different architecture.

Symptom:

docker run lightstack:latest
# standard_init_linux.go:228: exec user process caused: exec format error

Solution:

# Option 1: Rebuild for your platform
docker compose build lightstack

# Option 2: Specify platform explicitly
docker run --platform linux/$(uname -m | sed 's/x86_64/amd64/;s/aarch64/arm64/') lightstack:latest

# Option 3: Use multi-arch image from registry
docker pull yourusername/lightstack:latest  # Auto-selects correct arch

Slow Build Times on Cross-Compilation

Cause: QEMU emulation overhead when building for different architecture.

Solution:

# Option 1: Build only for your platform (fast)
./deployments/docker/build-local.sh

# Option 2: Use native builders for each architecture (in CI/CD)
# - ARM runner for ARM builds
# - x86 runner for x86 builds

# Option 3: Accept slower cross-compilation (ok for occasional builds)

Buildx Not Found

Error: docker: 'buildx' is not a docker command

Solution:

# Option 1: Install buildx plugin
docker buildx install

# Option 2: Use Docker Desktop (includes buildx)
# Download from: https://www.docker.com/products/docker-desktop

# Option 3: Install manually
mkdir -p ~/.docker/cli-plugins
curl -SL https://github.com/docker/buildx/releases/download/v0.12.0/buildx-v0.12.0.linux-amd64 \
  -o ~/.docker/cli-plugins/docker-buildx
chmod +x ~/.docker/cli-plugins/docker-buildx

QEMU Not Registered

Error: image operating system "linux" cannot be used on this platform

Solution:

# Register QEMU handlers
docker run --privileged --rm tonistiigi/binfmt --install all

# Verify
docker buildx ls

Architecture Benefits

ARM64 Advantages

  • ✅ Better power efficiency - Ideal for edge deployments, IoT
  • ✅ Cost-effective - AWS Graviton instances up to 40% cheaper
  • ✅ Apple Silicon - Native performance on M1/M2/M3 Macs
  • ✅ Future-proof - Industry trend toward ARM
  • ✅ Lower heat - Better thermal characteristics

Best for:

  • Apple Silicon development machines
  • AWS Graviton instances (cost savings)
  • Edge computing deployments
  • ARM-based Kubernetes clusters

AMD64 Advantages

  • ✅ Wide compatibility - Works on most cloud providers
  • ✅ Mature ecosystem - More tools and libraries available
  • ✅ High single-thread performance - Better for CPU-intensive tasks
  • ✅ Established platform - Decades of optimization

Best for:

  • Traditional cloud VMs (AWS EC2, GCP, Azure)
  • On-premises data centers
  • Legacy infrastructure
  • Maximum software compatibility

Lightstack Multi-Arch Benefits

  • ✅ Zero code changes - Pure Go, no architecture-specific code
  • ✅ Single codebase - Same source builds for both platforms
  • ✅ Automatic detection - Docker selects correct architecture
  • ✅ No CGO dependencies - Clean cross-compilation
  • ✅ Production-ready - Tested on both platforms

Best Practices

1. Development

# Local development: Use native build (fast)
docker compose build lightstack
docker compose up -d

2. CI/CD

# CI/CD: Build both architectures
docker buildx build \
  --platform linux/amd64,linux/arm64 \
  --push \
  -t yourusername/lightstack:${VERSION} \
  .

3. Production Deployment

# docker-compose.yml
services:
  lightstack:
    image: yourusername/lightstack:latest  # Multi-arch manifest
    # No platform specified - auto-detects!

4. Testing

# Test both architectures before release
docker buildx build --platform linux/amd64,linux/arm64 -t lightstack:test .

# Load and test each
docker buildx build --platform linux/amd64 --load -t lightstack:amd64 .
docker run --rm lightstack:amd64 /app/lightstack --help

docker buildx build --platform linux/arm64 --load -t lightstack:arm64 .
docker run --rm lightstack:arm64 /app/lightstack --help

Summary

What Works

  • ✅ ARM64 native builds (Apple Silicon, Graviton)
  • ✅ AMD64 native builds (Intel/AMD)
  • ✅ Cross-compilation (ARM ↔ AMD64)
  • ✅ Docker Compose auto-detection
  • ✅ Multi-arch registry images
  • ✅ CI/CD integration

Implementation Status

  • ✅ Dockerfile with multi-arch support
  • ✅ Build scripts (local and multi-arch)
  • ✅ Docker Compose integration
  • ✅ Verified on both architectures
  • ✅ Production-ready

Key Takeaways

  1. Just works - docker compose up -d on any platform
  2. Fast native builds - ~10 seconds on native architecture
  3. Single codebase - No platform-specific code
  4. Production-ready - Tested and verified
  5. Cost-effective - Use cheaper ARM instances when beneficial

Ready to deploy on any architecture! 🚀


Additional Resources


Next Steps: