Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
108 changes: 108 additions & 0 deletions .github/ISSUE_TEMPLATE/bug.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
name: Bug report
description: Report a deviation from upstream OpenStack API behavior or a lightstack defect
labels: ["bug", "triage"]
body:
- type: markdown
attributes:
value: >
Thanks for taking the time to file a bug. Per [SPEC-000](docs/specs/000-api-compliance),
OpenStack API compliance is priority zero: every bug report should be anchored in what
upstream OpenStack actually does. If you can cite the upstream API reference (or the
source service) for the expected behavior, triage gets much faster.
- type: textarea
id: summary
attributes:
label: Bug summary
description: One or two sentences. What is broken, and which OpenStack API does it deviate from?
placeholder: >
POST /v2.1/{tenant_id}/servers returns 500 instead of 202 when block_device_mapping_v2
references a non-existent volume; upstream Nova returns a 400 with a structured fault.
validations:
required: true
- type: dropdown
id: service
attributes:
label: Affected service
options:
- Keystone (35357)
- Nova (8774)
- Neutron (9696)
- Cinder (8776)
- Glance (9292)
- Placement
- Metadata
- Cross-service
- Other
validations:
required: true
- type: input
id: endpoint
attributes:
label: API endpoint and HTTP method
description: The exact route you called, including the version prefix.
placeholder: "POST /v2.1/{tenant_id}/servers"
validations:
required: true
- type: input
id: microversion
attributes:
label: Microversion (if applicable)
description: Nova 2.1–2.90, Cinder 3.0–3.71. Leave empty if the API is not versioned.
placeholder: "X-OpenStack-Nova-API-Version: 2.90"
- type: textarea
id: repro-steps
attributes:
label: Reproduction steps
description: What you did, in order. Include the client you used (openstack CLI, Terraform provider, Horizon, gophercloud, raw curl).
placeholder: |
1. Authenticate as a non-admin tenant user
2. Create a server with ...
3. Observe ...
validations:
required: true
- type: textarea
id: curl-repro
attributes:
label: curl reproduction (optional)
description: A self-contained request we can replay against a running lightstack. Redact tokens and secrets.
render: shell
placeholder: |
curl -si -X POST http://localhost:8774/v2.1/$TENANT/servers \
-H "X-Auth-Token: $TOKEN" \
-H "Content-Type: application/json" \
-d '{...}'
- type: textarea
id: expected
attributes:
label: Expected behavior (upstream OpenStack reference)
description: >
What upstream OpenStack does here — status code, response body, error semantics. Cite the
docs.openstack.org API reference, the gophercloud contract, or the upstream service source.
Compliance with upstream is priority zero (SPEC-000), so this is the anchor for the fix.
placeholder: >
https://docs.openstack.org/api-ref/compute/... — upstream returns 202 with the server
representation and sets the OS-EXT-STS:task_state to 'scheduling'; a missing image ref yields
400 itemNotFound fault.
validations:
required: true
- type: textarea
id: actual
attributes:
label: Actual behavior
description: Status code, response body, and/or panic log you got from lightstack.
validations:
required: true
- type: textarea
id: logs
attributes:
label: Logs (optional)
description: Relevant lightstack log output or panic stack trace. Wrap long output in a code block; redact tokens.
render: text
- type: input
id: environment
attributes:
label: Environment
description: >
lightstack version/release you deployed (see `CHANGELOG.md`), deployment mode (stub / real), storage mode,
networking mode — see docs/STORAGE_MODES.md and docs/NETWORKING_MODES.md.
placeholder: "lightstack v0.x.y, real mode, storage=ceph-rgw, networking=vxlan"
11 changes: 11 additions & 0 deletions .github/ISSUE_TEMPLATE/config.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
blank_issues_enabled: false
contact_links:
- name: Discussions
url: https://github.com/cobaltcore-dev/lightstack/discussions
about: Questions, design ideas, and deployment help — start here before opening an issue.
- name: Security vulnerabilities
url: https://github.com/cobaltcore-dev/lightstack/security/advisories/new
about: >
Report vulnerabilities privately via a GitHub security advisory — do NOT open a public issue.
For triage of SBOM findings (dependency advisories), see docs/security/SBOM_TRIAGE.md;
known/not-applicable findings are tracked in docs/security/openvex.json.
64 changes: 64 additions & 0 deletions .github/ISSUE_TEMPLATE/feature.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
name: Feature request
description: Propose new API surface or behavior — ideally to close a gap with upstream OpenStack
labels: ["enhancement", "triage"]
body:
- type: markdown
attributes:
value: >
Before proposing something new, check the [roadmap](ROADMAP.md) and
[STATUS.md](STATUS.md) — much of the plan is driven by upstream parity. If this request
closes an API gap with upstream OpenStack, say so up front; per [SPEC-000](docs/specs/000-api-compliance)
compliance is priority zero and parity work is prioritized accordingly.
- type: textarea
id: problem
attributes:
label: Problem statement
description: What can't you do today, and who is blocked by it? If a real client (openstack CLI, Terraform provider, Horizon, gophercloud) fails against lightstack, name it.
placeholder: >
The Terraform openstack provider calls GET /v2.1/{tenant_id}/servers/detail?changes-since=...
during refresh; lightstack ignores the changes-since filter so the provider sees stale state.
validations:
required: true
- type: textarea
id: upstream-parity
attributes:
label: Upstream OpenStack parity target
description: >
Is there an upstream API this should match? Cite the docs.openstack.org API reference
(or upstream service source). SPEC-000 makes upstream compliance priority zero, so
"upstream does X, we don't" is the strongest possible justification here. If this is
deliberately lightstack-specific, say so and why.
placeholder: >
https://docs.openstack.org/api-ref/compute/... — changes-since is a standard list filter
since microversion 2.1; upstream returns only servers updated after the timestamp.
validations:
required: true
- type: textarea
id: api-surface
attributes:
label: Proposed API surface (optional)
description: Endpoint(s), request/response shapes, microversion gating. Match upstream naming exactly — clients depend on it.
render: text
placeholder: |
GET /v2.1/{tenant_id}/servers/detail?changes-since=<ISO8601>
→ 200 {"servers": [...]} filtered server-side; 400 on malformed timestamp (upstream behavior).
- type: checkboxes
id: client-impact
attributes:
label: Client impact
description: Which OpenStack clients would exercise this? Helps us prioritize contract tests in test/contract/.
options:
- label: OpenStack CLI (python-openstackclient)
- label: Terraform provider (terraform-provider-openstack)
- label: Horizon dashboard
- label: gophercloud SDKs
- type: textarea
id: alternatives
attributes:
label: Alternatives considered (optional)
description: Other ways to solve this, including "wait for upstream parity work" — and why they fall short.
- type: textarea
id: context
attributes:
label: Additional context (optional)
description: Links, sketches, related specs in docs/specs/ or ADRs in docs/adr/.
30 changes: 30 additions & 0 deletions .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
## What & why

<!-- What does this PR do and why? Link the issue it closes. -->
Fixes #

<!-- If this closes or narrows an API-compliance gap, say what upstream OpenStack does. -->

## Spec / ADR references

<!-- If this implements or amends a spec in docs/specs/ or an ADR in docs/adr/, cite it. -->
- [ ] Spec/ADR referenced (e.g. `docs/specs/000-api-compliance`) or "none — bugfix/docs only"

## Checklist

- [ ] Tests added/updated — unit tests for logic, contract tests in `test/contract/` (gophercloud-based) for any API surface change
- [ ] `make build` passes
- [ ] Changelog entry added to `CHANGELOG.md` under `[Unreleased]` (user-visible changes only)
- [ ] Docs updated under `docs/` (see `docs/CONTRIBUTING.md`) where behavior or configuration changed
- [ ] Every commit carries `Signed-off-by: Name <name@sap.com>` — DCO; the "SAP Sign-off Check" CI job enforces this. Use `git commit -s`.
- [ ] No secrets, tokens, or credentials in the diff

## Testing evidence

<!-- How you verified this — commands and results. For API changes, include the curl or contract-test run against a running lightstack. -->
```
$ make build
...
$ go test ./test/contract/... -run TestNovaServers
...
```
27 changes: 13 additions & 14 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version: '1.27.2'
go-version: '1.27.3'

- name: Cache Go modules
uses: actions/cache@v5
Expand Down Expand Up @@ -83,7 +83,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version: '1.27.2'
go-version: '1.27.3'

- name: Cache Go modules
uses: actions/cache@v5
Expand Down Expand Up @@ -118,7 +118,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@v6
with:
go-version: '1.27.2'
go-version: '1.27.3'

- name: Cache Go modules
uses: actions/cache@v5
Expand Down Expand Up @@ -322,25 +322,24 @@ jobs:
name: Helm Charts
runs-on: ubuntu-latest
timeout-minutes: 10
env:
HELM_VERSION: v3.22.1
HELM_UNITTEST_VERSION: v1.2.1

steps:
- name: Checkout code
uses: actions/checkout@v6

# Same mechanism as `make install-tools`: the official get-helm-3 script plus the
# helm-unittest plugin, which `make chart-test` hard-requires (Makefile:103-104).
- name: Install Helm and the helm-unittest plugin
- name: Install pinned Helm and helm-unittest
run: |
if ! command -v helm >/dev/null 2>&1; then
# Downloaded, not piped: the default run shell is `bash -e` without pipefail,
# so `curl ... | bash` would swallow a failed download.
curl -fsSL -o "$RUNNER_TEMP/get-helm-3" \
https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
bash "$RUNNER_TEMP/get-helm-3"
fi
if ! helm plugin list 2>/dev/null | grep -q '^unittest'; then
helm plugin install https://github.com/helm-unittest/helm-unittest
fi
export HELM_PLUGINS="$RUNNER_TEMP/helm-plugins"
echo "HELM_PLUGINS=$HELM_PLUGINS" >> "$GITHUB_ENV"
curl -fsSL -o "$RUNNER_TEMP/get-helm-3" \
https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3
bash "$RUNNER_TEMP/get-helm-3" --version "$HELM_VERSION"
helm plugin install https://github.com/helm-unittest/helm-unittest --version "$HELM_UNITTEST_VERSION"
helm version --short

# Lints + renders + unit-tests both product charts: charts/lightstack, then
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/security.yml
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ jobs:
- name: Set up Go
uses: actions/setup-go@b7ad1dad31e06c5925ef5d2fc7ad053ef454303e # v7.0.0
with:
go-version: '1.27.2'
go-version: '1.27.3'
cache: true

- name: Scan Go code and pinned development tool
Expand Down
104 changes: 104 additions & 0 deletions AGENTS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,104 @@
# AGENTS.md

Guidance for any coding agent (Claude Code, Codex, Cursor, …) working in this repository.
For deeper Claude Code-specific guidance see `CLAUDE.md`; contribution process: `docs/CONTRIBUTING.md`.

## What this is

Lightstack is a single Go binary that reimplements the five core OpenStack services
(Keystone, Nova, Neutron, Cinder, Glance) plus a Placement stub and an EC2-compatible
metadata service — inspired by how K3s simplified Kubernetes. One process, one PostgreSQL
database, zero message queues, synchronous APIs.

**SPEC-000: OpenStack API compliance is priority zero.** Upstream OpenStack behavior is the
reference implementation — never invent semantics. If Lightstack and upstream disagree,
Lightstack is wrong. All other specifications must comply with SPEC-000
(`docs/specs/SPEC-000-COMPLIANCE-ADDENDUM.md`).

One process, seven HTTP listeners: Keystone `:35357`, Nova `:8774`, Neutron `:9696`,
Cinder `:8776`, Glance `:9292`, Placement `:8778`, Metadata `:8775` (unauthenticated).

Each service supports **stub mode** (fake data, no external dependencies — default, works on
macOS) and **real mode** (libvirt, netlink, Ceph RBD, S3 — Linux only), selected per service in
`config/lightstack.yaml` (`nova.libvirt_mode`, `neutron.networking_mode`,
`cinder.storage_mode`, `glance.storage_mode`). Storage modes may be comma-separated hybrids
(`local,rbd`). Note: the `rate_limiting.*` config block is **dead config** — no code reads it.

## Repo map

| Path | Contents |
|------|----------|
| `cmd/lightstack/` | Main binary: server plus subcommands `cleanup` (SPEC-010 teardown), `console-tokens` (SPEC-021), `healthcheck` (SPEC-013) |
| `cmd/lightstack-migrate/` | Standalone migration binary |
| `internal/<service>/` | One package per OpenStack service: `keystone`, `nova`, `neutron`, `cinder`, `glance`, plus `placement`, `metadata` |
| `internal/common/` | Config loading, error helpers/pagination, `BaseURL`, endpoint `scheme`, SQL builder, roles |
| `internal/database/` | pgx pool usage, models, `WithTx` transactions, migration wiring (golang-migrate) |
| `pkg/paths/` | `state_path` resolution (node-local state root, default `/var/lib/lightstack`) |
| `pkg/hypervisor`, `pkg/networking`, `pkg/storage` | libvirt, netlink/VXLAN, and storage backends (local/RBD/S3) |
| `test/contract/` | gophercloud contract tests — the OpenStack API compliance gate |
| `test/*.sh`, `test/terraform/` | Bash integration suites (OpenStack CLI) and Terraform conformance |
| `docs/specs/NNN-*/` | Feature specs: `spec.md`, `plan.md`, `tasks.md`, `contracts/`, `checklists/` |
| `docs/adr/` | Architecture decision records |
| `docs/review/` | Comprehensive review reports (fix loops run until CLEAN) |
| `charts/lightstack/`, `charts/horizon/` | Helm charts |
| `deployments/` | Docker compose stacks, horizon config, systemd units |
| `migrations/` | Numbered `NNN_name.up.sql` / `.down.sql` migrations |

## Ground truth commands

```bash
make build # Build bin/lightstack
make lint # golangci-lint (pinned via tools/ module)
make test # Unit tests (excludes test/contract)
make test-contract # gophercloud contract tests — needs a RUNNING server
# (docker compose -f deployments/docker-compose.yml up -d, or make run)
make chart-test # Helm lint + template + unittest (needs helm + helm-unittest; make install-tools)
```

DB-backed unit tests need an existing scratch database they may **migrate**: the
harness connects to the database named in `LIGHTSTACK_TEST_DB_URL` and applies all
migrations — it does **not** create the database (see `internal/common/maindb_test.go`),
so never point it at anything shared:

```bash
make db-up # start dev PostgreSQL in Docker
# create the scratch database once (the harness migrates it but does not create it)
docker exec lightstack-postgres psql -U lightstack -c 'CREATE DATABASE lightstack_test'
LIGHTSTACK_TEST_DB_URL=postgres://lightstack:secret@localhost:5432/lightstack_test make test
make migrate # apply migrations (golang-migrate)
```

Without `LIGHTSTACK_TEST_DB_URL`, DB-backed unit tests skip rather than fail.

## Hard rules

- **DCO sign-off on every commit**: `git commit -s`, `Signed-off-by: Name <name@sap.com>`.
The "SAP Sign-off Check" CI job rejects any PR commit missing a valid `@sap.com` trailer.
- **Conventional commits**: `feat(nova): …`, `fix(cinder): …`, `refactor(glance): …`,
`docs`, `test`, `chore`.
- **Branches**: `main`, `stable*`, and `releases*` are protected — work on PR branches only.
- **API changes need contract tests**: any endpoint added or behavior changed gets a test in
`test/contract/` (gophercloud). This is the SPEC-000 gate CI runs.
- **Tenant isolation and admin-only surfaces are security-critical** — see
`docs/specs/022-project-tenant-isolation/` and `docs/specs/023-tenant-scoping-authz/`.
Resources are scoped by `project_id`; the `admin` role override and `?all_tenants` follow
upstream `admin_or_owner` semantics. Do not loosen scoping predicates.
- **Never commit secrets** (JWT secrets, passwords, keys). The seeded `admin`/`secret`
credentials are development-only defaults.

## Working agreements

- **Significant features follow the spec pattern** (`docs/specs/README.md`): a numbered
`docs/specs/NNN-name/` directory with `spec.md` → `plan.md` → `tasks.md` → implementation →
validation. Read the spec covering the code you touch before changing it.
- **Expect review-fix loops**: comprehensive reviews (`docs/review/`) produce findings that are
fixed and re-reviewed until the report is CLEAN. Budget for iteration.
- **Changelog entries** for user-visible changes (`CHANGELOG.md`).
- **Never edit an applied migration's meaning** — add a new numbered migration instead.
- Prefer the shared helpers over re-implementations: `common.BaseURL`, `common.HasRole`,
`nova.targetInstance`, `database.WithTx`. The recurring review finding is "the right shared
thing exists and one caller bypasses it."
- **Test-first**: write the failing test (unit, contract, or integration script) before the
implementation; confirm it fails, then make it pass.
- Errors: wrap with `fmt.Errorf("...: %w", err)` and surface HTTP errors via the helpers in
`internal/common/error_helpers.go` — never leak raw error strings to API clients.
Loading
Loading