Skip to content

getting-started.md

Latest commit

 

History

History
275 lines (214 loc) · 10.5 KB

getting-started.md

File metadata and controls

275 lines (214 loc) · 10.5 KB
Raw

Getting started

Documentation index · Configuration · Operations

This guide runs one local control plane, retains its state in SQLite, and uses ordinary AWS clients against it. It does not launch containers, virtual machines, or resources in real AWS.

1. Build from source

Install Go 1.26.5 or newer, Git, and optionally AWS CLI v2 and curl. The Go version is declared in go.mod. Runtime-specific Linux, Docker, systemd, KVM, image, and helper requirements are separate; see Runtime prerequisites.

git clone https://github.com/radkawar/stackd.git
cd stackd
go version
go build -o bin/stackd ./cmd/stackd
./bin/stackd -h

Skip cloning if you already have the repository. Run subsequent shell commands from its root unless stated otherwise. A normal build uses checked-in generated code; it does not require the ignored AWS SDK source checkout in clones/. The initial build needs network access for Go modules unless your module cache has already been populated. No ready-made container image or release installer is required by this guide.

For actual Lambda execution, make build also builds the Linux telemetry helper executables. That does not install Docker, runtime images, or the other native prerequisites.

2. Start a persistent local instance

In terminal A:

mkdir -p data
./bin/stackd \
  -listen 127.0.0.1:4566 \
  -database ./data/stackd.sqlite

Leave it in the foreground. Wait for the AWS endpoint listening startup log. In terminal B:

curl --fail http://127.0.0.1:4566/_stackd/health

A successful health response means the HTTP control plane is available. It is not proof that an optional database, guest, function, or other resource is ready. Those resources have service-specific readiness states.

Omit -database for an ephemeral in-memory instance. Its resource state is lost when the process exits. Do not run two controllers against the same database.

3. Configure a local client shell

Use a dedicated terminal rather than changing your normal AWS profile:

export AWS_ACCESS_KEY_ID=test
export AWS_SECRET_ACCESS_KEY=test
export AWS_DEFAULT_REGION=us-east-1
export AWS_EC2_METADATA_DISABLED=true
export AWS_PAGER=""
unset AWS_SESSION_TOKEN AWS_PROFILE AWS_DEFAULT_PROFILE

aws --endpoint-url http://127.0.0.1:4566 sts get-caller-identity

Fresh state generates a random 12-digit account ID. The returned identity is its local root. To choose the ID, use --account-id 123456789012 at creation. SQLite keeps the ID across restarts. A conflicting flag returns an error. For old databases with no saved selection, see account selection and upgrades. A 12-digit access key with secret test selects another local account without moving resources between accounts. These are fixture/bootstrap identities, not real AWS credentials. IAM-created keys and STS sessions instead use their issued secrets and session tokens and are subject to current local policy.

Keep the endpoint override on every CLI command. Omitting it normally sends the request to AWS. SDK endpoint overrides must be configured separately for each service client. Do not use production credentials for these examples.

4. Store and retrieve an S3 object

aws --endpoint-url http://127.0.0.1:4566 s3api create-bucket \
  --bucket hello-stackd

printf 'hello from stackd\n' > data/hello.txt
aws --endpoint-url http://127.0.0.1:4566 s3api put-object \
  --bucket hello-stackd --key hello.txt --body data/hello.txt

aws --endpoint-url http://127.0.0.1:4566 s3api get-object \
  --bucket hello-stackd --key hello.txt data/hello.download.txt
cmp data/hello.txt data/hello.download.txt

cmp exits successfully when the retrieved bytes match. For this us-east-1 example, no CreateBucketConfiguration is needed. S3 naming and authorization rules still apply; this is not a generic HTTP file server.

5. Send and receive an SQS message

QUEUE_URL=$(aws --endpoint-url http://127.0.0.1:4566 sqs create-queue \
  --queue-name docs-jobs --query QueueUrl --output text)

aws --endpoint-url http://127.0.0.1:4566 sqs send-message \
  --queue-url "$QUEUE_URL" --message-body 'hello from stackd'

aws --endpoint-url http://127.0.0.1:4566 sqs receive-message \
  --queue-url "$QUEUE_URL" --max-number-of-messages 1 --visibility-timeout 0

The response contains the message body and a receipt handle. In an application, delete successfully processed messages using that handle. Visibility timeout zero makes this demonstration immediately readable again; it is not a normal worker acknowledgment strategy.

6. Use an SDK

Python / boto3

Python is needed only for this client example, not for the Go control plane. Install boto3 in your application's environment, then run:

import boto3
from botocore.config import Config

s3 = boto3.client(
    "s3",
    endpoint_url="http://127.0.0.1:4566",
    region_name="us-east-1",
    aws_access_key_id="test",
    aws_secret_access_key="test",
    config=Config(s3={"addressing_style": "path"}),
)
response = s3.get_object(Bucket="hello-stackd", Key="hello.txt")
with response["Body"] as body:
    assert body.read() == b"hello from stackd\n"

Path-style addressing avoids needing wildcard DNS for bucket hostnames. Other SDKs should likewise use the local endpoint, explicit local credentials, a region, and path-style S3 unless you deliberately configure virtual-host routing.

AWS SDK for Go v2

Save this as /tmp/stackd-client.go and run go run /tmp/stackd-client.go from the repository root, using its pinned SDK dependencies:

package main

import (
    "context"
    "fmt"
    "log"

    "github.com/aws/aws-sdk-go-v2/aws"
    "github.com/aws/aws-sdk-go-v2/credentials"
    "github.com/aws/aws-sdk-go-v2/service/sts"
)

func main() {
    client := sts.New(sts.Options{
        Region:       "us-east-1",
        BaseEndpoint: aws.String("http://127.0.0.1:4566"),
        Credentials:  credentials.NewStaticCredentialsProvider("test", "test", ""),
    })
    identity, err := client.GetCallerIdentity(context.Background(), &sts.GetCallerIdentityInput{})
    if err != nil {
        log.Fatal(err)
    }
    fmt.Println(aws.ToString(identity.Account), aws.ToString(identity.Arn))
}

For in-process tests, see the embedding reference. Call Close on the stack and on caller-owned runtimes; an HTTP test server's shutdown alone does not own all external resources.

7. Restart and clean up

Press Ctrl-C in terminal A, wait for the process to exit, and start it again with the same command and database path. Repeat get-object above: the stored object should still be present. Keep the same account and region in your clients.

Remove the demonstration resources while the controller is running:

aws --endpoint-url http://127.0.0.1:4566 s3api delete-object \
  --bucket hello-stackd --key hello.txt
aws --endpoint-url http://127.0.0.1:4566 s3api delete-bucket \
  --bucket hello-stackd
aws --endpoint-url http://127.0.0.1:4566 sqs delete-queue --queue-url "$QUEUE_URL"

Then press Ctrl-C again. Preserve the database if you want to keep other state. For backups, resets, external-runtime shutdown and common errors, use the operations guide. Do not apply blanket Docker pruning or delete state directories to stop an instance.

Terraform provider API requirements

The provider API map covers production source in all service directories of HashiCorp AWS provider 6.68.0. The pinned commit is c9b83d813022cdd05fc955480b07e269c6e18023. You do not need a checkout to read the retained map.

To regenerate it, prepare the exact source revision:

git clone --branch v6.68.0 --depth 1 \
  https://github.com/hashicorp/terraform-provider-aws.git clones/terraform-provider-aws
make generate-terraform-apis
make generate-terraform-apis-check

Use priority_counts and top_selected_scope_api_gaps to find missing APIs. Each requirement includes SDK operation names, source locations and evidence type. Paginator constructors are included. Tests and sweeper files are excluded. Dynamic calls, wrappers and ambiguous receivers can remain unresolved.

lifecycle_inventory.provider_types lists every managed resource and data source from the provider's generated registrations. Aliases keep their actual Terraform names. lifecycle_inventory.implemented_services includes all 72 stackd services with registered owner behavior, including data-only APIs and services with no direct provider resource. It distinguishes direct resources from dependencies.

On 2026-10-14, generation found all 1,730 resource types and 685 data-source types. Their names matched the installed, pinned provider's terraform providers schema -json output exactly. This verifies inventory coverage, not resource behavior. The per-type API links cover the defining source files; they do not claim a complete transitive call graph. Shared helpers and initialization can add calls. Lifecycle and native-engine evidence default to unverified; registered APIs do not change that value to passed.

Without the checkout, -retained-source refreshes classifications from the retained source snapshot and current stackd operation inventory. It validates the snapshot structure but cannot check provider source drift. Use the pinned checkout for source verification:

go run ./cmd/terraformapisgen -retained-source -check
  • registered-partial: an operation is registered, not behaviorally complete.
  • unimplemented: a selected modeled operation has no registration.
  • outside-selected-service-scope: the API is outside the current service selection.
  • unresolved-or-ambiguous: the retained inputs do not establish a clear mapping.

This is a static source map, not a Terraform conformance score or IAM policy. Source frequency is not runtime frequency or resource popularity. Validate real create/read/update/delete workflows and their native data planes. For examples, see the current-provider EIP lifecycle and RDS capacity and SQL checks.

Next steps