Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/sdk-server.md
Original file line number Diff line number Diff line change
Expand Up @@ -184,7 +184,7 @@ The tables below list the main handlers. They are not the full list: the generat
| **RDS / Aurora** *(query protocol)* | DBInstances (Create/Describe/Modify/Delete/Start/Stop/Reboot), DBClusters (Create/Describe/Modify/Delete/Start/Stop), DBSnapshots + DBClusterSnapshots (Create/Describe/Delete/Restore). One handler also serves the **Neptune** and **DocumentDB** engines: both reuse the same `aws-sdk-go-v2/service/{neptune,docdb}` client surface. |
| **Redshift** *(query protocol)* | CreateCluster, DescribeClusters, ModifyCluster, DeleteCluster, RebootCluster, CreateClusterSnapshot, DescribeClusterSnapshots, DeleteClusterSnapshot, RestoreFromClusterSnapshot |
| **MemoryDB** *(JSON 1.1, `AmazonMemoryDB.*`)* | Clusters (Create/Describe/Update/Delete/FailoverShard/ListAllowedNodeTypeUpdates), ACLs & Users, Parameter Groups, Subnet Groups, Snapshots (Create/Describe/Copy/Delete + restore), tags, engine-version & event catalogs. Optional (type-asserted): Multi-Region clusters, Reserved Nodes. Server-side `MaxResults`/`NextToken` pagination on every Describe. |
| **Keyspaces** *(JSON 1.0, `KeyspacesService.*`)* | Keyspaces (Create/Get/List/Update/Delete, single/multi-region replication), Tables (Create/Get/List/Update/Delete/Restore: full schema, capacity, encryption, PITR, TTL, CDC), user-defined types, tags. Optional: `GetTableAutoScalingSettings`. Responses use lower-camel keys so the case-sensitive SDK deserializer decodes them; pagination on every list. |
| **Keyspaces** *(JSON 1.0, `KeyspacesService.*`, and Smithy rpc-v2-cbor)* | Keyspaces (Create/Get/List/Update/Delete, single/multi-region replication), Tables (Create/Get/List/Update/Delete/Restore: full schema, capacity, encryption, PITR, TTL, CDC), user-defined types, tags. Optional: `GetTableAutoScalingSettings`. Responses use lower-camel keys so the case-sensitive SDK deserializer decodes them; pagination on every list. |
| **EKS** *(REST + JSON)* | Clusters (Create/Describe/List/UpdateConfig/UpdateVersion/Delete), NodeGroups (Create/Describe/List/UpdateConfig/UpdateVersion/Delete), Fargate Profiles (Create/Describe/List/Delete), Addons (Create/Describe/List/Update/Delete). The kubeconfig points at the shared [Kubernetes data plane](#kubernetes). |
| **IAM** *(query protocol)* | Users (Create/Get/List/Delete), Roles (Create/Get/List/Delete), Policies (Create/Get/List/Delete), Attach/Detach/ListAttached for both Users and Roles, Groups (Create/Get/List/Delete + AddUserToGroup/RemoveUserFromGroup/ListGroupsForUser), AccessKeys (Create/List/Delete), InstanceProfiles (Create/Get/List/Delete + AddRoleToInstanceProfile/RemoveRoleFromInstanceProfile). Errors surface as typed `*types.NoSuchEntityException` / `*types.EntityAlreadyExistsException`. |
| **Resource Explorer 2** *(JSON)* | Search: free-text plus filter expression over the cross-service inventory; results include ARN, resource type, region, owning account, and tags |
Expand Down Expand Up @@ -302,13 +302,13 @@ Each handler matches on a different signal, so dispatch within a provider is una
| AWS DynamoDB | `X-Amz-Target: DynamoDB_20120810.*` header |
| AWS SQS | `X-Amz-Target: AmazonSQS.*` header |
| AWS MemoryDB | `X-Amz-Target: AmazonMemoryDB.*` header |
| AWS Keyspaces | `X-Amz-Target: KeyspacesService.*` header |
| AWS Keyspaces | `X-Amz-Target: KeyspacesService.*` header, or `Smithy-Protocol: rpc-v2-cbor` on `/service/KeyspacesService/operation/*` (registered before CloudWatch) |
| AWS Lambda | URL prefix `/2015-03-31/functions` |
| AWS EKS | URL prefix `/clusters` |
| AWS RDS | Form-encoded POST whose `Action=` is a known RDS operation (registered before EC2) |
| AWS Redshift | Form-encoded POST whose `Action=` is a known Redshift operation (registered before EC2) |
| AWS EC2 | `Action=…` in URL query or `Content-Type: application/x-www-form-urlencoded` POST |
| AWS CloudWatch | `Smithy-Protocol: rpc-v2-cbor` header |
| AWS CloudWatch | `Smithy-Protocol: rpc-v2-cbor` header (any other `/service/` path) |
| AWS Bedrock | URL prefix `/foundation-models`, `/model-customization-jobs`, `/custom-models`, `/guardrails`, `/provisioned-model-throughput`, `/logging/modelinvocations`, `/tagResource`, `/untagResource`, `/listTagsForResource`, `/model-import-jobs`, `/model-copy-jobs`, `/evaluation-jobs`, `/evaluation-job/`, `/inference-profiles`, `/prompt-routers`, `/marketplace-model/endpoints`, `/automated-reasoning-policies`, `/create-foundation-model-agreement`, `/delete-foundation-model-agreement`, `/list-foundation-model-agreement-offers/`, `/foundation-model-availability/`, or bedrock-runtime `/model/{id}/{invoke,converse,converse-stream,invoke-with-response-stream,count-tokens}`, `/guardrail/{id}/version/{version}/apply`, and `/async-invoke` |
| AWS Bedrock Agent | Runtime routes are registered before the control plane and the query-protocol fallback, so they match whatever the Content-Type (except the roots that are also valid S3 bucket names, `/sessions`, `/rerank` and `/optimize-prompt`, which are claimed only for requests signed for `bedrock` or unsigned JSON requests): `/agents/{id}/agentAliases/{a}/sessions/{s}/text` (InvokeAgent), POST `/agents/{sessionId}` without a trailing slash (InvokeInlineAgent), `/agents/{id}/agentAliases/{a}/memories`, `/knowledgebases/{id}/retrieve`, `/knowledgebases/{id}/datasources/{ds}/documents/{doc}/content`, `/retrieveAndGenerate[Stream]`, `/rerank`, `/generateQuery`, `/optimize-prompt`, `/agenticRetrieveStream`, `/sessions...`, `/flows/{id}/aliases/{a}[/executions...]`, `/flows/{id}/executions`, and `/tags/{bedrock session ARN}`. Everything else under `/agents`, `/knowledgebases`, `/flows`, `/prompts` is the control plane |
| AWS S3 | Fallback (everything else REST-shaped) |
Expand Down
2 changes: 1 addition & 1 deletion providers/aws/keyspaces/keyspaces_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,7 @@ func TestTableLifecycleAndValidation(t *testing.T) {
t.Fatalf("CreateTable: %v", err)
}

// Default capacity mode is PAY_PER_REQUEST; PITR/TTL default DISABLED.
// Default capacity mode is PAY_PER_REQUEST; PITR defaults to DISABLED.
if tbl.CapacitySpecification.ThroughputMode != ksdriver.ThroughputPayPerRequest {
t.Fatalf("default throughput mode: %+v", tbl.CapacitySpecification)
}
Expand Down
21 changes: 21 additions & 0 deletions providers/aws/keyspaces/snapshot.go
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import (
"fmt"

"github.com/stackshy/cloudemu/v2/internal/snapshot"
ksdriver "github.com/stackshy/cloudemu/v2/services/keyspaces/driver"
)

var _ snapshot.Snapshottable = (*Mock)(nil)
Expand Down Expand Up @@ -86,6 +87,10 @@ func (m *Mock) Restore(_ context.Context, data json.RawMessage) error {
}
}

m.mu.Lock()
m.normalizeRestoredTablesLocked()
m.mu.Unlock()

if snap.Tags != nil {
m.mu.Lock()
m.tags = m.canonicalTagsLocked(snap.Tags)
Expand All @@ -95,6 +100,22 @@ func (m *Mock) Restore(_ context.Context, data json.RawMessage) error {
return nil
}

// normalizeRestoredTablesLocked clears the ttl and clientSideTimestamps
// status of restored tables that never enabled them. An older build stored
// DISABLED for both on every table, which GetTable then reported although real
// Keyspaces omits the members (see enabledOnly). The caller holds the write
// lock, and the tables store is already restored.
func (m *Mock) normalizeRestoredTablesLocked() {
for _, key := range m.tables.Keys() {
m.tables.Update(key, func(t ksdriver.Table) ksdriver.Table {
t.TTLStatus = enabledOnly(t.TTLStatus)
t.ClientSideTimestamps = enabledOnly(t.ClientSideTimestamps)

return t
})
}
}

// canonicalTagsLocked keys restored tags by the canonical ARN of the keyspace
// or table they belong to. Tag reads and writes resolve to the canonical ARN,
// but an older build stored tags under whatever ARN the caller passed (for
Expand Down
151 changes: 151 additions & 0 deletions providers/aws/keyspaces/table_settings_test.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
package keyspaces

import (
"context"
"encoding/json"
"testing"

ksdriver "github.com/stackshy/cloudemu/v2/services/keyspaces/driver"
)

// The ttl and clientSideTimestamps enums hold only ENABLED, so a table that
// never enabled them records "" (not set) and GetTable omits the members. An
// explicit DISABLED is the documented default and records "" too, and once
// enabled neither setting can be turned off.
func TestTTLAndClientSideTimestampsTrackSet(t *testing.T) {
ctx := context.Background()
m := newTestMock()
mustKeyspace(t, m, "app")

create := func(name, ttl, cst string) *ksdriver.Table {
t.Helper()

tbl, err := m.CreateTable(ctx, ksdriver.CreateTableConfig{
KeyspaceName: "app", Name: name, SchemaDefinition: sampleSchema(),
TTLStatus: ttl, ClientSideTimestamps: cst,
})
if err != nil {
t.Fatalf("CreateTable %s: %v", name, err)
}

return tbl
}

check := func(name, wantTTL, wantCST string) {
t.Helper()

got, err := m.GetTable(ctx, "app", name)
if err != nil {
t.Fatalf("GetTable %s: %v", name, err)
}

if got.TTLStatus != wantTTL || got.ClientSideTimestamps != wantCST {
t.Fatalf("%s: ttl=%q cst=%q, want ttl=%q cst=%q", name, got.TTLStatus, got.ClientSideTimestamps, wantTTL, wantCST)
}
}

create("unset", "", "")
check("unset", "", "")

create("disabled", "DISABLED", "DISABLED")
check("disabled", "", "")

create("enabled", "ENABLED", "ENABLED")
check("enabled", "ENABLED", "ENABLED")

// Enabling later records the setting.
if _, err := m.UpdateTable(ctx, ksdriver.UpdateTableConfig{
KeyspaceName: "app", Name: "unset", TTLStatus: "ENABLED", ClientSideTimestamps: "ENABLED",
}); err != nil {
t.Fatalf("UpdateTable enable: %v", err)
}

check("unset", "ENABLED", "ENABLED")

// DISABLED never turns an enabled setting back off.
if _, err := m.UpdateTable(ctx, ksdriver.UpdateTableConfig{
KeyspaceName: "app", Name: "enabled", TTLStatus: "DISABLED", ClientSideTimestamps: "DISABLED",
}); err != nil {
t.Fatalf("UpdateTable disable: %v", err)
}

check("enabled", "ENABLED", "ENABLED")
}

// A snapshot keeps whether each setting was set, and a snapshot from an older
// build, which stored DISABLED on every table, restores as not set.
func TestSnapshotTracksTTLAndClientSideTimestamps(t *testing.T) {
ctx := context.Background()
src := newTestMock()
mustKeyspace(t, src, "app")

for name, status := range map[string]string{"plain": "", "on": "ENABLED"} {
if _, err := src.CreateTable(ctx, ksdriver.CreateTableConfig{
KeyspaceName: "app", Name: name, SchemaDefinition: sampleSchema(),
TTLStatus: status, ClientSideTimestamps: status,
}); err != nil {
t.Fatalf("CreateTable %s: %v", name, err)
}
}

raw, err := src.Snapshot(ctx, false)
if err != nil {
t.Fatalf("snapshot: %v", err)
}

legacy := legacyDisabledSnapshot(t, raw)

for label, data := range map[string]json.RawMessage{"current": raw, "legacy": legacy} {
dst := newTestMock()
if err := dst.Restore(ctx, data); err != nil {
t.Fatalf("%s restore: %v", label, err)
}

for name, want := range map[string]string{"plain": "", "on": "ENABLED"} {
got, err := dst.GetTable(ctx, "app", name)
if err != nil {
t.Fatalf("%s GetTable %s: %v", label, name, err)
}

if got.TTLStatus != want || got.ClientSideTimestamps != want {
t.Fatalf("%s %s: ttl=%q cst=%q, want %q", label, name, got.TTLStatus, got.ClientSideTimestamps, want)
}
}
}
}

// legacyDisabledSnapshot rewrites raw the way an older build stored it: every
// table that never set ttl or clientSideTimestamps carries DISABLED.
func legacyDisabledSnapshot(t *testing.T, raw json.RawMessage) json.RawMessage {
t.Helper()

var snap keyspacesSnapshot
if err := json.Unmarshal(raw, &snap); err != nil {
t.Fatalf("parse snapshot: %v", err)
}

var tables map[string]ksdriver.Table
if err := json.Unmarshal(snap.Tables, &tables); err != nil {
t.Fatalf("parse tables: %v", err)
}

for k, tbl := range tables {
tbl.TTLStatus = disabledIfEmpty(tbl.TTLStatus)
tbl.ClientSideTimestamps = disabledIfEmpty(tbl.ClientSideTimestamps)
tables[k] = tbl
}

b, err := json.Marshal(tables)
if err != nil {
t.Fatalf("encode tables: %v", err)
}

snap.Tables = b

out, err := json.Marshal(snap)
if err != nil {
t.Fatalf("encode snapshot: %v", err)
}

return out
}
29 changes: 24 additions & 5 deletions providers/aws/keyspaces/tables.go
Original file line number Diff line number Diff line change
Expand Up @@ -51,7 +51,26 @@ func cloneTable(in *ksdriver.Table) ksdriver.Table {
// disabledIfEmpty defaults an unset status field to "DISABLED".
func disabledIfEmpty(v string) string {
if v == "" {
return "DISABLED"
return statusDisabled
}

return v
}

// statusDisabled is the not-enabled value callers may send for the TTL and
// client-side timestamps settings.
const statusDisabled = "DISABLED"

// enabledOnly normalizes the ttl and clientSideTimestamps status. Their model
// enums (TimeToLiveStatus, ClientSideTimestampsStatus) hold only ENABLED, so
// real Keyspaces never reports DISABLED: a table that never enabled the
// setting omits the member from GetTable. "" records "not set"; DISABLED is
// the documented default and maps to "" as well. The settings cannot be
// turned off once enabled, so on UpdateTable a DISABLED request is kept as a
// no-op by orKeep.
func enabledOnly(v string) string {
if strings.EqualFold(v, statusDisabled) {
return ""
}

return v
Expand Down Expand Up @@ -146,9 +165,9 @@ func (m *Mock) CreateTable(_ context.Context, cfg ksdriver.CreateTableConfig) (*
CapacitySpecification: resolveCapacity(cfg.CapacitySpecification),
EncryptionSpecification: resolveEncryption(cfg.EncryptionSpecification),
PointInTimeRecoveryStatus: disabledIfEmpty(cfg.PointInTimeRecovery),
TTLStatus: disabledIfEmpty(cfg.TTLStatus),
TTLStatus: enabledOnly(cfg.TTLStatus),

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: both enums hold only ENABLED, so real CreateTable/UpdateTable probably reject DISABLED with a ValidationException rather than treating it as unset. Accepting it seems fine until that is confirmed against real AWS. Could you add it to #1666 so it isn't lost?

DefaultTimeToLive: cfg.DefaultTimeToLive,
ClientSideTimestamps: disabledIfEmpty(cfg.ClientSideTimestamps),
ClientSideTimestamps: enabledOnly(cfg.ClientSideTimestamps),
CdcStatus: disabledIfEmpty(cfg.CdcStatus),
Comment: cfg.Comment,
ReplicaRegions: append([]string(nil), cfg.ReplicaRegions...),
Expand Down Expand Up @@ -287,8 +306,8 @@ func (m *Mock) UpdateTable(_ context.Context, cfg ksdriver.UpdateTableConfig) (*
}

t.PointInTimeRecoveryStatus = orKeep(cfg.PointInTimeRecovery, t.PointInTimeRecoveryStatus)
t.TTLStatus = orKeep(cfg.TTLStatus, t.TTLStatus)
t.ClientSideTimestamps = orKeep(cfg.ClientSideTimestamps, t.ClientSideTimestamps)
t.TTLStatus = orKeep(enabledOnly(cfg.TTLStatus), t.TTLStatus)
t.ClientSideTimestamps = orKeep(enabledOnly(cfg.ClientSideTimestamps), t.ClientSideTimestamps)
t.CdcStatus = orKeep(cfg.CdcStatus, t.CdcStatus)
t.Comment = orKeep(cfg.Comment, t.Comment)

Expand Down
2 changes: 1 addition & 1 deletion server/aws/authz_completeness_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -93,7 +93,7 @@ func TestHandlerIAMServicesMatchTable(t *testing.T) {
"*athena.Handler": "athena", "*cognito.Handler": "cognito-idp", "*cognito.WellKnown": "cognito-idp", "*configservice.Handler": "config",
"*wafv2.Handler": "wafv2", "*ecs.Handler": "ecs", "*ecr.Handler": "ecr", "*route53resolver.Handler": "route53resolver",
"*eventbridge.Handler": "events", "*cloudwatchlogs.Handler": "logs", "*secretsmanager.Handler": "secretsmanager",
"*keyspaces.Handler": "cassandra", "*memorydb.Handler": "memorydb", "*networkfirewall.Handler": "network-firewall",
"*keyspaces.Handler": "cassandra", "*keyspaces.CBORHandler": "cassandra", "*memorydb.Handler": "memorydb", "*networkfirewall.Handler": "network-firewall",
"*resourcegroupstaggingapi.Handler": "tag", "*transfer.Handler": "transfer", "*timestreamwrite.Handler": "timestream",
"*healthlake.Handler": "healthlake", "*apprunner.Handler": "apprunner", "*globalaccelerator.Handler": "globalaccelerator",
"*costexplorer.Handler": "ce", "*servicequotas.Handler": "servicequotas", "*emr.Handler": "elasticmapreduce",
Expand Down
11 changes: 10 additions & 1 deletion server/aws/aws.go
Original file line number Diff line number Diff line change
Expand Up @@ -587,7 +587,8 @@ func NewFromProvider(p *awsprovider.Provider) *server.Server {
// non-nil driver in d. Handlers are registered most-specific-first so the
// dispatch is unambiguous:
//
// - CloudWatch matches on Smithy-Protocol: rpc-v2-cbor header.
// - Keyspaces rpc-v2-cbor matches its own /service/KeyspacesService/ path;
// CloudWatch then matches any other Smithy-Protocol: rpc-v2-cbor request.
// - DynamoDB matches on X-Amz-Target header (JSON-RPC).
// - RDS matches form-encoded POSTs whose Action is one of the known RDS
// operations. It must register before EC2 because both speak the AWS
Expand Down Expand Up @@ -658,6 +659,14 @@ func newServer(d Drivers) (*server.Server, authzSets) {
srv.Register(bedrockagentruntime.New(d.BedrockAgentRuntime))
}

// Keyspaces rpc-v2-cbor (aws-sdk-go-v2/service/keyspaces v1.37.0+, so
// hashicorp/aws 6.69+) matches only /service/KeyspacesService/operation/*.
// It registers before CloudWatch, which claims every other rpc-v2-cbor
// request under /service/. The AWS JSON 1.0 handler registers below.
if d.Keyspaces != nil {
srv.Register(keyspacessrv.NewCBOR(d.Keyspaces))
}

if d.CloudWatch != nil {
// The VPC driver optionally supplies derived AWS/IPAM metrics; surface
// them through CloudWatch when it implements the capability.
Expand Down
Loading
Loading