Repository navigation
fix(aws-lambda): version immutability, async state, LogType=Tail, GetAccountSettings, code signing configs (#1518) - #1646
Conversation
NitinKumar004
left a comment
There was a problem hiding this comment.
Thanks, this is careful work. The six items are real gaps and the evidence in the description holds up. Here is what I ran against the pushed SHA:
- Gates: build, gofmt,
go mod tidy,go generatedocs (no diff) and-raceon./providers/aws/lambda/... ./server/aws/lambda/... ./server/aws/all pass. vet/test were re-run scoped and pass, and golangci-lint--new-from-revreports 0 new issues. - Live Terraform (hashicorp/aws 6.66.0, default endpoint override,
serve -async-settle): I usedaws_lambda_code_signing_config, plusaws_lambda_function(publish=true,code_signing_config_arn, env, tags),aws_lambda_alias,aws_lambda_function_urlon the alias, andaws_lambda_permissionon the alias.- apply: 5 created, no waiter hang,
version = "1". plan -detailed-exitcode: 0.- Changed the zip and env, then apply: 2 changed,
version = "2". TheFunctionUpdatedV2/FunctionActiveV2waiters completed. plan: 0.- Afterwards
list-versions-by-functionshows v1 frozen (CodeSize 193, STAGE=one, its own CodeSha256) and v2/$LATEST at 196/two. - destroy: 5 destroyed.
get-functionthen givesResourceNotFoundException: Function not found: arn:aws:lambda:us-east-1:000000000000:function:tf1646, andlist-code-signing-configsis empty.
- apply: 5 created, no waiter hang,
- AWS CLI against
serve -async-settle:- create gives
Pending/Creating/null, thenwait function-active-v2exits 0. - Invoke while Pending gives
ResourceConflictExceptionwith the documented message. --log-type Taildecodes to the START/END/REPORT lines.- An Event invoke gets no LogResult.
- The
get-account-settingsshape matches API_GetAccountSettings exactly.
- create gives
- Regression probes for #1535 and #1530 (throwaway tests under
-race), all green:- The existing
TestDeleteAliasNotUndoneByConcurrentUpdate(2000 rounds) andTestEngineDeployDoesNotHoldLock. - A slow-engine probe. While a gated engine Deploy is in flight for one function, these calls on another function return in under 100 ms:
GetFunction,ListFunctions,GetAccountSettings,GetFunctionCodeSigningConfig,ListCodeSigningConfigs,GetVersionAWSConfig. The new settle overlay (observe) only takes thesettle.SetRWMutex, neverm.mu. - A 3-way race of
PutFunctionCodeSigningConfig,UpdateFunctionandTagFunction(2000 rounds). The attachment is never lost, so there is no stale write-back offuncData. PutFunctionCodeSigningConfigvsDeleteCodeSigningConfig(1000 rounds). A function is never left attached to a deleted config.- A
PutFunctionCodeSigningConfigthat lands during an engine code deploy survives the deploy's commit. - Every new
funcDatawriter (PutFunctionCodeSigningConfig,DeleteFunctionCodeSigningConfig, the version snapshot inpublishVersion) is a read-modify-write underm.mu.beginCreateSettle/beginUpdateSettlerun inside the existing critical sections, and the engine deploy stays outside the lock.
- The existing
- Authz: the
--enforce-authqualifier→ARN mapping is untouched.tagChecksonly diverts when the ARN contains:code-signing-config:, which a function ARN cannot. The authz matrix tests pass (including the newTestAuthzMatrixLambdaAccountAndCodeSigning). - Checked against the docs: the Pending-blocked set (Invoke, UpdateFunctionCode, UpdateFunctionConfiguration, PublishVersion) and the InProgress-blocked set (the same minus Invoke, plus TagResource) match Lambda function states exactly.
Medium
A published version still reports $LATEST's State, LastUpdateStatus and LastModified (server/aws/lambda/handler.go toVersionConfiguration, ~L1556).
toVersionConfiguration starts from toConfiguration(info, …) (the live $LATEST) and overrides only the version fields. State, StateReason(Code), LastUpdateStatus and LastModified therefore leak from $LATEST into every published version. Reproduced on serve -async-settle:
publish-version -> v1 LastModified 19:42:27Z, Successful
update-function-code ($LATEST) -> InProgress, LastModified 19:42:30Z
get-function --qualifier 1 -> Version 1, Active, LastUpdateStatus InProgress, LastModified 19:42:30Z
list-versions-by-function -> "1" InProgress 19:42:30Z
In real Lambda a published version is immutable. It keeps its own LastModified (the publish time) and reports Active/Successful whatever $LATEST is doing. Impacts:
- The
LastModifieddrift happens in default mode too, not only under--async-settle. aws lambda wait function-updated-v2 --qualifier 1, or any tool polling a version or alias ARN, seesInProgressbecause of an unrelated$LATESTupdate.
This PR's stated goal is "a published version keeps … the AWS-only settings it was cut with", and versionData.createdAt already holds the right timestamp. Suggested fix:
- In
toVersionConfiguration, whenver.Version != $LATEST, setcfg.LastModified = ver.CreatedAt,cfg.LastUpdateStatus = Successfulandcfg.State = Active, and clearStateReason/StateReasonCode. - Pending should only carry over for the version cut atomically by
Publish=trueon create. In real AWS that version is also Pending until creation completes, so that case is fine to keep. - Add a wire test asserting that
GetFunction --qualifier 1is unchanged across a$LATESTupdate.
Low
- Invoke during an InProgress update runs the new code (
providers/aws/lambda/lambda.go~L545). The docs say "Invocations continue to use the function's previous code and configuration until the update completes." This is already listed under Known limits and is opt-in only, so I'm not asking for it here. A tracking issue would be good so it doesn't get lost. - CreateFunction with
CodeSigningConfigArnis check → create → attach, with no rollback (server/aws/lambda/handler.go~L1078).checkCodeSigningConfigExistsruns beforeCreateFunction, but the attach runs afterwards. If the config is deleted between the two, the request returns 404ResourceNotFoundExceptionwhile the function already exists, and a retry then hitsResourceConflictException. The window is narrow. Either delete the function when the attach fails, or returnCodeSigningConfigNotFoundExceptionand keep the function (AWS validates before creating anything). The error type also differs: the provider's NotFound maps toResourceNotFoundExceptionhere, notCodeSigningConfigNotFoundException. - Not-found wording is only partly converted.
GetAlias/DeleteAliasstill returnalias %s not found(providers/aws/lambda/aliases.goL134/L177), andDeleteFunction --qualifier 9still returnsfunction version %s not found(versions.goL217). The description scopes this deliberately to messages where real text was found. Still, the summary line ("a missing function, version or alias now saysFunction not found: …") overstates it for alias APIs and DeleteFunction-with-qualifier. Please tighten the description, or convert the version path, which uses the sameFunction not found: <arn>:<qualifier>form the PR already cites. - Displaced doc comment (
server/aws/lambda/iam_actions.goL149). The newcodeSigningConfigTagswas inserted betweenresourceTags' doc comment and its func. As a result,// resourceTags sets …now sits on top of thecodeSigningConfigTagscomment, andresourceTagshas none.
Cascade / blast radius
services/serverless/driver: additive struct fields only, and theServerlessinterface is unchanged. Azure/GCP leaveLastUpdateStatusempty, and the handler falls back toSuccessful, so their output is identical. Only the AWS Lambda handler consumes it.docs/coveragehas no diff.internal/settle: two new constants only.- Persistence: new snapshot fields are
omitempty, and old snapshots restore with an empty version AWSConfig. After such a restore, a published version reports empty AWS-only settings (e.g. noArchitectures) instead of$LATEST's. That is acceptable and not a Terraform drift source, since TF reads$LATEST. - Mirror rule: AWS-only is justified, because versions, code signing configs and log tails have no Azure/GCP equivalent.
| cfg.Role = ver.Role | ||
| cfg.MemorySize = ver.Memory | ||
| cfg.Timeout = ver.Timeout | ||
| cfg.CodeSize = ver.CodeSize |
There was a problem hiding this comment.
A published version still inherits State / StateReason / LastUpdateStatus / LastModified from $LATEST via toConfiguration(info, …) above. On serve -async-settle: publish v1, then update-function-code, then get-function --qualifier 1 shows LastUpdateStatus: InProgress and LastModified = the $LATEST update time. Real Lambda versions are immutable: own LastModified (publish time), Active/Successful. Suggest, for ver.Version != $LATEST: cfg.LastModified = ver.CreatedAt, cfg.LastUpdateStatus = lastUpdateStatusSuccessful, cfg.State = Active, clear StateReason/StateReasonCode (keep Pending only for the version cut by Publish=true on create). The LastModified drift also happens with settle off.
There was a problem hiding this comment.
Good catch, confirmed with your sequence. Fixed in f6700ed. The provider now reports each published version's own lifecycle on FunctionVersion (Active/Successful; Pending/Creating only for the version cut by Publish=true on create, and only until creation completes), and toVersionConfiguration uses it plus ver.CreatedAt (publish time) as LastModified for any non-$LATEST version. Tests: TestSDKPublishedVersionKeepsItsOwnLifecycle (default and async-settle; checks both GetFunction --qualifier 1 and ListVersionsByFunction across a $LATEST update), TestSDKPublishOnCreateVersionIsPendingUntilCreated, TestPublishedVersionReportsItsOwnLifecycle.
|
|
||
| if req.CodeSigningConfigArn != "" { | ||
| if csm, ok := h.fn.(codeSigningManager); ok { | ||
| if err := csm.PutFunctionCodeSigningConfig(r.Context(), req.FunctionName, req.CodeSigningConfigArn); err != nil { |
There was a problem hiding this comment.
The function is already created when this attach runs. If the code signing config is deleted between checkCodeSigningConfigExists and here, the client gets a 404 ResourceNotFoundException (not CodeSigningConfigNotFoundException) while the function exists, and a retry then 409s. Narrow window, but either roll the function back on failure or map this error the same way as the pre-check.
There was a problem hiding this comment.
Fixed in f6700ed. If the attach fails, the handler deletes the just-created function (no m.mu held; it goes through the normal DeleteFunction) and reports the error the pre-check would have: CodeSigningConfigNotFoundException. The standalone PutFunctionCodeSigningConfig had the same check/attach window, so it maps that case the same way. Test: TestSDKCreateFunctionRollsBackWhenCodeSigningAttachFails uses a backend that deletes the config between the check and the attach, then asserts the typed error, that GetFunction gives ResourceNotFoundException, and that a retry succeeds.
| @@ -147,6 +147,24 @@ func (c *checkSet) requestTags(tags map[string]string) { | |||
| } | |||
|
|
|||
| // resourceTags sets aws:ResourceTag/* from the tags of the function name. | |||
There was a problem hiding this comment.
nit: this resourceTags doc comment is now attached to codeSigningConfigTags (the new func was inserted between the comment and resourceTags). Move it back down above func (c *checkSet) resourceTags.
There was a problem hiding this comment.
Thanks, moved back above func (c *checkSet) resourceTags in f6700ed.
|
|
||
| // A function still Pending creation cannot be invoked | ||
| // (ResourceConflictException), matching real Lambda. | ||
| if err := m.checkInvocable(&fd); err != nil { |
There was a problem hiding this comment.
Matches the docs for Pending. For InProgress, the docs also say "Invocations continue to use the function's previous code and configuration until the update completes", but here the new code is already live. It's in Known limits, so this is fine for now. Worth a tracking issue so it doesn't get lost.
There was a problem hiding this comment.
Agreed. Leaving it as a known limit in this PR and tracking it as a follow-up: during an InProgress window Invoke should keep serving the previous code/config until the update completes.
|
Thanks for the thorough review, @NitinKumar004. All addressed in f6700ed. Medium: published version inherits
The wire tests fail on 0e333c7 and pass now. I re-ran your sequence on One thing I left alone: a version cut by Low 1: Invoke during InProgress runs the new code. Follow-up. It stays under Known limits. The follow-up is to keep serving the previous code/config until the update completes. Low 2: CreateFunction check → create → attach. Fixed. When the attach fails, the handler deletes the just-created function and returns Low 3: not-found wording. Partly changed, and the description now matches:
Low 4: displaced doc comment. Fixed. Locking is unchanged. The new version-lifecycle read only takes the Checks: |
NitinKumar004
left a comment
There was a problem hiding this comment.
Thanks for the follow-up. I re-checked every point from the last round against f6700ed and re-ran the gates and the live flows on the new head.
What I checked
- Earlier findings:
- Medium (published version reports
$LATEST's State, LastUpdateStatus and LastModified): fixed.toVersionConfigurationnow callsapplyVersionLifecyclefor any non-$LATESTversion. That setsLastModifiedfromver.CreatedAtand takes State/StateReason/LastUpdateStatus from the provider's newobserveVersion, which reports Active/Successful, or Pending/Creating only for the version cut byPublish=trueon create while creation is still running. I re-ran my earlier sequence onserve -async-settle. After publish-version (v1 at 20:58:37Z) and then update-function-code ($LATESTInProgress at 20:58:41Z):get-function --qualifier 1gives1, Active, Successful, 20:58:37Z, CodeSize 201, STAGE=one.get-function-configuration --qualifier 1gives the same.list-versions-by-functionshows$LATEST InProgressand1 Successful.wait function-updated-v2 --qualifier 1exits 0 while$LATESTis still updating.- An alias on v1 (
--qualifier live) reports v1's lifecycle and LastModified with the alias ARN.
- Low 1 (Invoke during InProgress runs the new code): still open, now tracked in #1665. That's fine by me for this PR.
- Low 2 (CreateFunction check, create, attach with no rollback): fixed.
attachCreateCodeSigningConfigdeletes the just-created function when the attach fails, then re-runscheckCodeSigningConfigExists, so the client getsCodeSigningConfigNotFoundExceptionand not the function'sResourceNotFoundException.putFunctionCodeSigningConfigmaps the same race the same way.TestSDKCreateFunctionRollsBackWhenCodeSigningAttachFailscovers the typed error, the absent function and a successful retry. - Low 3 (not-found wording): addressed.
GetAliasnow returnsCannot find alias arn: <function ARN>:<alias>(confirmed via the CLI). The description now says plainly that DeleteAlias/UpdateAlias andDeleteFunction --qualifierkeep emulator text (alias ghost not found,function version 9 not found, both confirmed). Those are tracked in #1665. - Low 4 (displaced doc comment in iam_actions.go): fixed. The
resourceTagscomment is back above its func.
- Medium (published version reports
- Gates on f6700ed:
go build ./...passes.go vetpasses on settle, aws lambda provider and handler, and serverless.- Scoped
go test -count=1passes oninternal/settle,providers/aws/lambda/...,server/aws/lambda/...,services/serverless/...,persist/andserver/aws -run Lambda. -racepasses onproviders/aws/lambda/...andserver/aws/lambda/....go mod tidyandgo generate ./...leave no diff, and no touched file fails gofmt.
- Live Terraform (hashicorp/aws v6.70.0, default lambda endpoint override,
serve -async-settle): I usedaws_lambda_code_signing_config, plusaws_lambda_function(publish=true,code_signing_config_arn, env, tags),aws_lambda_aliasandaws_lambda_function_urlon the alias.- apply: 4 added,
version = "1". plan -detailed-exitcode: 0.- Changed the zip and set
STAGE=two, then apply: 2 changed,version = "2". plan: 0.list-versions-by-functionthen shows each version with its own lifecycle:1: CodeSize 201,one, Active/Successful, LastModified 20:59:47Z.2and$LATEST: CodeSize 211,two.
- destroy: 4 destroyed.
get-functionthen returnsFunction not found: arn:aws:lambda:us-east-1:000000000000:function:tf1646, andlist-code-signing-configsis empty.
- apply: 4 added,
- AWS CLI under
-async-settle:- create-function gives
Pending/Creating/null, thenwait function-active-v2exits 0. create-function --publishreturns v1 asPending/Creating, andwait function-active-v2 --qualifier 1exits 0.update-function-code --publishgives v2Active/Successful. That is the known limit listed in #1665.- publish-version during InProgress gives
ResourceConflictException.
- create-function gives
- Architecture: the version lifecycle rule lives in the provider (
observeVersion). The handler only maps it, with an Active/Successful fallback for a backend that reports nothing.FunctionVersiongains four additive fields and theServerlessinterface is unchanged, so Azure/GCP are unaffected.cutWhilePendingis not persisted, which is consistent with settle windows not being persisted (a restored function is terminal, so its version is Active). The create rollback is a compensating step in the handler. That is acceptable because theCodeSigningConfigArnon CreateFunction is itself composed in the handler (the portable CreateFunction has no such field). - No AI attribution in the PR body or commits.
High
None.
Medium
-
New functions over the complexity limit of 10 (
server/aws/lambda). I ran golangci-lint (v2.14.0) on./providers/aws/lambda/... ./server/aws/lambda/...without--new-from-revand compared it with the merge base: 32 issues there, 37 here.- gocyclo:
ServeHTTPgoes 12 to 15,create11 to 13,invoke11 to 13, andclassify(12) andMatches(12) are new over the limit. - goconst:
CodeSigningConfig,CodeSigningConfigArnandTagsincode_signing.go, andTagsinhandler.goL442.
--new-from-revmisses the gocyclo ones because the function signature lines did not change. The repo keeps cyclomatic complexity at 10 or below, so please split the code-signing and account-settings branches out ofServeHTTP/Matches/classifyinto small helpers (asclassifyCodeSigningConfigsalready does), and the same forcreateandinvoke. This is the last thing I would like to see before approving. - gocyclo:
Low
- Invoke during InProgress,
UpdateFunctionCode --publishlifecycle, and DeleteAlias/UpdateAlias/DeleteFunction --qualifierwording remain as documented in #1665. No action needed in this PR.
| return | ||
| } | ||
|
|
||
| if isCodeSigningConfigARN(a.item) { |
There was a problem hiding this comment.
Non-blocking: without --new-from-rev, golangci-lint shows ServeHTTP going from cyclomatic 12 to 15 with this branch and the account-settings one, and create/invoke going from 11 to 13. Matches and classify are new over 10. --new-from-rev hides these because the func signature lines didn't change. Moving the code-signing/account-settings dispatch into a helper, like classifyCodeSigningConfigs, would bring them back under the limit.
There was a problem hiding this comment.
Done in dca3dcf. The code signing and account settings branches now live in small helpers: isCodeSigningOrAccountPath (Matches), classifyCodeSigningOrAccount (classify), and serveCodeSigningOrAccount plus serveTagsRoute and serveConcurrencyRoute (ServeHTTP). create and invoke are split the same way. gocyclo, full run without --new-from-rev: ServeHTTP 15 → 10, Matches 12 → 10, classify 12 → 10, create 13 → 8, invoke 13 → 7. Behaviour is unchanged and the existing tests pass. Details are in the summary comment.
- A published version now freezes its CodeSize, description, environment and AWS-only settings (VpcConfig, DeadLetterConfig, TracingConfig, Architectures, EphemeralStorage, Layers) at publish time, so UpdateFunctionCode/Configuration on $LATEST no longer changes what GetFunction --qualifier N or ListVersionsByFunction report for it. - Under AsyncSettle a new function reports State Pending (StateReasonCode Creating, no LastUpdateStatus) and an update reports LastUpdateStatus InProgress. While Pending, Invoke, UpdateFunctionCode, UpdateFunctionConfiguration and PublishVersion return ResourceConflictException; while InProgress the updates, PublishVersion and TagResource do. The InProgress gate also covers an in-flight engine deploy. Publish=true on Create/UpdateFunctionCode is not blocked by the window its own write opened. Settle off keeps the old behaviour. - Invoke with LogType=Tail returns X-Amz-Log-Result: the base64 of the last 4 KB of the execution log the provider already renders. - GetAccountSettings (GET /2016-08-19/account-settings). - Code signing configs: Create/Get/List/Update/DeleteCodeSigningConfig, Put/Get/DeleteFunctionCodeSigningConfig, ListFunctionsByCodeSigningConfig, CodeSigningConfigArn on CreateFunction, and tagging a code signing config ARN, with IAM checks per the Service Authorization Reference. State is in the Lambda snapshot. - ResourceNotFoundException for a missing function, version or alias reads "Function not found: <function ARN>[:<qualifier>]". Part of stackshy#1518
…CreateFunction on code signing attach failure - A published version no longer inherits $LATEST's State, StateReason(Code), LastUpdateStatus and LastModified. The provider now reports each version's own lifecycle on FunctionVersion (Active/Successful, or Pending only for the version CreateFunction(Publish=true) cut, until creation completes), and the handler renders LastModified from the version's publish time. GetFunction --qualifier N and ListVersionsByFunction are unchanged by a later $LATEST update, in default mode and under --async-settle. - CreateFunction with CodeSigningConfigArn deletes the just-created function when the attach fails (config deleted after the existence check) and reports CodeSigningConfigNotFoundException, so a retry does not hit ResourceConflictException. PutFunctionCodeSigningConfig maps the same race to CodeSigningConfigNotFoundException. - GetAlias for a missing alias returns real Lambda's text: "Cannot find alias arn: <function ARN>:<alias>". - Move resourceTags' doc comment back above its func.
Split the code signing and account settings branches out of ServeHTTP, Matches and classify into small helpers, group the two concurrency routes, and move CreateFunction's validation/config build and Invoke's DryRun and synchronous response writing into their own functions. Name the repeated response field strings as constants. No behaviour change.
f6700ed to
dca3dcf
Compare
|
Thanks @NitinKumar004. I rebased onto Medium: complexity and goconst in
|
| func | base (development) |
f6700ed | now |
|---|---|---|---|
(*Handler).ServeHTTP |
12 | 15 | 10 |
(*Handler).Matches |
≤10 | 12 | 10 |
classify |
≤10 | 12 | 10 |
(*Handler).create |
11 | 13 | 8 |
(*Handler).invoke |
11 | 13 | 7 |
The new helpers are all 6 or lower. Full lint on ./providers/aws/lambda/... ./server/aws/lambda/...:
- v2.14.0: 32 on
development, 28 on this branch. - v2.4.0: 6 on
development, 2 on this branch.
Every finding left on the branch is also on development. The drop comes from the three pre-existing gocyclo hits (ServeHTTP/create/invoke) and one gocritic hugeParam in policy.go that the branch already removed.
The change is a refactor only, with no behaviour change. Testing:
go test -racepasses onproviders/aws/lambda/...,server/aws/lambda/...,server/aws(including the authz matrix),persist/...andinternal/settle/....- The full
go test -p 1 ./...passes, run in chunks. go build ./...passes, andgo generate ./...leaves no diff.- AWS CLI smoke test on
serve -async-settle:- create gives
Pending/Creating, thenwait function-active-v2exits 0, and invoke returns 200. - publish-version, then update-function-code (
$LATESTInProgress), thenget-function --qualifier 1gives1 Active Successfulwith its own LastModified. - get-account-settings works.
- Code signing config create/get/update/list/tags/put/get/delete all behave as before.
- DryRun returns 204, and an unknown qualifier gives
ResourceNotFoundException.
- create gives
Low
No change in this PR; it stays tracked in #1665.
Rebase
The branch is rebased onto development (a624a95). The only conflict was internal/settle/settle.go: I kept both the Batch constants (DefaultBatchJobPhase, DefaultBatchJobRunDwell) and the Lambda ones. Nothing else upstream touched providers/aws/lambda, server/aws/lambda or services/serverless, and go generate ./... gives no diff after the rebase. I also updated the lint lines in the description.
Summary
Part of #1518. Closes all six AWS Lambda parity items:
$LATESTchanges. It also keeps its own lifecycle:LastModifiedis its publish time and it reportsActive/Successfulwhatever$LATESTis doing (only the version cut byPublish=trueon create isPending, until creation completes).--async-settle/config.WithAsyncSettle(), functions reportState: Pendingafter create andLastUpdateStatus: InProgressafter an update, and the operations AWS blocks in those states returnResourceConflictException. Default behaviour (settle off) is unchanged.InvokewithLogType=TailreturnsX-Amz-Log-Result.GetAccountSettingsis implemented.CodeSigningConfigArnon CreateFunction, and tagging a code signing config ARN.ResourceNotFoundExceptionfor a missing function now saysFunction not found: <function ARN>, and for a missing version or alias qualifier onGetFunction/GetFunctionConfiguration/Invokeit saysFunction not found: <function ARN>:<qualifier>.GetAliasfor a missing alias saysCannot find alias arn: <function ARN>:<alias>. Other not-found texts (DeleteAlias/UpdateAliasfor a missing alias,DeleteFunction --qualifierfor a missing version, layers, statements, event source mappings) are unchanged because I couldn't verify real AWS text for them.Item status
$LATESTCodeSize after UpdateFunctionCodeversionData(providers/aws/lambda/lambda.go) now freezescodeSize,descriptionand the AWS-only config (VpcConfig, DeadLetterConfig, TracingConfig, Architectures, EphemeralStorage, Layers) at publish time. Environment was already snapshotted inconfigbut never surfaced:driver.FunctionVersiongainsCodeSizeandEnvironment, and the handler renders a version from its snapshot through the new AWS-onlyGetVersionAWSConfig. Before this,get-function --qualifier 1after an UpdateFunctionCode/UpdateFunctionConfiguration returned$LATEST's CodeSize, Environment and Architectures.providers/aws/lambda/lifecycle.go, usinginternal/settle(DefaultLambdaCreateSettle,DefaultLambdaUpdateSettle, 1 s each). Create:State: Pending,StateReason: "The function is being created.",StateReasonCode: Creating, noLastUpdateStatus(the API reference says it "is first set to Successful after function creation completes"). Update:State: Active,LastUpdateStatus: InProgress. Blocked operations, per Lambda function states: while Pending, Invoke, UpdateFunctionCode, UpdateFunctionConfiguration and PublishVersion; while InProgress, UpdateFunctionCode, UpdateFunctionConfiguration, PublishVersion and TagResource. Invoke is not blocked during InProgress. The InProgress gate is shared with the existing engine-deploy reservation from #1535 (checkNotUpdating= settle window orengineBusy, reusingupdateInProgress). Publish=true on CreateFunction/UpdateFunctionCode goes throughPublishVersionOnWrite, so the atomic publish AWS does with the write is not refused by the window the write itself opened.LogType=Tailnever returnsX-Amz-Log-ResultInvokeOutput.LogTail, and the handler base64-encodes that intoX-Amz-Log-Resultfor synchronous invokes that ask forTail(Invoke: "The last 4 KB of the execution log, which is base64-encoded"). Event and DryRun invokes don't get the header, and neither does a call withoutLogType=Tail.GET /2016-08-19/account-settings. Limits: ConcurrentExecutions 1000, UnreservedConcurrentExecutions = 1000 minus the sum of reserved concurrency (the same poolPutFunctionConcurrencyalready enforces), CodeSizeZipped 52428800 (50 MB), CodeSizeUnzipped 262144000 (250 MB), TotalCodeSize 322122547200 (300 GB, the current quota page). Usage: FunctionCount, and TotalCodeSize =$LATEST+ every published version + every layer version ("Each function version and layer version consumes storage"). Note: the CLI reference example still shows the older 75 GB (80530636800). I used the quota page because it is the current documented limit./2020-04-22/code-signing-configs(POST 201 / GET),/2020-04-22/code-signing-configs/{arn}(GET / PUT / DELETE 204),/2020-04-22/code-signing-configs/{arn}/functions(GET),/2020-06-30/functions/{name}/code-signing-config(GET / PUT / DELETE 204). Validation from the model: 1 to 20SigningProfileVersionArns,UntrustedArtifactOnDeploymentin `WarnFunction not found: arn:aws:lambda:<region>:<account>:function:<name>(HashiCorp Discuss) and the qualified form...:function:GetCustomerProfile:RAM256for an alias (aws-lambda-power-tuning#93). Every function lookup in the provider uses this text througherrFunctionNotFound, with the qualifier for a missing version or alias on Invoke and onGetFunction/GetFunctionConfiguration --qualifier.GetAliasfor a missing alias uses the real GetAlias textCannot find alias arn: <function ARN>:<alias>(captured from a real SDK v2 GetAlias error in lambroll#386). Other not-found messages (DeleteAlias/UpdateAlias alias, DeleteFunction-with-qualifier version, layer, statement, event source mapping) are unchanged because I couldn't find real text for them.Changes
services/serverless/driver:FunctionVersion.CodeSize/Environment;FunctionInfo.StateReason/StateReasonCode/LastUpdateStatus;InvokeOutput.LogTail; the AWS-only data typesCodeSigningConfig,CodeSigningConfigUpdateandAccountSettings. No portable interface method changed, sogo generate ./...leavesdocs/coverageunchanged. The new operations are AWS-only optional interfaces, the same pattern as event invoke config and Function URLs.providers/aws/lambda:lifecycle.go(settle windows, gates, not-found error),code_signing.go,account_settings.go, the per-version snapshot inversions.go, and the log tail inexeclogs.go. All business rules live here.server/aws/lambda: routes, classify and IAM rules for the new operations;X-Amz-Log-Result; version rendering from the snapshot, including the version's ownLastModified/State/LastUpdateStatus(FunctionVersioncarries the lifecycle the provider computes); CreateFunction withCodeSigningConfigArndeletes the just-created function and returnsCodeSigningConfigNotFoundExceptionif the config disappears between the check and the attach;StateReason/StateReasonCode/LastUpdateStatusfrom the provider (falling back toSuccessfulfor a backend that doesn't report it).Provider Coverage
Checklist
go test -p 2 ./...run caught one stale expectation,TestAuthzMatrixLambdaUnroutedOps, which listed account-settings and code-signing-configs as unrouted. I moved those two into a newTestAuthzMatrixLambdaAccountAndCodeSigning, then re-rango test -p 2 . ./server/aws/... ./providers/aws/... ./persist/... ./services/... ./compat/..., which is green.--new-from-rev) againstdevelopmentas well as--new-from-revcloudemu_test.go: not added, covered by SDK wire testsgo generate ./...andgo run ./internal/compatgenproduce no diffTest Plan
New tests, all of which fail on
developmentand pass here. I ran the wire file against an extracted copy of the merge base: every test fails there, for exampleversion 1 CodeSize = 26, want 7,log tail missing "START RequestId:",GetAccountSettings ... StatusCode: 501,message = "function ghost not found",create = "Active"/""/"Successful".providers/aws/lambda/parity_1518_test.go:TestPublishedVersionKeepsItsCodeSizeAndConfig(including a snapshot round-trip),TestAsyncSettlePendingThenActive,TestAsyncSettleUpdateInProgress,TestDefaultSettleIsImmediate,TestInvokeReturnsLogTail,TestLogTailKeepsLast4KB,TestGetAccountSettings,TestCodeSigningConfigLifecycle(including a snapshot round-trip),TestFunctionNotFoundMessageNamesARN,TestGetAliasNotFoundMessage,TestPublishedVersionReportsItsOwnLifecycle.server/aws/lambda/parity_1518_sdk_test.go(real aws-sdk-go-v2 client):TestSDKPublishedVersionIsImmutable,TestSDKInvokeLogTypeTail,TestSDKGetAccountSettings,TestSDKCodeSigningConfigs(typedCodeSigningConfigNotFoundException,ResourceConflictException,ResourceNotFoundException, tags),TestSDKFunctionNotFoundMessage,TestSDKPublishedVersionKeepsItsOwnLifecycle(default and async-settle:GetFunction --qualifier 1andListVersionsByFunctionunchanged across a$LATESTupdate),TestSDKPublishOnCreateVersionIsPendingUntilCreated,TestSDKCreateFunctionRollsBackWhenCodeSigningAttachFails, andTestSDKAsyncSettleWaiters, which drives the SDK's ownFunctionActiveV2andFunctionUpdatedV2waiters through Pending to Active and InProgress to Successful and asserts that Invoke while Pending is aResourceConflictException.classify_test.goandiam_actions_test.go: rows for every new operation (action and resource ARN).Commands:
go build ./...go vet ./providers/aws/lambda ./server/aws/lambdago test -p 2 ./...(plus the re-run above)server/aws:TestAuthzMatrixLambdaAccountAndCodeSigning(signed requests under enforce-auth: GetAccountSettings allowed, CreateCodeSigningConfig denied then allowed, GetCodeSigningConfig denied on the config ARN)go test -race ./providers/aws/lambda ./server/aws/lambdago test ./persist/(completeness check)golangci-lint run --timeout=9m --new-from-rev=upstream/development --allow-parallel-runners --concurrency 2 ./providers/aws/lambda/... ./server/aws/lambda/... ./services/serverless/... ./internal/settle/...: 0 issuesgolangci-lint run(no--new-from-rev) on./providers/aws/lambda/... ./server/aws/lambda/...: 28 issues on this branch vs 32 ondevelopmentwith v2.14.0 (2 vs 6 with v2.4.0). Every finding on the branch is also ondevelopment; no function inserver/aws/lambdais over gocyclo 10.Real-user run (AWS CLI v1 / botocore 1.42 against
cloudemu serve -aws-port 4608)Default mode (no
--async-settle):With
serve -async-settle:With
serve -enforce-auth: an IAM user whose policy allows onlylambda:GetAccountSettingsandlambda:CreateCodeSigningConfigon*can call those two operations.list-code-signing-configsis denied withAccessDeniedException ... lambda:ListCodeSigningConfigs on resource: *, andget-code-signing-configis denied onresource: arn:aws:lambda:us-east-1:000000000000:code-signing-config:csc-....The server was stopped after each run, and nothing is left listening on 4608. Terraform is not installed here, so I did not run it.
Known limits
Enforcedoes not reject unsigned code).--async-settle, Invoke runs the new code. Real Lambda keeps serving the previous code until the update completes (follow-up).UpdateFunctionCode --publishreportsActive/Successfulright away rather than following the update's InProgress window.Duration/Billed Durationfields. It does not addMemory Size/Max Memory Used.Related Issues
Part of #1518