Skip to content

chore(serverless): publish web request-start event - #19898

Merged
gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
mainfrom
tianning.li/2-wsgi-asgi-integration
Oct 8, 2026
Merged

gh-worker-dd-mergequeue-cf854d[bot] merged 1 commit into
mainfrom
tianning.li/2-wsgi-asgi-integration

Conversation

@litianningdatadog

@litianningdatadog litianningdatadog commented Aug 27, 2026 •

Copy link
Copy Markdown
Contributor

Stacked PRs:

Description

This PR provides the framework side of the MicroVM identity-refresh flow. WSGI,
ASGI, and Django publish web.request.starting before they create a root span,
passing the request method and normalized path.

The publisher is deliberately product-neutral: it does not import MicroVM
helpers, recognize the MicroVM /run endpoint, or decide whether to refresh a
runtime ID. PR #19939 registers the MicroVM-only listener and owns that policy.
When no product listens for the event, the publisher returns after a single
listener check, before reading the request or normalizing the path.

Two helpers in trace_utils own the publishing:

  • dispatch_wsgi_web_request_starting(environ) is used by DDWSGIMiddleware
    and Django's get_response hook. WSGI and Django share an environ marker so
    nested layers publish one event. The Django hook also covers Django 3.0 ASGI
    requests, which reach the sync get_response because get_asgi_application
    is only wrapped on Django 3.1+.
  • dispatch_asgi_web_request_starting(scope) publishes only from the outer ASGI
    application. It accepts scopes whether or not path already includes
    root_path, so mounted or server-prefixed apps do not produce a duplicated
    prefix.

Scope: the event is published for the generic WSGI/ASGI middleware and Django.
Frameworks whose automatic instrumentation creates the root span without
DDWSGIMiddleware (Falcon, Bottle, Pyramid, CherryPy, Molten) are out of scope
for this PR and will follow separately.

This implements the requested split in review comment r4146791727.

Reference

Testing

  • scripts/lint fmt and scripts/lint typing on the changed files
  • focused WSGI request-start tests (py3.13): 7 passed
  • focused ASGI request-start tests (py3.13): 48 passed
  • focused Django request-start tests: 3 passed on Django 3.0 (py3.9) and on
    Django 5.1 (py3.13), including a real ASGIRequest reaching get_response

Risks

There is no public API change. The only production listener in this stack is
registered by PR #19939 in MicroVM processes, so non-MicroVM requests do not
dispatch this event.

🤖 Generated with Claude Code

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 27, 2026 •

Copy link
Copy Markdown

Codeowners resolved as

Resolved from the full PR diff against main using the target branch CODEOWNERS file.
CODEOWNERS team requests not listed below are not required by the current file set.

No remaining files require a CODEOWNERS review.

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 27, 2026 •

Copy link
Copy Markdown

Circular import analysis

⚠️ Existing circular imports

There are 1 circular imports that already exist on the base branch and have not been changed by this PR.

ddtrace.errortracking._handled_exceptions.bytecode_injector -> ddtrace.errortracking._handled_exceptions.callbacks -> ddtrace.errortracking._handled_exceptions.collector -> ddtrace.errortracking._handled_exceptions.bytecode_reporting -> ddtrace.errortracking._handled_exceptions.bytecode_injector

@cit-pr-commenter-54b7da

cit-pr-commenter-54b7da Bot commented Aug 27, 2026 •

Copy link
Copy Markdown

Dependency direction analysis

⚠️ Existing dependency direction violations

There are 191 dependency direction violations that already exist on the base branch and have not been changed by this PR.

Show existing violations (showing 5 of 191 highest severity)
ddtrace.internal.tracemethods -×-> ddtrace.trace  (internal-core -> product:tracing, score=125)
ddtrace.internal.openfeature._span_enrichment -×-> ddtrace.trace  (product:openfeature -> product:tracing, score=123)
ddtrace.internal.ci_visibility.recorder -×-> ddtrace.trace  (product:ci_visibility -> product:tracing, score=123)
ddtrace.llmobs._integrations.vllm -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=123)
ddtrace.llmobs._integrations.openai_agents -×-> ddtrace.trace  (product:llmobs -> product:tracing, score=123)

To see all violations, download the layers-base.json and layers-pr.json artifacts from this CI job and run:

uv run --script scripts/import-analysis/layers.py compare layers-base.json layers-pr.json

@datadog-datadog-us1-prod

datadog-datadog-us1-prod Bot commented Aug 27, 2026 •

Copy link
Copy Markdown

Tests

✅ All CI checks and tests passed.

🎉 All green!

🧪 All tests passed
❄️ No new flaky tests detected

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: 1e76b73 | Docs | View more details | Give us feedback!

@litianningdatadog litianningdatadog changed the title feat(serverless): emit MicroVM request start feat(serverless): emit MicroVM request start for WSGI & ASGI-based applications Aug 27, 2026
@litianningdatadog litianningdatadog added the aws-microvm Work related to AWS MicroVM onboarding label Aug 31, 2026
@litianningdatadog
litianningdatadog force-pushed the tianning.li/2-wsgi-asgi-integration branch 2 times, most recently from 4aebb33 to 206cd22 Compare September 3, 2026 20:32
@litianningdatadog
litianningdatadog force-pushed the tianning.li/2-wsgi-asgi-integration branch from 206cd22 to f3ea773 Compare September 3, 2026 20:47
@litianningdatadog
litianningdatadog force-pushed the tianning.li/2-wsgi-asgi-integration branch from f3ea773 to 68bf61e Compare September 4, 2026 15:37
@litianningdatadog
litianningdatadog requested a balanced review from Copilot September 4, 2026 16:23
@litianningdatadog
litianningdatadog force-pushed the tianning.li/2-wsgi-asgi-integration branch from 68bf61e to 8e26c22 Compare September 4, 2026 16:27

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The environment detector lacks direct coverage, and the release note announces behavior completed only by a follow-up PR.

Once you've addressed the issues Copilot identified, you can request another Copilot review.

Pull request overview

Adds pre-span MicroVM request detection to WSGI and ASGI instrumentation, supporting the runtime identity refresh stack.

Changes:

  • Dispatches web.request.starting for matching MicroVM requests.
  • Adds explicit runtime identity refresh callbacks while preserving fork lineage.
  • Adds focused integration/runtime tests and a release note.
File summaries
File Description
ddtrace/contrib/_events/web_framework.py Defines the request-start event.
ddtrace/contrib/internal/web.py Matches and dispatches MicroVM requests.
ddtrace/contrib/internal/asgi/middleware.py Dispatches before ASGI span creation.
ddtrace/contrib/internal/wsgi/wsgi.py Dispatches before WSGI span creation.
ddtrace/internal/serverless/__init__.py Detects the MicroVM environment.
ddtrace/internal/runtime/__init__.py Adds explicit identity refresh support.
tests/contrib/asgi/test_asgi.py Tests ASGI matching and ordering.
tests/contrib/wsgi/test_wsgi.py Tests WSGI request matching.
tests/tracer/runtime/test_runtime_id.py Tests refresh and fork semantics.
releasenotes/notes/add-lambda-microvm-web-request-starting-ffa6841a15e3c57c.yaml Announces MicroVM request detection.
Review details
  • Files reviewed: 8/8 changed files
  • Comments generated: 2
  • Review effort level: Balanced

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment thread ddtrace/internal/serverless/__init__.py Outdated
Comment thread releasenotes/notes/add-lambda-microvm-web-request-starting-ffa6841a15e3c57c.yaml Outdated
@litianningdatadog
litianningdatadog force-pushed the tianning.li/2-wsgi-asgi-integration branch from 8e26c22 to cdc53d4 Compare September 4, 2026 16:39
@litianningdatadog
litianningdatadog force-pushed the tianning.li/2-wsgi-asgi-integration branch 2 times, most recently from d28bc4e to 0d5cf10 Compare September 21, 2026 20:46
@litianningdatadog litianningdatadog added the changelog/no-changelog A changelog entry is not required for this PR. label Sep 23, 2026
@litianningdatadog
litianningdatadog force-pushed the tianning.li/2-wsgi-asgi-integration branch from 0d5cf10 to 4654514 Compare September 23, 2026 14:38
@litianningdatadog
litianningdatadog force-pushed the tianning.li/2-wsgi-asgi-integration branch from bfe3d91 to a44c5ce Compare September 25, 2026 15:50
@litianningdatadog
litianningdatadog requested a review from a team as a code owner September 25, 2026 21:37
@litianningdatadog
litianningdatadog force-pushed the tianning.li/2-wsgi-asgi-integration branch from a44c5ce to 7398b18 Compare September 25, 2026 21:40
Base automatically changed from tianning.li/1-runtime-identity-refresh to main September 28, 2026 15:37
gh-worker-dd-mergequeue-cf854d Bot pushed a commit that referenced this pull request Sep 28, 2026
Stacked PRs:
- 👉 This #19778 — runtime identity foundation
  - #19898 — WSGI/ASGI request-start hook
    - #19939 — central MicroVM /run identity refresh
      - #19820 — tracer/native writer exporter refresh
        - #19821 — telemetry worker refresh
          - #19822 — runtime metrics runtime-id tags

## Description

Adds `ddtrace.internal.runtime.refresh_identity()` for runtimes that need a fresh runtime ID without going through an OS fork.

This introduces `on_runtime_identity_refresh()`, a dedicated callback registry for consumers that should react only to explicit identity refreshes. Existing `on_runtime_id_change()` behavior remains available for fork-related and general runtime-ID change handling.

`refresh_identity()` rotates the runtime ID without recording parent or ancestor lineage because a resumed MicroVM run is not a real fork. Existing fork behavior and lineage tracking remain unchanged.

Refresh callbacks are isolated by default so one failed consumer does not block others. Callers coordinating a refresh transaction can opt into error propagation with `raise_on_error=True` and retry the same identity refresh when a rebuild fails.

## Reference

- [RFC](https://docs.google.com/document/d/17lde4Zak2YRBuDvf32KFanXj9TdpOv24GYaEY8swA1w/edit?tab=t.0#heading=h.wwu37decnk8j)

## Testing

Added coverage for:

- runtime ID rotation through `refresh_identity()`
- preservation of parent and ancestor fork lineage
- explicit refresh subscriber notification
- refresh subscribers not being invoked by fork handling
- callback failure isolation by default
- optional refresh callback failure propagation
- successful callbacks under `raise_on_error=True`

Validation:

- `scripts/run-tests -s --venv 190fcc7 -- -q tests/tracer/runtime/test_runtime_id.py` — 21 passed
- `scripts/lint checks` — passed

## Risks

Low. The new path is opt-in and is not wired to traffic in this PR. Existing fork behavior is preserved.


Co-authored-by: tianning.li <tianning.li@datadoghq.com>

@lym953 lym953 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving blindly as I don't have knowledge on any of the files, and since apm-serverless has approved it.
Will discuss whether we should remove serverless-aws as owner.

@litianningdatadog
litianningdatadog force-pushed the tianning.li/2-wsgi-asgi-integration branch from 7398b18 to 6e98bbd Compare September 29, 2026 18:06
Comment thread tests/contrib/asgi/test_asgi.py Outdated

@quinna-h quinna-h left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM overall for the test files that IDM owns; left one comment about a test that was supposed to prove the request-start event happens before the request span is created, but instead the span marker records when the middleware retrieves the span

@avara1986 avara1986 left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@litianningdatadog
litianningdatadog force-pushed the tianning.li/2-wsgi-asgi-integration branch from 6e98bbd to a251b4d Compare September 30, 2026 14:07
Comment thread ddtrace/contrib/internal/django/response.py Outdated
Comment thread ddtrace/contrib/internal/wsgi/wsgi.py Outdated
Comment thread ddtrace/contrib/internal/web.py Outdated
@litianningdatadog
litianningdatadog force-pushed the tianning.li/2-wsgi-asgi-integration branch from a251b4d to 3d8d4ca Compare September 30, 2026 18:23
@litianningdatadog litianningdatadog changed the title chore(serverless): emit MicroVM request start for WSGI & ASGI-based applications feat(serverless): publish web request-start event Sep 30, 2026

@dubloom dubloom left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Some small comments but overall LGTM

Comment thread ddtrace/contrib/_events/web_framework.py Outdated
Comment thread ddtrace/contrib/internal/web.py Outdated
Comment thread ddtrace/contrib/internal/web.py Outdated

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 11ab2d5fe7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread ddtrace/contrib/internal/wsgi/wsgi.py Outdated
Comment thread ddtrace/contrib/internal/asgi/middleware.py Outdated
Comment thread ddtrace/contrib/internal/django/response.py Outdated
…plications

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@litianningdatadog

Copy link
Copy Markdown
Contributor Author

/merge

@gh-worker-devflow-routing-ef8351

gh-worker-devflow-routing-ef8351 Bot commented Oct 8, 2026 •

Copy link
Copy Markdown

View all feedbacks in Devflow UI.

2026-10-08 15:20:55 UTC ℹ️ Start processing command /merge


2026-10-08 15:21:00 UTC ℹ️ MergeQueue: pull request added to the queue

The expected merge time in main is approximately 57m (p90).


2026-10-08 16:24:07 UTC ℹ️ MergeQueue: This merge request was merged

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

aws-microvm Work related to AWS MicroVM onboarding changelog/no-changelog A changelog entry is not required for this PR.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants