Repository navigation
Preserve namespace/application ownership through module lookup and indexing #435
Description
Module lookup can resolve a scanner path to the wrong service when file keys overlap
Problem
Module lookup can lose service ownership when several services in one application contain the same project-relative file key.
For example, the Online Boutique microservice application contains currencyservice and paymentservice. Each service is analyzed from its own source directory:
online-boutique/
src/currencyservice/server.js
src/paymentservice/server.js
The analyzers already emit distinct service-qualified module IDs:
can://currencyservice/javascript/server.js
can://paymentservice/javascript/server.js
However, both modules have the local key server.js. cldk.analysis.commons.keys.resolve_module_key accepts a caller path when it ends with a known local key. Consequently, currencyservice/server.js resolves to server.js in both services' backends. Local path normalization does not establish which service owns the original scanner path.
A multi-service caller that probes every backend and accepts the first callable can therefore select paymentservice code for a currencyservice source position. Indexes that deduplicate by local filename hide the distinct owners, and filename-tail joins can attach a finding to both modules.
Minimal reproduction
The shared helper demonstrates the ownership loss without needing a database:
from cldk.analysis.commons.keys import resolve_module_key
currency_keys = ["server.js"]
payment_keys = ["server.js"]
assert resolve_module_key("currencyservice/server.js", currency_keys) == "server.js"
assert resolve_module_key("currencyservice/server.js", payment_keys) == "server.js"To reproduce the downstream failure, use two fixture files with these source ranges:
| Service | File | Callable range | Meaning of line 28 |
|---|---|---|---|
| currencyservice | server.js |
_loadProto, starting at line 91 |
Module scope; no callable contains it |
| paymentservice | server.js |
HipsterShopServer.constructor, lines 24–34 |
Inside the constructor |
Route the scanner position src/currencyservice/server.js:28 through paymentservice first using filename-suffix matching. It can be attributed to the paymentservice constructor even though the source position belongs to currencyservice. These ranges form the regression fixture; the helper reproduction alone does not exercise callable selection.
Expected behavior
- Resolve an ordinary SDK request within one selected namespace and analyzed application, retaining its language view.
- Preserve a canonical module ID through lookup, indexing, analysis, deduplication, and finding joins. A relative filename is a lookup key within a binding, not a cross-service identity.
- Route scanner paths using declared service/source-root mappings. Reject a conflicting owner; report ambiguous or missing ownership instead of trying every backend.
- Return the currencyservice module-scope/no-callable result for its line 28. Never substitute paymentservice's constructor.
- Preserve both modules in combined indexes, and join resolved module/callable references by canonical ID equality.
- Verify local/Neo4j parity and independence from backend iteration order.
The proposed qualified identity is:
can://online-boutique/currencyservice/javascript/server.js
can://online-boutique/paymentservice/javascript/server.js
Adding a namespace segment alone does not fix a consumer that drops service ownership.
Existing implementation work to repurpose and extend
The workspace-root PRs contain related input validation, path handling, emission, and regression-test work:
Extend that work for namespace/application-qualified identity and scoped SDK lookup. Workspace-relative paths can support scanner mapping; they must not replace explicit application ownership.
Namespace identity contract
Application-owned IDs use can://<namespace>/<application>/<existing-language-and-entity-suffix>. An omitted application name defaults to the selected input root basename; an omitted namespace defaults to the resolved application name. Thus inventory emits can://inventory/inventory, while currencyservice in online-boutique emits can://online-boutique/currencyservice/javascript/server.js. Both root segments remain present.
Root inputs are case-sensitive raw strings: encode UTF-8 bytes outside ASCII letters/digits/-._~ with uppercase percent escapes, without trimming or Unicode normalization. Reject empty/whitespace-only names, controls, unpaired surrogates, slash/backslash, and whole-name ./... For example application a%b, namespace team space yields can://team%20space/a%25b.
Require integer identity_version: 2 in existing analysis envelopes and application anchors, while preserving schema_version: "2.0.0". Retain raw application_name and identity_namespace on application records. Keep language suffixes, input-relative file keys, and pkg: identities unchanged. Qualify all embedded application-owned references; include namespace in cache identity; reject incompatible outputs read-only and regenerate from source without aliases or ID translation.
Acceptance and scope
Bind ordinary SDK lookup to one selected namespace/application and its existing language view. Preserve canonical IDs in module indexes and equality joins; deduplicate currency/payment/currency rows to exactly two modules despite both file keys being server.js. Retain relationship kinds and existing site/context discriminators. Reject missing/conflicting/ambiguous ownership without namespace-wide fallback or unrestricted suffix search. Verify local and Neo4j parity, cache separation, read-only graph attachment, and the line-28 wrong-service regression above. Scanner/source-root routing and downstream finding joins remain explicit integration responsibilities.
Implementation delivery and remaining integration work
Implemented in SDK #440 (Draft), Python #235, TS/JS #220, Java #275, and Go #41. Existing producer PRs were repurposed in place.
Actual Online Boutique live validation — 2026-10-31
Cloned upstream revision c7cb6d5 over SSH and projected 11 supported services into one Neo4j5.26 database; skipped C# cartservice. All SDK reads used Neo4j backends, not analyzer JSON. After three follow-up fixes, all11 SDK views preserve actual module/declaration IDs and exact service ownership. The main namespace owns67,898 nodes and217,873 outgoing relationships.
- Currency server.js:28 yields module scope/no callable; payment:28 yields its own constructor; currency:92 resolves _loadProto. Reverse backend order and foreign-owner rejection pass.
- Real canonical DISTINCT keeps both same-name modules; a currency finding joins exactly to currency. No duplicate canonical IDs or cross-service canonical relationships in the main namespace. Two real payment calls retain separate IDs.
- Same application in another namespace and omitted/default repeated namespace root pass. Eager currency regeneration preserves the other12 service/control scopes. SDK attachment/read checks are mutation-free.
- Java and all4 Go services pass strict full-depth projection after build/toolchain setup. SDK suite1952passed422skipped; TS/JS namespace/Neo4j suites70passed23skipped. Skips are disclosed, not claimed as passing Docker tests.
Live testing found and fixed TS/JS relative graph module keys, TS/JS class ownership during eager reconciliation (6d80997), and Python SDK dropping canonical IDs in graph reconstruction (928ce57). Regressions failed before the fixes, passed after, and received independent review.
This issue remains open for:
- Compatible producer releases, SDK pins/lock and installed-package verification are complete: Python/TypeScript1.7.0, Java3.4.0, Go0.3.0, SDK2.1.0. All task PRs merged; all packages published on GitHub/PyPI. Fresh installed SDK and producer wheels passed the11-service live Neo4j checks; the full SDK suite passed1960tests with414skips. Separate SDK#439 retains the older superseded Go pin.
- Downstream scanner service/source-root routing, combined indexes and finding joins must retain canonical IDs in the actual integration repositories. Those repositories were not modified here; database equality-join tests do not certify their code.
- General sparse TS/JS graph attachment: a fresh currency-only graph without TS_HAS_METHOD still trips the existing fingerprint; the completed11-service database passes. Existing module-source hydration diagnostics also remain outside the ownership fix.
No production regeneration or business RPC deployment was performed. Coordinated merges and releases are now complete as recorded above.
Activity