Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 9 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Changelog

## [3.4.0] - 2026-10-31

- Add `--namespace` and namespace-qualified application IDs.
- Scope embedded references, graph reconciliation and caches to the selected application.
- Validate stored identity metadata before connected writes.

Regenerate existing analyses and graphs from source for the new ID roots.
56 changes: 43 additions & 13 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -126,6 +126,7 @@ Usage: codeanalyzer [-hvV] [--no-build] [--no-clean-dependencies]
[-a=<analysisLevel>] [-b=<build>] [-f=<projectRootPom>]
[-i=<input>] [-o=<output>] [-s=<sourceAnalysis>]
[--emit=<emit>] [--app-name=<appName>]
[--namespace=<namespace>]
[--neo4j-uri=<uri>] [--neo4j-user=<user>]
[--neo4j-password=<password>] [--neo4j-database=<db>]
[-t=<targetFiles>]...
Expand All @@ -145,6 +146,7 @@ Analyze java application.
schema (the Neo4j schema.neo4j.json contract).
--app-name=<name> Logical application name for the graph :JApplication
anchor (default: input dir name).
--namespace=<name> Identity namespace (defaults to application name).
--neo4j-uri=<uri> Push the graph to a live Neo4j over Bolt (incremental);
omit to write graph.cypher. Falls back to the
NEO4J_URI environment variable.
Expand All @@ -157,6 +159,24 @@ Analyze java application.
-V, --version Print version information and exit.
```

**Application identity.** `--app-name` defaults to the basename of `--input` and `--namespace`
defaults to the resolved application name. The canonical root always contains both segments:
`can://<namespace>/<application>`, including `can://inventory/inventory` for a default monolith.
Names are case-sensitive raw strings encoded as UTF-8 URI segments without trimming or Unicode
normalization. Explicit empty/whitespace-only names, controls, unpaired surrogates, slashes,
backslashes, `.` and `..` are rejected. `--namespace` is unavailable with `--schema v1`.

```bash
codeanalyzer -i monorepo/services/orders --app-name orders --namespace commerce
# root: can://commerce/orders
# symbol_table key: src/main/java/com/acme/Order.java
```

File and artifact keys remain relative to `--input`. The draft `--workspace-root` option has
been removed. Analysis envelopes and graph application anchors carry `identity_version: 2`;
anchors also retain raw `application_name` and `identity_namespace`. Namespace changes invalidate
the L1 cache. Legacy outputs must be regenerated from source; retire legacy graph data explicitly.


## 3. Installing `codeanalyzer` as a native binary (once built, no JVM will be required for running `codeanalyzer`)

Expand Down Expand Up @@ -241,23 +261,22 @@ build manifest or a configuration key is not a Java concept — a sibling-langua
the same repository lands on the same nodes instead of a per-language duplicate.
`SCHEMA_VERSION` is stamped onto the `:JApplication` node of every emitted graph.

Every node id starts with `can://<app>/` — `can://<app>/java/…` for code, `can://<app>/artifact/…`
for build manifests and config keys, `can://<app>/@external/…` for library symbols — so
`can://<app>` is a prefix of every node the application emits, which is what the destructive
Every node id starts with `can://<namespace>/<app>/` — `can://<namespace>/<app>/java/…` for code, `can://<namespace>/<app>/artifact/…`
for build manifests and config keys, `can://<namespace>/<app>/@external/…` for library symbols — so
`can://<namespace>/<app>` is a prefix of every node the application emits, which is what the destructive
statements scope on. (`:Package` is the exception: it is keyed on a `pkg:` purl, sits under no
application, and no wipe reaches it.) `:JApplication` is keyed on that id, not on the free-text
`--app-name`, so the root is addressable by the same id its descendants are prefixed with. The id
is derived from `--app-name` (`can://<app-name>`), so it does
**not** disambiguate two services analyzed under the same name — those still merge onto one root.
Give each service its own `--app-name` if they share a database.
is derived from the pair (`--namespace`, `--app-name`). Services with the same raw application name
remain distinct in different namespaces. Services selecting the same pair merge onto the same root.

**Polyglot ids.** The `/java/` segment is what makes a code node this analyzer's. Three id families
deliberately omit it and are therefore **language-neutral, shared merge targets**: `:Artifact`,
`:ConfigKey` and `@external` symbols. A sibling analyzer over the same `<app>` mints byte-identical
ids for them, so a `pom.xml`, a config key or `java.util.Map#get` is *one* node in a merged graph
rather than a per-language duplicate. That sharing is the point, and its cost is accepted: two
analyzers' notions of a library symbol are not necessarily the same thing, and merging them says
they are. The consequence for writes: because these shared nodes sit inside `can://<app>/`, a Cypher
they are. The consequence for writes: because these shared nodes sit inside `can://<namespace>/<app>/`, a Cypher
snapshot's prefix wipe rebuilds them, so a cross-language edge into a shared `:Artifact` is dropped
by one analyzer's snapshot and restored on the other's next push.

Expand Down Expand Up @@ -297,14 +316,13 @@ codeanalyzer -i /path/to/project -a 2 --emit neo4j -o ./out
cypher-shell -u neo4j -p <password> < ./out/graph.cypher
```

The snapshot is **not** incremental: it constraints, scopes-wipes this application's prior subgraph,
then `UNWIND … MERGE`-loads the full truth. The wipe is everything under `can://<app>/`, plus a
containment traversal that also replaces a prior v1 (pre-`3.0.0`) graph of the same application.
Offline snapshots validate incoming identity metadata; they do not inspect an existing database. Execute them only as part of explicit regeneration. The snapshot is **not** incremental: it constraints, scopes-wipes this application's prior subgraph,
then `UNWIND … MERGE`-loads the full truth. The wipe is everything under `can://<namespace>/<app>/`, plus a
containment traversal scoped to the selected canonical application. Legacy roots are not retired automatically.
That scope **includes** this application's `:Artifact` and `:ConfigKey` nodes — the snapshot rebuilds
them, rather than leaving stale ones to accumulate with nothing ever cleaning them up. The one
consequence: a sibling-language analyzer's edge into a shared `:Artifact` is dropped by a Java
snapshot and restored on that analyzer's next push. The Bolt writer does **not** do this — its
deletions are scoped per module, not per application.
snapshot and restored on that analyzer's next push. The Bolt writer is narrower; see §4.2.

### 4.2. Live incremental push over Bolt

Expand All @@ -313,11 +331,23 @@ codeanalyzer -i /path/to/project -a 2 --emit neo4j \
--neo4j-uri bolt://localhost:7687 --neo4j-user neo4j --neo4j-password <password>
```

The Bolt writer reads the database's current state and updates **only what changed**: it diffs each
The Bolt writer checks the selected existing anchor's identity marker and raw names before DDL or writes, then updates **only what changed**: it diffs each
compilation unit's `content_hash`, replaces just the changed units' subgraphs (idempotent
`MERGE` upserts), and — on a full run — prunes units whose source file vanished. Combine with
`--target-files` for a targeted, partial re-push (orphan pruning is then skipped).

On a full run it also reconciles the application's artifacts, so a deleted or renamed file, a
or removed dependency leaves nothing stale within the selected namespace/application:

- Java-owned artifact edges are rebuilt from the run: `HAS_ARTIFACT`, `DECLARES_DEPENDENCY`/`LOCKS`
to `pkg:maven/…`, and code reads of them (`J_USES_CONFIG`, `J_DISPATCHES_TO`,
`J_READS_CONFIG_UNRESOLVED`), so a config-only change updates them too.
- An `:Artifact`/`:ConfigKey` the run no longer emits loses its `:JCanNode` label. An artifact is
then deleted only if no other analyzer's `HAS_ARTIFACT` still points at it; a config key only if
no other analyzer's marker (`:PyCanNode`, `:TSCanNode`) still claims it. A sibling's edges are
never removed.
- Legacy artifacts require explicit retirement; reconciliation never falls back to a legacy root.

### 4.3. Schema contract

```sh
Expand Down
28 changes: 28 additions & 0 deletions docs/design/specs/2026-10-24-workspace-root.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# Application identity namespaces (supersedes draft workspace-root design)

The unmerged `--workspace-root` proposal is withdrawn. File keys, module suffixes and artifact
paths remain relative to the selected `--input` directory. No path prefix rekeying or URI
translation is performed.

`application_name` is the supplied `--app-name`, otherwise the input basename. `identity_namespace`
is the supplied `--namespace`, otherwise the resolved application name. Both segments are always
present: `can://<namespace>/<application>`, including `can://inventory/inventory`.

Names retain case and Unicode spelling. UTF-8 bytes outside ASCII letters/digits/`-._~` are
percent-encoded with uppercase escapes. Empty or whitespace-only names, controls, unpaired
surrogates, slashes/backslashes and whole-name `.`/`..` are invalid. Explicit invalid flags never
fall back to defaults. Legacy schema v1 rejects `--namespace`.

The existing schema family remains `2.0.0`. The analysis envelope and graph application anchors
carry `identity_version: 2`; anchors retain raw `application_name` and `identity_namespace`.
Missing persisted metadata is unsupported, not defaulted. Regenerate from source and retire legacy
outputs explicitly. Cache identity includes the canonical namespace/application root and marker.

Java keeps its existing `/java/<file>/<type>/<signature>` suffixes and body ordinals. External
symbols and artifacts share the canonical application root; `pkg:` identities remain unchanged.
Canonical-parent builders forward their root unchanged. No mutable global namespace is used.

Bolt checks the selected existing application anchor before DDL or data writes. Incremental hashes,
module purges, orphan pruning and artifact reconciliation stay within that selected application.
Shared-artifact retention rules remain intact. Offline snapshots validate incoming metadata but
cannot inspect the target database; execution is an explicit regeneration operation.
2 changes: 1 addition & 1 deletion gradle.properties
Original file line number Diff line number Diff line change
@@ -1 +1 @@
version=3.3.4
version=3.4.0
2 changes: 1 addition & 1 deletion packaging/python/src/codeanalyzer_java/__init__.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@
from pathlib import Path
from typing import List

__version__ = "0.0.0"
__version__ = "3.4.0"

__all__ = ["jar_path", "java_path", "command", "__version__"]

Expand Down
3 changes: 3 additions & 0 deletions schema.neo4j.json
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@
"properties": {
"id": "string",
"name": "string",
"application_name": "string",
"identity_namespace": "string",
"identity_version": "integer",
"schema_version": "string",
"analyzer_name": "string",
"analyzer_version": "string",
Expand Down
53 changes: 35 additions & 18 deletions src/main/java/com/ibm/cldk/CodeAnalyzer.java
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
import com.ibm.cldk.neo4j.BoltConfig;
import com.ibm.cldk.neo4j.Neo4jEmitter;
import com.ibm.cldk.schema.Analysis;
import com.ibm.cldk.schema.CanId;
import com.ibm.cldk.schema.JArtifact;
import com.ibm.cldk.schema.JDependency;
import com.ibm.cldk.schema.JEntrypointReport;
Expand Down Expand Up @@ -146,6 +147,9 @@ public class CodeAnalyzer implements Runnable {
// values between CommandLine instances in the same JVM. New flags do not add to that.
private String schema = "v2";

@Option(names = "--namespace", description = "Identity namespace (defaults to application name)")
private String namespace;

@Option(names = {"-c",
"--cache-dir" }, description = "Directory holding the incremental analysis cache. When set, "
+ "unchanged files are reused from analysis_cache.json instead of being reparsed.")
Expand Down Expand Up @@ -261,6 +265,11 @@ private void analyze() throws Exception {
analyzeV2();
return;
}
if (namespace != null) {
// v1 keys are absolute paths, so there is nothing for a workspace root to relativize.
throw new ParameterException(spec.commandLine(),
"error: --namespace is not supported with --schema v1");
}

JsonObject combinedJsonObject = new JsonObject();
Map<String, JavaCompilationUnit> symbolTable;
Expand Down Expand Up @@ -399,9 +408,11 @@ private boolean isV2Schema() {
* which {@code --artifact-text}/{@code --artifact-text-max-bytes} may have emptied or truncated —
* delegates to {@link DependencyView#readFromDisk} rather than re-reading the file itself, so that
* from-disk logic exists exactly once in this codebase.
*
* @param keyRoot the directory artifact paths are relative to (the workspace root, else --input)
*/
private static String readFully(JArtifact artifact) {
return DependencyView.readFromDisk(Paths.get(input), artifact.getPath());
private static String readFully(Path keyRoot, JArtifact artifact) {
return DependencyView.readFromDisk(keyRoot, artifact.getPath());
}

/**
Expand Down Expand Up @@ -455,10 +466,16 @@ private void analyzeV2() throws Exception {
if (input == null) {
throw new ParameterException(spec.commandLine(), "error: --input is required");
}

String application = appName != null && !appName.isBlank()
? appName
: Paths.get(input).toAbsolutePath().normalize().getFileName().toString();
Path inputRoot = Paths.get(input).toAbsolutePath().normalize();
Path keyRoot = inputRoot;
String application = spec.commandLine().getParseResult().hasMatchedOption("--app-name")
? appName : inputRoot.getFileName().toString();
String applicationId;
try {
applicationId = CanId.applicationId(application, namespace);
} catch (IllegalArgumentException e) {
throw new ParameterException(spec.commandLine(), "error: invalid --app-name or --namespace: " + e.getMessage());
}

// Always attempt library type resolution: without the dependency jars on the solver's path,
// third-party types degrade to bare spellings (`Model` rather than `org.springframework.ui.Model`)
Expand All @@ -485,7 +502,7 @@ private void analyzeV2() throws Exception {
// L1 artifact that must not carry L3 overlays. So bypass the cache entirely at level >= 3.
Map<String, JModule> cached = (eager || analysisLevel >= 3)
? new java.util.LinkedHashMap<>()
: L1Cache.load(cache, application, version);
: L1Cache.load(cache, applicationId, version);

Map<String, JModule> modules;
List<L2CallGraph.RtaEndpoint> rtaEndpoints = null;
Expand All @@ -496,8 +513,8 @@ private void analyzeV2() throws Exception {
// is what turns them into a non-zero exit for a caller that cannot read stderr.
List<String> degraded = new ArrayList<>();
try {
modules = L1Extractor.extractAll(
Paths.get(input), application, dependencyDir, cached,
modules = L1Extractor.extractAllForApplication(
Paths.get(input), applicationId, dependencyDir, cached,
analysisLevel, graphFieldDepth, l3Engine, entrypointReport);
// The WALA L3 engine needs the RTA build; --no-rta suppresses it, so a level-3 wala run with
// --no-rta would silently carry no overlays. Warn rather than degrade without a signal.
Expand Down Expand Up @@ -559,21 +576,21 @@ private void analyzeV2() throws Exception {
// (an L2 refinement) must not be persisted into it. At level >= 3 the tree carries L3 overlays,
// which are not an L1 artifact, so the cache is not written.
if (analysisLevel < 3) {
L1Cache.save(cache, application, version, modules);
L1Cache.save(cache, applicationId, version, modules);
}

// The repository-artifact layer (build manifests, config files, declared dependencies) sits
// beside the call-graph/SDG assembly below because both are application-scope data built once
// -- but unlike them it is L1 data and runs at EVERY analysis level, so it is computed here,
// ahead of the level gate, rather than inside either branch of it.
Map<String, JArtifact> artifacts =
ArtifactDiscovery.discover(Paths.get(input), application, artifactText, artifactTextMaxBytes);
List<JDependency> dependencies = DependencyView.build(Paths.get(input), artifacts);
ArtifactDiscovery.discover(inputRoot, application, artifactText, artifactTextMaxBytes, namespace);
List<JDependency> dependencies = DependencyView.build(keyRoot, artifacts);
for (JArtifact a : artifacts.values()) {
if (ConfigKeys.isEligible(a)) {
// Re-read from disk: `source` may be truncated or suppressed, and extraction
// must not silently degrade with a capture flag.
ConfigKeys.Result r = ConfigKeys.extract(a, readFully(a), artifactText);
ConfigKeys.Result r = ConfigKeys.extract(a, readFully(keyRoot, a), artifactText);
a.setConfigKeys(r.keys);
// A pre-existing "partial" from the dependency pass is never overwritten.
if (!r.ok) {
Expand All @@ -589,7 +606,7 @@ private void analyzeV2() throws Exception {
// engine-free, so they run whenever analysisLevel >= 4 regardless of the WALA build's fate.
Analysis analysis;
if (analysisLevel >= 2) {
L2CallGraph.Result l2 = L2CallGraph.build(application, modules, rtaEndpoints, externalCalls);
L2CallGraph.Result l2 = L2CallGraph.buildForApplication(applicationId, modules, rtaEndpoints, externalCalls);
// SdgVertices needs the callee ids L2CallGraph.build just backfilled onto call body nodes,
// and no dependency jars, so it runs here rather than inside the try block above.
SdgVertices.Result sdg = null;
Expand All @@ -604,10 +621,10 @@ private void analyzeV2() throws Exception {
analysis = V2Emitter.emit(application, analysisLevel, modules, version,
l2.callGraph(), l2.externalSymbols(),
sdg == null ? null : sdg.paramIn, sdg == null ? null : sdg.paramOut,
artifacts, dependencies);
artifacts, dependencies, namespace);
} else {
analysis = V2Emitter.emit(application, analysisLevel, modules, version,
null, null, null, null, artifacts, dependencies);
null, null, null, null, artifacts, dependencies, namespace);
}

// Enforced here, after every overlay pass has had its chance and before anything is written:
Expand All @@ -629,7 +646,7 @@ private void analyzeV2() throws Exception {
// Config reads join the L1 tree to the artifact layer's declared keys, so it runs after both
// exist and at every analysis level -- the literal tier needs no call graph. Absence means
// "no fact", so an empty result leaves both keys off the envelope.
ConfigUses.Result configReads = ConfigUses.detect(application, modules, artifacts,
ConfigUses.Result configReads = ConfigUses.detectForApplication(applicationId, modules, artifacts,
analysisLevel, analysis.getApplication().getCallGraph());
if (!configReads.uses.isEmpty()) {
analysis.getApplication().setConfigUses(configReads.uses);
Expand All @@ -640,7 +657,7 @@ private void analyzeV2() throws Exception {

// View dispatches (#259) join dispatch sites to the artifact layer's view templates -- the
// same shape as config reads, at every level, with the dataflow tiers widening from -a 3.
ViewDispatches.Result views = ViewDispatches.detect(application, modules, artifacts,
ViewDispatches.Result views = ViewDispatches.detectForApplication(applicationId, modules, artifacts,
analysisLevel, analysis.getApplication().getCallGraph());
if (!views.dispatches.isEmpty()) {
analysis.getApplication().setViewDispatches(views.dispatches);
Expand Down
Loading