Skip to content

Go L3/L4: consume the dataflow substrate (models, accessors, both backends) #436

Open

Description

@sinha108

Is your feature request related to a problem? Please describe.

codeanalyzer-go v0.2.0 (release, PR codellm-devkit/codeanalyzer-go#36) emits the full L3/L4 substrate in both projections: per-callable cfg/cdg/ddg at -a 3, and at -a 4 the interprocedural SDG — parameter vertices, param_in/param_out, per-call-site summary, VTA dispatch edges, library stand-ins and degraded[] — with --emit neo4j projecting the same graph at L4.

The SDK cannot consume any of it. #434 (#426's M7) delivered the L1/L2 facade only, and the models stop at L2. Two concrete failures, both verified against v0.2.0 output:

  1. Current output does not parse at all. GoType.kind is Literal["struct", "interface"] (cldk/models/go/models.py:203), but the analyzer emits four kinds. On a fixture with type Celsius float64 and type Alias = int, GoAnalysis(**doc) raises:

    2 validation errors for GoAnalysis
    application.symbol_table.`k.go`.types.Alias.kind
      Input should be 'struct' or 'interface' [input_value='alias']
    application.symbol_table.`k.go`.types.Celsius.kind
      Input should be 'struct' or 'interface' [input_value='defined']
    

    This is independent of L3/L4 — it blocks -a 1 output on any project declaring a defined type or an alias.

  2. L4 output parses, then silently drops the graph. A -a 4 document loads without error, but GoApplication has no param_in/param_out/external_symbols/degraded, and GoBodyNode keeps only kind/span/callee/is_goroutine — no id, parent, of, is_deferred. A consumer gets a valid object with the dataflow missing, which is worse than a parse error.

Describe the solution you'd like

The L3/L4 consumer train for Go, mirroring what the Python backend already exposes. Scope is specified in codeanalyzer-go/docs/design/specs/v2-l3-l4-dataflow.md §7 (the analyzer-side decisions it builds on are in that repo's CLAUDE.md).

  • Models (cldk/models/go/): widen GoType.kind to all four kinds first, then add the L3 body fields (id, parent), the L4 vertex fields (of, is_deferred), summary, application-scope param_in/param_out, external_symbols (incl. receiver and body stand-ins), and degraded[]. extra="ignore" per Relax the schema-v2 mirrors from extra="forbid" to extra="ignore" #386.
  • Accessors with the Python backend's names and signatures — get_cfg, get_cdg, get_ddg, resolve_callable, resolve_value, slice_backward, slice_forward, paths_between, reaches, backward_cone, flows_to_call, flows_to_argument, taint, get_source, get_external_symbols — routed through commons/graphs.py, commons/resolve.py, commons/results.py exactly as cldk/analysis/python/backend.py does.
  • Both backends: local GoCodeanalyzer (subprocess + JSON → networkx) and GoNeo4jBackend (Cypher with node_label="GoBodyNode", P="GO_"), with a parity contract test — both answer identically, miss paths included.
  • Gates: _require_dataflow at level 3 (as the Java backend declares it), plus the port-connectivity probe — refuse slicing/taint when no dependence edge leaves any formal_in.
  • Bump the pin: pyproject.toml still has codeanalyzer-go==0.1.0rc1; v0.2.0 is on PyPI.

Go-specific details that will bite (from §7):

  • Never split a body-node id on its first @ — external ids contain /@external/ before the vertex @. Strip the known callable-id prefix instead.
  • of, not parameters[], resolves formals; formal_in:0 is the receiver for a method.
  • Global names: value_candidate turns <global>:example.com/lib::V into example.com/lib.V, so a lib.V query fails segment_match. Normalise the import path or match on / too.
  • degraded[] is the honesty signal. Non-empty means part of the graph is approximated (range_func, unmapped_ssa, …), so the SDK must not certify "no flow" over it.
  • Known gap: global → global summary edges are omitted (L4: global → global summary edges are omitted; reverse for context-sensitive walks or finer wiring codeanalyzer-go#30). Reachability is unchanged, but depth-bounded walks (DEFAULT_DEPTH = 5) may stop short where the shortcut is gone.

Describe alternatives you've considered

Additional context

Evidence that the substrate is real and worth consuming:

  • 8 open-source Go apps complete -a 4 deterministically (-j 1 = -j 16 = rerun, byte-identical). Kubernetes: 748k lines, 38 s, 7.9 GB, 1.71M SDG nodes and 5.78M dependence edges.
  • The Neo4j snapshot's row counts equal the JSON's on all eight.
  • An HTTP → SQL fixture is gated end to end: a handler's request parameter reaches database/sql.DB.Query's query through an interface call only VTA resolves, and unrelated pairs stay unreachable (TestGateL4_HTTPToSQLPath). That fixture is the natural first integration test for taint.
  • 15 metamorphic relations pass with 0 violations on memos, nats-server, chi and the fixtures.
  • Numbers: codeanalyzer-go/docs/design/notes/l3-dataflow-metrics.md § L4.

Caveat for large apps: --emit neo4j currently peaks at 21–25 GB on nats-server and Kubernetes (codellm-devkit/codeanalyzer-go#38), and analysis.json at -a 4 reaches 1.78 GB, which the local backend would load whole.

Activity

  1. sinha108 commented on Oct 30, 2026

    @sinha108
    CollaboratorAuthor

    Precision inherited from the shared walk. The Go accessors route through commons/graphs.py like Python's, so they inherit its context-insensitivity: a walk can enter a callee through one call site's param_in and leave through another's param_out (#437). Nothing to fix in the Go backend — noted here so the docstrings say what a path proves, and so consumers read TaintResult.exhausted / .unresolved rather than treating a returned path as a feasible flow.

    #437 is also the gate on reinstating the global → global summary edges the analyzer omits (codellm-devkit/codeanalyzer-go#30).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions