Skip to content

feat(parity): stackiq's capability matrix against five competitors - #1072

Merged
rubenvdlinde merged 2 commits into
developmentfrom
feat/parity-capability-matrix
Sep 25, 2026
Merged

rubenvdlinde merged 2 commits into
developmentfrom
feat/parity-capability-matrix

Conversation

@rubenvdlinde

Copy link
Copy Markdown
Contributor

Adds openspec/parity/capabilities.json, stackiq's capability matrix. Data file only, no page wired. Read from development at c9760e07454e8ae6d4e87d52ce31daa5c04492c8.

Two security-shaped findings (code reading, needs a live check)

  • Active supplier-registered modules are readable anonymously, published or not. The module read rule includes {group: public, match: {registeredBy: Supplier, status: Active}} next to the publication-date rule (lib/Settings/softwarecatalogus_register.json:2501). If OpenRegister applies the rules as an OR, any active module a supplier registered is public, whatever its publicationDate.
  • POST /api/archimate/test-round-trip is @NoAdminRequired and @NoCSRFRequired, and it writes to the live register. It imports a test model into the live AMEF register. It can also never pass: it compares an imported_count nothing sets against the literal string 'calculated_in_export_service'.

Live-defect candidates (code reading)

  • Cancelling an ArchiMate import crashes. The route calls ArchiMateService::cancelArchiMateImport(), which does not exist. The resulting \Error is not caught by the surrounding catch (\Exception).
  • The contract-expiry notification can never fire. The x-openregister-notifications rule filters on status Actief, while the enum is Active / Expired / In negotiation.
  • The concept-organisations dashboard widget is always empty. It filters on concept while the values are Draft / Active / Inactive. Its accept button also writes actief.
  • Federation "pull now" mirrors the instance's own listings. FederationService::fetchPeerCatalog() passes a url that OpenCatalogi's getDirectory ignores.
  • openapi.json has 0 paths.
  • Some detail pages do not show their data. The ModuleDetail and SuiteDetail data widgets list stale Dutch keys, so description and contact person do not render.
  • The email template editor saves nothing. Its Save button shows success anyway.
  • Contact-to-account reads a field that no longer exists. It reads email, which the contactPerson schema no longer has.

Backends no page reaches, and pages nothing opens

  • The connection and usage schemas have no page at all. So no stackiq page can register a koppeling or create the usage a contract requires.
  • ModuleDetail cannot be opened from the Modules list (src/views/FacetedCatalogIndexView.vue:108 has no row click). SuiteDetail cannot be opened from Suites, nor KwetsbaarheidDetail from Vulnerabilities.
  • Settings.vue and StackiqSettingsPage are built but never mounted.
  • These API paths have no caller in src/: api/aanbod, api/aangeboden-gebruik, api/koppelingen-gebruik, api/gebruik, api/intake, api/preferences, api/dashboard, api/setup. Moderation and eol-sync are reached, from admin settings.
  • Many of these endpoints exist for the external VNG Softwarecatalogus frontend, which is not in this repo. Their reachedOn says "external frontend, not in this repo" or "API only". The pending rows that depend on that frontend name it in their question.

Judgement calls

Category. Stackiq does two jobs in one product. It is the back office and API of the successor to the VNG GEMMA Softwarecatalogus: suppliers publish what they offer, and municipalities record what they use and how it connects, all plotted on GEMMA. It is also a portfolio view for one organisation: contracts, lifecycle, licences, vulnerabilities and compliance. So it competes first with the VNG catalogue, second with application portfolio and EA tools, and third with CMDB and IT asset tools. The paragraph sits in the file's category field, so correct it there.

Columns. Five competitors:

  • VNG GEMMA Softwarecatalogus: the incumbent and the requirement source.
  • SAP LeanIX: the application portfolio management reference.
  • BlueDolphin (ValueBlue): the Dutch ArchiMate-first EA tool.
  • GLPI: the open-source asset and CMDB tool with the deepest research on file.
  • TOPdesk: the service management and CMDB suite most municipalities already run.

The other 66 linked competitors stay out. They are developer portals, enterprise-priced asset and licence tools, duplicate EA tools, discovery and hardware tools, or noise such as Atlas Governance, Kong, Gravitee, Apigee and Sensus BPM. The reasons are in columnsWhy.

Areas. 12 areas of 9 to 20 rows. The operations area (discovery, tickets, change, SLA) exists on purpose. It records what a catalogue does not do, and most of its rows honestly rate no.

What the research held (tagging checked in both directions)

  • On disk: no research file for the product under any name (stackiq, softwarecatalog, softwarecatalogus). Only a command reference, a scorecard line and a SOURCES line about the GEMMA import. The richest requirement source is the product's own issues.md: 160 VNG issues with acceptance criteria.
  • Tagged in: competitor_apps links 71 competitors to softwarecatalog. The 153 tagged competitor_features are 12 competitors × 12 to 15 rows, all from one docs pass on 2026-07-23. Of the 250 canonical_features, 144 are uncategorised release-note slugs and about 29 are stackiq's own specs tagged back in.
  • Tagged out: about 1,500 untagged rows belong to the linked competitors. Another 330 GLPI and iTop rows are tagged procest; those come from a GLPI source reading on 2026-09-14, and they are cited here for general features.
  • GEMMA: the gemma_* tables hold 254 components and 422 services and were used for vocabulary only.

Every competitor is graded docs-only, with readOn 2026-07-23 and a system-level unknownReason. Nobody drove any of them.

Rows and split

173 rows: 134 rated, 39 pending.

  • 110 rows were taken from stackiq's own code and specs, 63 from what competitors do.
  • Of the 63 competitor-derived rows, 42 rate us no.
  • Stackiq's column, rated rows: 38 yes, 44 partial, 52 no.
  • Pending rows carry a provisional rating (36 partial, 2 no, 1 yes). Each has a question that would settle it. Most ask whether the installed OpenRegister executes a declared rule, or whether the external frontend reaches an endpoint.

On sibling rows, built.owner names the repo that holds the logic: 20 openregister, 2 decidiq, 2 integriq, 2 opencatalogi and 1 portaliq.

Claimed differentiators nobody has checked

These 18 rows rate stackiq yes while every competitor reads unknown. They are the next pass's work:

  • a suite made of several applications
  • a demo data loader
  • SBOM components per version
  • evidence attached to a compliance claim
  • the compliance matrix
  • a DPIA flag
  • a supplier directory
  • reviews
  • the rating aggregate
  • contract renewal as a decision
  • portfolio licence posture
  • contract documents
  • SBOM import
  • the vulnerability list
  • a moderation queue for self-registered organisations
  • inviting colleagues
  • multi-organisation membership
  • an in-app feature roadmap

Verify

python3 parity_verify.py openspec/parity/capabilities.json (hydra development):

parity-verify: openspec/parity/capabilities.json
  134 rated rows, 39 pending, 6 systems

unknown-cells  (1)
    710 cells rate unknown. That is legitimate, and it is listed so it cannot hide: capabilities[land-application-modules] vng-softwarecatalogus, capabilities[land-application-modules] bluedolphin, capabilities[land-application-modules] glpi, capabilities[land-application-modules] topdesk, capabilities[land-module-versions] vng-softwarecatalogus, capabilities[land-module-versions] sap-leanix, capabilities[land-module-versions] bluedolphin, capabilities[land-module-versions] topdesk

1 finding(s) in 1 check(s)
report-only: pass --strict to fail on these

With --strict it exits 1, and the only finding is the unknown census. The strict-profile counts are all zero: system-without-grade 0, grade-not-in-enum 0, cell-without-evidence 0, own-rating-without-evidence 0, unknown-without-reason 0.

Not run

A JSON file under openspec/ cannot affect stackiq's suites, so I did not run PHPUnit, check:strict, lint or Playwright.

173 rows in 12 areas against five competitor columns, competitor cells
from the intelligence database. Stackiq's own column is not read yet.
134 rated rows and 39 pending across 12 areas. Every own rating carries
a path and a line, a reachedOn and, on sibling rows, the owning repo.
@rubenvdlinde
rubenvdlinde merged commit 38b9938 into development Sep 25, 2026
37 of 38 checks passed
@github-actions

Copy link
Copy Markdown
Contributor

Quality Report — ConductionNL/stackiq @ d257451

Check PHP Vue Security License Tests
lint ✅
phpcs ✅
phpmd ✅
psalm ✅
phpstan ✅
phpmetrics ✅
eslint ✅
stylelint ✅
build ✅
check-manifest ✅
check-vue-demi ✅
test-l10n ✅
format ✅
check-schema-l10n ✅
check-l10n-js ✅
composer ✅ ✅ 130/130
npm ✅ ✅ 807/807
app:check-code ⏭️
info.xml ✅
REUSE ❌
lockfile sync ✅
PHPUnit ⏭️ not run for this diff — no file in this diff matches the code globs, and none carries a source extension — the heavy tier has nothing to decide about it.
Newman ⏭️
Playwright ⏭️ deferred: E2E runs locally and on the promotion path only. This pull request targets development, so the suite is asked once per promotion into beta and main rather than once per push per open pull request. Run it on any branch from the Actions tab, or locally with npx playwright test.
Hydra gates ❌

Quality workflow — 2026-09-25 18:44 UTC

Download the full PDF report from the workflow artifacts.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant