Skip to content

Return structured artifacts and execution manifests instead of loose buffers and locations #14

Description

@Super-Genius

Problem

ProcessingResult currently returns a hash, a pair of output names and byte buffers, and a list of output-location strings. SuperGenius then joins output locations into one newline-delimited protobuf string.

That loses type, shape, format, pass provenance, runtime identity, and per-artifact verification data needed by multi-pass jobs, GCS, rendering, caching, and EIS.

Scope

Replace or extend the loose result shape with typed records for each output artifact and one execution manifest.

Each artifact should include, where applicable:

  • Declared resource name and stable artifact ID.
  • Producing pass and output binding.
  • Data type, format, shape/dimensions, byte size, and media type.
  • Content hash and chunk hashes.
  • Storage URI/CID and local-cache identity.
  • Partial/final status and sequence number for streamed outputs.
  • Privacy/tenant scope copied from validated job metadata.

The execution manifest should include:

  • Execution, attempt, task, subtask, and pass IDs.
  • Selected executor and runtime compatibility identity.
  • Model, tokenizer, adapter, shader, and quantization identities when used.
  • Input and output artifact hashes.
  • Start/end times, terminal state, error details, and resource-use summary.
  • Verification/EIS metadata extension points and worker signature input.

Coordinate required protobuf changes with SuperGenius rather than encoding several values into one string.

Acceptance criteria

  • Multi-output jobs return independently typed and hashed artifact records.
  • Final and intermediate artifacts retain their producing pass and binding identity.
  • No newline parsing is required to discover output locations.
  • The manifest can be serialized deterministically for hashing, signing, caching, and verification.
  • Existing callers have a migration adapter until the SuperGenius protobuf integration is updated.

Supports issues #9, #10, #12, and #13 and GCS/EIS integration.

Base implementation and review work on the branch consumed by SuperGenius/develop.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions