Repository navigation
Conversation
Introduce orb-pcp with explicit 2.7, 2.8 and 3.0 manifest formats, fixed-size SHA-256 inputs, and duplicate/reserved-name validation. Serialize compact JSON with stable sorted keys. Validation: seven manifest tests, one doctest, formatting and scoped Clippy pass. Synthetic tests cover wire bytes, ordering, escaping and invalid entries. AI assistance: Codex assisted with implementation, tests and review.
Preserve ordered GNU tar entries and explicit gzip metadata. Reject duplicate or invalid filenames and out-of-range gzip timestamps. Keep archive construction in memory with caller-owned buffers. Validation: all 15 integration tests and two doctests, scoped Clippy and formatting pass. Synthetic archive tests cover headers, framing, ordering, filenames, gzip round trips and timestamp boundaries. AI assistance: Codex assisted with implementation, tests and review.
Assemble prepared payloads with fixed filenames, ordering, and v2/v3 routing. Separate auxiliary-tier construction from tier 0 so encrypted tier hashes can be signed first. Preserve optional fraud and empty DI payloads. This low-level component does not validate opaque payloads, encryption, signatures, or full biometric redaction. Production consumers are unchanged. Validation: 19 integration tests and 2 doctests, consumer feature variants, Clippy, formatting, Linux ARM64 cargo check, and byte-identical frozen v2.8 outer tier reconstruction. AI assistance: Codex assisted with implementation, tests and review.
Add borrowed inputs and compact sorted encoders for face embeddings, iris codes, individual iris-code shares and backend-key metadata. Preserve nulls, array order, opaque strings and standard padded Base64 without adding dependencies. Validated 24 integration tests and 2 doctests on Linux ARM64 with default and consumer feature variants; Clippy, formatting, ARM64 compile check and standalone byte comparisons against six reference JSON payloads passed. Native payload test launch is blocked by an AMFI signature error. AI assistance: Codex assisted with implementation, tests and review.
Accept portable borrowed eye records and emit one embedding protobuf plus exactly three same-index share protobufs. Preserve missing-eye empty output, metadata-check order, integer widening, and floating-point bits. Errors identify fields without exposing their values. Reuse orb-pcp-defs and document its protoc build prerequisite. Six new tests ran red then green. Linux ARM64 passed all 30 crate integration tests, 2 doctests, 3 schema tests and alternate consumer features; Clippy and formatting passed. Standalone consumer reproduced four frozen protobufs exactly. AI assistance: Codex assisted with implementation, tests and review.
Retain sodiumoxide 0.2.7 for core compatibility. Reject low-order recipients using its safe X25519 API before sealing, preserve original recipient bytes, and check output-size bounds. Use platform libsodium through pkg-config. Replace the alkali-only dependency smoke test as approved; the native bindings cannot coexist. No production caller changes, new cipher construction, private fixtures, or plaintext fallback. Validated 35 integration tests, one unit test and two doctests on Linux ARM64, alternate consumer features, Clippy and a linked standalone consumer. Broader deps-tests is blocked by missing ALSA; host build by missing libsodium.pc; cached cargo-deny cannot parse existing license configuration. AI assistance: Codex assisted with implementation, tests and review.
Hash the exact encoded manifest once with SHA-256, invoke the caller once with the raw 32-byte digest, and retain signature bytes unchanged. Encoding errors skip signing; signer errors propagate without retries. Reuse workspace ring without package upgrades. Four tests ran red then green. Linux ARM64 passed 39 integration tests, one unit test, two doctests and three schema tests; alternate consumer features, Clippy, formatting and standalone consumer checks passed. No production caller changes or private artifacts. AI assistance: Codex assisted with implementation, tests and review.
Return sorted info.json and its nine or ten salted identity hashes together. Inject a secure RNG with fallible 16-byte draws; preserve legacy timestamps, salt encoding, certificate Base64 and optional device binding. Redacted metadata has no image IDs. Partial-ready included groups currently return field-only errors when required IDs are absent, without adopting draft wire semantics. Six tests ran red then green. Linux ARM64 passed 45 integration tests, one unit test, two doctests and three schema tests, default and alternate-feature Clippy, and formatting. A private standalone consumer reproduced frozen v2.8 metadata bytes and all ten salted hashes exactly; no baseline changes or private artifacts committed. AI assistance: Codex assisted with implementation, tests and review.
Generate commitments through the exact pinned Hyrax implementation, preserve small-input behavior, construct named inner archives and their hashes, and compose metadata, payloads, signing, compression and recipient encryption into all three PCP versions. One redaction choice controls files, hashes and image IDs. Checked import and partial-data wire changes remain deferred. Fourteen new component and failure-path tests ran red then green. Linux ARM64 default and alternate-feature tests, Clippy and formatting passed. Add the existing P-256 development dependency for the accompanying synthetic prototype signer; no production signature algorithm is imposed. AI assistance: Codex assisted with implementation, tests and review.
Exercise full and redacted packages in 2.7, 2.8 and 3.0 with synthetic images, fresh recipient keys and an example-only P-256 prehash signer. Open all tiers and inner archives, verify signatures, exact manifest coverage, metadata redaction, backend key envelopes, tier routing and ciphertext checksums. Persist no keys or payloads. Linux ARM64 debug and optimized release builds both ran successfully, including an independent consumer outside the workspace. Full suite passes 60 integration tests, one unit test, two doctests and three schema tests; default/alternate-feature Clippy and formatting pass. Document build prerequisites and remaining production-integration limitations. AI assistance: Codex assisted with implementation, tests and review.
Allow absent normalization on additional captured frames while preserving primary requirements, image IDs, and hashes. Cover the regression and all-version encrypted round trips. AI-assisted implementation and tests.
Group archive/layout, payload encoders, and cryptographic helpers. Export only construction inputs and errors with distinct domain names; keep prepared intermediates private. Group existing checks as unit tests and retain public builder round trips. AI-assisted implementation, tests, and review.
Expose iris and di as separate named Included inputs without a BiometricData wrapper. Store modality metadata once and keep values with their recipient shares. Document source DI agreement as a consumer-adapter responsibility while preserving missing-data wire behavior. Retain exact payload fixtures and exercise populated iris/DI data in encrypted round trips for all supported versions. AI-assisted implementation, tests, and review.
Keep normal builds encrypted while exposing an opt-in diagnostic result type for local comparison. Preserve hashing, signing and versioned layout, with all-version plaintext tests and an encryption regression test. Validation: 65 focused tests pass with diagnostics enabled; default tests, native dependency Clippy and formatting also passed. Only synthetic test data is included. AI-assisted implementation, tests and review.
Keep exact-version validation, labels and envelope selection in private PcpVersion methods. Reuse the V2/V3 envelope for inner protection and manifest tier entries without changing public inputs, wire formats or preparation order. Add characterization coverage for valid device-key combinations and encryption-error precedence. Validation: 64 default and 67 diagnostic public tests, focused Clippy, formatting, and 33 private core PCP tests passed; one real-engine export test was intentionally ignored. AI-assisted implementation, tests and review.
Demonstrate explicit version and metadata inputs and verify v3 with and without a device key. Keep the example verifier independent of builder internals, covering eight included/redacted round trips. Reproduced the missing-device assertion before fixing the example. Default and diagnostic-feature prototype tests, both runnable examples, example Clippy and formatting passed. Only synthetic data is used. AI-assisted implementation, tests and review.
Rename the public input to DaugmanData and DaugmanEyeData and the included field to daugman. Encode codes and recipient shares together, and keep both Daugman and DI grouped through preparation and borrowed archive inputs. Preserve wire fields, filenames, ordering and null/empty semantics. Existing byte-level expectations now exercise the grouped encoder. Default and all-feature tests, all-target Clippy and formatting passed before commit. Examples and public API documentation updated; only synthetic fixtures are included. AI-assisted implementation, tests and review.
Restore alkali, propagate native sealing failures, centralize manifest duplicate checks, strengthen regression assertions, and wipe owned archive intermediates. Enforce reviewed Git sources in CI and document remaining Hyrax maintenance and memory-erasure limitations. AI-assisted implementation and validation with Codex. Private integration dependency migration remains separate.
Pin the explicit sealed-box module in the dependency smoke test, preserve a synthetic sodiumoxide ciphertext fixture, exercise payloads through 1 MiB, and assert exact authentication failures. Validated the full PCP suite and release example in AArch64 Linux, plus all eight synthetic package variants with an independent sodiumoxide decryptor. AI-assisted implementation and validation with Codex.
Remove the new source-check CI step and Git allowlist from the extraction PR, along with the README claim. AI-assisted with Codex.
Keep build commands, the runnable example, input requirements, and concise diagnostics safety notes. AI-assisted with Codex.
Preserve orb-hil removal and required PCP/Hyrax dependencies. Validate locked PCP tests with and without diagnostics, release example, Clippy, formatting, dependency policy, and independent sodiumoxide interoperability.
…n.pb - Take `info.json` as `v1::Info` and write it as given. The builder owns only the `*_salt` fields, the multiframe image ID lists and, under redaction, the image IDs, so fields added to pcp-defs need no builder change and absent fields are omitted. - Take face embeddings, iris codes and shares, and DI embeddings and shares as their `v1` messages, serialized with sorted keys like orb-core. Unset iris version and codes are now omitted instead of written as `null`. - Write an optional `migration.pb` after `info.json` and hash it in `hashes.json`; the PCP version is unchanged. - Re-export `orb_pcp_defs::v1`. Primary frames take their image ID from `info`, and their normalized data is required by type. - Clarify which hashes remain in redacted packages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`pcp.v1` describes 2.x packages and evolves additively, so the builder no longer selects a version: every manifest says 2.8 and `device_public_key` is an optional `info` field like any other. - Remove `PcpVersion`, `BuildRequest::version`, the device-key/version check and the 3.0 layout with its `tier_1` to `tier_5` manifest entries. - Keep the three-tier `Package`; tiers 1 and 2 are the empty archives the 2.x layout always produced, so 2.8 output is unchanged. - Reserve only `version` in the manifest. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Drop history-based wording ("legacy", "prototype", "current wire profile"),
defensive negations and version or device-key mentions that nothing depends
on. Keep the comments that explain non-obvious constraints and the
diagnostics safety warnings.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Take multiframe image IDs as `orb_wld_data_id::ImageId`, re-exported as `orb_pcp::ImageId`. They always render as 32 lowercase hex characters, so the file names derived from them are valid and cannot collide with fixed names; duplicate IDs are still rejected by the manifest. - Make `PackageImages::left` and `right` required and drop `MissingLeftEye` and `MissingRightEye`. - Document the primary frame, multiframe captures, their optional per-frame normalization and the aggregate image ID lists. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Need to agree on naming convention and address the EncodedDi extension.
Additionally, we should probably go through and wherever we can (meaning it is not exposed externally) we should also rename current shares into something meaningful. E.g.
pub(crate) struct EncodedDi {
pub embeddings: Option<Vec<u8>>,
pub shares: [Option<Vec<u8>>; 3],
pub orbit5_shares: Option<[Vec<u8>; 5]>,
}
should be something like
pub(crate) struct EncodedDi {
pub embeddings: Option<Vec<u8>>,
pub <smart_name>_shares: [Option<Vec<u8>>; 3],
pub orbit5_shares: Option<[Vec<u8>; 5]>,
}
We have to be careful tho and make suer we are not breaking publicly exposed APIs. if we do - not a big deal, of course, but one would have to go and fix them downstream in TEE and priv-orb-core, probably...
|
Please keep the DI embedding and both share families together in
For the existing internal |
ShevchenkoRostyslav
left a comment
There was a problem hiding this comment.
Use the DI-qualified naming consistently through schema, archive filenames, builder inputs, and tests. The suggestions are coordinated edits and should be applied together. AI-assisted review.
ShevchenkoRostyslav
left a comment
There was a problem hiding this comment.
Follow-up naming pass: these suggestions propagate Orbit5DiIrisEmbeddingShares / Orbit5DiIrisEmbeddingShareV1, orbit5_di_embedding_shares, and orbit5_di_iris_embeddings_shares_* through the schema, manifest, encoder, archive, decoder example, builder example, and tests. They also wire the ORBIT5 bytes through EncodedDi.
Apply these together with the existing message-name, build-script, public-input, and README suggestions. Rename the protobuf source file to orbit5_di_iris_embedding_shares.proto. Declarations outside the commentable diff still need the new EncodedDi field, the third encode_di argument, None in the existing encoder test calls, and the new optional field in both archive-test EncodedDi literals. Run formatting and the targeted tests afterward.
Keep protobuf tags unchanged. The filename and JSON-key rename must be consistent because the signed manifest binds those names. The internal grouping change does not add a downstream API change; the explicit public-input and protobuf renames require callers to use the agreed names when adopting this PR.
AI-assisted review.
## Motivation Extract PCP construction into a reusable public `orb-pcp` crate so Orb firmware and [TEE migration](https://github.com/worldcoin/di-migration-tee) can share format rules instead of maintaining separate builders. This PR does not switch production consumers. ## What changes - Builds PCP 2.8: payload encoding, legacy Hyrax commitments, tier layout, salted metadata, manifests, signing, compression and encryption. Callers supply encoded images, the shared `pcp-defs` messages (re-exported as `orb_pcp::v1`) for `info.json` and the biometric payloads, authorized recipient keys and a signer for the exact SHA-256 digest. Messages are written as given, so fields added to `pcp-defs` need no builder change; the builder owns only salts, salted hashes and multiframe image ID lists. JSON is serialized with sorted keys, matching orb-core. - Every package is stamped 2.8: `pcp.v1` evolves additively, so the builder does not select a version, and `device_public_key` is an optional `info` field. Tiers 1 and 2 are empty archives outside the manifest, as in the existing 2.x layout. PCP 3.0 is out of scope; it will get its own proto version. - Writes the optional `migration.pb` (#1442) for TEE migrations and hashes it in `hashes.json`. - Supports redaction and separately feature-gated plaintext diagnostics; the normal builder always encrypts. - Centralizes duplicate-name validation and wipes owned archive intermediates. Memory cleanup remains best-effort; the pinned [Hyrax](https://github.com/worldcoin/hyrax-pcs-commit) implementation and its transitive dependency debt are unchanged. ## Validation - 65 `orb-pcp` tests (unit, integration, and the synthetic round trip that decrypts all four device-key/redaction combinations), with and without features, plus `orb-pcp-defs` tests. - A schema test decodes every tier 0 JSON file into the `pcp-defs` types without loss; only per-frame manifest keys fall outside `Hashes`. - Clippy (`-D warnings`), formatting, docs and `cargo deny` pass locally (macOS, Rust 1.98.1); the full workspace runs in CI. AI-assisted implementation and validation with Codex and Claude Code. 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Rostyslav Shevchenko <rostyslav.shevchenko@toolsforhumanity.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: Elena Gryaznova <elena.gryaznova@toolsforhumanity.com> Co-authored-by: vmenge <victor.ferreira.menge@toolsforhumanity.com>
### Notes - pin HPKE and QR-link to [merged relay verification](worldcoin/orb-relay-messages#140) so their message types match [Oxide pairing](worldcoin/oxide#1263) - brings in P-256 0.14, matching the [Oxide dependency upgrade](worldcoin/oxide#1295), without changing HPKE code - ran formatting, scoped Clippy, 20 HPKE and QR-link tests, and HPKE license/advisory checks - AI-assisted implementation and validation with Codex
removes alkali as a runtime dependency of `orb-pcp` so orb-core can use its existing sodiumoxide implementation without a libsodium dependency conflict. callers now provide `.sealer(...)` to `Builder`. sealing errors keep their concrete type and propagate through `BuildError::Encryption`. the PCP format stays unchanged. alkali remains available for tests and the example.
ORBIT5 DI bundles use owned inputs and zeroize their share payloads on drop. Keep both share families in the DI payload with signed manifest coverage and biometric redaction. AI-assisted-by: Codex
Adds optional manifest hash fields (tags 61–65) for `di_iris_embeddings_5_party_shares_0.pb` through `_4.pb`, a five-party family of DI embedding shares. Each file holds one recipient's `DiIrisEmbeddingShares`, the message the three-party family already uses; the filename identifies the family, and index i holds Shamir ID i + 1. `Di` accepts an optional owned array of five recipient bundles through `Di::embeddings(embeddings).di_embedding_5_party_shares(bundles).shares(shares)`. When supplied, all five files are written with signed hashes alongside the existing three-party DI files. Omitting the input or the DI group, or redacting biometrics, omits the family. `Di` checks both families the same way: each share needs its record and the embedding's model and embedding versions. Share payloads and encoded buffers are zeroized on drop. PCP version remains 2.8. Callers own share generation, dimension and value validation, and enabling five-party shares only for compatible consumers. Replaces #1466 by @evagelia, rebuilt on current `main`. The earlier review discussion is in #1466. ### Validation - `cargo test -p orb-pcp -p orb-pcp-defs --all-features --all-targets --locked`: 89 passed; the two existing fixture tests remain ignored. - Clippy with `-D warnings`, rustfmt, rustdoc with `-D warnings`, the `build_pcp` example, `buf lint`, `buf format` and `buf breaking` against `main` pass. - Not run locally: Linux/Android builds and end-to-end consumer acceptance. AI-assisted-by: Codex, Claude Code 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Co-authored-by: Evangelia <evagerg@gmail.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Adds the
Orbit5DiIrisEmbeddingSharesprotobuf payload and optional manifest hash fields fororbit5_di_iris_embeddings_shares_0.pbthrough_4.pb.The shared PCP builder accepts an optional owned array of five recipient bundles through
Di::embeddings(embeddings).orbit5_di_embedding_shares(bundles).shares(shares). Each bundle carries left, left mirrored, right, and right mirrored share vectors. Both share families belong to the DI payload; all five ORBIT5 files are included with signed hashes alongside the existing three-party DI files. Biometric redaction omits the files and their hashes. ORBIT5 share payloads and encoded buffers are zeroized on drop.PCP version remains 2.8. Callers own share generation, dimension/value/metadata validation, and enabling ORBIT5 only for compatible consumers.
Validation: 88 tests passed across
orb-pcpandorb-pcp-defswith all features and targets, including synthetic encrypted round trips, redaction, schema/manifest coverage, and zeroization. Two existing fixture tests remain ignored. Focused Clippy with warnings denied, workspace Rust formatting, documentation generation, protobuf lint/format, and protobuf breaking-change checks passed. Linux/Android builds and end-to-end consumer acceptance were not run locally.AI-assisted-by: Codex