fix(ingest): correct --write-mode to the real load-mode vocabulary - #279
fix(ingest): correct --write-mode to the real load-mode vocabulary#279eddietejeda wants to merge 1 commit into
Conversation
| /// `hotdata ingest sources fields <family>` — the authoritative source. The | ||
| /// flag forwards any value and lets the service validate, rather than | ||
| /// hardcoding a list that drifts from what families accept (that drift is | ||
| /// exactly what showed a family advertising `append` while this flag | ||
| /// refused it: see issue #278). |
There was a problem hiding this comment.
super nit: this doc comment is clap long-help, so hotdata ingest create --help will print "the flag forwards any value and lets the service validate, rather than hardcoding a list that drifts... see issue #278" to end users. The other issue-number references in this repo (indexes.rs:91, sdk.rs:1717, query.rs:314) are all in internal function docs and // comments, not in user-facing help strings. Consider keeping the user-relevant half and dropping the implementation rationale + tracker ref (or moving them to a // comment above the field, which is where the same argument in --record-shape effectively lives). (not blocking)
| /// hardcoding a list that drifts from what families accept (that drift is | ||
| /// exactly what showed a family advertising `append` while this flag | ||
| /// refused it: see issue #278). | ||
| #[arg(long = "write-mode")] |
There was a problem hiding this comment.
nit: nothing pins the new behavior. The existing write_mode tests (destination_flags_default_schema_and_write_mode, ~line 2396) call build_destination directly and never go through clap, so re-adding a value_parser here would not fail any test — the exact regression #278 describes. src/commands/results.rs:165-173 already has the pattern for a clap-level parse test (Wrapper + try_parse_from); a one-liner asserting ingest create ... --write-mode append parses would guard it. (not blocking)
Codecov Report✅ All modified and coverable lines are covered by tests. 📢 Thoughts on this report? Let us know! |
1cd5b53 to
9b05476
Compare
There was a problem hiding this comment.
Correctness of the change looks fine: the enum now covers the modes the server-reported write_modes can advertise (src/client/ingest.rs:1136,1197,1234 show replace/append), build_destination still defaults to replace (src/commands/ingest.rs:938), and the --destination JSON path is unaffected. SKILL.md text matches.
The two prior non-blocking threads (issue ref in the clap long-help, no clap-level parse test pinning --write-mode append) are still open; both remain non-blocking.
The --write-mode flag hardcoded value_parser = ["replace", "upsert"], which contradicted what the CLI itself shows and what the service accepts: - `ingest sources fields <family>` renders the server-reported per-family write_modes (e.g. filesystem: replace, append), so the CLI advertised `append` while the create flag rejected it with 'invalid value append'. - The real service-wide load-mode vocabulary is replace/append (keyless) plus the key-based upsert/update/delete (confirmed against the OpenAPI spec and the dlt worker, which maps upsert -> dlt merge/upsert). The old pair both omitted valid modes (append, update, delete) and offered `upsert` as if universal, though it is key-gated. Correct the enum to the full set so clap still catches typos up front while accepting every mode a family may advertise; per-family/per-table support is still enforced by the service. Fix the SKILL.md note likewise (it wrongly said upsert needs a continuous bucket ingest; it needs a declared key). Closes #278
9b05476 to
658d863
Compare
There was a problem hiding this comment.
Reviewed the full diff (the branch was force-pushed, so no since-last-review diff was available). The corrected value_parser list matches the server-reported write_modes vocabulary used elsewhere in the CLI (src/client/ingest.rs:1136,1197,1222,1234), the replace default in build_destination (src/commands/ingest.rs:938) is unchanged, and the SKILL.md text is consistent with the new flag help. No blocking issues.
The two nits from the prior cycle (issue-number reference inside the user-facing clap long-help, and no clap-level test pinning the accepted values) are still open but remain non-blocking.
Live parse verification
No conflicts with main (GitHub reports CLEAN); CI green. LGTM. |
Closes #278.
Problem
ingest create --write-modehardcodedvalue_parser = ["replace", "upsert"], which contradicted the rest of the CLI and the service:ingest sources fields <family>renders the server-reported per-familywrite_modes(e.g. the filesystem family:replace, append). So one CLI surface advertisedappendwhileingest create --write-mode appendfailed withinvalid value 'append' [possible values: replace, upsert].replace/append(keyless) plus the key-basedupsert/update/delete(the worker mapsupsert→ dlt{disposition: merge, strategy: upsert}). The old pair both omitted valid modes (append,update,delete) and offeredupsertas if universal, though it is key-gated.Fix
[replace, append, upsert, update, delete]. clap still catches typos up front, and every mode a family may advertise is now accepted; per-family/per-table support (e.g. a key-based mode on a keyless table) is enforced by the service.ingest sources fieldsas the authoritative per-family source.skills/hotdata/SKILL.md: thewrite_modenote listed onlyreplace | upsertand wrongly saidupsertneeds a continuous bucket ingest — it needs a declared key on the destination table.Before / after
Testing
cargo buildclean;cargo test --bin hotdata ingest→ 82 passed, 0 failed.appendnow parses and reaches the service; typos are still caught locally against the full enum.cc @anoop-narang