perf: remove two redundant object-store calls in datasets metadata fetch

[prabhaks]

Summary

Two small, low-risk fixes in the object-store I/O behind POST /api/prism/v1/datasets, which has been observed taking 50s+ to respond on multi-TB, multi-day deployments even on a 44 vCPU / 192GB query node.

• get_manifest_list (src/query/mod.rs) unconditionally fetched the base stream.json via get_stream_json, but on Mode::Query/Mode::Prism nodes that result is never read (those modes merge every ingestor's stream.json instead via get_all_stream_jsons). Moved the fetch into the else branch that actually needs it (Mode::All).
• get_first_and_latest_event_from_storage (src/storage/object_storage.rs) awaited its two independent extract_timestamp_for_date calls sequentially. Replaced with tokio::try_join! to run them concurrently, matching the existing tokio::join! pattern already used one call up the stack in get_prism_logstream_info.

Both changes are behavior-preserving on the success path (same I/O, fewer/faster round trips). Two notes worth flagging for review:

• The get_manifest_list change means a failed/corrupt base stream.json on a Query-mode node no longer short-circuits the whole call — it now proceeds to get_all_stream_jsons instead, which can succeed where the old code would fail. This seems like a desirable side effect but is a real change in failure-path behavior, not just a speedup.
• These are part of a larger investigation into /datasets latency (bundles stats/retention/hottier/info/counts/query per stream); the dominant cost — a manifest-list scan that's unbounded by stream history when no retention policy is configured — needs a larger catalog-level change and is intentionally not part of this PR. Happy to open a separate issue for that once there's an associated tracking issue for this one (the Parseable team mentioned they'd be opening one).

Test plan

• cargo test --lib — 426 passed, 0 failed (full existing suite, no regressions)
• cargo fmt --check — clean
• cargo clippy --lib — clean
• Validate against a real multi-TB test cluster (in progress, pending network access)

[coderabbitai]

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 75905ac5-db54-4190-a549-834519da7ebd

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

• 🔍 Trigger review

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}