Document TSDB backfill behaviour - #7558
Conversation
Elastic Docs AI PR menuCheck the box to run an AI review for this pull request.
Powered by GitHub Agentic Workflows and docs-actions. For more information, reach out to the docs team. |
Elastic Docs Style Checker (Vale)Summary: 1 warning, 4 suggestions found
|
| File | Line | Rule | Message |
|---|---|---|---|
| manage-data/data-store/data-streams/time-bound-tsds.md | 31 | Elastic.DontUse | Don't use 'just'. |
💡 Suggestions (4): Optional style improvements. Apply when helpful.
| File | Line | Rule | Message |
|---|---|---|---|
| manage-data/data-store/data-streams.md | 39 | Elastic.WordChoice | Consider using 'can, might' instead of 'may', unless the term is in the UI. |
| manage-data/data-store/data-streams/load-historical-tsds.md | 20 | Elastic.WordChoice | Consider using 'deactivated, deselected, hidden, turned off, unavailable' instead of 'disabled', unless the term is in the UI. |
| manage-data/data-store/data-streams/time-bound-tsds.md | 93 | Elastic.WordChoice | Consider using 'can, might' instead of 'may', unless the term is in the UI. |
| manage-data/lifecycle/data-stream.md | 46 | Elastic.WordChoice | Consider using 'run, start' instead of 'execute', unless the term is in the UI. |
The Vale linter checks documentation changes against the Elastic Docs style guide. To use Vale locally or report issues, refer to Elastic style guide for Vale.
gmarouli
left a comment
gmarouli
left a comment
There was a problem hiding this comment.
Great start, I will investigate the two follow up actions I noted:
- Set-up should also use DLM instead of ILM.
- Reindexing needs to be reworked.
|
@lcawl we realised that changing the cluster setting is not accessible to serverless users. So for now let's keep this as a stack feature, and we will add serverless when a user will be able to enable it. |
I think we can make a lot of improvements to that page layout and it likely requires more discussion so for now I've deferred that to #8021 |
Co-authored-by: Mary Gouseti <mgouseti@gmail.com> Co-authored-by: Lisa Cawley <lcawley@elastic.co>
leemthompo
left a comment
leemthompo
left a comment
There was a problem hiding this comment.
LGTM with a few mostly minor language and structure suggestions. I didn't test the API calls. :)
| Historical data must fit on the target tier as a whole before you enable data stream lifecycle. | ||
| If you're importing a large data set, split it into batches. | ||
| Each batch should fit within available disk space at indexing time. | ||
| ::: |
There was a problem hiding this comment.
are there any relevant links we could add here?
|
|
||
| :::{image} /manage-data/images/elasticsearch-reference-time-bound-indices.svg | ||
| :alt: time bound indices | ||
| :alt: Time bound indices |
There was a problem hiding this comment.
This is a big visual that takes up a lot of real estate but isn't introduced or explained, which contributes to this section being kinda hard to read.
|
|
||
| - An `index.time_series.start_time` value set to `now - index.look_back_time` | ||
| - An `index.time_series.end_time` value set to `now + index.look_ahead_time` | ||
| ::::{tip} |
There was a problem hiding this comment.
this tip feels too big, consider moving the info to a subsection
| Timestamps outside the eligible write window or in the future are still rejected. | ||
| If a [failure store](/manage-data/data-store/data-streams/failure-store.md) is enabled, rejected timestamp failures can be redirected there. | ||
|
|
||
| :::{admonition} Lifecycle age for past indices |
There was a problem hiding this comment.
the placement of this admonition is a little jarring to my eyes, maybe this flow better as a plain paragraph in the configuration section
|
|
||
| ```console | ||
| GET _data_stream/my-tsds | ||
| ## Past index creation [tsds-past-index-creation] |
There was a problem hiding this comment.
I wonder if this section could be reordered a bit:
what is it → what limits it → how to tune it → edge case → next steps
Co-authored-by: Liam Thompson <leemthompo@gmail.com>
yetanothertw
left a comment
yetanothertw
left a comment
There was a problem hiding this comment.
Looks good, @lcawl!
Left a few suggestions suggestions for linking concepts, but nothing blocking the PR.
| --- | ||
| navigation_title: "Load historical data" | ||
| applies_to: | ||
| stack: ga 9.5 |
There was a problem hiding this comment.
Because the parent page contains two dimensions, I think we should explicitly call out serverless: unavailable here
|
|
||
| ## Protect the cluster during large loads | ||
|
|
||
| Loading months of historical data can trigger significant storage use, force merge activity, and lifecycle processing in parallel. |
There was a problem hiding this comment.
This entire section sounds like a prerequisite or advice that one would need to know before loading historical data.
There was a problem hiding this comment.
Or maybe these are considerations for production environments?
| # Time-bound indices and dimension-based routing [time-bound-indices] | ||
|
|
||
| Unlike regular data streams that only write to the most recent backing index, time series data streams (TSDS) use time-bound backing indices that accept documents based on their timestamp values. This page provides details and best practices to help you work with time-bound indices. | ||
| Unlike regular data streams that write only to the most recent backing index, {{tsds}} ({{tsds-init}}) are backed by a set of time-bound indices, each covering a contiguous, non-overlapping time range. |
There was a problem hiding this comment.
Minor nit:
{{tsds}} resolves to time series data stream, and I think it might need to be plural here ...data streams
| fedramp-mod: "FedRAMP Moderate" | ||
| fedramp-high: "FedRAMP High" | ||
| fedramp-il5: "FedRAMP IL5" | ||
| tsds: "time series data stream" |
There was a problem hiding this comment.
You might want to add a plural, too?
Co-authored-by: Vlada Chirmicci <vlada.chirmicci@elastic.co>
4 participants
Summary
Addresses #7252
Generative AI disclosure
Tool(s) and model(s) used: Cursor Grok 4.5, Composer 2.5