Land no line a reader may refuse, and index any label and any number of parts

[wu-sheng]

A review of every reader against what asz writes found two ways our own data could be refused.

What changes

• No landed line nests deeper than 256 objects and lists. A reader may refuse a deeper line, and the OAP does: its typed reader stops at that line and drops every later record of the file. asz landed data up to Go's own limit of about 10,000 levels.
  • A part whose data nests deeper than its 253 levels of the line is taken out the way a payload too large to carry is, whatever its size. It says too_large, or truncated while its text is left, and keeps its size and SHA-256.
  • A changes/1 or execution/1 record is never taken out whole, since a step joins to it by its tool call. It is written again in its own format, without the field its format does not name, and says truncated.
  • The provider body cutter takes out a piece nested too deep the same way.
  • The writer changes a record only when it then encodes, so a record that cannot be written stays as the caller gave it.
• The index takes a label or identifier of any length, and a record of any number of parts. A string of 64 KiB or more, such as a long title, stopped the session's index from being written, so the session never parsed again. The 65,537th part of a record took the first part's position, so its step replaced the first one's. String lengths and part positions are now 32-bit, and the index schema moves to 16; an older index is built again from the landed files.

The renderer side of the same review is apache/skywalking-horizon-ui#198. This PR does not need it. The pinned renderer moves in its own change once #198 merges.

Tests

• pkg/sessiondata:
  • data at the line limit lands as is; deeper data, and data past Go's limit, is taken out, and no line exceeds 256 levels;
  • a deep field leaves a change or execution record in place, truncated;
  • a record that cannot be written is not changed.
• pkg/providerbody: a tool nested too deep is taken out, and the body still rebuilds from its source.
• internal/index: a 70,000-byte label and a record of 65,538 parts read back.
• internal/adapters/langsmith: a request holding 9,998-deep data lands with the field taken out, where it was set aside whole before.
• internal/view: the skeleton test of the files route now tests what the route serves today.

Each new guard was proven by reverting it. make test, the JSON v1 step (GOEXPERIMENT=nojsonv2), make scenarios, lint and the Windows build pass.