Skip to content

Feature/sparse chunk cache - #211

Merged
korzh merged 19 commits into
masterfrom
feature/sparse-chunk-cache
Aug 4, 2026
Merged

korzh merged 19 commits into
masterfrom
feature/sparse-chunk-cache

Conversation

@korzh

@korzh korzh commented Aug 4, 2026 •

Copy link
Copy Markdown
Collaborator

This pull request introduces a new sparse row caching mechanism to the EasyDataTable class in the EasyData.JS core package. The main goal is to improve memory management and performance when handling large datasets by caching only needed windows of rows and evicting older ones as necessary. The changes also add configurability for the cache size and refactor row loading logic to support these improvements.

Key changes include:

Sparse Row Cache Integration

  • Replaced the old contiguous array row cache with a new SparseRowCache class in EasyDataTable, allowing for non-contiguous, windowed caching of rows. This enables efficient memory use and supports loading and evicting windows of rows as needed. (easydata.js/packs/core/src/data/easy_data_table.ts, [1] [2] [3]
  • Added logic to evict older cached windows when the maximum number is exceeded, controlled by the new maxCachedWindows option. (easydata.js/packs/core/src/data/easy_data_table.ts, [1] [2]

Row Loading and Window Management

  • Refactored the row loading logic to request and cache windows of rows aligned to the chunk size, ensuring requests are reproducible and cacheable. Introduced planLoadWindows and loadWindow methods to manage this process. (easydata.js/packs/core/src/data/easy_data_table.ts, easydata.js/packs/core/src/data/easy_data_table.tsL146-R290)
  • Added tracking of pending chunk requests and cache generations to prevent race conditions and ensure data consistency when cache is cleared or updated. (easydata.js/packs/core/src/data/easy_data_table.ts, [1] [2]

API and Type Improvements

Miscellaneous

These changes collectively make row access more efficient and scalable, especially for large datasets, and provide more control over memory usage.

korzh and others added 19 commits May 18, 2026 06:31
…onditional cursor)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…or, simplify guard)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
- suppress the click synthesized after a resize drag so finishing a resize
  (even past the min-width point) never triggers a column sort
- in resizeColumns, set column.width to the natural header width when there are
  no data cells, so a resize on an empty grid does not snap the column width

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Save bodyViewport.scrollLeft before clearDOM wipes the DOM and restore it
after the body is rendered, so sorting/paging no longer jumps the grid to 0.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Column resizing and sorting in EasyGrid
Two width-stability fixes for EasyGrid:

- Always render the sort-indicator box (fixed 12px, empty when unsorted)
  on sortable headers, so the arrow appearing/disappearing no longer
  changes the header's natural width used by the auto-fit lower bound.
- Make AutoResizeColumns.Once survive refetches: the grid keeps a map of
  auto-computed widths per column id and carries them over to the fresh
  DataColumn instances each server response brings, so columns are
  measured once on first render instead of being re-fit to every new
  page of data. Always mode is unchanged.

Covered by 4 new latte tests in easy_grid_interactions.test.ts.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The header label is a flex item, so without min-width: 0 it refuses to
shrink below its longest word and the text bleeds into the neighboring
header cell when a column is resized down to its minimum. Let it shrink
and clip the overflow with an ellipsis (data cells already do the same).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The mobile UI (full-page menus instead of drop-downs, no drag and drop,
no hover-driven interactions) was chosen by the window size alone, so any
window narrower than 840px got it - including a narrow desktop window, a
browser side pane or a half-screen window, where it only gets in the way.

The screen size is still required, but now the device must also have no
hover and a coarse primary pointer. Browsers which don't understand those
media features report the query as "not all" and fall back to the size
check, and setIsMobileMode() keeps overriding everything.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…inned widths), the allowColumnResize option and the column resize event.

[new]: EasyGrid: clickable sortable header with a sort indicator, the sortable option and the columnSort event, plus per-column sort direction / width state and helpers.
[upd]: The mobile mode is turned on for touch devices only - besides the screen size the device must have no hover and a coarse pointer, so a narrow desktop window or a browser side pane keeps the normal UI (the size check remains as a fallback, setIsMobileMode() still overrides everything).
[fix]: The horizontal scroll position of the grid is preserved across re-renders (sorting, paging).
[fix]: The header label is clipped with an ellipsis at narrow column widths instead of bleeding into the neighboring header cell.
[fix]: An exception was thrown on the conversion to text for numeric fields in FullTextSearchQuery (EF Core 5+ uses the instance ToString(), on EF Core 2/3 such properties are skipped).
[fix]: Numeric fields are included into the full text filter only when the search text is a number.
The rows used to be cached as one contiguous run starting at the very
first one, so reading a distant page pulled in everything before it -
jumping to page 200 meant loading pages 1 to 200.

The rows are now held as a set of disjoint segments (SparseRowCache), and
a read only asks the loader for the parts that are really missing. The
requested windows are aligned to chunkSize, so reading any row of the same
chunk asks for the same window and it is loaded once. The windows that are
not adjacent to the first row are dropped on an LRU basis once there are
more than maxCachedWindows (8 by default).

getCachedRows() and getCachedCount() keep their meaning - the rows loaded
in one uninterrupted run from the very first one. That run has to stay
gap-free and in order, as the aggregate calculators, the pivot table and
the charts walk it from the beginning. The rows of a detached window are
deliberately not exposed there.

Also:
- the chunk requests for the same window are shared instead of being sent
  twice, and a chunk requested before the rows were dropped is no longer
  merged into the table it no longer belongs to;
- "the total is known" is told apart from "the total has been asked for".
  They were one flag, so a read made while the first one was still on its
  way saw the total as 0 and returned no rows at all;
- ChunkInfo is exported as an alias of DataChunkDescriptor - the tests
  already referred to the chunk parameters under that name.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sonarqubecloud

sonarqubecloud Bot commented Aug 4, 2026

Copy link
Copy Markdown

Quality Gate Failed Quality Gate failed

Failed conditions
C Reliability Rating on New Code (required ≥ A)

See analysis details on SonarQube Cloud

Catch issues before they fail your Quality Gate with our IDE extension SonarQube for IDE

@korzh
korzh merged commit 91389e9 into master Aug 4, 2026
6 of 7 checks passed
@korzh
korzh deleted the feature/sparse-chunk-cache branch August 4, 2026 15:28
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants