Scrollback has a gap right above the live screen until scrolling to the very top triggers a full-history re-pull (OMP session) #814

Activity

1. Ark0N commented on Nov 14, 2026

   

   Ark0N

   on Nov 14, 2026

   Owner

   More actions

   Thanks for the detailed report.

   What you describe matches the code. The history re-pull (_maybeRefetchFullHistory in src/web/public/app.js) is started only by _maybeLoadMoreHistoryOnScroll in src/web/public/terminal-ui.js, which returns early unless the scroll has reached viewportY === 0. The partial-history notice is revealed by that same gesture, so a hole in the middle of the buffer gives no sign until you scroll past it. That is deliberate today: test/history-notice-reveal.test.ts has a case named "does not appear on the way up, only at the top".

   The likely cause of the hole is the one described in the comment on _maybeRefetchFullHistory: when output arrives faster than tmux flushes it to the attached client, tmux repaints the pane instead of scrolling it, so lines that scrolled off inside tmux never reach the browser's scrollback. tmux still has them, which is why the full pull gives you a continuous history. I have not reproduced it with OMP itself.

   Notes on the suggested direction, from the code:

   • _paneHistoryLines is updated from terminal capture responses only (tab switch, server-triggered refresh, the history pull itself), never from live output. After a burst in a tab you stayed on, it still holds the count from before the burst, and a fresh #{history_size} currently only comes back together with a capture.
   • The two counts are not directly comparable: the browser keeps at most DEFAULT_SCROLLBACK (50,000) rows while tmux keeps tmuxHistoryLimit (100,000 by default), and the capture joins wrapped lines (capture-pane -J) that xterm then re-wraps.
   • For a non-shell session the pull is an unbounded ?full=1 capture, run as a synchronous capture-pane on the server. The comment on _maybeRefetchFullHistory gives that cost as the reason it runs on demand rather than automatically.

   Whether the pull (or the notice) should fire before the top, and on which signal, is a design decision that is still open, since it changes behaviour the current tests pin.

   One detail would help narrow down which path leaves the hole: does it show up in a tab you never left, or only after switching to another tab and back? After the first load of a page, switching back to a non-shell tab replays a 1 MiB tail of the recorded output stream (selectSession in app.js) rather than tmux's rendered history, and that recording carries the same hole the live stream had.
2. HadrienKerlero commented on Nov 14, 2026

   

   HadrienKerlero

   on Nov 14, 2026

   Author

   More actions

   Hello @Ark0N, Thanks for your fast answer !

   Yes it does show up even when I stay in the same tab.
   Here is a short video of the issue : https://imgur.com/a/XbE3FaY
3. Ark0N commented on Nov 14, 2026

   

   Ark0N

   on Nov 14, 2026

   Owner

   More actions

   Thanks for checking, that answers my question.

   Since it happens in a tab you never left, the tab-switch replay is not the cause. After a tab's first load (a full ?full=1 capture), new rows only reach that terminal through the live stream, so that is where the hole comes from.

   I reproduced the mechanism with plain tmux 3.4 outside Codeman. I used seq rather than OMP, so it is not specific to OMP. When a pane prints a burst, tmux sends its attached client a single scroll-up sequence (CSI 24 S on a 24-row pane) and then repaints the final screen. The lines in between are never sent. xterm.js does not put rows scrolled by that sequence into its scrollback, so the rows that were on screen when the burst started are lost too. In one run I printed 40 lines slowly and then 500 at once. The browser's terminal engine (xterm.js 6.0.0, the version Codeman ships) ended up with lines 1 to 17 in scrollback and 518 to 540 on screen, while tmux's own history held all 540. When the lines were printed slowly, every one of them arrived. That matches the gap right above the live screen in your report. OMP sessions use altScreen: 'strip-mux-only', which strips only the alternate-screen switches, so the scroll-up reaches the browser unchanged.

   The missing rows can't be rebuilt from the stream, so they have to come back from tmux. Today that happens in two cases: when a scroll reaches the top of the browser buffer (_maybeLoadMoreHistoryOnScroll in src/web/public/terminal-ui.js), and when the browser itself dropped output, since that recovery re-pulls the full history for a non-shell session.

   Whether the pull or the partial-history notice should fire earlier, and on which signal, is still an open design decision. The points from my previous comment still apply: _paneHistoryLines is stale after a burst in the same tab, the browser and tmux counts are not directly comparable, and the pull is a synchronous full capture-pane on the server. For now, scrolling to the top once, as you found, is what brings the rows back.
4. HadrienKerlero

   added a commit that references this issue on Nov 14, 2026

   fix(web): keep fetch-time output dirty and clamp replay travel per mo…

   ...

   a9a3c6f
5. HadrienKerlero commented on Nov 14, 2026

   

   HadrienKerlero

   on Nov 14, 2026

   Last edited by HadrienKerlero

   Author

   More actions

   @Ark0N The issue is that scrolling up on big big conv isn't always feasible (on mobile for exemple) / practical. Scrollbar dragging doesn't always work.

   I've asked my agents to fix this issue, and they did, but I can't vouch for the quality of the code obviously.
   Here is a PR of the code to keep that information : #815
   Pretty sure it's not mergeable as is, but I'll keep it on my working copy and report if I see any issues with it.
6. HadrienKerlero commented on Nov 14, 2026

   

   HadrienKerlero

   on Nov 14, 2026

   Last edited by HadrienKerlero

   Author

   More actions

   FYI :

   I opened the PR: #815 (fixes #814). It has two commits on top of current master and touches src/web/public/app.js, src/web/public/terminal-ui.js and test/shell-scroll-history-pull.test.ts.

   What the change does: tmux sometimes repaints the screen instead of scrolling, so recent rows never reach the browser's scrollback and leave a gap. Today the repair (a full-history re-pull) only runs once you scroll to the very top. Now the browser marks a session (except shell sessions) whenever new output arrives. The first scroll-up from the bottom after that runs the same re-pull straight away, and keeps your distance from the live screen. Scrolling you do while it reloads is applied afterwards.

   Why this way: the issue suggested comparing the browser's buffer with tmux's history size. That number is only recorded on tab switch, so it is out of date right after a burst, and a fresh value would cost a server round trip on every scroll. Detecting tmux repaints in the output stream wouldn't be reliable, because OMP redraws in place all the time anyway. Shell sessions are left out because their history can be many MB.

   Downsides:

   • It fires more often than needed. Any new output marks the session, so in an active session the first scroll-up after new output costs a full history capture on the server and a full reload in the browser. The 4 s cooldown limits it, but it is much more traffic than before. This is the main thing to review.
   • It can flicker. The reload happens as you start scrolling. If a large capture is slow, it can land after you have already scrolled further and started reading.
   • Some scrolling doesn't trigger it. Only mouse wheel and touch scrolling do. Scrollbar dragging and keyboard scrolling still wait for the old scroll-to-top repair. It also only fires if the scroll starts at the bottom.
   • Some sessions pay for nothing. Where tmux keeps less history than the browser, the pull is refused, but it still costs one wasted capture per minute per session.
   • Shared state. The scroll-during-reload record is one app-wide field; this is safe because only one pull can run at a time.
7. Ark0N commented on Nov 14, 2026

   

   Ark0N

   on Nov 14, 2026

   Owner

   More actions

   Thanks for opening the PR.

   PR #815 is linked to this issue, so the fix is tracked there now. I read it against current master (the 1.51.0 code). It changes only src/web/public/app.js and src/web/public/terminal-ui.js, plus tests. Every live write marks the session, and the first wheel or touch scroll-up from the bottom of a marked non-shell session runs the existing _maybeRefetchFullHistory, which then keeps the position relative to the bottom. Touch scrolling goes through the same _maybeLoadMoreHistoryOnScroll hook, so the phone case you mention would be covered.

   I ran its tests from a copy of the branch. test/shell-scroll-history-pull.test.ts, test/history-notice-reveal.test.ts and eight other scroll, history and flush test files pass (262 tests). CI has not run on the branch yet.

   The code shows two more points, beyond the downsides your PR description already lists:

   • The ?full=1 capture is a synchronous capture-pane on the server (execSync through captureRunner() in src/tmux-manager.ts). Each pull blocks the whole Codeman server, not only that tab, while tmux serializes up to tmuxHistoryLimit lines. Any live write marks the session and an OMP pane writes almost constantly, so most first scroll-ups would run a full capture (at most one per 4 s per session).
   • Grid tiles and the split view's second pane have their own history pull (_pullHistory in src/web/public/terminal-tile.js). The PR does not change it, so they keep the scroll-to-top behaviour.

   Whether the pull should fire automatically on any new output, at that cost, is the design question left open earlier in this thread. It is still open, and the PR is a concrete version of it.

   One thing would help, since you are running the branch on your working copy: a before and after observation on an OMP session. Does the first scroll-up after a burst show the recent rows without a jump, and does the reload visibly flicker? That would cover the manual check the PR says has not been done yet.
8. HadrienKerlero commented on Nov 14, 2026

   

   HadrienKerlero

   on Nov 14, 2026

   Author

   More actions

   Hello @Ark0N,

   Here is a small test video : https://imgur.com/a/kP2sg1d

   So when the generation is over, scrolling up works perfectly.
   When the generation is in progress, scrolling up does show previous conversation, but introduce a gap because the windows won't auto scroll.

   That been said, it's much easier to scroll down a bit, then scroll back up, or wait for a pause in the text generation.

   From my usage yesterday and today, it's made the process much much easier, and I didn't had to ask for missed information again.
9. Ark0N commented on Nov 14, 2026

   

   Ark0N

   on Nov 14, 2026

   Owner

   More actions

   Thanks for the video and for running the branch.

   Both cases you describe match what the code on #815 does.

   Once generation has finished, little or nothing reaches the browser while the pull downloads. The capture then holds everything, and the replay keeps your distance from the bottom, which is why scrolling up works in that case.

   While generation is running, the code has two ways to leave a gap.

   The first is output that arrives while the capture downloads. The early pull fetches the capture first (src/web/public/app.js:9654 on the branch) and only opens the buffer-load queue once it starts writing (_beginBufferLoad inside chunkedTerminalWrite, src/web/public/terminal-ui.js). Rows that reach the browser in between go into the old buffer and are then wiped by _resetTerminalForReplay() (app.js:9751 on the branch), and the capture was taken before they existed. This is point 1 of the review on #815. selectSession avoids it by opening the load before its fetch.

   The second is output that arrives after the pull while you stay scrolled up. While you read history, Codeman keeps the viewport where you are instead of following the live output (preserveViewportY in flushPendingWrites, terminal-ui.js, from #259). That is the window not auto-scrolling, and it is intended. A burst during that time can open a new hole between your position and the live screen, and the branch only re-pulls on a scroll-up that starts within a couple of rows of the bottom (viewportY >= baseY - 2 - |lines|, terminal-ui.js:4086 on the branch). Scrolling down and back up helps for that reason: the second scroll-up starts at the bottom and, once the 4 s cooldown has passed, runs a new pull.

   To tell the two apart: during generation, is the gap already there on the first scroll-up from the bottom, or does it only appear after you have stayed scrolled up while more output arrived?

   The design question from the review on #815 is still open: whether the repair may run before the top of the buffer, and on which signal (any live output, as on the branch, a burst signal such as the full-screen CSI <rows> S tmux sends when it falls behind, or a bounded pull of the recent rows). Your report that the branch made long sessions much easier to follow is useful input for that decision.