Skip to content

Touchscreen on boxes that have one: menu taps, petting, TOUCH status - #149

Merged
longzhi merged 5 commits into
mainfrom
claude/touchscreen
Oct 21, 2026
Merged

longzhi merged 5 commits into
mainfrom
claude/touchscreen

Conversation

@longzhi

@longzhi longzhi commented Oct 14, 2026 •

Copy link
Copy Markdown
Collaborator

Part 1 of #148 (spec). Decision record: docs/adr/0019-touch-is-optional-and-everything-stays-on-the-keys.md; new terms Touchscreen, Tap and Pet in CONTEXT.md.

What changes

  • Probe. At boot the device layer reads the CHSC5432's ID register at 0x2E. The box prints TOUCH CHSC5432 or TOUCH NONE at boot and in answer to device.hello, and the STATUS view gets a TOUCH row. A box without a touchscreen behaves exactly as before.
  • Taps (firmware-rs/core/src/touch.rs). The 28-byte report at 0x2000002C is parsed as ALIENTEK's driver does, read every 20 ms. A tap is down and up within 400 ms, staying within 12 px. Every tap prints TAP <x> <y>.
  • What a tap does. A menu row is selected and acted on (K1 to it, then K0). A key hint at the bottom of the menu is that key. The buddy on duty is petted: it hops with a heart for 1.2 s and nothing else changes. In Leisure a tap only calls the buddy back, like a key. The Pomodoro screen has nothing to tap.
  • Hit testing uses the rectangles the display draws with (card_rect, panel_row_rect, hint_rects, buddy_rect), so the targets follow the layout. The tapped target is outlined for 150 ms.
  • Docs: device-menu.md (Touch section, images), protocol.md (TOUCH, TAP), hardware.md, and both READMEs.

A tapped row The buddy petted

Differences from the spec

  • Poll interval is 20 ms, not 50 ms. At 50 ms a quick tap could fall between two reads.
  • No petting in Pomodoro. The Pomodoro screen doesn't draw the buddy, so there is nothing there to pet.

Mac and Linux

No App changes on either side. Neither the daemon nor the Apps use the TOUCH line; it is ignored like any other diagnostic line. Card taps, which need a daemon change shared by both platforms, come in part 2.

Testing

  • cargo test -p vibebuddy-firmware-core: report parsing, the tap recognizer and Menu::tap_row unit tests. The integration tests use a FakeBoard with a finger and cover: TOUCH at boot and on hello, a box without a touchscreen, row taps, hint taps, petting, a slide, and Leisure.
  • tools/build-firmware.sh builds.
  • On box 30:ED:A0:A4:0D:08 (2026-10-15, with the App quit, reading the serial port):
    • The box prints TOUCH CHSC5432.
    • The coordinates match the screen with no adjustment: the four corners read (19, 40), (285, 29), (28, 202) and (279, 208).
    • Menu row taps toggle MUTE on and off; a tap opens STATUS; the BACK and CLOSE hints work; petting works.
    • Two fixes came out of it, both in this PR:
      • A tap 15 px under a row now still picks the row: a row takes its whole pitch, and the last row reaches down to the hints.
      • The last hint now takes the rest of the bar.
  • Added after that check: a tap outside the menu panel is now K2 and closes the menu, because dragon instinctively tapped the dimmed edge to close it.
  • Checked again on 2026-10-21, rebased on main 68cb887 (firmware firmware-v0.5.0-41-ge06f):
    • A tap on the dimmed edge (TAP 0 138) closes the menu.
    • A tap on the right of the hint bar (TAP 319 121) closes the menu.

🤖 Generated with Claude Code

longzhi and others added 5 commits October 21, 2026 10:57
Some boxes of the ATK-DNESP32S3-BOX V1.1 have a CHSC5432 at 0x2E on I2C0,
others don't. The firmware probes it at boot, reports TOUCH CHSC5432 or
TOUCH NONE (at boot, on hello and on the STATUS view), and on a box with
one reads the report every 20 ms. A tap on a menu row selects it and acts,
a tap on a key hint is that key, a tap on the buddy pets it, and a tap in
Leisure calls the buddy back like any key. The keys are unchanged
(ADR-0017). Hit testing uses the rectangles the display draws with.

Card taps come in a second PR with the protocol change (#148).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On box 30:ED:A0:A4:0D:08 a finger aimed at STATUS landed 15 px under it,
below the row's bar, and nothing happened. A row now takes taps across its
whole pitch, and the last one down to the line above the hints.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On box 30:ED:A0:A4:0D:08 it took several taps to close the STATUS view:
the finger went for the empty right half of the hint bar, past CLOSE.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
0017 went to voice commands (#133) and 0018 is held for accounts.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On box 30:ED:A0:A4:0D:08 dragon tapped the dimmed edge around the menu to
close it, again and again; now that is K2.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@longzhi
longzhi force-pushed the claude/touchscreen branch from ceffa54 to 48653e8 Compare October 21, 2026 02:58
@longzhi
longzhi marked this pull request as ready for review October 21, 2026 04:18
Copilot AI balanced review requested due to automatic review settings October 21, 2026 04:18
@longzhi
longzhi merged commit 514bd45 into main Oct 21, 2026
2 checks passed
@longzhi
longzhi deleted the claude/touchscreen branch October 21, 2026 04:18

Copilot AI left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

An organization owner can choose the repository organization as the billing source for reviews requested by members with a paid Copilot license. Learn about Copilot code review billing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants