Repository navigation
Conversation
No firmware of ours has used the box's microphone. The I2C0 scans found no ES7210 at 0x40-0x43, so the mic is most likely on the ES8311's own ADC, which our ES8311 sequence (esp_codec_dev's WORK_MODE_BOTH) already powers, with samples on GPIO47 as in xiaozhi 1.9.4's atk-dnesp32s3-box config. - core: device.mic.record (cable only) reports chip IDs at 0x18 and 0x40-0x43, sets the ES8311 mic gain (register 0x16, as es8311_set_mic_gain), records, then sends per-100 ms RMS/peak levels for both slots and the samples as base64 lines, one per poll, with a CRC. - device: I2S0's receiver shares the transmitter's BCLK/WS (signal loopback) and reads GPIO47 into PSRAM only while a capture runs; 24 kHz 16-bit stereo, since the ES8311 has one LRCK for ADC and DAC. - tools/mic-record.py writes a WAV and says whether a slot is live. - docs/firmware-bringup.md: facts table with sources, how to run and read it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The first mic build boot-looped on a stack guard panic while drawing the look at boot. esp-hal's main stack is the internal DRAM left between the end of .bss and the reclaimed heap; the static 24 KB RX ring and the task's 2.4 KB local buffer took it from 47,784 to 21,032 bytes. The ring (12 KB, 4 descriptors) is now allocated from internal heap only while a capture runs and freed after, and the task copies straight from the ring into the PSRAM capture, so its pool is 184 bytes. The stack is 47,576 bytes, 208 below main. A lesson records how to check this. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the box the first capture ended in MIC ERROR STALLED with nothing to say why. The 12 KB ring of four descriptors had to be relinked by the mic task before the DMA reached its end, and the task only runs when the main loop yields; once the DMA hits the end of the chain it stops for good. The DMA now writes the whole recording into one PSRAM buffer (esp-hal's DmaRxBuf handles the cache), so nothing depends on the main loop. Its descriptors (~1 KB for 5 s) come from the internal heap for the capture. Each capture also prints MIC DEBUG lines at 100 ms and at the end: descriptors filled, I2S0 and GDMA channel 1 registers, and how often GPIO47/21/13 changed level, so the next flash says where it stops. Main stack: 47,472 bytes (main: 47,784). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the box all 71 descriptors filled and the capture still came back MIC ERROR SHORT. The I2S receiver signals an EOF every 4095 samples (esp-hal leaves RXEOF_NUM at its 12-bit maximum, 8190 bytes), and each EOF closes the descriptor it lands in, so about every third 4080-byte descriptor holds 30 bytes and the buffer held about two thirds of the samples. GDMA IN_INT_RAW 0x13 (suc_eof set) fits this. The buffer is now 1.5x the capture plus two descriptors, and the capture ends once the handed-back descriptors hold enough bytes. MIC DEBUG adds the byte count and the first six descriptor lengths, so the next flash shows whether the 4080 4080 30 pattern is the cause. The BCLK/WS level counts are gone: those pins are outputs with the input path off, so the input register reads 0 however fast they run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On the box the descriptors came back as 4080, 14, 4080, 14: the I2S EOF cuts one every 4094 bytes, so a capture needs about twice its size in descriptors. 2.5 times leaves room. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- docs/hardware.md: box 30:ED:A0:A4:0D:08 recorded 5 s on 2026-10-13. The mic is the ES8311's ADC, samples on GPIO47, left slot; no ES7210; about -31 dBFS for a close voice at 30 dB gain. - device mic.rs and docs: RXEOF_NUM (0xFFE from esp-hal) counts bytes on the ESP32-S3, as ESP-IDF uses it, hence descriptors of 4080, 14, ...; the buffer stays at 2.5x, since lining EOFs up would need a raw register write racing esp-hal's that hasn't been tried on a box. - tools/mic-record.py judges slots from the samples with DC removed and the first 100 ms dropped, needs a window above -70 dBFS to call one LIVE, and bases the overall verdict on the left slot. - LESSONS.md: count received data by descriptor lengths, and make a bring-up diagnostic report its own state from the first build. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
longzhi
added a commit
that referenced
this pull request
Oct 20, 2026
…e Character's voice (#133) Hold K0 and speak: a resident session answers as the buddy, in the Character's voice and persona (ADR-0017, #132). Experimental, off unless the daemon runs with VIBEBUDDY_RESIDENT_SESSION=1. - Box: push-to-talk on K0 long while voice is offered, the mic (ES8311 ADC on GPIO47) streamed as IMA ADPCM, LISTENING/THINKING/reply overlay, MP3 reply playback, a 12 px CJK font, Agent news held during a voice command, RESET <reason> at boot. - Daemon: recognition through the voice service (Groq, Workers AI fallback) or the user's own recognizer; one resident session for good on Codex (own CODEX_HOME, only our tools) or Claude Code (the user's own API key); spoken replies in the Character's ElevenLabs or Doubao voice, paced into the link. - Mac and Linux Apps: a Conversation tab, the brain picker and the Anthropic API key. - Includes the microphone bring-up (#145): device.mic.record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Collaborator
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A bring-up diagnostic for the box's microphone, which no firmware of ours had used (part of #132). Not a user feature; neither app sends it.
device.mic.record(cable only;ms100–5000,gain_db0–42): reports chip IDs at 0x18 and 0x40–0x43, sets the ES8311 mic gain (register 0x16, as esp_codec_dev'ses8311_set_mic_gain), records, then sends per-100 ms RMS/peak for both I2S slots and the samples asMIC DATAbase64 lines (one per main-loop pass) with a CRC.MIC DEBUGlines report the DMA and I2S state for when it fails.tools/mic-record.py --port …writes a WAV and says whether the microphone (left slot) works.Result on a box
Box
30:ED:A0:A4:0D:08, 2026-10-13:MIC CAPTURED 5175 MS 480000 BYTES, CRC ok, and a person speaking close to the box is clearly audible in the recording. The microphone is the ES8311's own ADC (no ES7210 at 0x40–0x43), samples on GPIO47, left slot; the right slot is the DAC reference and stays silent. A close voice is about -31 dBFS at the default 30 dB gain, so real use needs more gain. Other boxes, including the daily box, are untested.Notes for review
with_signal_loopback(true)). The box booted and ran normally with it, but announcements and the Pomodoro chime were not listened to on this build: check them before merging.Tests
cargo test --workspace,cargo clippy --workspace --all-targets(no new warnings), firmware builds.core/src/mic.rs; end-to-end recording and clamping/probe tests incore/tests/firmware.rs.🤖 Generated with Claude Code