Skip to content

refactor: carry common ASIC jobs through submission and results - #1972

Open
johnny9 wants to merge 17 commits into
bitaxeorg:masterfrom
johnny9:upstream-refactor/03-common-jobs-upstream
Open

johnny9 wants to merge 17 commits into
bitaxeorg:masterfrom
johnny9:upstream-refactor/03-common-jobs-upstream

Conversation

@johnny9

@johnny9 johnny9 commented Sep 14, 2026

Copy link
Copy Markdown
Collaborator

Job creation, active-job storage, nonce validation, and pool submission use asic_job_t, which owns the Bitcoin header fields and submission metadata. Bitmain drivers build their final wire packets immediately before transmission, removing bm_job and the conversion back to the common job.

Each nonce result carries the matched common job, captured under the active-slot lock. Its rolled version is derived from that same snapshot. Validation and pool submission consume the result directly, removing the public job getter and the second slot lookup that could observe a replacement job. The result's job snapshot survives active-slot reuse, and hardware job IDs stay inside the drivers.

BM1366, BM1368, BM1370, and BM1373 share a packet encoder. BM1397 has a separate encoder that generates its software midstates directly into the packet. Chip packet formats are preserved. Version rolling also now carries and wraps within the negotiated mask, fixing the sparse-mask and wraparound bug exposed by the new regression tests.

This keeps Stratum independent of Bitmain internals and provides a common job interface for other ASIC implementations.

The QEMU test task uses an 8 KB stack with the stack-end watchpoint enabled. Job-building and hashing tests exceed ESP-IDF's 3,584-byte default; that overflow could corrupt heap metadata and surface later as a JSON parser allocation crash. A diagnostic run with heap integrity checks passed with about 6 KB peak stack usage.

Validation: all 145 QEMU tests pass, including exact packet bytes for all five drivers, BM1397 midstate/version mapping, result snapshots across slot replacement for every driver and job protocol, invalid response IDs, and allocation failure recovery. Result-task tests run without an active-job store. The ESP32-S3 firmware builds with ESP-IDF 6.0.2 using existing AxeOS assets. Hardware validation has not been run for this revision.

@github-actions

github-actions Bot commented Sep 14, 2026

Copy link
Copy Markdown

Test Results

  2 files  ± 0    2 suites  ±0   1s ⏱️ -1s
231 tests +18  231 ✅ +18  0 💤 ±0  0 ❌ ±0 
233 runs  +18  233 ✅ +18  0 💤 ±0  0 ❌ ±0 

Results for commit 492392e. ± Comparison against base commit 43e9b97.

This pull request removes 7 and adds 25 tests. Note that renamed tests count towards both.
test_asic_result_task ‑ result task keeps owned snapshots through submission for every protocol
test_asic_result_task ‑ result task separates registers and rejects unavailable job slots
test_job_building ‑ Bitmain job defaults and explicit values preserve metadata precedence
test_job_building ‑ Bitmain software midstate count honors zero mask and the buffer limit
test_mining ‑ Validate midstate generation
test_mining_pipeline ‑ SV1 notify reaches the ASIC job boundary byte exact
test_mining_pipeline ‑ SV2 standard messages reach the ASIC job boundary byte exact
test_asic_job ‑ Bitcoin header encodes hash bytes and little-endian integers
test_asic_result_job ‑ ASIC results preserve complete matched jobs across slot reuse for every driver
test_asic_result_job ‑ BM1366 and BM1397 reject response job IDs outside the active slots
test_asic_result_task ‑ result task separates registers and empty ASIC responses
test_asic_result_task ‑ result task submits embedded job snapshots without a job store for every protocol
test_asic_submit ‑ job submission owns metadata and recovers from allocation failures
test_asic_submit ‑ job submission rejects unterminated metadata before allocation
test_bitmain_job_packet_building ‑ BM1397 packet limits midstates and clears unused entries
test_bitmain_job_packet_building ‑ Bitmain packet builders support unaligned outputs without overwriting guards
test_bitmain_job_packet_building ‑ Validate midstate generation
…

♻️ This comment has been updated with latest results.

@johnny9
johnny9 force-pushed the upstream-refactor/03-common-jobs-upstream branch from 14c730f to 2effe48 Compare September 14, 2026 16:38
@johnny9 johnny9 changed the title refactor: introduce common ASIC jobs refactor: separate common mining jobs from Bitmain work Sep 14, 2026
Comment thread components/asic/private_include/bm_job.h Outdated
Comment thread components/asic/bm_job.c Outdated
@johnny9
johnny9 force-pushed the upstream-refactor/03-common-jobs-upstream branch from a1c202c to 80564b2 Compare September 16, 2026 13:23
@johnny9
johnny9 marked this pull request as ready for review September 16, 2026 13:24
QEMU validation: 143 tests passed, with 0 failures and 0 ignored.
Port the relevant fixes from UBSan PR bitaxeorg#1976 (a3e5540), retaining the existing byte-safe target conversion. Validate alignment at compile time and overlapping buffers in the ordinary QEMU suite.

Validation: 144 QEMU tests passed; full ESP-IDF firmware build passed.
@johnny9
johnny9 requested a review from mutatrum September 16, 2026 14:38

@johnny9 johnny9 left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

double check the correctness of stratum->bm_job

Comment thread components/asic/include/asic.h Outdated
Comment thread components/asic/include/asic_common.h Outdated
Comment thread components/asic/bm_job.c Outdated
free(job);
}

bool bm_job_to_asic_job(const bm_job *source, asic_job_t *destination)

@mutatrum mutatrum Sep 16, 2026

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Conceptually I don't understand why we would need a transformation from bm_job to asic_job. The pipeline should only go in one direction? Maybe I'm misreading the code, not sure :-)

@johnny9 johnny9 Sep 16, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was a storage decision which I also don't think i agree with. Need to do another round to see if I can get away with not duplicating both. But it's going back to asic job for share submissions

@johnny9 johnny9 Sep 16, 2026

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think Im going to remove bm job and just convert from the generic asic job before sending

Retain asic_job_t in active slots and build final Bitmain packets immediately before sending. Share the newer-chip encoder and keep BM1397 midstate generation in its packet builder, removing bm_job and reverse conversion.

Preserve packet bytes, snapshot ownership, and result version mapping with regression tests. Fix masked version increment carry and wraparound exposed by the BM1397 sparse-mask fixture.

Validation: 145 QEMU tests pass; ESP32-S3 firmware builds with ESP-IDF 6.0.2.
@johnny9 johnny9 changed the title refactor: separate common mining jobs from Bitmain work refactor: retain common ASIC jobs and encode Bitmain packets on send Sep 17, 2026
Capture the retained common job while decoding each nonce response and derive its rolled version from that same snapshot. Validation and pool submission consume the result context directly, removing the public job getter and the second slot lookup.

Keep hardware job IDs internal and reject response IDs outside the active slots. Exercise snapshot lifetime across slot replacement for every Bitmain driver and job protocol, and run result-task tests without an active-job store.

Validation: 145 QEMU tests pass; ESP32-S3 firmware builds with ESP-IDF 6.0.2.
@johnny9 johnny9 changed the title refactor: retain common ASIC jobs and encode Bitmain packets on send refactor: carry common ASIC jobs through submission and results Sep 17, 2026
Unity runs production job building and SHA-256 below large test fixtures on its main task. The default 3584-byte stack can overflow into heap metadata, causing a later cJSON allocation to assert in remove_free_block.

Use the same 8192-byte stack budget as the firmware job and result tasks, and enable the stack-end watchpoint. Diagnostic runs with comprehensive heap poisoning and integrity checks before and after each test pass at 8192 bytes and show approximately 6 KB peak usage.

Validation: a fresh ESP32-S3 QEMU configuration passes all 145 tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants