Repository navigation
Quorum reduction - #617
Merged
Merged
Quorum reduction#617
Conversation
Rounds could dead-end when a juror dropped out mid-round: modify_jurors refused to shrink the jury below quorum, and modify_quorum rejected any decrease (NotImplementedResponse, unchanged since 2017). The blocker was "whose completed vote do you discard?"; the answer is "none": results are a per-entry mean over completed votes, rounds close when no active votes remain, and unequal per-entry vote counts are already a supported state. Lowering quorum therefore only cancels excess not-yet-cast tasks, treating quorum as a coverage floor (incomplete block design posture), never discarding completed votes. New capabilities: - modify_quorum now supports decreases (cancel excess active tasks, then rebalance without reassign_all, so jurors keep what they hold) and its increase path actually creates tasks mid-round (see bug 1 below). - New module-level helpers: create_rating_tasks_for_coverage (top every non-DQ entry up to a target of non-cancelled votes, fewest-open-tasks juror first) and cancel_excess_rating_tasks (cancel above-target active votes from the most-loaded jurors first; max-min entry coverage is preserved by construction). - CoordinatorDAO.remove_juror + POST /admin/round/<id>/remove_juror: remove one juror from a paused round, cancelling their open tasks; their completed votes are kept unless discard_completed is set (values retained on the rows for audit; scoring only reads completed votes). Quorum implicitly drops to the remaining jury size when needed (or to an explicit new_quorum); affected entries are re-covered by remaining jurors who have not seen them. - edit_round applies quorum before jurors, so one RoundEdit.vue save can lower the quorum and shrink the jury together; error responses roll back both (covered by a rollback test). No frontend change needed: the quorum input has no client-side lower bound. - get_round_damped_rating_map: IMDb-style damped (Bayesian) mean with prior weight m = quorum, shown as damped_ratings beside the plain ratings in preview_results, to spot low-coverage entries whose score is unstable. Advancement still uses the plain mean. - tools/admin.py: round edit-quorum rewritten (it called nonexistent MaintainerDAO methods; now uses CoordinatorDAO.from_round like its siblings, prints created/cancelled counts) and new round remove-juror command. Latent bugs fixed in the 2016-17 assignment plumbing (5 and 6 were demonstrated red-first; 1 is pinned by a task-count regression test): 1. modify_quorum increases silently created zero tasks mid-round (create_initial_rating_tasks only covers vote-less entries). 2. reassign_ranking_tasks returned None when juror sets matched; callers subscript the result. 3. create_initial_rating_tasks' quorum ceiling counted inactive jurors. 4. get_rating_advancing_group averaged votes of any status; now filters completed like get_round_average_rating_map (this also guarantees discarded votes stay out of advancement). 5. modify_jurors' same-size swap path assumed exactly one added and one removed juror; replacing two-plus jurors at once stranded a removed juror's tasks. Multi-swaps now fall through to reassign_tasks. 6. reassign_rating_tasks crashed (ValueError in weighted_choice) when an active vote had no eligible juror; such orphaned tasks are now cancelled (coverage is already met) and reported via a new cancelled_task_count key. Also: Round.to_details_dict juror_details is no longer a 1-tuple (trailing comma). Tests: red-first regressions for bugs 1/5/6 (test_reassign.py is a new DAO-free unit test against in-memory sqlite), quorum decrease and combined-edit lifecycle coverage, remove_juror keep/discard flows, error paths (active round, unknown juror, last juror, oversized new_quorum, rollback), and damped-ratings presence/bounds. Full suite green (15 passed, 1 xfailed). CLI smoke-tested against a scratch DB. Design references: incomplete block designs (Niu & Rosenberger, arXiv:1806.00034), PeerReview4All max-min fairness (JMLR 22(163)), IMDb weighted rating / Bayesian average, and slaporte's 2017 Montage work-allocation gist (its "preserve" default is adopted; its discard question is dissolved by keep-everything). Workstream-Session-Id: 01a01955-97bb-7000-82f4-f3188a675ce2
Discovered live testing the quorum-decrease UI: the round-edit form was unusable for its own core cases, independent of the quorum-decrease feature itself. - Round.to_info_dict() never included quorum (only to_details_dict() did), but GET /admin/campaign/<id> -- the endpoint that populates the round list RoundEdit.vue reads its initial `round` prop from -- uses to_info_dict(). So the quorum field always opened blank. Worse, saving with a blank field sends quorum: '' to PATCH /round/<id>/edit, which edit_round's `if new_quorum and new_quorum != rnd.quorum` silently treats as "no change" (200 OK, nothing applied, no error toast) -- indistinguishable from the backend just not working. - RoundEdit.vue seeded its juror chips from the unfiltered jurors list (every juror ever added to the round, active or not -- intentional at the to_info_dict() layer, since the finalized-campaign report's all_jurors credit list at rdb.py depends on seeing removed jurors too). The edit form needs only the active roster; each juror already carries is_active, so filtered on the frontend instead of narrowing the shared serializer. - saveRound() never reloaded the page on success, unlike its own sibling deleteRound() (and RoundInfo's activate/pause). Round-level panels refresh independently via their own fetch, but the `round` prop RoundEdit reads from is fetched once at the parent and was otherwise never refreshed for the rest of the SPA session -- so even a successful save would show reverted-looking quorum/jurors on the next Edit open. Fix: add quorum to Round.to_info_dict() (drop the now-redundant reassignment in to_details_dict()); filter jurors to is_active before seeding RoundEdit's form; reload after a successful save. Verified live against the seeded scratch round from the earlier screenshot session: quorum now pre-fills with the true current value, a removed juror's chip and the new quorum both persist correctly on reopening Edit with no page refresh, matching the exact remove-juror-then-lower-quorum flow the quorum-decrease feature exists to support. Workstream-Session-Id: 01a01955-97bb-7000-82f4-f3188a675ce2
The quorum-reduction steps added to test_home_client end the yes/no round at quorum 1 and then reactivate it. With 20 entries and 5 jurors, the decrease (cancel_excess_rating_tasks plus a non- reassign_all rebalance) and the two-for-two juror swaps can leave Jimbo Wales with only 1-2 open tasks. The juror steps that follow need at least 3 (submit tasks[0], skip tasks[1], then expect the skipped vote not to come back), so the test failed nondeterministically: 3/15 runs on e72ae58 (before the master merge) and 8/20 runs after merging origin/master, as either `assert <id> not in [<id>]` (only the skipped task left, which #371 serves again by design) or an IndexError on tasks[1]. Raising the quorum back to 4 (exercising the coverage-based increase path once more) gives every juror a full queue. 25/25 runs pass. No production code changes.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Long awaited, and finally here. Draft description soon.