Repository navigation
October 2026 integration: import check (#656), quorum reduction (#617), final-review PRs, sentry/timeout/export/import-lock fixes - #659
Merged
Conversation
- README: add GitHub Issues link, mark Phabricator as archived, add pointer to dev.md - dev.md: fix Node.js version (v16→v18), rdb.py line number (113→119), and project tree (filenames, removed non-existent config/ dir) - .gitignore: add .claude/ and tmp/ - app.py: fix /a/ StaticFileRoute to point to static/index.html (not static/a/index.html which no longer exists) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…static files - dev.md: document debug:true and userid options for bypassing OAuth locally - config.default.yaml: add debug:false as an optional documented field - Remove montage/static/a/index.html and static/dist/ (obsolete pre-Vue frontend files) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…keys Add debug_userid and debug_username config options so developers can set which account they are auto-logged in as when debug: true, instead of always defaulting to Slaporte. Document the /complete_login workaround for bypassing OAuth locally. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- deployment.md: correct db_url format (mysql+pymysql://), document log paths for both production and beta instances separately - dev.md: remove non-existent config/ directory from project tree, fix Dockerfile -> dockerfile reference in Docker files section Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
The juror round listing was filtering on campaign status only, so finalized and cancelled rounds inside active campaigns still appeared. Jurors clicking them got a confusing 400 "round not active" error. Now only active and paused rounds are shown in the active view. Suspected cause: e3069dd (PR #350, merged 2025-10-23) added a finalize_round endpoint that can finalize individual rounds without advancing, creating a state the juror listing never filtered for. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Paused rounds were still appearing in the juror round list because the only_active filter allowed both ACTIVE_STATUS and PAUSED_STATUS. However confirm_active() only permits ACTIVE_STATUS, so clicking a paused round always produced a 400 "round not active" error. Fix restricts the juror index to strictly active rounds only. Suspected introduced by e3069dd (#350). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
get_index() accepted an only_active parameter but hard-coded True when calling get_all_rounds_task_counts, so get_all_campaigns() (the archive endpoint at /juror/campaigns/all) was always returning an empty list instead of finalized rounds. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
…and added entries in i18n
Rounds could dead-end when a juror dropped out mid-round: modify_jurors refused to shrink the jury below quorum, and modify_quorum rejected any decrease (NotImplementedResponse, unchanged since 2017). The blocker was "whose completed vote do you discard?"; the answer is "none": results are a per-entry mean over completed votes, rounds close when no active votes remain, and unequal per-entry vote counts are already a supported state. Lowering quorum therefore only cancels excess not-yet-cast tasks, treating quorum as a coverage floor (incomplete block design posture), never discarding completed votes. New capabilities: - modify_quorum now supports decreases (cancel excess active tasks, then rebalance without reassign_all, so jurors keep what they hold) and its increase path actually creates tasks mid-round (see bug 1 below). - New module-level helpers: create_rating_tasks_for_coverage (top every non-DQ entry up to a target of non-cancelled votes, fewest-open-tasks juror first) and cancel_excess_rating_tasks (cancel above-target active votes from the most-loaded jurors first; max-min entry coverage is preserved by construction). - CoordinatorDAO.remove_juror + POST /admin/round/<id>/remove_juror: remove one juror from a paused round, cancelling their open tasks; their completed votes are kept unless discard_completed is set (values retained on the rows for audit; scoring only reads completed votes). Quorum implicitly drops to the remaining jury size when needed (or to an explicit new_quorum); affected entries are re-covered by remaining jurors who have not seen them. - edit_round applies quorum before jurors, so one RoundEdit.vue save can lower the quorum and shrink the jury together; error responses roll back both (covered by a rollback test). No frontend change needed: the quorum input has no client-side lower bound. - get_round_damped_rating_map: IMDb-style damped (Bayesian) mean with prior weight m = quorum, shown as damped_ratings beside the plain ratings in preview_results, to spot low-coverage entries whose score is unstable. Advancement still uses the plain mean. - tools/admin.py: round edit-quorum rewritten (it called nonexistent MaintainerDAO methods; now uses CoordinatorDAO.from_round like its siblings, prints created/cancelled counts) and new round remove-juror command. Latent bugs fixed in the 2016-17 assignment plumbing (5 and 6 were demonstrated red-first; 1 is pinned by a task-count regression test): 1. modify_quorum increases silently created zero tasks mid-round (create_initial_rating_tasks only covers vote-less entries). 2. reassign_ranking_tasks returned None when juror sets matched; callers subscript the result. 3. create_initial_rating_tasks' quorum ceiling counted inactive jurors. 4. get_rating_advancing_group averaged votes of any status; now filters completed like get_round_average_rating_map (this also guarantees discarded votes stay out of advancement). 5. modify_jurors' same-size swap path assumed exactly one added and one removed juror; replacing two-plus jurors at once stranded a removed juror's tasks. Multi-swaps now fall through to reassign_tasks. 6. reassign_rating_tasks crashed (ValueError in weighted_choice) when an active vote had no eligible juror; such orphaned tasks are now cancelled (coverage is already met) and reported via a new cancelled_task_count key. Also: Round.to_details_dict juror_details is no longer a 1-tuple (trailing comma). Tests: red-first regressions for bugs 1/5/6 (test_reassign.py is a new DAO-free unit test against in-memory sqlite), quorum decrease and combined-edit lifecycle coverage, remove_juror keep/discard flows, error paths (active round, unknown juror, last juror, oversized new_quorum, rollback), and damped-ratings presence/bounds. Full suite green (15 passed, 1 xfailed). CLI smoke-tested against a scratch DB. Design references: incomplete block designs (Niu & Rosenberger, arXiv:1806.00034), PeerReview4All max-min fairness (JMLR 22(163)), IMDb weighted rating / Bayesian average, and slaporte's 2017 Montage work-allocation gist (its "preserve" default is adopted; its discard question is dissolved by keep-everything). Workstream-Session-Id: 01a01955-97bb-7000-82f4-f3188a675ce2
Discovered live testing the quorum-decrease UI: the round-edit form was unusable for its own core cases, independent of the quorum-decrease feature itself. - Round.to_info_dict() never included quorum (only to_details_dict() did), but GET /admin/campaign/<id> -- the endpoint that populates the round list RoundEdit.vue reads its initial `round` prop from -- uses to_info_dict(). So the quorum field always opened blank. Worse, saving with a blank field sends quorum: '' to PATCH /round/<id>/edit, which edit_round's `if new_quorum and new_quorum != rnd.quorum` silently treats as "no change" (200 OK, nothing applied, no error toast) -- indistinguishable from the backend just not working. - RoundEdit.vue seeded its juror chips from the unfiltered jurors list (every juror ever added to the round, active or not -- intentional at the to_info_dict() layer, since the finalized-campaign report's all_jurors credit list at rdb.py depends on seeing removed jurors too). The edit form needs only the active roster; each juror already carries is_active, so filtered on the frontend instead of narrowing the shared serializer. - saveRound() never reloaded the page on success, unlike its own sibling deleteRound() (and RoundInfo's activate/pause). Round-level panels refresh independently via their own fetch, but the `round` prop RoundEdit reads from is fetched once at the parent and was otherwise never refreshed for the rest of the SPA session -- so even a successful save would show reverted-looking quorum/jurors on the next Edit open. Fix: add quorum to Round.to_info_dict() (drop the now-redundant reassignment in to_details_dict()); filter jurors to is_active before seeding RoundEdit's form; reload after a successful save. Verified live against the seeded scratch round from the earlier screenshot session: quorum now pre-fills with the true current value, a removed juror's chip and the new quorum both persist correctly on reopening Edit with no page refresh, matching the exact remove-juror-then-lower-quorum flow the quorum-decrease feature exists to support. Workstream-Session-Id: 01a01955-97bb-7000-82f4-f3188a675ce2
…tainers only /meta and /meta/json/ are public. clastic's resource section shows the first 70 characters of repr() of each app resource, which for 'config' showed the start of the config dict to anyone. The resource section now lists each resource by name and type only (ResourceNamesPeripheral); the rest of /meta (start time, routes, host, process) is unchanged, so deploy.sh's smoke test keeps working. GET /v1/logs/feel (the front-end error log tail: usernames and error payloads) had no permission check; it now requires a maintainer, like /logs/api and /logs/api_exc. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- utils: ImportSourceInvalid/CheckBlocked/CheckExpired/CheckRequired (400 with error_type); config key import_check_path (MONTAGE_IMPORT_CHECK_PATH) - labs: one query text for lookups by name and by file_id; get_files_info_by_ids - loaders: fetch_source_text, parse_source_rows (filename > img_name, optional file_id, BOM/case-insensitive headers), parse_name_list, lookups; old full-CSV/name-list loaders removed (#509, #517); only gist links are rewritten (#208) Refs #510, #509, #208 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- POST /admin/campaign/<id>/import/check, GET .../import/check/<token>/download - import_entries requires check_token for csv/gistcsv/category/selected and imports exactly the checked list; blocked check -> 400 before writes - add_checked_entries replaces the three DAO loaders; file_id lookup in every campaign (FILE_ID_LOOKUP_CUTOFF removed) - app refuses to start on beta/prod/devlabs without MONTAGE_IMPORT_CHECK_PATH - no 'all disqualified' warning for an empty import (#208) - tests import through check + token Refs #510, #208 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- Check button + ImportCheckResult panel (text only), download link - Save only after a non-blocking check; source change clears the check; stale check responses ignored; source locked while checking - the round is created once; a failed import is retried into it, with the round settings locked - i18n (en + qqq) for the new texts and the import warning dialog - Cypress: check before Add Round (not run) Refs #510 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ll (#447) A first round was created by one request and filled by a second one; when the import brought no files (an empty category, a CSV with no valid names) or failed, an empty round stayed behind and blocked creating another one. - add_round takes an optional import {import_method, check_token}: the round and its files are saved in one transaction; an import that fails, is blocked or brings no files (import_empty) rolls the round back - the round form sends the checked import with the round and keeps Save disabled when the check found no importable file; on an error nothing was saved and the form stays as entered (the locked-settings retry of #510 is no longer needed and is removed) Based on feat/import-check-510. Fixes #447 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- check folder: cleanup deletes only Montage's own check files (token- named, .tmp-), at most MAX_CHECK_FILES kept; an existing folder keeps its mode; startup refuses a relative, unwritable or shared folder and any unconfigured folder outside dev/devtest; warns outside /data/project - source links: https only, every address of the host (and of each of at most 3 redirects) must be public; 20 MB and 100,000-row caps; source fields must be text - same-name keys: database keys on mysql/mariadb, the test-only stand-in only for SQLite in dev/tests, otherwise an error; NULL keys refused - file names canonical like MediaWiki File: titles (prefix in any case, runs of spaces/underscores, first letter upper case) - round sources store a check fingerprint (check_id), not the token, and a new check of the same source reuses the round source - issues report download (every row that is not ok) - add_round accepts only checked import methods - checked imports skip add_entries' lower-case dedupe (matched by file_id); Entry.file_id declared indexed - tools/admin.py create_round: source and advance params - unused imports and cutoff names removed; _ImportError renamed Refs #510, #447 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- deploy.sh stops before building when a required envvar is missing - deployment.md: $HOME-based commands, folder of its own, cleanup rules, troubleshooting (missing variable, timeout, old files) and rollback - docs/api.md: import/check, download, issues, add_round's import, the new error types Refs #510, #447 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
- while a round is saved: source fields and Cancel disabled, Save says what it is doing; Cancel reloads the campaign - save errors handled by type: blocked -> check again, no files -> nothing to import, expired / stale page -> check again or reload, other errors -> a message next to Save, no answer -> reload the campaign and say the round may exist - a check older than 7 days is refused before saving - the warnings dialog closed with X/Escape closes the form too (dialogService onClose); dialog text keeps its line breaks - screen readers: a live region announces check results, errors and a cleared check; focus moves to the result; Save is described by the reason it is disabled; the issues table is a named, focusable region with a caption; blocking rows are marked in text; server English is marked lang=en; the pair separator is hidden - category checks: own summary, no 'row' labels, same-name rows not shown as blocking; a typed but unpicked category gets a hint - issues report link; download links use the download attribute - i18n: no concatenated strings; no plurals (none in the project) - Cypress: the save is intercepted, its body asserted, a double click saves once, a refused import keeps the form (not run) Refs #510, #447 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… days Files are renamed and deleted on Commons all the time, and the check and the save happen in one sitting, so a check can be imported for 1 hour (was 7 days). The check file and its downloads (upload file, issues report) stay at most 7 days: every check, import, download and the app start delete older files, so they also go after a quiet spell. Refs #510 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…help texts The File List box read every line as a file name, so a pasted CSV (e.g. the check's download, meant to be the next source) came back as "not on Commons" line by line. Now: - a first line that is a header with filename, img_name or file_id makes the box read as a CSV, exactly like a File List URL source; - a line of digits only is a file_id (a Commons name always has an extension), so a plain list of file_ids works, mixed with names; - a CSV may have only a file_id column (names come from Commons). The same-name rule and the file_id lookup apply as for every list. Help texts: file list explains names, file_ids and pasted CSVs; File List URL names the columns and the sharing setting; the download says it can be pasted into File List; category example is 2026. Found while testing on montage-dev (2026-10-07). #510 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…heck Where entries.name has a unique index (beta, installs built from the model), a new file whose name the database treats as equal to that of a different file already in Montage cannot be stored: saving a round with the WLM 2026 Russia category on beta failed with IntegrityError 1062 on "Челябинск_2026,_пр._Ленина_71А.jpg" (beta already had "...71а.jpg"), 2026-10-10. The check compared names only within one import. The check now also looks the names up in entries (the database's own comparison, through that unique index) and marks the clashes as same_name_existing: left out of a category with a warning, blocking a list. The same file (same file_id) and an old row of exactly this name without a file_id are reused by the import, as before, and are not clashes. Production has no index on entries.name (#650): there both rows are stored, so the lookup (a full table scan there) is skipped. New status label and reasons (translated); tests for the app-level case, the database comparison and the skip without a unique index. #510, #645 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Testing on beta (2026-10-10) showed the "ambiguous id" check blocking an
ordinary file ID: almost every file_id is also the page ID of some other
file page (Commons API: 4 of 4 file_ids from our tests), so a bare
number cannot be told apart and the check blocked nearly every list of
file IDs.
Now a bare number that is a file_id is taken as one, and shown as
"Found by file ID" with the file's name (status by_file_id, imported),
so a wrong number is seen before saving. Only a number that is no
file_id is looked up as a page ID / revision ID of a file page, as a
courtesy in the error message ("... is the page ID of File:X, not its
file ID"). The ambiguous_id status is gone.
The help texts and errors now say plainly that a file ID is the number
of the file itself (the file_id column), not the page ID shown under
Page information.
#510
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…file ID The yellow message after a check with only warnings said rows were "left out or changed". Rows found by a bare file ID are also listed there and are neither: they are imported, and shown so a wrong number is seen. The message now reads "You can save the round. Check the rows listed below: some are left out, changed, or found by file ID." The qqq entry explains the three kinds. #510 Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The Procfile ran gunicorn with no --timeout, so the 30 s default applied. Sync workers do not heartbeat during a request, so this flag is effectively a per-request deadline; prod's exception log shows imports killed at 30 s (SystemExit from handle_abort on POST /admin/round/<id>/import; #618, #650). With prod's ix_entries_name restored, Lodewijk measured a 26.4k-file import at ~18 s in the request (#650), so 120 s is headroom, not the plan. Set in the Procfile rather than a GUNICORN_CMD_ARGS envvar so it is versioned and applies to montage, montage-beta and montage-dev alike (CLI flags also override that envvar). Caveat: the Toolforge front-proxy request timeout is undocumented; measure on beta whether it cuts before 120 s. Workers stay at 4; the 3Gi memory is a Toolforge setting, not code. Verified: gunicorn --print-config shows timeout = 120, workers = 4. Workstream-Session-Id: 01a126ed-002a-7000-99d5-a8700aed81a9
…per entry (#640) GET /admin/round/<id>/entries and /entries/download loaded every RoundEntry, then lazy-loaded its Entry (and RoundSource) one row at a time via RoundEntry.to_export_dict(). On a 15k-entry round that is ~15k sequential SELECTs, which outlasts gunicorn's request timeout (#640). CoordinatorDAO.get_round_entry_export_rows() replaces get_round_entries() plus Entry/RoundEntry.to_export_dict(): one joined column query for the round's non-disqualified entries (now ordered by round_entries.id), and one query for the referenced round sources, whose params are json.dumps'd once per source instead of once per row. Output keys and values are unchanged; a round entry with a NULL round_source_id now exports source_method=None / source_params='null' instead of raising AttributeError. Both endpoints use it; the CSV is still built in memory. New test test_round_entries_export_query_count (n=450, three import chunks) counts SELECTs per request: before: list 457, download 458 after: list 7, download 8 and checks row counts, CSV fieldnames, source_method and source_params. Full suite: 57 passed, 1 xfailed (master: 56 passed, 1 xfailed). Still open: #639, file-list imports repeat their (large) source_params in every exported row. Workstream-Session-Id: 01a126ed-002a-7000-99d5-a8700aed81a9
… the deprecated Hub API On Python 3.13 (buildservice, since #564) sentry-sdk 2.8.0 crashes while building events: it does copy(frame.f_locals), and on 3.13 f_locals is a FrameLocalsProxy, which can't be copied. Every exception report dies, so Sentry has shown nothing for prod's import timeouts and OOMs (#642, #653). Fixed upstream from 2.11; pin the latest 2.x, 2.71.0. Only the sentry-sdk line in requirements.txt changes (no recompile; urllib3 2.5.0 and certifi 2024.2.2 already satisfy 2.71's requirements). clastic_sentry.py moves to the 2.x scope API: the per-request isolation scope that SentryWsgiMiddleware already opens, the transaction name set on the current scope (setting it on the isolation scope loses to the WSGI middleware's "generic WSGI request"), and sentry_sdk.capture_event with mechanism handled=False. The old module no longer imports under 2.71 (_should_send_default_pii is gone from sentry_sdk.hub). The sentry_hub dependency is dropped from provides; nothing consumes it. mw/__init__.py's sentry_scope._user use still works (Scope._user exists). Smoke test (clastic app with SentryMiddleware, /boom raises ValueError): - before, 2.8.0: /boom 500 with FrameLocalsProxy in the body, 0 events - after, 2.71.0: /boom 500, 1 event, ValueError, transaction '/boom'; /ok 200, 0 events pytest: 56 passed, 1 xfailed, same as origin/master in the same venv. Workstream-Session-Id: 01a126ed-002a-7000-99d5-a8700aed81a9
…ry file twice (#635) A slow import (minutes on prod without ix_entries_name, #650) outlives the browser; coordinators retry, and two imports of the same round ran side by side. Neither locked the round, so both inserted every file (#635). A row lock alone isn't enough on MySQL/MariaDB. Under InnoDB REPEATABLE READ a request's snapshot is fixed at its first plain read (the user lookup in UserMiddleware), so a request that waits on the lock and then reads plainly still sees the round's entries as they were before the other import committed. add_round_entries and add_entries both read plainly, so the waiting import would still double-insert. So CoordinatorDAO.lock_round_for_import: - counts the round's entries in the request's snapshot, - takes the round's row lock (_lock_round, FOR UPDATE, as #626 does for activation) and refuses unless the round is paused, - counts again on a separate connection (its own fresh snapshot, so it sees what was committed while this request waited), - refuses with a 400 if the two counts differ: another import finished meanwhile; the coordinator refreshes and re-imports only if files are missing. import_entries takes this lock first and holds it across the fetch, the inserts and autodisqualify, ending with a flush, all inside lock_errors_as_busy. Only that round's row is locked: a second import or activation of the same round waits (innodb_lock_wait_timeout, 50 s by default) and then gets a "server is busy, please retry" 400 instead of a 500; other rounds are unaffected. activate_round gets the same guard plus a trailing flush, so activating while an import holds the lock is a 400, not a 500. MYSQL_LOCK_ERROR_CODES, is_lock_error and lock_errors_as_busy are a verbatim port from #630 (commit 1a19b5d, rdb.py) so a later rebase of the worker merges cleanly. Test: test_import_waiting_on_concurrent_import_is_refused commits one extra round entry from another connection right after the request takes the round lock (the other import finishing during the wait). The import is refused (400, nothing added), and a later import without the race returns 200 with the usual "duplicate import" warning. With master's rdb.py/admin_endpoints.py the same test fails: the second import returns success (200). Full suite: 57 passed, 1 xfailed (master: 56 passed, 1 xfailed). Interplay with #656: it rewrites import_entries around check tokens (_import_into_round, also used by add_round's first-round import). Whichever lands second must keep lock_round_for_import as the first statement of the import path, inside lock_errors_as_busy, including the check-token import into an existing round that #656 adds. Workstream-Session-Id: 01a126ed-002a-7000-99d5-a8700aed81a9
… artifacts, update dev docs (beta integration) PR #468 (docs/improve-setup-docs, @lgelauff, labelled 'final review'), head 575f93a. Merged for the montage-beta drill of the WLM 2026 import fixes; the PR itself still lands on master through GitHub. Conflicts, resolved to master's side: - montage/public_endpoints.py complete_login: master already reads debug_userid/debug_username and, since the OAuth 2.0 + dev-only impersonation change, gates debug login on env_name == 'dev' with defaults 0 / '__montage_debug__'. #468's version (config-driven Slaporte defaults, no env gate) predates that; master's wins so a stray debug flag can't bypass OAuth on beta/prod. - dev.md repo tree: master lists Procfile; both sides dropped config/*.ini. - config.default.yaml (auto-merged from #468): the comment's defaults updated to master's __montage_debug__ / 0 to match the code. Kept from #468: /a/ serves the main index.html, stale montage/static/a and logo_white_fat.svg removed, README/dev.md setup docs. pytest montage/tests/ -q: 56 passed, 1 xfailed.
…ration) PR #571 (fix/juror-round-visibility, @lgelauff, labelled 'final review'), head 69d6903. Juror index: pass only_active through, hide paused/finalized/cancelled rounds from the juror's active view (juror_endpoints.py, rdb.py, test). Conflict: frontend/src/components/Round/RoundEdit.vue, the unused vue-router import. Master (#588) and #571 both deleted it; kept master's text (one blank line after the imports). pytest montage/tests/ -q: 56 passed, 1 xfailed.
…ilters in get_campaign_rounds (#399) (beta integration) PR #500 (fix-get-campaign-rounds-399, @ayushshukla1807, labelled 'final review'), head af95219. Merged for the montage-beta drill of the WLM 2026 import fixes; the PR itself still lands on master through GitHub. Changes (2 files changed, 95 insertions(+), 2 deletions(-)): - montage/rdb.py - montage/tests/test_dao_unit.py
…ed entries in i18n (beta integration) PR #593 (block-edit-when-active-round, @SSpirate11, labelled 'final review'), head 2d27134. Merged for the montage-beta drill of the WLM 2026 import fixes; the PR itself still lands on master through GitHub. Changes (3 files changed, 13 insertions(+), 5 deletions(-)): - frontend/src/components/Round/RoundView.vue - frontend/src/i18n/en.json - frontend/src/i18n/qqq.json
The quorum-reduction steps added to test_home_client end the yes/no round at quorum 1 and then reactivate it. With 20 entries and 5 jurors, the decrease (cancel_excess_rating_tasks plus a non- reassign_all rebalance) and the two-for-two juror swaps can leave Jimbo Wales with only 1-2 open tasks. The juror steps that follow need at least 3 (submit tasks[0], skip tasks[1], then expect the skipped vote not to come back), so the test failed nondeterministically: 3/15 runs on e72ae58 (before the master merge) and 8/20 runs after merging origin/master, as either `assert <id> not in [<id>]` (only the skipped task left, which #371 serves again by design) or an IndexError on tasks[1]. Raising the quorum back to 4 (exercising the coverage-based increase path once more) gives every juror a full queue. 25/25 runs pass. No production code changes.
… legacy Py2 code (beta integration) PR #551 (refactor/modernize-utils-typing, @ayushshukla1807, labelled 'final review'), head 35d88d6. Type hints and Py2 cleanup in montage/utils.py only. Conflicts in montage/utils.py, resolved by keeping master's behavior with #551's signatures: - get_env_name() -> str: master's body (MONTAGE_ENV first, a clear RuntimeError when getpass fails in containers), #551's annotation. - load_env_config(env_name: str | None = None) -> dict[str, Any]: #551's signature; master's _load_config_from_env (envvar config, not touched by #551) kept unchanged above it. Tests do NOT pass at this commit: #551 deletes utils.basestring, which rdb.py and mw/__init__.py still import (4 collection ImportErrors, on #551's own head too). Fixed in the next commit.
…oved #551 deletes the Py2 aliases `basestring` and `unicode` from montage/utils.py (both were `str` on Python 3), but five modules still imported them, so test collection failed with 4 ImportErrors after the #551 merge (the same on #551's own head). Replaced each use with `str`, which is exactly what the aliases were: - montage/rdb.py, montage/mw/__init__.py: basestring - montage/imgutils.py, montage/loaders.py, montage/tests/test_web_basic.py: unicode No behavior change. TimingMiddleware's `isinstance(ret.data, str)` is still never true for werkzeug's bytes body, as before. pytest montage/tests/ -q: 57 passed, 1 xfailed. pyflakes over montage/ and tools/ finds no undefined names besides tools/trim_csv.py's pre-existing `file`. Workstream-Session-Id: 01a12672-ec39-7000-9f0c-c9b9269890b3 Workstream-Plan-Id: 2026-10-10-c5476fcaee
…-end error log for maintainers only (beta integration) PR #647 (fix/meta-no-resource-values, @lgelauff, labelled 'final review'), head e358a26. Merged for the montage-beta drill of the WLM 2026 import fixes; the PR itself still lands on master through GitHub. Changes (3 files changed, 59 insertions(+), 3 deletions(-)): - montage/app.py - montage/meta_endpoints.py - montage/tests/test_web_basic.py
…c_sentry on the 2.x scope API (#642, #653) (beta integration) Sentry has been blind on Python 3.13: sentry-sdk 2.8.0 crashes copying a FrameLocalsProxy while building events. Branch tip 489c813. Not yet a PR; goes to master as its own PR after the beta checks. Changes: 2 files changed, 24 insertions(+), 32 deletions(-).
…t round with its files (#510, #509, #447) (beta integration) PR #656 (feat/import-check-510, @lgelauff, labelled 'final review'), head 4f12cf2. Mandatory import Check (POST /admin/campaign/<id>/import/check, check files under MONTAGE_IMPORT_CHECK_PATH), MediaWiki-style name canonicalization, lookup by file_id, https/public-only fetch guard, atomic first-round save in add_round (#447), WEIGHT_STRING same-name check. Removes the gist path that calls prod's own /v1/utils//file ~79 times per 15k-row CSV (today's prod kills on rounds 5051/5052). Conflict: montage/loaders.py, both hunks taken from #656: - imports: #656's list (it drops utils.unicode too, matching the previous integration fix); - load_partial_csv/load_name_list: #656 replaces them with parse_source_rows and friends; the only change on our side was the unicode -> str swap in code #656 deletes. Result is byte-identical to pr/656's loaders.py. Tests at this commit: 1 failed, 161 passed, 3 xfailed. The failure is test_round_entries_export_query_count (from perf/round-entries-export-640), which imports through POST /round/<id>/import without a check_token; #656 makes that a 400. Adapted in the next commit.
test_round_entries_export_query_count (perf/round-entries-export-640) set up its round with a bare POST /admin/round/<id>/import, which #656 turns into a 400 (import_check_required). It now uses the suite's fetch_checked_import helper from #656: check the category, then import with the check_token, as the round form does. #656 also stores a fingerprint of the check in round_sources.params (import_check.source_params adds CHECK_ID_KEY), so the CSV's source_params is no longer exactly {"category": "Batch_import"}. The test now checks the category key and that every row carries the same params string (serialized once per source, which is the point of #640). SELECT counts are unchanged after #656: list 7, download 8 at n=450. pytest montage/tests/ -q: 162 passed, 3 xfailed. Workstream-Session-Id: 01a12672-ec39-7000-9f0c-c9b9269890b3 Workstream-Plan-Id: 2026-10-10-c5476fcaee
… (beta integration) Branch tip 997a619 (not yet a PR). Ported onto #656's import path, per the integration rule that #656's import semantics win: - montage/admin_endpoints.py: #656 moved the import into _import_into_round, used by both POST /round/<id>/import and add_round's first-round import. Its body (round advance + checked methods, add_round_entries, warnings, autodisqualify) now runs inside lock_errors_as_busy('import into this round'), with coord_dao.lock_round_for_import(round_id) first and rdb_session.flush() last. G's version of the old csv/gist/category/ selected dispatch is dropped; #656 replaced it with check tokens. For add_round the round is new and uncommitted, so the lock is on the request's own row and both counts are 0. - montage/rdb.py: G's MYSQL_LOCK_ERROR_CODES / is_lock_error / lock_errors_as_busy kept verbatim; the ENV_NAME = get_env_name() line that sat next to them stays deleted, as #656 has it. - test_import_waiting_on_concurrent_import_is_refused: all three imports go through fetch_checked_import (check, then import with the check_token). It still asserts the patched _lock_round ran, the 400 text from lock_round_for_import, and that the refused import added nothing. pytest montage/tests/ -q: 163 passed, 3 xfailed. Workstream-Session-Id: 01a12672-ec39-7000-9f0c-c9b9269890b3 Workstream-Plan-Id: 2026-10-10-c5476fcaee
…gian names match Proposed change to #656, found in the backend review. normalise_name upper-cased the first character with str.upper(). Since Unicode 11 that maps Georgian Mkhedruli (U+10D0-U+10FF) to the Mtavruli capitals (U+1C90-), but MediaWiki's Language::ucfirst falls back to mb_convert_case($first, MB_CASE_TITLE) (core includes/Language/ Language.php, read 2026-10-10), and Mkhedruli's titlecase is the letter itself. So Commons stores 'ქართული.jpg' lower-case first (11,123 File: pages on the replica; prod Montage has 1,488 such entries), and normalise_name produced 'Ქართული.jpg', which file.file_name (binary(255)) never matches. Effect before this fix: every Georgian-named row of a CSV, gist, Sheet or pasted list without a file_id was classified unknown_name and left out (with a file_id it was imported but flagged renamed). Categories were unaffected (names come from Commons). master's load_name_list never touched the first letter, so this was a regression in #656. str.title() on one character is the same mapping as MB_CASE_TITLE for the cases that matter here; the len(first) == 1 guard still leaves 'ß' alone ('ß'.title() is 'Ss'). New case in test_names_are_canonical_like_mediawiki: 'ქართული.jpg' stays as is (failed before, passes now). pytest montage/tests/ -q: 164 passed, 3 xfailed. Workstream-Session-Id: 01a12672-ec39-7000-9f0c-c9b9269890b3 Workstream-Plan-Id: 2026-10-10-c5476fcaee
…er, as Commons does Follow-up to 329a85b. MediaWiki's Language::ucfirst applies mb_convert_case($first, MB_CASE_TITLE), the full Unicode titlecase mapping including the multi-character SpecialCasing expansions, unless $wgOverrideUcfirstCharacters says otherwise. Wikimedia's override table (wmf-config/UcfirstOverrides.php) pins 'ß' => 'ß' (T292552); its other 55 entries are Unicode 16/17 lowercase letters that Python 3.13's tables (Unicode 15.1) do not case either. normalise_name kept a `len(first) == 1` guard, which happened to be right for ß but wrong for the 47 other code points whose titlecase is more than one character: ʼn, ǰ, ẖ ẗ ẘ ẙ ẚ, the ff..st ligatures, Armenian և and ﬓ..ﬗ, and 27 polytonic Greek forms. Commons expands those (checked with the API today: File:fixyz.jpg -> File:Fixyz.jpg, ʼn -> ʼN, և -> Եւ, ß stays ß), so a list naming such a file could never match the replica. Now: first.title() unless first is ß. Exhaustive comparison of this rule against the Unicode 14 full titlecase data plus the WMF overrides gives no difference for any code point. Tests: dž -> Dž (digraph), an already-capital Mtavruli first letter, ʼn, fi and և expansions, and ß kept; the three expansion cases failed with the previous guard. pytest montage/tests/ -q: 170 passed, 3 xfailed. Workstream-Session-Id: 01a12715-1528-7000-961b-c78ee7776f73
…d-mean display (#617) (beta integration) Branch tip 39f9367 (PR #617, draft; master 073f0bf merged in f49e2a2, test flake fix 39f9367). Added to the integration on Mahmoud's call: the integration carries everything but the worker (#630/#631). No conflicts: git merge-tree --write-tree reported a clean tree before the merge, and the merge touches only #617's six files (rdb.py, admin_endpoints.py, tools/admin.py, test_web_basic.py, test_reassign.py, RoundEdit.vue). #632's _insert_tasks and #617's active-juror guard compose in create_initial_rating_tasks. Known follow-up (review doc section 6): edit_round/modify_quorum/ remove_juror/modify_jurors take no round lock yet; a double-submitted quorum raise on a paused round creates coverage tasks twice.
…file ID (beta integration) #656 moved after 24e747d merged it at 1206105: one commit, 4f12cf2 (en.json/qqq.json, the warnings-only check message). Merged so the PR's head is an ancestor of the integration branch again and GitHub marks #656 merged when the integration PR merges with a merge commit. No conflicts, i18n only.
This was referenced Oct 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Everything with the
final reviewlabel plus #617 and four small ops/perf fixes, integrated on master (073f0bf) with one merge per PR so each PR's conflict resolution is in its merge commit message. Merge with a merge commit so GitHub marks the contained PRs merged.Contains
strfor the removedbasestring/unicodeimports), Show resource names, not config values, on /meta; front-end error log for maintainers only #647normalise_nametitlecases the first letter the way MediaWiki'sucfirstdoes (Georgian names were "not on Commons"; ß stays, as on Wikimedia)clastic_sentryon the 2.x scope API; Sentry has been blind on Python 3.13 (sentry-sdk crashes on Python 3.13 and hides the real error (cannot pickle 'FrameLocalsProxy') #642, Sentry SDK fails on worker timeouts under Python 3.13 (cannot pickle 'FrameLocalsProxy' object) #653)--timeout=120instead of the 30 s default that kills large imports (Large category import killed by gunicorn's 30 s worker timeout (buildservice deploy) #618, Category imports time out on production: every file-name lookup reads the whole entries table #650)Not included: the import worker (#630, #631); #587 and #533.
Verified
pytest montage/tests/ -q: 171 passed, 3 xfailed at 33c0c00; frontendformat:check,lint:check,buildclean.Deploy notes
MONTAGE_IMPORT_CHECK_PATHmust be set (it is on all three tools).ix_entries_namerestored (Category imports time out on production: every file-name lookup reads the whole entries table #650).Known follow-ups
add_roundwith an import is not serialized: a retry sent while the first Save is still running creates a second round with the same files. Low likelihood now that the Save is seconds; a campaign row lock increate_roundfixes it.edit_round/modify_quorum/remove_juror/modify_jurorstake no round lock yet (Lock the round when activating it, so a double click can't create every task twice #626 pattern).UserIPMiddleware's ip never reaches Sentry events withoutsend_default_pii.Closes #635, #640, #642, #653.