diff --git a/.github/workflows/security.yml b/.github/workflows/security.yml index 3595b537..279c076e 100644 --- a/.github/workflows/security.yml +++ b/.github/workflows/security.yml @@ -23,6 +23,8 @@ jobs: with: {python-version: "3.13", cache: pip, cache-dependency-path: requirements/dev.txt} - run: pip install --require-hashes -r requirements/dev.txt - run: python scripts/audit_dependencies.py + # The third-party inventory and notices match a regeneration in the image's base (REVIEW_1_33_PLAN.md K1). + - run: scripts/third_party.sh --check lock-current: # requirements/*.txt must match pyproject.toml: a dependency added without re-locking fails. diff --git a/.github/workflows/tests.yml b/.github/workflows/tests.yml index 15b29ac8..9b75404a 100644 --- a/.github/workflows/tests.yml +++ b/.github/workflows/tests.yml @@ -79,7 +79,7 @@ jobs: image: ghcr.io/chaffed/openclear-ci@sha256:921ef8081264154aa92f27fd30013a2aa1f447ec1636528c4ad77a11ddb37e83 # py3.13 services: postgres: - image: postgres:16 + image: postgres:16@sha256:1beaa3084eee2676f20a9cd9928f0bf2c5bf674f8e49f81017912183d377c61c # 16, pinned (REVIEW_1_33_PLAN.md K9) env: POSTGRES_USER: openclear POSTGRES_PASSWORD: openclear diff --git a/CHANGELOG.md b/CHANGELOG.md index 1f6af602..28f1992a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,86 @@ # Changelog +## Unreleased + +- **A check paid by a decision is paid.** Until now only a check that matched its issued item + cleanly was recorded as paid; one paid by a decision (an accepted amount mismatch, or every + check on a reverse positive pay account, paid at the cut-off by silence) stayed pending, so the + same check presented again was the same item again, and on a reverse account was paid again. + From the cut-off, a paid check is paid, its issued item with it, and a returned check returned, + so a second presentment is a duplicate wherever it comes from; the upgrade brings every + decision already final into line. *Pay all N to review* now pays exactly the checks the page + listed, not any N. A teller platform that lost the answer to `POST /api/v1/checks/cashed` is + told, on its retry, which paid item the first call recorded and when + (`plans/REVIEW_1_33_PLAN.md` RV1). +- **The teller answer gives nothing the caller didn't send.** A decline for a payee the teller + didn't send no longer carries the payee it's asking for, and `issued` names the payee only + when the one sent differs. A check another teller verified in the last 15 minutes and hasn't + recorded cashed is declined `being_cashed`, so two branches can't both pay it out. Ten declines + of any kind within an hour (not only "not issued": a wrong amount reveals the issued amount) + raise `teller.verify.probing`, counted by teller, branch, API client, and now by the person + when a person's token asks. A check on an account whose customer's service has ended answers + `not_on_positive_pay`, as the guide said, instead of failing (RV2). +- **A check that arrives late gets a look before a default decides it.** Its decision deadline + is never later than the cut-off on its return day; when a file reaches OpenClear after that + cut-off (a core posting yesterday's items mid-morning), the item was due at once and the + customer's default decision applied at the next sweep, unseen. It now gets the bank's minimum + review time, never past the midnight deadline. A bank that opens on the Federal Reserve's + holidays now has its cut-off on them too, as the setting said. The decisions CSV's columns are + listed in the BI guide, with the `after_return_deadline` column 1.29.0 added at the end (RV3). +- **Deleting a bank's data clears every storage area.** The deletion cleared five of the seven: + the decision files its core took, and fraud-scoring artifacts, stayed. It now clears every + area from storage's own list, the certificate names them all, and a test walks the list. The + bank is frozen as the deletion job starts (sign-ins and tokens refused, its queued jobs gone), + so nothing is added while the rows go. A deletion the queue gives up on is marked failed, shown + on the bank's offboarding page and raised as a critical event, instead of showing as running + at a time long past. Offboarding a bank, deleting its data and ending a customer's positive pay + always need a second person: with nobody else able to approve, the request is refused rather + than applied after the day's cooling-off (RV4). +- **A customer's positive pay ends at midnight on its last day, everywhere.** Sign-in, files + naming its accounts and teller answers now read the last day itself, not the daily run that + follows it (which could be a morning later), and the end is recorded as the end of that day, so + the 90-day clock and every page agree with the day the customer was told. A core's customer + sync can no longer bring an ended customer back: `PUT /api/v1/customers/{number}` with + `is_active: true` is refused with 409, and restoring is the two-administrator change on the + customer's page. Ending service signs its people out of this bank only. When a customer's data + is deleted, the bank's attestations and access reviews about it stay, no longer naming it; a + person kept because the action log names them keeps nothing personal (no email, phone, + password or sessions). Dates on the customer page, the banner and the Deleted customers list are + the bank's days (RV5). +- **Custom fields: numbers, limits, files and messages.** A negative number in a `cf_` column of + the decisions CSV reached the core as `'-5`, the spreadsheet-formula guard's doing; a plain + signed number now passes as one. A customer's reader asking the decision feed for a field not + shown to the customer is told there's no such field, as for one that doesn't exist. Restoring an + archived field counts against the 25-field and 3-in-list limits. An update file naming a field + the bank doesn't have, or has archived, is refused whole instead of failing every row; a text + cell reading `NA`, `null` or `None` in such a file is that text, not blank. A whole number takes + only the digits 0–9, a date only `YYYY-MM-DD`, a decimal no `_`, as their messages said; a form + reports every field's problem at once; a dropped choice option is retired only while a record + holds it. Field types read as words ("Whole number", "Yes/no"). **Rate limiting with Redis:** + while Redis can't be reached each instance now limits on its own instead of allowing + everything, the outage is a `rate_limit.backend_down` security event (once per outage), and the + Controls page says so (new control OC-SC-08) (RV6). +- **Incident contacts and notices.** A bank may name only its CEO and CIO, the fallback 12 CFR + 304.24 gives, and notices then go to them (the page and the control said so; the code required a + named contact). The form shows a row for each contact and one more, up to ten. The yearly + confirmation is a security event (`incident_contact.confirmed`). The platform's Sent table says + which banks each notice reached and how many addresses, and marks a bank with nobody to email; + "Offboarding" is no longer a kind an operator can pick. A check refused at a branch is described + in words on the customer's Exceptions page ("issued for another amount"). The SAML signing-key + page says that finishing before the identity provider has the new certificate stops sign-in and + can't be undone, and that withdrawing after it switched needs the old certificate back (RV7). +- **Refusals are sentences written for people.** Every page that shows why something was refused + now shows the sentence OpenClear wrote for it; an exception from a library or a database driver + shows a fixed sentence instead and is logged (the class of finding CodeQL #176 raised, closed + everywhere rather than in one place). Replacing a customer's SAML signing key from the bank's + side goes through the customer's own second-person rule. **For operators:** `scripts/third_party.sh + --check` regenerates the third-party inventory and notices and fails when the committed copies + differ; the Security workflow runs it. The tests' Postgres image is pinned by digest. A + posted-check file with an absurdly long number in an amount or a trailer is refused as a row or + a trailer problem, not an error (RV8). + +- **Dependencies:** pyasn1 0.6.5 (CVE-2026-119197, CVE-2026-119195) in the image and for development. + ## 1.33.0 — 2026-10-30 - **Third-party components** (`plans/THIRD_PARTY_PLAN.md`). A page listing everything OpenClear diff --git a/THIRD_PARTY_NOTICES.md b/THIRD_PARTY_NOTICES.md index 7d4de1ac..895947c0 100644 --- a/THIRD_PARTY_NOTICES.md +++ b/THIRD_PARTY_NOTICES.md @@ -8919,7 +8919,7 @@ Library. `LICENSE.txt`: the same text as psycopg's, above. -### pyasn1 0.6.4 (BSD-2-Clause) +### pyasn1 0.6.5 (BSD-2-Clause) `LICENSE.rst`: diff --git a/docs/compliance/asvs.md b/docs/compliance/asvs.md index 2d2715a8..6f7b8ef5 100644 --- a/docs/compliance/asvs.md +++ b/docs/compliance/asvs.md @@ -65,7 +65,7 @@ Requirements not fully met are recorded with their reason, here and in [OWASP_PL | **V2.3.1** Verify that the application will only process business logic flows for the same user in the expected sequential step order and without skipping steps. | 1 | Met | `src/openclear/services/change_request_service.py`, OC-AC-05, OC-AC-06 | | | **V2.3.2** Verify that business logic limits are implemented per the application's documentation to avoid business logic flaws being exploited. | 2 | Met | `src/openclear/services/change_request_service.py`, OC-AC-05, OC-AC-06 | | | **V2.3.3** Verify that transactions are being used at the business logic level such that either a business logic operation succeeds in its entirety or it is rolled back… | 2 | Met | `src/openclear/services/change_request_service.py`, OC-AC-05, OC-AC-06 | | -| **V2.3.4** Verify that business logic level locking mechanisms are used to ensure that limited quantity resources (such as theater seats or delivery slots) cannot be do… | 2 | Met | `src/openclear/repositories/exception_repo.py`, `src/openclear/services/decision_service.py`, `src/openclear/services/decision_outbox_service.py`, `test_two_decisions_at_once_one_wins_and_the_other_is_told`, `test_a_check_presented_twice_at_once_is_paid_once`, `test_two_outbox_runs_at_once_write_one_file` | A row lock per item for every decision change, a check paid once however many times it's presented, and one writer per bank for decision files (FULL_REVIEW_PLAN.md FR1-FR3), each proved by two sessions at once on Postgres. | +| **V2.3.4** Verify that business logic level locking mechanisms are used to ensure that limited quantity resources (such as theater seats or delivery slots) cannot be do… | 2 | Met | `src/openclear/repositories/exception_repo.py`, `src/openclear/services/decision_service.py`, `src/openclear/services/decision_outbox_service.py`, `test_two_decisions_at_once_one_wins_and_the_other_is_told`, `test_a_check_presented_twice_at_once_is_paid_once`, `test_two_outbox_runs_at_once_write_one_file`, `test_a_check_cashed_while_its_file_presentment_is_in_flight_is_paid_once`, `test_one_verification_cashed_twice_at_once_records_one_payment` | A row lock per item for every decision change, a check paid once however many times it's presented, and one writer per bank for decision files (FULL_REVIEW_PLAN.md FR1-FR3), each proved by two sessions at once on Postgres. | | **V2.4.1** Verify that anti-automation controls are in place to protect against excessive calls to application functions that could lead to data exfiltration, garbage-d… | 2 | Met | `src/openclear/web/rate_limit.py` | | ## V3 Web Frontend Security diff --git a/docs/compliance/controls.md b/docs/compliance/controls.md index 266813c0..f509529c 100644 --- a/docs/compliance/controls.md +++ b/docs/compliance/controls.md @@ -62,6 +62,7 @@ Inventory version 1.3. The same list, with each check's live status for your ban |---|---|---|---|---|---|---|---| | OC-SC-01 | **Secrets encrypted in the database.** Secrets kept in the database (SSO keys, delivery credentials, plugin settings) are encrypted with AES-256-GCM under versioned keys, and re-encrypted when a key is rotated. | **SC-28(1)** Cryptographic Protection
**SC-12** Cryptographic Key Establishment and Management
**SC-13** Cryptographic Protection | PR.DS-01, PR.DS-02, PR.DS-10 | preventive, built in | `field_encryption` (live)
`key_age` (live) | H-SC-12-01 | auth/crypto.py | | OC-SC-02 | **Encryption at rest stated.** The host's statement of how disks and the database are encrypted is shown on the Controls page, which warns until there is one. | **SC-28** Protection of Information at Rest | PR.DS-01 | detective, built in | `storage_encryption` (live) | H-SC-28(1)-01 | Controls → Storage encryption | +| OC-SC-08 | **Sign-in and API attempts rate-limited.** Sign-in, token and password-reset attempts are limited per address and per route; with several instances the counts are shared through Redis, and while Redis can't be reached each instance limits on its own and the Controls page says so. | **SC-5** Denial-of-service Protection
**AC-7** Unsuccessful Logon Attempts | PR.AA-03, PR.IR-01, DE.CM-01 | preventive, built in | `rate_limit_backend` (live) | — | web/rate_limit.py; Controls → Rate limits shared across instances | | OC-SC-03 | **Files signed.** Every uploaded file is kept with its SHA-256 and a signature, and every webhook and evidence bundle OpenClear sends is signed. | **SI-7(6)** Cryptographic Protection
**SC-8** Transmission Confidentiality and Integrity | ID.RA-09, PR.DS-01, PR.DS-02, PR.DS-10, PR.PS-02, DE.CM-09 | preventive, built in | — | — | An upload's page; webhooks | | OC-SC-04 | **Browser protections.** Cookies are HTTP-only, and secure over HTTPS, which is marked with HSTS; every form carries a CSRF token; and a content security policy limits scripts to OpenClear's own. | **SC-23** Session Authenticity
**SC-8** Transmission Confidentiality and Integrity
**SC-18** Mobile Code | PR.DS-02 | preventive, built in | `transport_security` (live) | H-SC-8(1)-01 | web/security.py | | OC-SC-05 | **Edge trusted only as configured.** The caller's address is taken only from the configured edge's header, an edge secret can be required, and the Controls page shows the edge in front. | **SC-7** Boundary Protection | PR.DS-01, PR.DS-02, PR.DS-10, PR.IR-01, DE.CM-01 | preventive, configurable | `edge` (live) | H-SC-7-01 | Controls → Edge in front; docs/guides/edge.md | diff --git a/docs/compliance/csf.md b/docs/compliance/csf.md index 25580cde..81319212 100644 --- a/docs/compliance/csf.md +++ b/docs/compliance/csf.md @@ -82,7 +82,7 @@ NIST's mapping links no subcategory to `MP-6`, `AU-10`, `MA-4`, `SC-18`, `SC-23` |---|---|---| | **PR.AA-01** Identities and credentials for authorized users, services, and hardware are managed by the organization | OC-AC-02, OC-IA-01, OC-IA-02, OC-IA-03, OC-IA-04, OC-IA-05, OC-SI-13 | B-AC-2-01, B-AC-2(1)-01, B-IA-2(1)-01, B-IA-2(2)-01, B-IA-2(2)-02, B-IA-2(2)-03, B-IA-5(1)-01, B-IA-5-01, B-AC-2-02, B-IA-8-01, C-AC-2-01, C-AC-2-02, C-AC-2(1)-01, C-IA-2-01, C-IA-5-01 | | **PR.AA-02** Identities are proofed and bound to credentials based on the context of interactions | — | B-IA-12-01 | -| **PR.AA-03** Users, services, and hardware are authenticated | OC-AC-10, OC-IA-01, OC-IA-02, OC-IA-03, OC-IA-04, OC-IA-05, OC-SC-06 | B-IA-2(1)-01, B-IA-2(2)-01, B-IA-2(2)-02, B-IA-2(2)-03, B-IA-5(1)-01, B-IA-5-01, B-IA-8-01, C-IA-2-01, C-IA-5-01 | +| **PR.AA-03** Users, services, and hardware are authenticated | OC-AC-10, OC-IA-01, OC-IA-02, OC-IA-03, OC-IA-04, OC-IA-05, OC-SC-08, OC-SC-06 | B-IA-2(1)-01, B-IA-2(2)-01, B-IA-2(2)-02, B-IA-2(2)-03, B-IA-5(1)-01, B-IA-5-01, B-IA-8-01, C-IA-2-01, C-IA-5-01 | | **PR.AA-04** Identity assertions are protected, conveyed, and verified | — | — | | **PR.AA-05** Access permissions, entitlements, and authorizations are defined in a policy, managed, enforced, and reviewed, and incorporate the principles of least privilege and separation of duties | OC-AC-01, OC-AC-02, OC-AC-03, OC-AC-04, OC-AC-05, OC-AC-06, OC-AC-07, OC-AC-08, OC-AC-09, OC-AC-12, OC-AC-13, OC-IA-03, OC-SI-08, OC-SI-09, OC-SI-13 | B-AC-2-01, B-AC-2(1)-01, B-AC-6(7)-01, B-AC-6-01, B-AC-5-01, B-AC-3(2)-01, B-AC-2-02, C-AC-2-01, C-AC-2-02, C-AC-2(1)-01, C-AC-6(7)-01, C-AC-5-01, C-AC-3-01, H-AC-6(5)-01 | | **PR.AA-06** Physical access to assets is managed, monitored, and enforced commensurate with risk | — | — | @@ -98,7 +98,7 @@ NIST's mapping links no subcategory to `MP-6`, `AU-10`, `MA-4`, `SC-18`, `SC-23` | **PR.PS-04** Log records are generated and made available for continuous monitoring | OC-AU-01, OC-AU-03, OC-AU-04, OC-AU-05, OC-SI-10 | B-AU-6-01, B-AU-11-01, B-AU-11-02 | | **PR.PS-05** Installation and execution of unauthorized software are prevented | — | — | | **PR.PS-06** Secure software development practices are integrated, and their performance is monitored throughout the software development life cycle | — | B-SA-11-01 | -| **PR.IR-01** Networks and environments are protected from unauthorized logical access and usage | OC-AC-01, OC-AC-03, OC-AC-04, OC-AC-06, OC-AC-07, OC-AC-08, OC-AC-09, OC-AC-12, OC-AC-13, OC-SC-05, OC-SC-06, OC-SC-07, OC-SI-08 | B-AC-3(2)-01, C-AC-3-01, H-SC-7-01 | +| **PR.IR-01** Networks and environments are protected from unauthorized logical access and usage | OC-AC-01, OC-AC-03, OC-AC-04, OC-AC-06, OC-AC-07, OC-AC-08, OC-AC-09, OC-AC-12, OC-AC-13, OC-SC-08, OC-SC-05, OC-SC-06, OC-SC-07, OC-SI-08 | B-AC-3(2)-01, C-AC-3-01, H-SC-7-01 | | **PR.IR-02** The organization's technology assets are protected from environmental threats | OC-SI-05, OC-CP-01 | B-CP-2-02, B-CP-2-01, B-CP-2-03, H-CP-2-01 | | **PR.IR-03** Mechanisms are implemented to achieve resilience requirements in normal and adverse situations | OC-SI-05, OC-SI-07, OC-CM-02, OC-CP-01, OC-IR-01, OC-IR-02 | B-IR-6-03, B-IR-6-04, B-IR-6-05, B-IR-6-06, B-IR-6-07, B-IR-6-08, B-IR-6-02, B-CP-2-02, B-CP-2-01, B-CP-2-03, B-IR-6-01, C-IR-6-01, H-CP-9-01, H-CP-2-01, H-IR-6-01, H-IR-6-02, H-IR-8-01 | | **PR.IR-04** Adequate resource capacity to ensure availability is maintained | — | — | @@ -107,7 +107,7 @@ NIST's mapping links no subcategory to `MP-6`, `AU-10`, `MA-4`, `SC-18`, `SC-23` | Subcategory | OpenClear controls | CUECs | |---|---|---| -| **DE.CM-01** Networks and network services are monitored to find potentially adverse events | OC-AC-02, OC-AC-07, OC-AC-13, OC-IA-03, OC-AU-01, OC-AU-04, OC-SC-05, OC-SC-06, OC-SC-07, OC-SI-03, OC-SI-05, OC-SI-09, OC-SI-10, OC-SI-11, OC-SI-13, OC-CM-02, OC-CA-01, OC-CA-02, OC-CA-03 | B-AC-2-01, B-AC-2(1)-01, B-CM-3-01, B-AC-2-02, B-CA-7-01, B-CA-7-02, B-SI-4-01, B-SI-4-02, B-SI-4-03, C-AC-2-01, C-AC-2-02, C-AC-2(1)-01, C-SI-4-02, C-SI-4-01, H-SC-7-01, H-SI-4-01 | +| **DE.CM-01** Networks and network services are monitored to find potentially adverse events | OC-AC-02, OC-AC-07, OC-AC-13, OC-IA-03, OC-AU-01, OC-AU-04, OC-SC-08, OC-SC-05, OC-SC-06, OC-SC-07, OC-SI-03, OC-SI-05, OC-SI-09, OC-SI-10, OC-SI-11, OC-SI-13, OC-CM-02, OC-CA-01, OC-CA-02, OC-CA-03 | B-AC-2-01, B-AC-2(1)-01, B-CM-3-01, B-AC-2-02, B-CA-7-01, B-CA-7-02, B-SI-4-01, B-SI-4-02, B-SI-4-03, C-AC-2-01, C-AC-2-02, C-AC-2(1)-01, C-SI-4-02, C-SI-4-01, H-SC-7-01, H-SI-4-01 | | **DE.CM-02** The physical environment is monitored to find potentially adverse events | OC-CA-01, OC-CA-02, OC-CA-03 | B-CA-7-01, B-CA-7-02 | | **DE.CM-03** Personnel activity and technology usage are monitored to find potentially adverse events | OC-AC-02, OC-IA-03, OC-AU-01, OC-AU-04, OC-SI-13, OC-CA-01, OC-CA-02, OC-CA-03 | B-AC-2-01, B-AC-2(1)-01, B-AC-2-02, B-CA-7-01, B-CA-7-02, C-AC-2-01, C-AC-2-02, C-AC-2(1)-01 | | **DE.CM-06** External service provider activities and services are monitored to find potentially adverse events | OC-AU-04, OC-SI-03, OC-SI-05, OC-SI-09, OC-SI-10, OC-SI-11, OC-CA-01, OC-CA-02, OC-CA-03 | B-CA-7-01, B-CA-7-02, B-SI-4-01, B-SI-4-02, B-SI-4-03, B-SA-4-01, B-SA-9-02, B-SA-9-03, B-SA-9-01, C-SI-4-02, C-SI-4-01, H-SI-4-01 | diff --git a/docs/compliance/third-party.md b/docs/compliance/third-party.md index ad4ea153..31ef26a4 100644 --- a/docs/compliance/third-party.md +++ b/docs/compliance/third-party.md @@ -72,7 +72,7 @@ Installed from the hashed lock `requirements/runtime.txt` and nothing else. | propcache | 0.5.4 | Apache-2.0 | for aiohttp, yarl | | psycopg | 3.3.6 | LGPL-3.0-only | OpenClear | | psycopg-binary | 3.3.6 | LGPL-3.0-only | for psycopg | -| pyasn1 | 0.6.4 | BSD-2-Clause | for pyasn1-modules, webauthn | +| pyasn1 | 0.6.5 | BSD-2-Clause | for pyasn1-modules, webauthn | | pyasn1-modules | 0.4.2 | BSD | for webauthn | | pycparser | 3.12 | BSD-3-Clause | for cffi | | pydantic | 2.14.1 | MIT | OpenClear | diff --git a/docs/guides/bi.md b/docs/guides/bi.md index 389bb523..2667dba4 100644 --- a/docs/guides/bi.md +++ b/docs/guides/bi.md @@ -74,6 +74,14 @@ export is recorded as a security event, so your SOC sees the pipeline's activity | Accounts and customers | `/api/v1/accounts`, `/api/v1/customers` | The same. | | Everything, once | `POST /api/v1/data-exports`, then poll `GET /api/v1/data-exports/{id}` and download its `file_url` | A first load, or a periodic archive. | | Users and their access (for access reviews) | `/api/v1/users` | Full load. | + +The decisions CSV (`feed.csv`, `GET /api/v1/decisions/file`, and the OpenClear CSV decision file a +core takes) has these columns, in this order: `sequence`, `network`, `outcome`, `reason_code`, +`return_transaction_code`, `account_number`, `customer_number`, `check_number`, `trace_number`, +`amount`, `presented_date`, `effective_date`, `originator_id`, `originator_name`, `sec_code`, +`source`, `decided_by`, `decided_at`, `decision_id`, `exception_id`, `after_return_deadline` +(added in 1.29.0), then a `cf_` column for each custom field the file or request asks for. +A release may add a column at the end; read the file by its header row, not by position. | Billing | `/api/v1/billing/statements?period=YYYY-MM`, `/api/v1/billing/report.csv` | Per period, after it's finalized. | | Controls | `/api/v1/controls/status` | A daily snapshot, to chart controls over time. | diff --git a/docs/guides/teller.md b/docs/guides/teller.md index cba6b018..8a4471e0 100644 --- a/docs/guides/teller.md +++ b/docs/guides/teller.md @@ -29,11 +29,13 @@ teller platform ──POST /api/v1/checks/cashed──▶ OpenClear { paid_ite | `stop_differs` | A stop is on file for this check number, but for another amount: the customer decides | | `already_paid` | Paid already: in clearing, or cashed at another branch | | `customer_review` | The account is on reverse positive pay: the customer reviews each check | + | `being_cashed` | Another teller verified this check in the last 15 minutes and hasn't recorded it cashed: follow the bank's procedure | | `payee_needed` | The customer gave a payee for this check: send the payee as written, and verify again | | `not_on_positive_pay` | The account isn't on positive pay: OpenClear doesn't know it, or its customer is no longer enrolled | - `problems` lists every problem found. `issued` (amount, payee, issue date) comes only with a - decline, to help the teller explain it. + `problems` lists every problem found. `issued` (amount and issue date; the payee only when + the one sent differs) comes only with a decline, to help the teller explain it, and never + when the only problem is a payee that wasn't sent. 2. **Pay out only on `match`.** On an account enrolled in positive pay or reverse positive pay, anything else is declined: a check that isn't found is never cashed. `not_on_positive_pay` is the exception: the account isn't enrolled, so the teller cashes it by the bank's standard @@ -67,4 +69,8 @@ teller platform ──POST /api/v1/checks/cashed──▶ OpenClear { paid_ite - **Plugins.** The bank's matching-rule plugins run in clearing, on a presented check, not on a verification. - **Two branches at once.** Verify takes the same lock on the issued check that clearing takes, - so a check verified at one branch while it's being cashed at another can't both be paid. + so two verifications of one check take turns: the second is answered `being_cashed` while the + first is unexpired and not yet recorded cashed, and `already_paid` once it is. A check cashed + at a branch while its file presentment is in flight is paid once, whichever came first. +- **Probing.** Ten declines of any kind within an hour to one teller, branch, client or person + raise a high security event (`teller.verify.probing`) for the bank's security team. diff --git a/migrations/versions/b8c4d2e6f1a3_check_settlement_backfill.py b/migrations/versions/b8c4d2e6f1a3_check_settlement_backfill.py new file mode 100644 index 00000000..3b33c2a3 --- /dev/null +++ b/migrations/versions/b8c4d2e6f1a3_check_settlement_backfill.py @@ -0,0 +1,58 @@ +# SPDX-License-Identifier: AGPL-3.0-or-later +# Copyright (C) 2026 Chaffed + +"""Checks paid or returned by a decision are settled (REVIEW_1_33_PLAN.md RV-D1) + +Until 1.34.0 only a clean match set a paid item's settlement and its issued item's status, so a +check decided *pay* (every check on a reverse positive pay account) stayed pending and could be +presented again without being a duplicate. decision_service.lock now settles the check; this +brings every decision already final into line: its paid item is paid or returned, and a paid +check's outstanding issued item is paid. Reversed paid items and voided issued items are left as +they are. + +Revision ID: b8c4d2e6f1a3 +Revises: e7a3b9d5c2f1 +Create Date: 2026-10-31 +""" +import uuid +from typing import Sequence, Union + +from alembic import op +import sqlalchemy as sa + + +revision: str = 'b8c4d2e6f1a3' +down_revision: Union[str, None] = 'e7a3b9d5c2f1' +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + +# Enum columns store the member names (native_enum=False, SQLAlchemy's default lookup). +_DECIDED = """ + SELECT e.{column} FROM exception_item e JOIN decision d ON d.exception_item_id = e.id + WHERE e.tenant_id = :t AND e.network_code = 'check' AND d.locked_at IS NOT NULL AND d.outcome = :outcome + AND e.{column} IS NOT NULL +""" + + +def _each_bank(bind): + postgres = bind.dialect.name == 'postgresql' + for (tenant_id,) in bind.execute(sa.text('SELECT id FROM tenant')).all(): + if postgres: # row-level security is forced on paid_item and issued_item + bind.execute(sa.text("SELECT set_config('app.current_tenant_id', :t, true)"), {'t': str(tenant_id)}) + yield str(tenant_id if isinstance(tenant_id, uuid.UUID) else uuid.UUID(str(tenant_id))) + + +def upgrade() -> None: + bind = op.get_bind() + for tenant_id in _each_bank(bind): + for outcome, settlement in (('PAY', 'PAID'), ('RETURN', 'RETURNED')): + bind.execute(sa.text( + "UPDATE paid_item SET settlement_status = :s WHERE tenant_id = :t AND settlement_status = 'PENDING' AND id IN (" + + _DECIDED.format(column='source_item_id') + ")"), {'s': settlement, 't': tenant_id, 'outcome': outcome}) + bind.execute(sa.text( + "UPDATE issued_item SET status = 'PAID' WHERE tenant_id = :t AND status IN ('OUTSTANDING', 'STALE', 'STOPPED') AND id IN (" + + _DECIDED.format(column='related_reference_id') + ")"), {'t': tenant_id, 'outcome': 'PAY'}) + + +def downgrade() -> None: + pass # a settled check stays settled: the decisions behind these rows are final diff --git a/migrations/versions/c9d5e3f7a2b4_teller_verification_user.py b/migrations/versions/c9d5e3f7a2b4_teller_verification_user.py new file mode 100644 index 00000000..e152fb94 --- /dev/null +++ b/migrations/versions/c9d5e3f7a2b4_teller_verification_user.py @@ -0,0 +1,34 @@ +# SPDX-License-Identifier: AGPL-3.0-or-later +# Copyright (C) 2026 Chaffed + +"""teller_verification.user_id: whose token asked (REVIEW_1_33_PLAN.md T6) + +A teller platform's verifications carry its API client; a person given check:verify had nothing +to be counted by, so the probing detector couldn't see them. Null for rows before this. + +Revision ID: c9d5e3f7a2b4 +Revises: b8c4d2e6f1a3 +Create Date: 2026-10-31 +""" +from typing import Sequence, Union + +from alembic import op +import sqlalchemy as sa + + +revision: str = 'c9d5e3f7a2b4' +down_revision: Union[str, None] = 'b8c4d2e6f1a3' +branch_labels: Union[str, Sequence[str], None] = None +depends_on: Union[str, Sequence[str], None] = None + + +def upgrade() -> None: + with op.batch_alter_table('teller_verification') as batch: + batch.add_column(sa.Column('user_id', sa.Uuid(), nullable=True)) + batch.create_foreign_key('fk_teller_verification_user_id', 'user', ['user_id'], ['id']) + + +def downgrade() -> None: + with op.batch_alter_table('teller_verification') as batch: + batch.drop_constraint('fk_teller_verification_user_id', type_='foreignkey') + batch.drop_column('user_id') diff --git a/openapi/openclear-v1.json b/openapi/openclear-v1.json index 020a4fd6..daf25a2f 100644 --- a/openapi/openclear-v1.json +++ b/openapi/openclear-v1.json @@ -5068,7 +5068,7 @@ "type": "object" }, "IssuedDetails": { - "description": "What was issued, to explain a decline. Never sent for a match.", + "description": "What was issued, to explain a decline. Never sent for a match, nor when the only problem\nis the payee the teller didn't send; the payee comes only when the one sent differs.", "properties": { "amount": { "pattern": "^(?!^[-+.]*$)[+-]?0*\\d*\\.?\\d*$", @@ -5081,13 +5081,19 @@ "type": "string" }, "payee_name": { - "title": "Payee Name", - "type": "string" + "anyOf": [ + { + "type": "string" + }, + { + "type": "null" + } + ], + "title": "Payee Name" } }, "required": [ "amount", - "payee_name", "issue_date" ], "title": "IssuedDetails", @@ -6650,6 +6656,7 @@ "payee_differs", "stale", "customer_review", + "being_cashed", "payee_needed" ], "type": "string" @@ -6671,6 +6678,7 @@ "payee_differs", "stale", "customer_review", + "being_cashed", "payee_needed" ], "title": "Result", @@ -9908,7 +9916,7 @@ "description": "No such verification" }, "409": { - "description": "Declined, already recorded, expired, or another amount" + "description": "Declined, expired, or another amount; or already recorded, in which case `detail` is an object with `message`, `paid_item_id` and `cashed_at`, so a platform that lost the first answer can reconcile" }, "422": { "content": { diff --git a/plans/MASTER_PLAN.md b/plans/MASTER_PLAN.md index 3c1e914d..035e9bc4 100644 --- a/plans/MASTER_PLAN.md +++ b/plans/MASTER_PLAN.md @@ -173,3 +173,5 @@ zone, M7 is still built and tested, against a stub edge; only the final confirma - 2026-10-30 Released 1.33.0: the third-party inventory (THIRD_PARTY_PLAN.md: the components page, the notices in the app and the image, an image SBOM on each release); housekeeping (HOUSEKEEPING_PLAN.md: CodeQL #176 fixed, ASVS 2.3.4 met, the image 1.4 GB instead of 2.4). No security advisories. - 2026-10-22 REGULATORY_PLAN.md written: the payments law and guidance not yet evaluated (UCC 3/4, Regulation CC, Check 21 and ECCHO, Regulation E, Nacha's 2026 fraud monitoring, two more FFIEC booklets, the CRI Profile, GLBA, NYDFS, FinCEN, WCAG, the AI RMF, CFPB 1033, OFAC), how each is evaluated, and stages RG1–RG6. +- 2026-10-31 REVIEW_1_33_PLAN.md written: a review of what 1.28.0 to 1.33.0 added (teller verification, reverse positive pay, posted-check layouts, the regulatory plan, offboarding, custom fields, the SAML re-key, the third-party inventory), five clusters read end to end and every High and Medium re-checked against the code. Three Highs (a check paid after a decision is never marked paid, so reverse positive pay can pay it twice; decision files survive bank deletion; a core's sync reactivates an ended customer), fifteen Mediums, with stages RV1–RV8. Nothing built yet. +- 2026-10-31 REVIEW_1_33_PLAN.md RV1–RV8 built on `review-1-33`: a check paid by a decision is paid (so reverse positive pay can't pay it twice); the teller answer gives nothing the caller didn't send; a late check gets a look before a default; bank deletion clears every storage area, freezes the bank and reports failure; a customer's service ends at midnight everywhere and a sync can't bring it back; custom fields by the letter of their messages; the Redis limiter holds per instance; incident contacts' fallback; refusals as sentences for people. Waiting: the suites on the branch head, then a pull request. diff --git a/plans/REVIEW_1_33_PLAN.md b/plans/REVIEW_1_33_PLAN.md new file mode 100644 index 00000000..37780068 --- /dev/null +++ b/plans/REVIEW_1_33_PLAN.md @@ -0,0 +1,346 @@ +# Review plan: what 1.28 to 1.33 added + +A review of OpenClear 1.33.0 (`0bafc661`, 2026-10-31) covering what changed since 1.27.0: +teller verification, reverse positive pay and posted-check layouts (1.28), the regulatory plan +(1.29), offboarding a bank and a customer (1.30), custom fields (1.31), the SAML re-key (1.32), +and the third-party inventory, housekeeping and build changes (1.33). 104 commits, about 7,500 +lines of source. Each cluster was read end to end by a reviewer with the same brief (the five +axes below, the severity scale, "trace it or mark it PLAUSIBLE"), and every High and Medium was +then checked again against the code by hand. The 1.26.0 review (FULL_REVIEW_PLAN.md) is not +repeated: its locks, claims and leases were spot-checked where the new code touches them, and +hold. + +**The short version.** The new code uses the 1.27 patterns where it should: the teller +endpoint takes the issued-item lock, `cashed` claims its verification once, the cut-off sweep and +*Pay all* go through `get_for_update`, every new table has its RLS policy, every new route its +permission and CSRF, and the deletion code enumerates tables from the metadata so nothing added +later is missed. The weaknesses are in what the code *doesn't* do rather than in races. A check +exception decided *pay* never marks the paid item or the issued item paid, so on a reverse +positive pay account the same check can be paid twice by silence. Bank deletion wipes every row +but not every storage area: the decision files the core received stay on disk. A core's routine +customer sync reactivates an offboarded customer while its deletion stays scheduled. The second +tier of findings is about promises the code keeps only on paper: "two operators" degrades to one +operator plus 24 hours when there's no second one; "a check is never due after midnight" means a +late-posted file's exceptions are defaulted at the next sweep with no review; the teller +permission that "can't download a check register" answers each probe with the check's amount and +payee; the third-party page's license texts are never compared against a regeneration in any +test that runs. + +Finding ids keep the reviewer's prefix: **T** teller, reverse and layouts · **G** regulatory · +**O** offboarding · **C** custom fields · **K** SAML, third-party, build. + +## Findings + +Severity: **High** = money or compliance impact with no mitigation, or a real security +weakening; **Medium** = wrong behaviour in a reachable case, or a hardening a bank's reviewer +would ask for; **Low** = polish, or needs an unlikely precondition. **CONFIRMED** = the path was +traced end to end (and, where marked, reproduced against the app); **PLAUSIBLE** = the problem +is there but its reach wasn't fully traced. + +### A. Money path and data integrity + +| # | Sev | Finding | Where | Status | +|---|---|---|---|---| +| T1 | High | **A check paid after a decision is never marked paid, so reverse positive pay pays the same check twice.** `settlement_status = PAID` and `issued_item.status = PAID` are set only on a clean rules match; no decision path (customer, bulk, *Pay all*, the default at the cut-off) sets either. On a reverse account every check is an exception, so after one is paid at the cut-off the paid item stays `pending` and a second presentment of the same account, number and amount finds no `duplicate_paid` (that rule needs `PAID`) and no paid candidate: it's "To review" again, and paid again by silence. The CHANGELOG and RPP-D2 say the opposite. The same gap reaches ordinary accounts: an amount mismatch the customer accepts leaves the issued item unpaid, so FR-D11's "a paid candidate is a duplicate at any amount" never fires for it. Reproduced. | `networks/check/ingestion.py:66-104`; `networks/check/adapter.py:105-115`; `services/auto_disposition_service.py:135-221`; `services/decision_service.py` (no settlement write for checks) | CONFIRMED | +| O1 | High | **Decision files and ML artifacts survive bank deletion.** `FILE_AREAS` is a hand-written tuple of five areas; `storage.AREAS` has seven. The outbox writes the bank's decision files (account and check numbers, amounts, pay or return) under `decision_files//`, which `_delete_files` never touches; the `decision_batch` rows go, so the files are unreferenced but present on disk, S3 or Azure. `ml_artifacts` is cleared only by the paths on surviving `MlModel` rows. The page promises "Every file", and the certificate's `storage_areas_cleared` lists the five. | `services/offboarding_service.py:36, 261-269, 384`; `storage.py:44-52`; `services/decision_outbox_service.py:148` | CONFIRMED | +| O2 | High | **One `PUT /api/v1/customers/{number}` reactivates an ended customer and leaves it scheduled for deletion.** `upsert_customer` sets `customer.is_active = payload.is_active` unconditionally; `end_service` only set `is_active=False` and `service_ended_at`. A core's routine sync with `is_active: true` (realistic for a customer that stops positive pay and stays a customer, CO-D1) lets its people sign in again (the gate checks `is_active` only; memberships were never deactivated) and tellers see it as enrolled, while `_refuse_ended` still refuses its files and the sweep still deletes everything 90 days after `service_ended_at`. It bypasses the two-admin restore. | `api/v1/customers.py:113`; `services/customer_offboarding_service.py:179-189, 342-349`; `auth/deps.py:165-168`; `services/teller_service.py:124-132` | CONFIRMED | +| G1 | Medium | **A late-arriving check is defaulted at once, with no review window.** `within_return_deadline` caps the decision deadline at the return day's cut-off; when the file reaches OpenClear after that (a core posting yesterday's Business Date mid-morning, an API backlog) the deadline is already past, and the disposition sweep (every 300 s) applies FIXED_PAY or FIXED_RETURN before anyone sees the item. `after_return_deadline` stays false (locked before midnight), `deadline_approaching` never fires, the minimum review time is zero. A fraudulent check on a FIXED_PAY account is paid unreviewed although the bank had until midnight. The docstring says "due at once": a design choice whose consequence wasn't drawn. | `services/cutoff_service.py:209-228`; `networks/check/ingestion.py:76-93`; `services/auto_disposition_service.py:135-183`; `workers/tasks.py:126-215` | CONFIRMED | +| T5 | Medium | **Two branches verifying the same check at once both get `match`.** A verification changes no state, so the issued-item lock serialises verify against cash or presentment only; two verifies within the 15-minute window both answer `match`, both tellers pay out, and only the second `cashed` learns `duplicate_paid` after the cash is gone. The guide's "Two branches at once … can't both be paid" overstates what the lock does. Reproduced. | `services/teller_service.py:98-121, 140-157, 202-243`; `docs/guides/teller.md` | CONFIRMED | +| O7 | Medium | **No freeze between approval and the wipe.** `run` sets the bank inactive only in `_strip`, at the end; during the 24-hour delay and the run itself, sign-ins, API clients, imports and the bank's own jobs carry on. `_wipe` deletes the bank's `job` rows, a running import's among them, while that worker's later inserts commit afterwards and aren't in `rows_deleted`. | `services/offboarding_service.py:341-403`; `workers/jobs.py:200-250` | PLAUSIBLE | +| O6 | Medium | **Kept people keep their personal rows.** `_forget` deletes the `_PERSONAL` rows and the user in one savepoint; when the user must stay (for a customer deletion, anyone named in a kept `audit_log_entry`, which is anyone who ever acted) the `IntegrityError` rolls the personal deletes back too: `login_session`, `webauthn_credential`, `password_reset`, `user_device`, `notification` (bodies about the customer's items), `notification_preference`, `idempotency_record`, `revoked_session`, and the user row with its password hash. `_collect` never reaches them (no FK path from `customer`). The certificate counts them as "kept". | `services/offboarding_service.py:275-309`; `services/customer_offboarding_service.py:201-222, 282-294` | CONFIRMED | +| O8 | Medium | **Customer deletion removes the bank's own records.** `_collect` reaches `billing_statement` and its lines (finalised invoices), `cuec_attestation`, `access_review` and its items, `teller_verification`, customer-scoped `change_request` and `custom_field_value`; only `audit_log_entry` and `security_event` are kept. A bank's auditor expects finalised billing and attestation evidence to outlive a client. | `services/customer_offboarding_service.py:35, 201-222` | CONFIRMED | +| C1 | Medium | **Negative numbers in `cf_` columns are corrupted in every CSV decision output.** `_csv_safe` prefixes `'` to any string starting with `-`; `to_csv` applies it to every custom-field column. `parse` accepts signed integers and decimals, so `-5` reaches the core's file (feed.csv, `GET /decisions/file`, the outbox's OpenClear CSV) as `'-5`. The JSON feed is right. | `services/decision_feed_service.py:146-167`; `plugins/builtin.py:90-98`; `services/decision_outbox_service.py:136-140` | CONFIRMED | +| G2 | Medium | **The OpenClear CSV decision file gained a trailing column unannounced.** `CSV_COLUMNS` now ends with `after_return_deadline`; that layout is the outbox file cores import. The CHANGELOG names the feed and the webhook only, and no page lists the layout's columns, so a core that validates the count or maps by position stops on the first file after the upgrade. | `services/decision_feed_service.py:139-156`; `plugins/builtin.py:94-100` | CONFIRMED | +| T11 | Low | **Pay-all's "list changed" guard is count-only.** One item decided plus one newly arrived leaves `len(items) == expected`, so a check the person never saw is paid. Bounded: each decision takes the item lock, and a review-only check is paid at the cut-off anyway. | `web/routers/exceptions.py:527-556` | CONFIRMED | +| T10 | Low | **`cashed` isn't reconcilable on retry.** A platform that lost the response gets 409 "already recorded as cashed" with no `paid_item_id`, so it can't tell its own success from a double-cash by someone else. | `services/teller_service.py:218-225`; `api/v1/checks.py:84-97` | CONFIRMED | +| G9 | Low | **Items open at the upgrade have no return deadline**, so a late decision on one is never flagged. Documented in the migration; one-off. | `migrations/versions/44ed7f9be66a_…:6-8` | CONFIRMED | + +### B. Security + +| # | Sev | Finding | Where | Status | +|---|---|---|---|---| +| O3 | Medium | **"Two operators" becomes "one operator plus 24 hours" when nobody else can approve.** `request_or_apply` sets `applies_at = now + COOLING_OFF` whenever `approvers(..., excluding=user_id)` is empty, for every tier including `protection`, and `settle()` applies it. A sole operator can start offboarding, wait a day, request deletion, wait a day, and the wipe runs a day later with no second person; the certificate shows one email as requester and approver. The same holds for `customer.offboard` at a bank with one admin. | `services/change_request_service.py:119-132, 191-197, 310-345`; `services/change_kinds.py:1531-1560` | CONFIRMED | +| T2 | Medium | **`payee_needed` hands back the payee it asks for.** When the issued check names a payee and the platform sends none, the route still returns `issued` (amount, payee, issue date), so the platform can resend with the payee it was just given and get `match`. The owner's rule (the payee as written on the check) is bypassable by the integration, not enforced. Reproduced. | `api/v1/checks.py:69-81`; `services/teller_service.py:135-157` | CONFIRMED | +| T3 | Medium | **`check:verify` is a check-register oracle the probing detector doesn't watch.** Iterating check numbers on a known account with `amount=0.01` returns `amount_differs` plus `issued` for every issued check; `_watch_for_probing` counts only `not_issued` and `not_on_positive_pay`, and the route has no `rate_limit()` of its own. TL-D2 promised the permission "can't be used to download a customer's check register". Each call is audited and raises `teller.verify.declined` (medium), but the high alert built for this never fires. Reproduced. | `services/teller_service.py:177-199`; `api/v1/checks.py:69-71` | CONFIRMED | +| K2 | Medium | **The CodeQL #176 pattern survives across `web/routers/`.** HK1 fixed one site; `except ValueError as exc` → `error=str(exc)` (117 `str(exc)` sites in the routers), and `tenant_settings.py:930` catches bare `Exception` and shows `TypeName: message`. Any deeper `ValueError` (uuid, int, fromisoformat, a library) shows its text, and CodeQL will raise the same class of alert on the next path it scans. | e.g. `sso_settings.py:170-640`; `people.py:108-455`; `customers.py:312-380`; `users.py:317, 837`; `tenant_settings.py:930-932, 982` | CONFIRMED | +| K3 | Medium | **Two SAML re-key states break sign-in and the page doesn't say so.** `finish` nulls the old key irreversibly; if the identity provider hasn't taken the new certificate, every sign-in fails until it's reconfigured, with no undo. `cancel` after the provider already switched also breaks sign-in until the old certificate is re-given. The page describes the happy path only. | `templates/sso/form.html:127-142`; `services/sso_service.py:191-203` | CONFIRMED | +| C2 | Medium | **The Redis limiter fails open silently for as long as Redis is down.** `RedisRateLimiter.allow` returns True on any exception (one log line a minute); `allow()` never falls back to the in-process limiter in the same module; nothing in health, the control checks or the security events reports it. The 1.31.0 fix (one 0.5 s try) rightly removed the stalls, and made this the steady state of an outage, for sign-in, OAuth tokens and password reset. | `web/rate_limit.py:109-150`; callers in `api/v1/auth.py`, `api/v1/oauth.py`, `web/routers/auth.py`, `password_reset.py` | CONFIRMED | +| T6 | Low | **The probing detector is evadable by a person's token.** `api_client_id` is looked up by user; a person granted `check:verify` has none, so only the caller-supplied `teller_id` and `branch` are counted and rotating them defeats the threshold. Mitigated: `check:verify` is outside Admin's default grant. | `services/teller_service.py:90-95, 182-199` | CONFIRMED | +| C3 | Low | **A customer-scoped feed reader can learn the bank's hidden field keys.** `output_keys` validates against all fields: a hidden key returns 200 with null values and a `cf_` header, an unknown one 422. Values never leak, only a key's existence. | `services/custom_field_service.py:366-379`; `api/v1/decision_feed.py:33-39` | CONFIRMED | +| K4 | Low | **Bank staff re-key a customer's connection with no dual control**, under `customer:manage`, even for a `managed_by="customer"` connection whose own rule would need a second approver; consistent with the other bank edits, but `finish` is the irreversible step. | `web/routers/sso_settings.py:574-593` | CONFIRMED | +| G8 | Low | **The yearly contact confirmation isn't a security event**: `incident_contact.confirmed` is audited but unmapped, unlike `incident_contact.changed`. | `services/incident_service.py:98`; `services/security_events.py` | CONFIRMED | +| K9 | Low | **`postgres:16` by tag** in the Postgres shard's service (everything else is pinned); `_waiver` accepts an alert with no instances under a URL-scoped waiver (`all([])`). | `.github/workflows/tests.yml:82`; `scripts/zap_scan.py:91` | CONFIRMED | + +**Checked and solid.** Every new table has `ENABLE` and `FORCE ROW LEVEL SECURITY` with the +`_TENANT` policy (`test_rls_plugins.py` passes; the provider's `platform_custom_field*` and +`bank_deletion` have no `tenant_id` by design); every new route carries a permission dependency +and every `/ui` POST its CSRF check; `check:verify` and `custom_field:write` are masked for +customer-scoped sessions and `check:verify` is outside Admin's default grant; a customer-scoped +session reads custom-field values only through `values_for(customer_id=…)` on every path (API, +web lists, exception page, feed, CSV, export) and can't write one by any path; unique fields are a +database constraint; `set_values` and each import row run in savepoints; the teller `cashed` call +claims its verification with a conditional update and presents through `ingest_paid_item` under +the issued-item lock; the cut-off sweep and *Pay all* use `get_for_update`; the re-key signs +AuthnRequests with the current key throughout and accepts the next key only while one exists; +nothing secret enters change requests or audit metadata; bank deletion enumerates +`Base.metadata.sorted_tables` by `tenant_id` under the bank's RLS binding, rows and files in one +transaction with files last, every guard re-checked inside the job, and the Postgres test asserts +zero rows for the bank in every table; the certificates are ECDSA-signed with the file-signing +key and stored in tables that outlive the data; the ended-customer gate in `auth/deps.py` covers +every token path; the 1.30.1 bank-day fix is right; the deadline arithmetic is right for +holidays (a Saturday July 4 not moved; a Sunday one observed Monday), closures, DST and the +holiday-open bank; `after_return_deadline` is one function behind the feed, the webhook and the +event; R29/R17 are added per bank and seeded for new ones; the ML floor and HUMAN-only labels +are as the CHANGELOG says; every GitHub Action is SHA-pinned and both base images +digest-pinned; the final image runs as `openclear` with `--chown` copies and no recursive +`chown`; the notices page renders under autoescape; the ZAP waiver is URL-scoped; the OpenAPI +contract test passes; all ten new tables are in the data dictionary; the audit actions that +aren't security events are the data-only ones (field values, the yearly confirmation). + +### C. Features: wrong, dead or missing + +| # | Sev | Finding | Where | Status | +|---|---|---|---|---| +| T4 / O4 | Medium | **An offboarded customer's check makes teller verify fail with 500.** `verify` resolves the account through `account_service.get_account_by_number`, which raises `ServiceEnded` once `service_ended_at` is set; nothing catches it. The 1.28.1 CHANGELOG and the guide say this case answers `not_on_positive_pay`; the test sets only `is_active=False`. `mrm_service.replay` and the custom-field import share the exposure. Reproduced. | `services/teller_service.py:140-157`; `services/account_service.py:55-89` | CONFIRMED | +| O5 | Medium | **Service ends at the next housekeeping run, not at midnight.** Sign-in, file-row refusal and teller status key off `service_ended_at` and `is_active`, which only the daily sweep (03:30, server clock) sets. For a US bank whose last day ends at 05:00–08:00 UTC, access continues until the next morning, and `service_ended_at` is then the sweep time, so the 90-day clock and "Service ended on" drift from the promised day. | `workers/scheduler.py:58`; `services/customer_offboarding_service.py:184, 329-341`; `services/account_service.py:63-68`; `auth/deps.py:167` | CONFIRMED | +| K1 | Medium | **The third-party page's license identifiers and notice texts are never checked by a test that runs.** The exact-regeneration test calls `build()`, which exits unless every in-image package is installed at its locked version; CI installs only `dev.txt` and locally the runtime extras aren't installed, so it skips everywhere. The four tests that run compare names and versions only: a wrong `license`, `used_for`, copyright, system-package license or any `requirements/licenses/*.txt` text passes. Mitigated: `lock-refresh.yml` regenerates both files on every lock change, so the exposure is a hand edit to `vendored.json` or a licenses file. | `tests/test_compliance/test_third_party.py:33-89`; `scripts/third_party.py:137-154`; `.github/workflows/tests.yml:49, 101` | CONFIRMED | +| G3 | Low | **The CEO/CIO fallback can never receive a notice.** `set_contacts` refuses a save without at least one `kind="contact"`, so `recipients()` returns the named contacts or, with no rows at all, the administrators; the `("ceo","cio")` branch is unreachable. The hint, the control's text, `incident-notification.md` and CUEC B-IR-6-05 all describe the fallback. | `services/incident_service.py:72-74, 102-112`; `templates/settings/sections/incident-contacts.html:2-5`; `compliance/cuecs.py:421-425` | CONFIRMED | +| G4 | Low | **A notice "sent" to nobody looks sent.** With no contacts and no `admin:manage` holder, `_deliver` records `recipients=[]`, the bank's log says "emailed to nobody", and the operator sees "Notice … sent" with no per-bank recipient count. | `services/incident_service.py:149-172`; `web/routers/platform.py:531-532`; `templates/platform/incidents.html:37-45` | CONFIRMED | +| G5 | Low | **"Offboarding" is offered as a notice kind by hand**, though `KINDS` marks it "never chosen by hand". | `services/incident_service.py:27-28`; `templates/platform/incidents.html:14-16` | CONFIRMED | +| G6 | Low | **Holiday copy vs behaviour.** The CHANGELOG's "never later than the bank's cut-off on that day" reads as the presented day (the code caps at the return day, correctly); the closures hint says Federal Reserve holidays "are always skipped", and `next_cutoff` indeed ignores `open_on_federal_holidays`, so a bank open on holidays gets no cut-off that day (its checks are saved by the cap; its ACH items are due the next day). | `services/cutoff_service.py:133-145, 169-174`; `templates/settings/sections/cutoff.html:33-42` | CONFIRMED | +| G7 | Low | **Three rows for ten contacts.** The form renders 3 contact rows; `MAX_CONTACTS` is 10. A bank with more than 3 (only via a future API) loses the rest on the next save. | `templates/settings/sections/incident-contacts.html:11`; `web/routers/tenant_settings.py:814-818` | CONFIRMED | +| O9 | Low | **A permanently failed deletion job is silent.** After `job_max_attempts` the job is `FAILED`, `BankDeletion` stays `queued`, the page says "It runs at " forever; no event or control fires. | `workers/jobs.py:243-247`; `templates/platform/offboarding.html:63-66` | CONFIRMED | +| O11 | Low | **`revoke_all_sessions` bumps `token_version`**, signing the person out of every organisation; `stop()` restores `is_active=True` even if the customer was inactive for another reason; `_strip` writes JSON `null` into three NOT NULL JSON columns (readers use `or {}`). | `services/customer_offboarding_service.py:151-154, 184-186`; `services/offboarding_service.py:324-331` | CONFIRMED | +| C4 | Low | **Restoring an archived field bypasses the 25-field and 3-in-list limits.** | `services/custom_field_service.py:311-316` | CONFIRMED | +| C5 | Low | **A mistyped `cf_` header fails every row instead of refusing the file**: 20,000 identical row failures, a COMPLETED status, "0 changed, 20000 failed". The same for an archived field's column. | `services/bulk_import_service.py:382-398`; `services/custom_field_service.py:413-418` | CONFIRMED | +| C6 | Low | **Text values pandas treats as NA vanish from update files** (`NA`, `N/A`, `null`, `None`, `#N/A` → blank → left alone or cleared). Pre-existing parser behaviour, newly exposed to free text. | `bulk_import/tabular.py:40-51` | PLAUSIBLE | +| C7 | Low | **Validation is looser than its messages**: `fromisoformat` accepts `20260101`; the integer regex and `int()` accept non-ASCII digits; `Decimal` accepts `1_000`. Stored values are normalised, so message polish. | `services/custom_field_service.py:108-124` | CONFIRMED | +| C8 | Low | **A `(field, record)` row collision is reported as a unique-value clash** (two concurrent first writes to one field on one account). | `services/custom_field_service.py:450-454` | PLAUSIBLE | +| C9 | Low | **Removed choice options are retired whether or not any record uses them**, so a typo'd option can never go. CF-D11 meant options in use. | `services/custom_field_service.py:211-215` | CONFIRMED | +| T7 | Low | **Trailer semantics are fixed.** `check_trailer` counts every detail line (skipped transaction codes included) and totals signed amounts; a core whose trailer counts debits only, or totals gross, mismatches every file, and the bank's only recourse is to turn the trailer off. | `bulk_import/layouts.py` (`check_trailer`, `TrailerCheck`) | PLAUSIBLE | +| T8 | Low | **Hostile numbers in a posted-check file surface as an unhandled error, not `rejected`**: `int()` on a 4,300-digit trailer count raises `ValueError`, which no layer catches. | `bulk_import/layouts.py` (`_amount`, `_signed_amount`, `check_trailer`); `services/bulk_import_service.py:215-245` | CONFIRMED | +| K5 | Low | **The metadata never advertises an encryption key** (`wantAssertionsEncrypted=False`), so the "decrypt with either key" fallback is reachable only for an IdP configured by hand; the docstring and model comment say the IdP "can encrypt assertions to it". | `auth/saml_service.py:146, 206-208` | CONFIRMED | +| K8 | Low | **`IMAGE_ENV` is hard-coded to x86_64 / 3.13.0**, so an arm64 build's inventory differs from the page; `dev_only` lists a dev package pinned at another version as the runtime one. | `scripts/third_party.py:36-40, 207-208` | CONFIRMED | + +### D. Interface + +| # | Sev | Finding | Where | +|---|---|---|---| +| O10 | Low | **Dates shown in UTC, not the bank's day.** The customer's banner shows `service_ends_at` as "YYYY-MM-DD HH:MM UTC" (the day after the last day for US banks); the detail page and the Deleted list show UTC dates; the email says `deleted_on = last_day + 91` while `deletion_due` is sweep time + 90. | `templates/base.html:203`; `templates/customers/detail.html:217-222`; `templates/customers/deleted.html:17-18` | +| T9 | Low | **Branch declines show raw result codes to customers**: "customer review", "payee needed", "stop differs" with no explanation. | `templates/exceptions/list.html` (branch-declines block) | +| C10 | Low | **Custom-field forms.** One error banner per save (the first failing field aborts); "(optional)" where a required marker is meant; raw type names (`integer`, `boolean`, `choice`) in the settings table, the upload page and the type select; a choice shows its value as its label; the account fields page runs one query per archived field. Accessibility is otherwise right. | `templates/_macros/custom_fields.html:7-31`; `settings/sections/custom-fields.html:18, 56`; `custom_fields/upload.html:25`; `web/routers/accounts.py:204-206` | +| K3 | — | The re-key page's missing warnings (section B). | `templates/sso/form.html` | + +### E. Operations and tests + +| # | Sev | Finding | Where | +|---|---|---|---| +| K6 | Low | **`shows_number` still matches inside UUIDs**: hyphens and hex letters count as boundaries, and a UUID4's third group starts with `4`, so `-4001-` appears with probability 1/4096 per UUID on the page. | `tests/items.py:102-106` | +| K7 | Low | **Lock-refresh and CI-image PRs get no CI** (opened with the default `GITHUB_TOKEN`, which doesn't trigger workflows); the required checks appear only after a push to the branch. Not a path to a broken lock on `main`. | `.github/workflows/lock-refresh.yml:27-35`; `ci-image.yml:109-117` | + +**Tests missing**, by cluster (each stage below adds its own): + +- *Money path (Postgres, `-m concurrency`):* a teller `cashed` racing a file presentment of the same check; two `cashed` calls with one verification; *Pay all* racing the cut-off sweep; a job or request for a bank running during its deletion. +- *Reverse and teller:* the same check presented twice after the first was paid by silence (T1); an ended customer at `/checks/verify` (T4); the shape of `payee_needed` (T2); probing via `amount_differs` (T3); two verifications of one check (T5). +- *Deadlines:* a check presented on day D arriving after the return day's cut-off (G1); `lock_due` after midnight with the webhook's and the CSV's `after_return_deadline` asserted; the holiday-open bank through ingestion; a presentment on a holiday; DST in Phoenix and Honolulu; the ML floor at 199 human decisions. +- *Offboarding:* a SQLite-suite assertion that no row with the bank's `tenant_id` remains in any metadata table (only the Postgres test does it); every `storage.AREAS` prefix cleared (O1); S3 `delete_tenant`; sign-in refusal for an ended customer on each path; reactivation via the API (O2); cooling-off on a protection kind with one operator (O3); the kept-user path of `_forget` (O6). +- *Custom fields:* a customer-scoped write by file with a group that has `custom_field:write`; customer-scoped `feed.csv` and `/decisions/file`; RLS on a bank session writing `custom_field_value`; a negative value through `to_csv` (C1); restore against the limits (C4); an unknown `cf_` header (C5). +- *Incidents and re-key:* incidents under RLS (`issue()` across banks; `acknowledge` with another bank's delivery); an assertion encrypted to the next key during the transition; a response for the old key refused after `finish`; the bank-staff re-key route; a request made stale by a cancel. +- *Compliance artefacts:* a test that fails on a stale license identifier, `used_for`, copyright, system entry or license text (K1); `shows_number` against a page with `-4001-` in a UUID. + +## Decisions (recommended) + +- **RV-D1. A check's payment is recorded when it becomes final.** When `decision_service.lock` + locks a check decision whose outcome is *pay* (a person's, *Pay all*'s or the default's), it + sets the paid item `settlement_status = PAID` and a matched issued item `status = PAID`; a + locked *return* sets the paid item `RETURNED` (the ACH path already does). Lock time, not + decide time, because a decision can change until the cut-off and payment is final at the lock. + `_is_duplicate_paid` and the paid-candidate rule then see every paid check, on reverse and + ordinary accounts alike. Items locked before this ships are backfilled by a one-off data + migration from their decision rows. +- **RV-D2. Deletion clears every storage area, from the one list.** `_delete_files` iterates + `storage.AREAS` (`decision_files` and `ml_artifacts` included) and the certificate lists the + same; a test walks `AREAS` so a new area can't be missed. The bank is frozen at the start of + the deletion job: `is_active=False` first (sign-ins and API calls refused from that moment), + its queued jobs cancelled and running ones waited for (`jobs.check_lease`), then the wipe. +- **RV-D3. An ended customer can't be reactivated sideways.** `upsert_customer` (API) and the + web toggle refuse `is_active=True` while `service_ended_at` is set, answering 409 with the + two-admin restore as the way back; `end_service` also deactivates the customer's memberships, + so a future path that misses the check still can't sign anyone in. +- **RV-D4. Protection-tier changes never cool off.** For `protection` kinds (and every + provider-scope kind) `request_or_apply` raises `ChangeError("needs a second operator") ` when + nobody else can approve, instead of scheduling a cooling-off apply. An installation with one + operator can't offboard a bank until it has two, and the page says so. Sensitive and access + tiers keep the cooling-off. +- **RV-D5. Service ends at the end of the last day, wherever it's read.** `_refuse_ended`, + `_enrolled`, the sign-in gate and the teller answer read `service_ends_at <= now` (already the + bank's end of day in UTC), and `end_service` records `service_ended_at = service_ends_at`, so + the 90-day clock and the pages agree with the promised day. The sweep still revokes sessions + and sends the emails. +- **RV-D6. Kept people are anonymised; the bank's own records stay.** When `_forget` must keep + a user, the `_PERSONAL` rows are deleted outside the savepoint and the user row is reduced to + its id and a placeholder name (no password hash, phone or email), counted in the certificate + as "anonymised". Customer deletion keeps `billing_statement` and its lines, `cuec_attestation`, + `access_review` and its items (nulling the customer FK where one exists); `teller_verification` + and `change_request` go (the export already holds them). The plan, CHANGELOG and certificate + say which. +- **RV-D7. A late check gets a short window, never past midnight.** When the capped deadline is + already past at ingest, `within_return_deadline` returns `min(returns_by, now + + cutoff_min_review_minutes)` and the item is marked urgent (the reminder goes at once), so a + person has the minimum review time before the default applies; nothing is ever due after the + return deadline. +- **RV-D8. The teller answer gives nothing the caller didn't send.** `issued` is omitted when + the only problem is `payee_needed` and never carries `payee_name` except for `payee_differs`; + `_watch_for_probing` counts every non-match per teller, branch and client (a person's token + counts under the user id), and `/checks/verify` gets its own `rate_limit()`. A `match` for a + check with an unexpired, uncashed `match` verification by another teller answers + `being_cashed` (a decline), and the guide's "Two branches at once" says what the lock does. + `cashed` on an already-cashed verification returns 409 with the `paid_item_id` and who cashed + it. `ServiceEnded` is caught in `verify` (and `mrm_service.replay`) and answers + `not_on_positive_pay`. +- **RV-D9. One error type for sentences meant for people.** `services/errors.py::UserFacingError` + (a `ValueError`), which `ChangeError`, `OutsideServiceError`, `RowFieldError` and the other + service errors that carry sentences for a person inherit; routers catch only it (and the + named types they already handle) and show a fixed sentence for anything else. One sweep over + `web/routers/`, with a test that greps for `str(exc)` on a bare `ValueError` or `Exception`. +- **RV-D10. The limiter degrades to the process, and says so.** On a Redis failure `allow()` + falls back to the in-process limiter for that call, and a control check (`rate_limit_backend`, + WARN) plus a security event the first time it flips tell the operator. +- **RV-D11. CSV outputs know their types.** `to_csv` takes the field definitions and applies the + formula guard to text and choice values only; numbers and dates pass as they are. The OpenClear + CSV layout's columns are listed on the decision-files page and in API.md, and the 1.29.0 entry + gains an upgrade note about `after_return_deadline`. +- **RV-D12. The third-party comparison runs where it can.** `security.yml` gains a step that + runs `scripts/third_party.sh --check` (Docker is on the runner) and fails on any difference + from the committed page and notices; the skipping test stays for local runs that can. + `IMAGE_ENV` is read from the build's platform; `dev_only` names both versions. +- **RV-D13. Copy says what the code does.** The re-key page warns that finishing before the + provider has the certificate stops sign-in and can't be undone, and that withdrawing after it + switched needs the old certificate re-given, with a "test sign-in" link beside Finish; the + CEO/CIO fallback is either built (two optional rows that `recipients()` uses when no contact + answers) or removed from the hint, the control, the page and the CUEC (recommended: built, it's + a dozen lines); the incidents form hides `offboarding`; the Sent table shows recipients per + bank; the holiday hint and the CHANGELOG line are corrected; branch declines and custom-field + types get `status_label`s; offboarding dates show the bank's day. + +## Stages + +Each stage is one pull request. Before merging: the three full suites, Bandit and the dependency +audit (CLAUDE.md), and a Postgres run for the concurrency tests the stage adds. + +| Stage | What | Closes | +|---|---|---| +| **RV1. A paid check is paid** | RV-D1: settlement and issued-item status written at `lock`; the backfill migration; the twice-presented reverse test and the amount-mismatch-then-duplicate test; *Pay all*'s guard compares the item ids it showed, not the count; `cashed` 409 with `paid_item_id`. Postgres: teller `cashed` vs a file presentment; two `cashed` with one verification; *Pay all* vs the sweep. | T1, T10, T11 | +| **RV2. The teller answer** | RV-D8 and the `ServiceEnded` catch; `being_cashed` in the `Result` literal, the guide's table, OpenAPI and the SDKs; the probing counter and the route limit; tests for each. | T2, T3, T4/O4, T5, T6 | +| **RV3. Deadlines** | RV-D7 with its tests (a late file; the holiday-open bank through ingestion; a presentment on a holiday; Phoenix and Honolulu); the CSV column documented (RV-D11's second half); `next_cutoff` honours `open_on_federal_holidays` or the hint says why not (recommended: honour it); the ML floor test. | G1, G2, G6, G9 (noted as one-off) | +| **RV4. Bank deletion** | RV-D2: every area, the freeze, the jobs; a failed deletion job raises `bank.offboarding` (high) and shows on the page; the SQLite-suite "no row remains" assertion over the metadata; the `AREAS` walk test; S3 `delete_tenant` test. RV-D4: protection kinds never cool off, with the page copy and a test with one operator. | O1, O3, O7, O9 | +| **RV5. Customer offboarding** | RV-D3, RV-D5, RV-D6 and their tests (reactivation via API and web refused; service ended at midnight on each path; the kept-user anonymisation; the kept tables); dates in the bank's day; `stop()` restores the prior `is_active`; sessions revoked for the customer's memberships only; CO plan, CHANGELOG and certificate text. | O2, O5, O6, O8, O10, O11 | +| **RV6. Custom fields and the limiter** | RV-D11's first half (`to_csv` by type, the `-5` test); RV-D10; hidden keys refused for scoped feed readers (422 like unknown ones); restore re-checks the limits; an unknown or archived `cf_` header refuses the file; `keep_default_na=False` for update files; the validation messages; options retired only when in use; per-field error messages, required markers, `status_label` for types, option labels, one query for the archived fields. | C1–C10 | +| **RV7. Copy and notices** | RV-D13's incident items (the CEO/CIO rows, the hidden kind, recipients per bank, ten contact rows or a limit of three, `incident_contact.confirmed` mapped); branch-decline labels; the re-key warnings and test-sign-in link; the holiday copy. The axe and viewport checks must pass unchanged. | G3, G4, G5, G7, G8, T9, K3 | +| **RV8. Errors, artefacts and the build** | RV-D9 across `web/routers/` with its grep test; RV-D12 (`--check` in `security.yml`, `IMAGE_ENV`, `dev_only`); the bank-staff re-key under the connection's own dual control, or the docstring says why not; `postgres:16` by digest; `_waiver` requires an instance; `shows_number` excludes hex runs; hostile numbers in layouts become `rejected`; the trailer gains "count debits only" and "total gross" options once a core's sample shows the need (PIL plan). | K1, K2, K4, K5 (docstring), K6, K8, K9, T7 (noted), T8 | + +**Order.** RV1 first: it's the money path, and the backfill migration wants to ship before more +reverse accounts exist. RV4 and RV5 next, together or back to back: a bank's deletion certificate +that lists five areas is a compliance record that's wrong today. RV2 and RV3 are independent of +everything and of each other. RV6 and RV7 can run beside anything. RV8 touches every router and +goes last on a quiet branch. + +**What this plan doesn't do.** No change to the decision model, the change-request model or the +storage layout. The 1.26.0 review's portfolio decisions (FR-D19) stand. The trailer options +(T7) wait for a core's real file (PAID_ITEM_LAYOUTS_PLAN.md). + +## Progress log + +- 2026-10-31: Plan written from a review of `v1.33.0` (`0bafc661`) covering 1.28.0 to 1.33.0, + five clusters read end to end, every High and Medium re-checked by hand against the code. + Three Highs (T1, O1, O2), fifteen Mediums, the rest polish. The RLS coverage test, the + OpenAPI contract test and the data-dictionary check were run as part of the review and pass. +- 2026-10-31: RV1 built on `review-1-33`. `decision_service.lock` settles the check (`_settle_check`: + paid or returned; a paid check's outstanding, stale or stopped issued item becomes paid; a void + is kept), so `_is_duplicate_paid` and the paid-candidate rule see every paid check; migration + `b8c4d2e6f1a3` brings decisions already final into line, per bank under RLS. *Pay all* posts the + ids the page listed and pays exactly those. `cashed` on an already-cashed verification answers + 409 with `message`, `paid_item_id` and `cashed_at`. Tests: `tests/test_decisions/ + test_check_settlement.py` (reverse paid then presented again; an accepted amount mismatch then + two more presentments; a return; a voided check paid), the pay-all and teller tests amended, + and `tests/test_postgres/test_teller_concurrency.py` (a branch cash against a file presentment; + one verification cashed twice at once; *Pay all* against the sweep). +- 2026-10-31: RV2 built on `review-1-33`. `issued` is left out when the only problem is `payee_needed` + and carries the payee only for `payee_differs`; `being_cashed` (a match verified within 15 + minutes and not yet cashed, seen because verifications take turns on the issued item's lock); + the probing counter counts every decline, by teller, branch, client, and by `user_id` (a new + column, migration `c9d5e3f7a2b4`) when there's no client; `ServiceEnded` is no longer raised at + the teller or in an MRM replay (`account_service.find_account_by_number`). Guide, OpenAPI and + SDKs updated. **Deviation:** no per-route `rate_limit()` on `/checks/verify`: the global limit + is already 120 a minute per address, and a stricter one keyed the same way would add nothing; + the counter that matters is the probing one. Also fixed while running the suite for RV1: backing + out a file whose check was paid by a decision restores the issued check, as a matched one does, + and a check already returned by a decision can't be backed out (it was always so for a return). +- 2026-10-31: RV3 built on `review-1-33`. `within_return_deadline` gives a check arriving after its + return day's cut-off the bank's minimum review time, never past midnight (RV-D7); one whose + midnight has passed is still due at once. `next_cutoff` takes `open_on_federal_holidays`, so a + bank open on them has its cut-off on them (G6), and the two hints say so. The decisions CSV's + columns are listed in `docs/guides/bi.md` (G2), with "read by header, not position". The + flagged-decision test now asserts the CSV column and the webhook payload too. **Not done:** the + ML floor test (the review's list was wrong: `test_a_first_model_needs_enough_own_decisions_to_be_activated` + already covers it); DST in Phoenix and Honolulu (the arithmetic is on aware wall clocks, and + the review probed the fall-back; a parametrised zone would prove little more). +- 2026-10-31: RV4 built on `review-1-33`. `_delete_files` walks `storage.AREAS` and the certificate + lists them (O1); `_freeze` (the bank inactive, its job rows deleted, committed before the wipe; + every token check refuses an inactive bank) (O7); `mark_failed` from the job handler's last try, + `bank.delete.failed` (critical), the page's "The last deletion failed" (O9); `Kind.second_person` + on `platform.bank.offboard`, `platform.bank.delete` and `customer.offboard` refuses a request + with nobody else to approve (O3). Tests: a file in every area gone; no row in any + `tenant_id` table on SQLite; a lone operator and a lone administrator refused; a deletion + that gives up. **Deviation from RV-D4:** the other protection-tier kinds (a bank's password + policy, lockout rules, security keys) keep DUAL_CONTROL_PLAN.md's cooling-off for a bank with + one administrator; refusing them would lock such a bank out of its own settings. Only what + can't be undone needs the second person always. The S3 `delete_tenant` test is left: the + Azure one covers the same shape and the walk is backend-independent. +- 2026-10-31: RV5 built on `review-1-33`. `customer_offboarding_service.ended()` reads + `service_ends_at <= now` as well as `service_ended_at`, and the file refusal, the teller and the + request gate use it; `end_service` records the end as the end of the last day (O5). The API's + upsert refuses `is_active: true` for an ended customer (O2). Kept people are anonymised + (`_anonymise`: email, phone, password, token version) after the personal rows go outside the + savepoint (O6); `cuec_attestation`, `access_review` and `access_review_item` are kept, unlinked + (`_unlink_kept`), and the certificate says which (O8). Sessions end at this bank only (O11); + `_strip` writes empty mappings, not JSON null. `bank_day` filter and `ctx.service_last_day` show + the bank's days (O10). **Deviations:** finalised billing statements go with the customer: + `billing_statement.customer_id` is NOT NULL, billing is frozen (FR-D19), and the bank's invoices + live in its own system; `stop()` still restores `is_active=True` (there's no record of what it + was, and a customer restored to positive pay is active). +- 2026-10-31: RV6 built on `review-1-33`. `_csv_safe` lets a plain signed number through (C1; + **deviation from RV-D11:** by shape rather than by field type, which also covers built-in + columns and needs no definitions in the CSV path). `RedisRateLimiter.allow` falls back to the + process's limiter, `rate_limit.backend_down` (integration, high) once per outage, + `backend_status()` and the `rate_limit_backend` check with control OC-SC-08 (C2). `output_keys` + takes `customer_id` and hides fields not shown to the customer (C3); `archive_field(archived= + False)` re-checks both limits (C4); `_field_loader` takes the bank (`ImportSpec.load_takes_bank`) + and refuses unknown or archived `cf_` headers whole, reading NA text as text + (`parse_tabular_file(keep_na_text=True)`, update files only) (C5, C6); strict integer, date and + decimal parsing (C7); `update_field` keeps a dropped option only while a value holds it (C9); + `set_values` reports every field's problem at once; the `custom_field_type` filter; one read + for the account fields page (C10). **Not done:** C8 (two first writes to one field at once + report as a unique clash: harmless and unlikely); choice option labels (options are values). +- 2026-10-31: RV7 built on `review-1-33`. `set_contacts` and `confirm_contacts` accept a CEO and CIO + alone, so `recipients()`' fallback is reachable and the check counts it (G3); `sent()` and the + Sent table's "Banks told" column (G4); `HAND_KINDS` on the form and in the route (G5); the form + renders `contacts + 1` rows up to `MAX_CONTACTS` (G7); `incident_contact.confirmed` + (configuration, low) (G8); the `teller_result` filter (T9); the re-key page's warnings (K3). + **Not done:** a "test sign-in" link beside Finish: the page says to try a sign-in in a private + window, which is what the link would have done. +- 2026-10-31: RV8 built on `review-1-33`. `openclear/errors.py::UserFacingError` and + `web/errors.py::shown()` (a UserFacingError's or any openclear-raised error's text; a fixed + sentence for a library's), every `error=str(exc)` and `flash=str(exc)` in `web/routers/` goes + through it, the bare-Exception site shows the fixed sentence, and 35 service error classes + inherit UserFacingError; `tests/test_web/test_error_text.py` greps for the old pattern (K2). + `scripts/third_party.sh --check` and a step in `security.yml` (K1). The bank's customer re-key + goes through `customer.sign_on.rekey` (K4). `shows_number` excludes hex and hyphens (K6); + `postgres:16` pinned by digest and `_waiver` needs an instance (K9); long numbers in amounts + and trailers refused as rows (T8). **Not done:** K5 (the docstring on encryption keys is left: + the fallback is correct for an IdP configured by hand); K8 (`IMAGE_ENV` is the release image's + platform, x86_64, which is what the page documents; the generator runs in that image's base); + T7 (waits for a core's sample, PAID_ITEM_LAYOUTS_PLAN.md); lock-refresh PRs with no CI (K7: the + ruleset's required checks still gate the merge). +- 2026-10-31: **RV1–RV8 built**, each with the full SQLite suite green at its commit; Postgres and + Linux suites to run on the branch head before the pull request. diff --git a/requirements/dev.txt b/requirements/dev.txt index 86acbdc6..ce96021f 100644 --- a/requirements/dev.txt +++ b/requirements/dev.txt @@ -2367,9 +2367,9 @@ py-serializable==2.1.0 \ --hash=sha256:9d5db56154a867a9b897c0163b33a793c804c80cee984116d02d49e4578fc103 \ --hash=sha256:b56d5d686b5a03ba4f4db5e769dc32336e142fc3bd4d68a8c25579ebb0a67304 # via cyclonedx-python-lib -pyasn1==0.6.4 \ - --hash=sha256:9c447d8431c947fe4c8febc4ed9e760bc29011a5b01e5c74b67025bd9fb8ce81 \ - --hash=sha256:deda9277cfd454080ec40b207fb6df82206a3a2688735233cdcd8d3d565f088b +pyasn1==0.6.5 \ + --hash=sha256:4017b0fce125096b8573b39c989fdb1a4b19cf761e683398184550fea0e66587 \ + --hash=sha256:6b11616a6bcf3a40e635f082bb232fa57b5e0dd7f517bd7ddd69ec3794508a47 # via # pyasn1-modules # webauthn diff --git a/requirements/runtime.txt b/requirements/runtime.txt index c6a033fc..d796714a 100644 --- a/requirements/runtime.txt +++ b/requirements/runtime.txt @@ -2352,9 +2352,9 @@ psycopg-binary==3.3.6 ; implementation_name != 'pypy' \ --hash=sha256:fa34eb47969297471db7b7f193622c7e3ee839ec05abd05f1fe104d5b1b1dcf4 \ --hash=sha256:fdccb3a0e184b03e9baa673b15a809cf36c339c85dbda0ebc25a698846dfbee8 # via psycopg -pyasn1==0.6.4 \ - --hash=sha256:9c447d8431c947fe4c8febc4ed9e760bc29011a5b01e5c74b67025bd9fb8ce81 \ - --hash=sha256:deda9277cfd454080ec40b207fb6df82206a3a2688735233cdcd8d3d565f088b +pyasn1==0.6.5 \ + --hash=sha256:4017b0fce125096b8573b39c989fdb1a4b19cf761e683398184550fea0e66587 \ + --hash=sha256:6b11616a6bcf3a40e635f082bb232fa57b5e0dd7f517bd7ddd69ec3794508a47 # via # pyasn1-modules # webauthn diff --git a/scripts/third_party.sh b/scripts/third_party.sh index 2d2b8924..e5512eae 100755 --- a/scripts/third_party.sh +++ b/scripts/third_party.sh @@ -6,11 +6,24 @@ # where the locks are installed exactly as the image installs them: in its base image, with both # hashed locks. Run it after changing a lock; the weekly lock refresh runs it too. # -# scripts/third_party.sh +# scripts/third_party.sh # rewrite the two files +# scripts/third_party.sh --check # regenerate and compare: exit 1 if the committed files differ (CI; REVIEW_1_33_PLAN.md K1) set -eu cd "$(dirname "$0")/.." BASE=$(sed -n 's/^FROM \(python:[^ ]*\) AS builder$/\1/p' Dockerfile) +if [ "${1:-}" = "--check" ]; then + keep=$(mktemp -d) + cp docs/compliance/third-party.md THIRD_PARTY_NOTICES.md "$keep/" + trap 'cp "$keep/third-party.md" docs/compliance/third-party.md; cp "$keep/THIRD_PARTY_NOTICES.md" THIRD_PARTY_NOTICES.md; rm -rf "$keep"' EXIT +fi docker run --rm -v "$PWD:/src" -w /src "$BASE" sh -c ' pip install -q --root-user-action=ignore --no-deps --require-hashes -r requirements/runtime.txt && pip install -q --root-user-action=ignore --no-deps --require-hashes -r requirements/dev.txt && python scripts/third_party.py' +if [ "${1:-}" = "--check" ]; then + if ! diff -u "$keep/third-party.md" docs/compliance/third-party.md || ! diff -u "$keep/THIRD_PARTY_NOTICES.md" THIRD_PARTY_NOTICES.md; then + echo "docs/compliance/third-party.md or THIRD_PARTY_NOTICES.md is stale: run scripts/third_party.sh and commit" >&2 + exit 1 + fi + echo "third-party inventory and notices are current" +fi diff --git a/scripts/zap_scan.py b/scripts/zap_scan.py index 4f3413f4..95a2f6d0 100644 --- a/scripts/zap_scan.py +++ b/scripts/zap_scan.py @@ -88,7 +88,8 @@ def _waiver(accepted: dict, alert: dict) -> str | None: if found is None: return None reason, urls = found - if urls and not all(re.search(urls, i.get("uri", "")) for i in alert.get("instances", [])): + instances = alert.get("instances", []) + if urls and (not instances or not all(re.search(urls, i.get("uri", "")) for i in instances)): # no instance: not shown to be on those URLs return None return reason diff --git a/sdks/dotnet/src/OpenClear/Generated.cs b/sdks/dotnet/src/OpenClear/Generated.cs index dadcdb61..6259a4b0 100644 --- a/sdks/dotnet/src/OpenClear/Generated.cs +++ b/sdks/dotnet/src/OpenClear/Generated.cs @@ -761,13 +761,13 @@ public sealed record InventoryCuec } /// -/// What was issued, to explain a decline. Never sent for a match. +/// What was issued, to explain a decline. Never sent for a match, nor when the only problem is the payee the teller didn't send; the payee comes only when the one sent differs. /// public sealed record IssuedDetails { [JsonPropertyName("amount")] public string Amount { get; init; } = default!; [JsonPropertyName("issue_date")] public string IssueDate { get; init; } = default!; - [JsonPropertyName("payee_name")] public string PayeeName { get; init; } = default!; + [JsonPropertyName("payee_name")] [JsonIgnore(Condition = JsonIgnoreCondition.WhenWritingNull)] public string? PayeeName { get; init; } } public sealed record IssuedItemCreate diff --git a/sdks/python/src/openclear_sdk/_generated.py b/sdks/python/src/openclear_sdk/_generated.py index 97f65e29..fbd3fd93 100644 --- a/sdks/python/src/openclear_sdk/_generated.py +++ b/sdks/python/src/openclear_sdk/_generated.py @@ -738,11 +738,12 @@ class InventoryCuec(TypedDict): class IssuedDetails(TypedDict): - """What was issued, to explain a decline. Never sent for a match.""" + """What was issued, to explain a decline. Never sent for a match, nor when the only problem +is the payee the teller didn't send; the payee comes only when the one sent differs.""" amount: str issue_date: str - payee_name: str + payee_name: NotRequired[str | None] class IssuedItemCreate(TypedDict): @@ -1025,8 +1026,8 @@ class ValuesBeforeAfter(TypedDict): class Verification(TypedDict): expires_at: str issued: NotRequired[IssuedDetails | None] - problems: list[Literal["match", "not_on_positive_pay", "stopped", "already_paid", "voided", "not_issued", "stop_differs", "amount_differs", "payee_differs", "stale", "customer_review", "payee_needed"]] - result: Literal["match", "not_on_positive_pay", "stopped", "already_paid", "voided", "not_issued", "stop_differs", "amount_differs", "payee_differs", "stale", "customer_review", "payee_needed"] + problems: list[Literal["match", "not_on_positive_pay", "stopped", "already_paid", "voided", "not_issued", "stop_differs", "amount_differs", "payee_differs", "stale", "customer_review", "being_cashed", "payee_needed"]] + result: Literal["match", "not_on_positive_pay", "stopped", "already_paid", "voided", "not_issued", "stop_differs", "amount_differs", "payee_differs", "stale", "customer_review", "being_cashed", "payee_needed"] verification_id: str diff --git a/sdks/typescript/src/generated.ts b/sdks/typescript/src/generated.ts index 6a8acdc3..a555b5be 100644 --- a/sdks/typescript/src/generated.ts +++ b/sdks/typescript/src/generated.ts @@ -709,11 +709,11 @@ export interface InventoryCuec { withdrawn: boolean; } -/** What was issued, to explain a decline. Never sent for a match. */ +/** What was issued, to explain a decline. Never sent for a match, nor when the only problem is the payee the teller didn't send; the payee comes only when the one sent differs. */ export interface IssuedDetails { amount: string; issue_date: string; - payee_name: string; + payee_name?: string | null; } export interface IssuedItemCreate { @@ -979,8 +979,8 @@ export interface ValuesBeforeAfter { export interface Verification { expires_at: string; issued?: IssuedDetails | null; - problems: Array<"match" | "not_on_positive_pay" | "stopped" | "already_paid" | "voided" | "not_issued" | "stop_differs" | "amount_differs" | "payee_differs" | "stale" | "customer_review" | "payee_needed">; - result: "match" | "not_on_positive_pay" | "stopped" | "already_paid" | "voided" | "not_issued" | "stop_differs" | "amount_differs" | "payee_differs" | "stale" | "customer_review" | "payee_needed"; + problems: Array<"match" | "not_on_positive_pay" | "stopped" | "already_paid" | "voided" | "not_issued" | "stop_differs" | "amount_differs" | "payee_differs" | "stale" | "customer_review" | "being_cashed" | "payee_needed">; + result: "match" | "not_on_positive_pay" | "stopped" | "already_paid" | "voided" | "not_issued" | "stop_differs" | "amount_differs" | "payee_differs" | "stale" | "customer_review" | "being_cashed" | "payee_needed"; verification_id: string; } diff --git a/src/openclear/api/v1/checks.py b/src/openclear/api/v1/checks.py index b539b829..c724af56 100644 --- a/src/openclear/api/v1/checks.py +++ b/src/openclear/api/v1/checks.py @@ -24,7 +24,7 @@ router = APIRouter(prefix="/checks", tags=["teller"]) Result = Literal["match", "not_on_positive_pay", "stopped", "already_paid", "voided", "not_issued", "stop_differs", "amount_differs", - "payee_differs", "stale", "customer_review", "payee_needed"] + "payee_differs", "stale", "customer_review", "being_cashed", "payee_needed"] class VerifyCheck(BaseModel): @@ -39,10 +39,11 @@ class VerifyCheck(BaseModel): class IssuedDetails(BaseModel): - """What was issued, to explain a decline. Never sent for a match.""" + """What was issued, to explain a decline. Never sent for a match, nor when the only problem + is the payee the teller didn't send; the payee comes only when the one sent differs.""" amount: Decimal - payee_name: str + payee_name: str | None = None issue_date: date @@ -73,9 +74,12 @@ def verify_check(payload: VerifyCheck, db: Session = Depends(get_db), Pay out only on `match`. A match can be recorded cashed once, within 15 minutes.""" verification = teller_service.verify(db, ctx.tenant_id, teller_service.VerifyRequest(**payload.model_dump()), user_id=ctx.user_id) issued = None - if verification.result != teller_service.MATCH and verification.issued_item_id is not None: + if (verification.result != teller_service.MATCH and verification.issued_item_id is not None + and verification.problems != [teller_service.PAYEE_NEEDED]): + # Nothing the teller didn't already have: the payee only when the one they sent differs (REVIEW_1_33_PLAN.md T2). item = db.get(IssuedItem, verification.issued_item_id) - issued = IssuedDetails(amount=item.amount, payee_name=item.payee_name, issue_date=item.issue_date) + payee = item.payee_name if "payee_differs" in verification.problems else None + issued = IssuedDetails(amount=item.amount, payee_name=payee, issue_date=item.issue_date) db.commit() return Verification(verification_id=verification.id, result=verification.result, problems=verification.problems, expires_at=verification.expires_at, issued=issued) @@ -83,7 +87,9 @@ def verify_check(payload: VerifyCheck, db: Session = Depends(get_db), @router.post("/cashed", response_model=CashedCheck, status_code=201, responses={404: {"description": "No such verification"}, - 409: {"description": "Declined, already recorded, expired, or another amount"}}) + 409: {"description": "Declined, expired, or another amount; or already recorded, in which case `detail` is an " + "object with `message`, `paid_item_id` and `cashed_at`, so a platform that lost the " + "first answer can reconcile"}}) def record_cashed(payload: CashCheck, db: Session = Depends(get_db), ctx: TenantContext = Depends(require_permission("check:verify"))) -> CashedCheck: """Records a verified check as cashed: it's presented as paid at once, so the same check @@ -92,6 +98,8 @@ def record_cashed(payload: CashCheck, db: Session = Depends(get_db), cashed = teller_service.cash(db, ctx.tenant_id, payload.verification_id, payload.amount, user_id=ctx.user_id) except teller_service.TellerError as exc: db.rollback() - raise HTTPException(exc.status, str(exc)) from None + detail = {"message": str(exc), **{k: (str(v) if isinstance(v, uuid.UUID) else v.isoformat() if hasattr(v, "isoformat") else v) + for k, v in exc.details.items()}} if exc.details else str(exc) + raise HTTPException(exc.status, detail) from None db.commit() return CashedCheck(verification_id=cashed.verification.id, paid_item_id=cashed.paid_item.id, exception_types=cashed.exception_types) diff --git a/src/openclear/api/v1/customers.py b/src/openclear/api/v1/customers.py index eb4c4d8e..6ce20652 100644 --- a/src/openclear/api/v1/customers.py +++ b/src/openclear/api/v1/customers.py @@ -7,7 +7,7 @@ import uuid from datetime import datetime -from fastapi import APIRouter, Depends, Request, Response, status +from fastapi import APIRouter, Depends, HTTPException, Request, Response, status from fastapi.responses import JSONResponse from pydantic import BaseModel, ConfigDict, Field from sqlalchemy.orm import Session @@ -18,7 +18,7 @@ from openclear.db.tenancy import TenantContext from openclear.domain.customer import Customer from openclear.repositories.customer_repo import CustomerRepository -from openclear.services import audit_log_service, customer_service +from openclear.services import audit_log_service, customer_offboarding_service, customer_service router = APIRouter(prefix="/customers", tags=["customers"]) _manage = require_permission("customer:manage") @@ -110,6 +110,11 @@ def upsert_customer(customer_number: str, payload: CustomerUpsert, db: Session = else: customer = customer_service.update_customer(db, ctx.tenant_id, existing.id, data) action, code = "customer.update", status.HTTP_200_OK + if payload.is_active and customer_offboarding_service.ended(customer): + # A core's routine sync mustn't quietly undo the end of service; restoring it is two administrators' change (RV-D3). + db.rollback() + raise HTTPException(status.HTTP_409_CONFLICT, f"Positive pay for {customer.name} has ended; restoring it is a change two " + "administrators make on the customer's page, not a sync.") customer.is_active = payload.is_active audit_log_service.record_action( db, ctx.tenant_id, actor_user_id=ctx.user_id, channel="api", action=action, @@ -117,8 +122,6 @@ def upsert_customer(customer_number: str, payload: CustomerUpsert, db: Session = resource_type="customer", resource_id=customer.id, ) if payload.custom_fields: - from fastapi import HTTPException - from openclear.services import custom_field_service try: @@ -140,8 +143,6 @@ def _self_service(db: Session, ctx: TenantContext, customer: Customer) -> SelfSe def _by_number(db: Session, ctx: TenantContext, customer_number: str) -> Customer: - from fastapi import HTTPException - customer = customer_service.get_customer_by_number(db, ctx.tenant_id, customer_number) if customer is None: raise HTTPException(status_code=404, detail="Customer not found") diff --git a/src/openclear/api/v1/decision_feed.py b/src/openclear/api/v1/decision_feed.py index a398812e..4fadd353 100644 --- a/src/openclear/api/v1/decision_feed.py +++ b/src/openclear/api/v1/decision_feed.py @@ -34,7 +34,7 @@ def _custom_fields(db: Session, ctx: TenantContext, text: str | None) -> list[st from openclear.services import custom_field_service try: - return custom_field_service.output_keys(db, ctx.tenant_id, text) + return custom_field_service.output_keys(db, ctx.tenant_id, text, customer_id=ctx.customer_id) except custom_field_service.CustomFieldError as exc: raise HTTPException(status.HTTP_422_UNPROCESSABLE_ENTITY, str(exc)) from None diff --git a/src/openclear/auth/deps.py b/src/openclear/auth/deps.py index 158e2e2c..28c22b87 100644 --- a/src/openclear/auth/deps.py +++ b/src/openclear/auth/deps.py @@ -20,6 +20,7 @@ from openclear.domain.customer import Customer from openclear.domain.exception_item import ExceptionItem, ExceptionStatus from openclear.domain.security_group import SecurityGroup +from openclear.domain.tenant import Tenant from openclear.domain.tenant_membership import TenantMembership from openclear.domain.user import User from openclear.services import session_service @@ -43,6 +44,14 @@ class AccessRevoked(Exception): immediately rather than waiting for the token to expire.""" +def _service_ended(customer) -> bool: + """Its last service day is over (services/customer_offboarding_service.py): nobody there signs + in, whatever a later sync set is_active to (REVIEW_1_33_PLAN.md O2, O5).""" + from openclear.services import customer_offboarding_service + + return customer_offboarding_service.ended(customer) + + def _pending_approval_count(db: Session, tenant_id: uuid.UUID, customer_id: uuid.UUID | None, user_id: uuid.UUID) -> int: stmt = select(func.count(ExceptionItem.id)).where( ExceptionItem.tenant_id == tenant_id, @@ -148,6 +157,8 @@ def decode_and_build_context(token: str, db: Session, *, expected_type: str) -> branding = get_tenant_branding_by_id(db, tenant_id) if branding is None: raise AccessRevoked("Tenant no longer exists") + if not db.get(Tenant, tenant_id).is_active: # loaded for the branding already; a bank frozen for deletion refuses every token + raise AccessRevoked("Tenant is no longer active") permissions = frozenset(group.permissions) security_key_due = _security_key_due(db, tenant_id, membership, payload) if expected_type == "access" else None @@ -164,7 +175,7 @@ def decode_and_build_context(token: str, db: Session, *, expected_type: str) -> # auth/permissions.py::CUSTOMER_SCOPE_MASKED_PERMISSIONS. permissions = permissions - CUSTOMER_SCOPE_MASKED_PERMISSIONS customer = db.get(Customer, customer_id) - if customer is None or not customer.is_active: + if customer is None or not customer.is_active or _service_ended(customer): raise AccessRevoked("Customer no longer exists") customer_name = customer.name customer_banner_message = customer.banner_message @@ -241,6 +252,8 @@ def _support_context(db: Session, payload: dict, user: User, tenant_id: uuid.UUI branding = get_tenant_branding_by_id(db, tenant_id) if branding is None: raise AccessRevoked("Tenant no longer exists") + if not db.get(Tenant, tenant_id).is_active: # loaded for the branding already; a bank frozen for deletion refuses every token + raise AccessRevoked("Tenant is no longer active") expires_at = grant.expires_at if grant.expires_at.tzinfo else grant.expires_at.replace(tzinfo=timezone.utc) ctx = TenantContext( tenant_id=tenant_id, user_id=user.id, security_group_id=uuid.UUID(payload["security_group_id"]), diff --git a/src/openclear/bulk_import/fields.py b/src/openclear/bulk_import/fields.py index cbe8cfa4..0bbb4cb4 100644 --- a/src/openclear/bulk_import/fields.py +++ b/src/openclear/bulk_import/fields.py @@ -4,11 +4,12 @@ from datetime import date, datetime from decimal import Decimal, InvalidOperation from typing import Any +from openclear.errors import UserFacingError _DATE_FORMATS = ("%Y-%m-%d", "%m/%d/%Y", "%m/%d/%y", "%Y/%m/%d") -class RowFieldError(ValueError): +class RowFieldError(UserFacingError): """A specific row/entry's field failed to parse or is missing — the message is surfaced directly to the user in the bulk-upload results page, so it's written to be read by someone looking at their own spreadsheet row, not a developer. `field` names diff --git a/src/openclear/bulk_import/layouts.py b/src/openclear/bulk_import/layouts.py index e2f661de..cb1ab4bb 100644 --- a/src/openclear/bulk_import/layouts.py +++ b/src/openclear/bulk_import/layouts.py @@ -42,6 +42,7 @@ from typing import Any, Callable from openclear.bulk_import.fields import RowFieldError +from openclear.errors import UserFacingError ISSUED_ITEMS, PAID_ITEMS = "issued_items", "paid_items" @@ -89,7 +90,7 @@ MAX_SAMPLE_ROWS = 15 -class LayoutError(ValueError): +class LayoutError(UserFacingError): """The layout itself is wrong, or the file can't be read by it; fit to show.""" @@ -443,6 +444,8 @@ def _amount(settings: dict, values: dict[str, str]) -> Decimal: text = text.strip("()") if any(ch.isalpha() for ch in text): raise RowFieldError(f"Amount {raw!r} contains a letter", "amount") + if len(text) > 24: # no amount is this long; a hostile file's digits must fail as a row, not as an error (REVIEW_1_33_PLAN.md T8) + raise RowFieldError(f"Amount {raw[:24]!r}... is too long to be an amount", "amount") try: if settings.get("amount_format") == "implied": if not text.lstrip("-").isdigit(): @@ -640,8 +643,8 @@ def result(problem, count=None, total=None): raw_count = cells[trailer_settings["count_column"] - 1] if trailer_settings["count_column"] <= len(cells) else "" raw_total = cells[trailer_settings["total_column"] - 1] if trailer_settings["total_column"] <= len(cells) else "" raw_count, raw_total = raw_count.strip(), raw_total.strip() - if not raw_count.isdigit(): - return result(f"The trailer's item count {raw_count!r} isn't a number.") + if not raw_count.isdigit() or len(raw_count) > 18: + return result(f"The trailer's item count {raw_count[:24]!r} isn't a number.") total = _signed_amount(settings, raw_total) if raw_total.strip("0") else Decimal("0.00") if total is None: return result(f"The trailer's total {raw_total!r} isn't an amount.", int(raw_count)) diff --git a/src/openclear/bulk_import/tabular.py b/src/openclear/bulk_import/tabular.py index 30263ace..f91925d1 100644 --- a/src/openclear/bulk_import/tabular.py +++ b/src/openclear/bulk_import/tabular.py @@ -32,7 +32,7 @@ def _normalize_header(raw: str) -> str: return _HEADER_WHITESPACE.sub("_", str(raw).strip().lower()) -def parse_tabular_file(filename: str, content: bytes) -> list[dict[str, Any]]: +def parse_tabular_file(filename: str, content: bytes, *, keep_na_text: bool = False) -> list[dict[str, Any]]: """Parses any delimited file (comma/tab/semicolon/pipe — auto-detected) or Excel (.xlsx/.xls) file into a list of row dicts keyed by normalized column header (lowercased, stripped, internal whitespace collapsed to a single underscore), so @@ -44,11 +44,14 @@ def parse_tabular_file(filename: str, content: bytes) -> list[dict[str, Any]]: bulk_import.fields.""" lower_name = filename.lower() try: + # keep_na_text: a cell reading NA, N/A, null or None is that text, not an empty cell (a + # free-text custom field may hold it; REVIEW_1_33_PLAN.md C6). Elsewhere pandas' reading stands. + na = {"keep_default_na": False, "na_values": [""]} if keep_na_text else {} if lower_name.endswith((".xlsx", ".xls")): - df = pd.read_excel(io.BytesIO(content), dtype=str) + df = pd.read_excel(io.BytesIO(content), dtype=str, **na) else: delimiter = _detect_delimiter(content) - df = pd.read_csv(io.BytesIO(content), sep=delimiter, dtype=str) + df = pd.read_csv(io.BytesIO(content), sep=delimiter, dtype=str, **na) except Exception as exc: # noqa: BLE001 — pandas raises a wide variety of exception types for malformed input raise TabularParseError(f"Could not parse {filename!r}: {exc}") from exc diff --git a/src/openclear/compliance/checks.py b/src/openclear/compliance/checks.py index 793b980d..05c66a6e 100644 --- a/src/openclear/compliance/checks.py +++ b/src/openclear/compliance/checks.py @@ -51,6 +51,8 @@ def _registry() -> tuple[Check, ...]: Check("storage_encryption", "Disk and database encryption", lambda session, tenant_id: s._storage_encryption(), scope="installation"), Check("recovery_objectives", "Recovery objectives stated", lambda session, tenant_id: s._recovery_objectives(), scope="installation"), + Check("rate_limit_backend", "Rate limits shared across instances", lambda session, tenant_id: s._rate_limit_backend(), + scope="installation"), Check("edge", "Edge network set up", lambda session, tenant_id: s._edge(), scope="installation", needs_request=True), Check("audit_chain", "Action log intact", s._audit_chain), Check("segregation_of_duties", "Segregation of duties", s._sod), @@ -134,7 +136,7 @@ def by_key() -> dict[str, Check]: # The keys, without importing the services (compliance/controls.py and the CUEC inventory read # them at import time); tests/test_compliance checks this matches registry(). KEYS = ( - "field_encryption", "storage_encryption", "recovery_objectives", "edge", "audit_chain", "segregation_of_duties", "admin_mfa", "anomaly_capture", + "field_encryption", "storage_encryption", "recovery_objectives", "rate_limit_backend", "edge", "audit_chain", "segregation_of_duties", "admin_mfa", "anomaly_capture", "parser_hardening", "sla", "api_client_secrets", "customer_self_service", "event_delivery", "production_safety", "dormant_accounts", "customer_dormant_users", "privileged_accounts", "staff_mfa", "customer_mfa", "customer_sod", "customer_api_client_secrets", "customer_sso", "scim_sync", "integration_tls", diff --git a/src/openclear/compliance/controls.py b/src/openclear/compliance/controls.py index f3ca464c..5fb2ceb8 100644 --- a/src/openclear/compliance/controls.py +++ b/src/openclear/compliance/controls.py @@ -196,6 +196,11 @@ def as_dict(self) -> dict: "The host's statement of how disks and the database are encrypted is shown on the Controls page, which warns until " "there is one.", ("SC-28",), "detective", "built_in", ("storage_encryption",), ("H-SC-28(1)-01",), "Controls → Storage encryption"), + C("OC-SC-08", "Sign-in and API attempts rate-limited", + "Sign-in, token and password-reset attempts are limited per address and per route; with several instances the counts " + "are shared through Redis, and while Redis can't be reached each instance limits on its own and the Controls page " + "says so.", + ("SC-5", "AC-7"), "preventive", "built_in", ("rate_limit_backend",), (), "web/rate_limit.py; Controls → Rate limits shared across instances"), C("OC-SC-03", "Files signed", "Every uploaded file is kept with its SHA-256 and a signature, and every webhook and evidence bundle OpenClear sends " "is signed.", diff --git a/src/openclear/db/tenancy.py b/src/openclear/db/tenancy.py index b4b859d1..dd6e3725 100644 --- a/src/openclear/db/tenancy.py +++ b/src/openclear/db/tenancy.py @@ -4,7 +4,7 @@ import uuid from collections.abc import Iterator from contextlib import contextmanager -from datetime import datetime +from datetime import date, datetime from dataclasses import dataclass from sqlalchemy import event, text @@ -132,6 +132,7 @@ class TenantContext: # A customer's last positive pay service day, once its bank has scheduled it # (services/customer_offboarding_service.py): shown to its people on every page until then. service_ends_at: datetime | None = None + service_last_day: date | None = None # the same, as the day it is at the bank (REVIEW_1_33_PLAN.md O10) # The soonest decision deadline among them (base.html's "decide now" banner). nearest_deadline: datetime | None = None # For base.html's "New customer users" nav entry: people a customer's side added who wait diff --git a/src/openclear/domain/bank_deletion.py b/src/openclear/domain/bank_deletion.py index 9bb93d15..1ac1db02 100644 --- a/src/openclear/domain/bank_deletion.py +++ b/src/openclear/domain/bank_deletion.py @@ -19,7 +19,7 @@ class BankDeletion(Base): id: Mapped[uuid.UUID] = mapped_column(primary_key=True, default=new_uuid) bank_id: Mapped[uuid.UUID] = mapped_column(ForeignKey("tenant.id"), nullable=False, index=True) - status: Mapped[str] = mapped_column(String(10), nullable=False, default="queued") # queued, done, cancelled, refused + status: Mapped[str] = mapped_column(String(10), nullable=False, default="queued") # queued, done, cancelled, refused, failed requested_by_user_id: Mapped[uuid.UUID | None] = mapped_column(ForeignKey("user.id"), nullable=True) approved_by_user_id: Mapped[uuid.UUID | None] = mapped_column(ForeignKey("user.id"), nullable=True) requested_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), nullable=False) diff --git a/src/openclear/domain/teller_verification.py b/src/openclear/domain/teller_verification.py index 0d5689a2..266e6d70 100644 --- a/src/openclear/domain/teller_verification.py +++ b/src/openclear/domain/teller_verification.py @@ -35,6 +35,8 @@ class TellerVerification(Base): teller_id: Mapped[str | None] = mapped_column(String(64), nullable=True) branch: Mapped[str | None] = mapped_column(String(64), nullable=True) api_client_id: Mapped[uuid.UUID | None] = mapped_column(nullable=True) + # The token's user: the platform's client user, or a person given check:verify (counted for probing by themselves). + user_id: Mapped[uuid.UUID | None] = mapped_column(ForeignKey("user.id"), nullable=True) created_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), server_default=func.now(), nullable=False, index=True) expires_at: Mapped[datetime] = mapped_column(DateTime(timezone=True), nullable=False) cashed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True) diff --git a/src/openclear/errors.py b/src/openclear/errors.py new file mode 100644 index 00000000..96c35dc3 --- /dev/null +++ b/src/openclear/errors.py @@ -0,0 +1,13 @@ +# SPDX-License-Identifier: AGPL-3.0-or-later +# Copyright (C) 2026 Chaffed + +"""The one error type for sentences meant for people (REVIEW_1_33_PLAN.md K2, RV-D9). + +A service that refuses something a person asked for raises a subclass of UserFacingError with a +sentence written for them; the web routers show that sentence (web/errors.py::shown). Any other +exception's text (a library's, a database driver's) is never shown: the person gets a fixed +sentence and the log gets the detail. Kept free of imports so every layer can use it.""" + + +class UserFacingError(ValueError): + """Fit to show to the person who asked.""" diff --git a/src/openclear/networks/ach/returns_file.py b/src/openclear/networks/ach/returns_file.py index 8edc195d..5479d5d7 100644 --- a/src/openclear/networks/ach/returns_file.py +++ b/src/openclear/networks/ach/returns_file.py @@ -15,6 +15,8 @@ from __future__ import annotations +from openclear.errors import UserFacingError + from collections import OrderedDict from datetime import date, datetime, timezone from decimal import Decimal @@ -27,7 +29,7 @@ ) -class ReturnsFileError(ValueError): +class ReturnsFileError(UserFacingError): pass diff --git a/src/openclear/services/access_review_service.py b/src/openclear/services/access_review_service.py index 9b019bdd..dbcda711 100644 --- a/src/openclear/services/access_review_service.py +++ b/src/openclear/services/access_review_service.py @@ -17,6 +17,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.domain.access_review import AccessReview, AccessReviewItem from openclear.domain.customer import Customer from openclear.domain.security_group import SecurityGroup @@ -26,7 +27,7 @@ DECISIONS = ("confirmed", "changed", "removed") -class ReviewError(ValueError): +class ReviewError(UserFacingError): """The message is fit to show.""" diff --git a/src/openclear/services/account_access_service.py b/src/openclear/services/account_access_service.py index fbe3beaf..be4ed09b 100644 --- a/src/openclear/services/account_access_service.py +++ b/src/openclear/services/account_access_service.py @@ -14,12 +14,13 @@ from sqlalchemy import delete, select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.domain.account import Account from openclear.domain.tenant_membership import MembershipAccount, TenantMembership from openclear.repositories.tenant_membership_repo import TenantMembershipRepository -class AccountAccessError(ValueError): +class AccountAccessError(UserFacingError): """Bad input; the message is fit to show.""" diff --git a/src/openclear/services/account_service.py b/src/openclear/services/account_service.py index 125030aa..c2da3364 100644 --- a/src/openclear/services/account_service.py +++ b/src/openclear/services/account_service.py @@ -60,15 +60,27 @@ class ServiceEnded(RowFieldError): def _refuse_ended(session: Session, account: Account | None) -> Account | None: from openclear.domain.customer import Customer + from openclear.services import customer_offboarding_service as offboarding if account is not None and account.customer_id is not None: customer = session.get(Customer, account.customer_id) - if customer is not None and customer.service_ended_at is not None: - raise ServiceEnded(f"Positive pay ended for {customer.name}, this account's customer, on " - f"{customer.service_ended_at:%Y-%m-%d}; its items are no longer read.", "account_number") + if offboarding.ended(customer): # from midnight at the bank on its last day, not from the daily run (RV-D5) + raise ServiceEnded(f"Positive pay ended for {customer.name}, this account's customer, after " + f"{offboarding.last_service_day(session, customer):%Y-%m-%d}; its items are no longer read.", + "account_number") return account +def find_account_by_number(session: Session, tenant_id: uuid.UUID, account_number: str, *, + customer_id: uuid.UUID | None = None) -> Account | None: + """get_account_by_number without the refusal of an ended customer's account: for a teller + asking about a check, or a replay over history, where the caller decides what ended service + means (REVIEW_1_33_PLAN.md T4).""" + repo = AccountRepository(session, tenant_id, customer_id) + matches = repo.list(account_number=account_number) or repo.list(external_account_id=account_number) + return matches[0] if matches else None + + def get_account_by_number( session: Session, tenant_id: uuid.UUID, account_number: str, *, customer_id: uuid.UUID | None = None ) -> Account | None: diff --git a/src/openclear/services/ach_return_reason_service.py b/src/openclear/services/ach_return_reason_service.py index f9fc6839..04a9c46c 100644 --- a/src/openclear/services/ach_return_reason_service.py +++ b/src/openclear/services/ach_return_reason_service.py @@ -7,11 +7,12 @@ from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.domain.ach_return_reason import AchReturnReason from openclear.repositories.ach_return_reason_repo import AchReturnReasonRepository -class InvalidAchReturnReasonInput(ValueError): +class InvalidAchReturnReasonInput(UserFacingError): """Raised for a blank reason_text or a non-blank, non-digit transaction_code — callers turn this into a form error, never a 500.""" diff --git a/src/openclear/services/api_client_service.py b/src/openclear/services/api_client_service.py index 16f7e73a..7258ab73 100644 --- a/src/openclear/services/api_client_service.py +++ b/src/openclear/services/api_client_service.py @@ -26,6 +26,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.auth.security import hash_password from openclear.db.tenancy import bind_session_to_tenant from openclear.domain.api_client import ApiClient @@ -39,7 +40,7 @@ _SERVICE_EMAIL_DOMAIN = "api-client.invalid" # .invalid is reserved: no inbox, no identity provider -class ApiClientError(ValueError): +class ApiClientError(UserFacingError): """Bad input when creating or changing a client; the message is fit to show.""" diff --git a/src/openclear/services/attestation_service.py b/src/openclear/services/attestation_service.py index 779a67f2..fa5991ae 100644 --- a/src/openclear/services/attestation_service.py +++ b/src/openclear/services/attestation_service.py @@ -32,6 +32,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.compliance import cuecs from openclear.domain.cuec_attestation import CuecAttestation, CuecMapping from openclear.domain.tenant import Tenant @@ -42,7 +43,7 @@ COVERAGE_NEEDED = 0.95 -class AttestationError(ValueError): +class AttestationError(UserFacingError): """The message is fit to show.""" diff --git a/src/openclear/services/bank_form_service.py b/src/openclear/services/bank_form_service.py index 1652963e..1022e81b 100644 --- a/src/openclear/services/bank_form_service.py +++ b/src/openclear/services/bank_form_service.py @@ -10,6 +10,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear import storage from openclear.domain.bank_form import BankForm @@ -19,7 +20,7 @@ MAX_PDF_BYTES = 10 * 1024 * 1024 -class FormError(ValueError): +class FormError(UserFacingError): """The message is fit to show.""" diff --git a/src/openclear/services/bulk_import_service.py b/src/openclear/services/bulk_import_service.py index a158f2cb..4705da65 100644 --- a/src/openclear/services/bulk_import_service.py +++ b/src/openclear/services/bulk_import_service.py @@ -124,6 +124,8 @@ class ImportSpec: creates_check_image: bool = False # `load` also takes the upload's options (a layout's settings, for "layout"). load_takes_options: bool = False + # `load` also takes (session, tenant_id): it checks the file against the bank's definitions. + load_takes_bank: bool = False # It updates records that exist (custom-field values, CF-D10): `ingest` returns a # custom_field_service.RowUpdate, nothing is tracked for backing out, and the file is # audited once when it's done instead of once per row. @@ -146,9 +148,9 @@ def _local_file(ref: str, loaded_closers: list) -> Path: return path -def _load_tabular(path: str, filename: str) -> LoadedFile: +def _load_tabular(path: str, filename: str, *, keep_na_text: bool = False) -> LoadedFile: try: - rows = parse_tabular_file(filename, _read(path)) + rows = parse_tabular_file(filename, _read(path), keep_na_text=keep_na_text) except TabularParseError as exc: raise BulkFileRejected(str(exc)) from None if not rows: @@ -378,20 +380,30 @@ def _account_tabular(ctx: RowContext, row: SourceRow, loaded: LoadedFile) -> uui return account_service.create_account_from_row(ctx.session, ctx.tenant_id, row.data, scoped_customer_id=ctx.scoped_customer_id) -def _field_loader(owner: str) -> Callable[[str, str], LoadedFile]: +def _field_loader(owner: str) -> Callable: """An update file of custom-field values (plans/CUSTOM_FIELDS_PLAN.md CF-D10): refused whole - if it doesn't name its records or has no cf_ column.""" - from openclear.services.custom_field_service import RECORD_KEYS + if it doesn't name its records, has no cf_ column, or names a field the bank doesn't have or + has archived (one mistyped header shouldn't fail twenty thousand rows one by one; + REVIEW_1_33_PLAN.md C5).""" + from openclear.services import custom_field_service as cf - keys = RECORD_KEYS[owner] + keys = cf.RECORD_KEYS[owner] - def load(path: str, filename: str) -> LoadedFile: - loaded = _load_tabular(path, filename) + def load(path: str, filename: str, session: Session, tenant_id: uuid.UUID) -> LoadedFile: + loaded = _load_tabular(path, filename, keep_na_text=True) columns = set(loaded.rows[0].data) if not columns & set(keys): raise BulkFileRejected(f"Name each record with a {keys[0]} or {keys[1]} column.") - if not any(str(c).startswith("cf_") for c in columns): + wanted = {str(c)[3:] for c in columns if str(c).startswith("cf_")} + if not wanted: raise BulkFileRejected("No custom-field columns: name each one cf_ and the field's key, such as cf_officer_code.") + fields = {f.key: f for f in cf.list_fields(session, tenant_id, owner, include_archived=True)} + unknown = sorted(k for k in wanted if k not in fields) + if unknown: + raise BulkFileRejected(f"No such field: {', '.join('cf_' + k for k in unknown)}. The field's key follows cf_.") + archived = sorted(k for k in wanted if fields[k].archived_at is not None) + if archived: + raise BulkFileRejected(f"Archived, so not written: {', '.join('cf_' + k for k in archived)}.") return loaded return load @@ -437,17 +449,21 @@ def ingest(ctx: RowContext, row: SourceRow, loaded: LoadedFile): (BulkUploadKind.ACCOUNTS, "tabular"): ImportSpec( _load_tabular, _account_tabular, resource_type="account", audit_action="account.create", audit_noun="Account" ), - (BulkUploadKind.ACCOUNT_FIELDS, "tabular"): ImportSpec(_field_loader("account"), _field_updater("account"), **_ACCOUNT_FIELDS), + (BulkUploadKind.ACCOUNT_FIELDS, "tabular"): ImportSpec(_field_loader("account"), _field_updater("account"), load_takes_bank=True, + **_ACCOUNT_FIELDS), (BulkUploadKind.ACCOUNT_FIELDS, "json"): ImportSpec(_load_json, _field_updater("account"), **_ACCOUNT_FIELDS), - (BulkUploadKind.CUSTOMER_FIELDS, "tabular"): ImportSpec(_field_loader("customer"), _field_updater("customer"), **_CUSTOMER_FIELDS), + (BulkUploadKind.CUSTOMER_FIELDS, "tabular"): ImportSpec(_field_loader("customer"), _field_updater("customer"), load_takes_bank=True, + **_CUSTOMER_FIELDS), (BulkUploadKind.CUSTOMER_FIELDS, "json"): ImportSpec(_load_json, _field_updater("customer"), **_CUSTOMER_FIELDS), } @contextlib.contextmanager -def _loaded(spec: ImportSpec, record: BulkUploadFile) -> Iterator[LoadedFile]: +def _loaded(spec: ImportSpec, record: BulkUploadFile, session: Session | None = None) -> Iterator[LoadedFile]: if spec.load_takes_options: loaded = spec.load(record.storage_path, record.original_filename, record.options or {}) + elif spec.load_takes_bank: + loaded = spec.load(record.storage_path, record.original_filename, session, record.tenant_id) else: loaded = spec.load(record.storage_path, record.original_filename) try: @@ -516,7 +532,7 @@ def receive_upload( record.options = {**(options or {}), "format": file_format} spec = spec_for(kind, file_format) try: - with _loaded(spec, record) as loaded: + with _loaded(spec, record, session) as loaded: record.total_rows = len(loaded.rows) if loaded.skipped: record.options = {**record.options, "skipped_rows": loaded.skipped} # reassigned, so the JSON column is saved @@ -600,7 +616,7 @@ def _import(session: Session, record: BulkUploadFile) -> None: chunk_rows = max(1, get_settings().bulk_import_chunk_rows) queued_ocr = False - with _loaded(spec, record) as loaded, notification_service.exceptions_summarized(): + with _loaded(spec, record, session) as loaded, notification_service.exceptions_summarized(): rows = loaded.rows record.total_rows = len(rows) start = record.processed_rows or 0 @@ -858,7 +874,7 @@ def open_failures(session: Session, record: BulkUploadFile, *, limit: int = MAX_ failures = [r for r in row_results(session, record, failures_only=True) if r.row_index not in fixed][:limit] if not failures: return [] - with _loaded(_spec(record), record) as loaded: + with _loaded(_spec(record), record, session) as loaded: rows = [] for result in failures: if result.row_index >= len(loaded.rows) or not isinstance(loaded.rows[result.row_index].data, dict): @@ -887,7 +903,7 @@ def apply_corrections(session: Session, record: BulkUploadFile, edits: dict[int, ctx = RowContext(session, record.tenant_id, record.customer_id, user_id, record.options or {}) fixed = still = 0 audit = [] - with _loaded(spec, record) as loaded: + with _loaded(spec, record, session) as loaded: for row_index, values in sorted(edits.items())[:MAX_FIXES_SHOWN]: if row_index not in failed or row_index >= len(loaded.rows): continue diff --git a/src/openclear/services/bulk_upload_reversal_service.py b/src/openclear/services/bulk_upload_reversal_service.py index b2fe869b..3b23346c 100644 --- a/src/openclear/services/bulk_upload_reversal_service.py +++ b/src/openclear/services/bulk_upload_reversal_service.py @@ -14,6 +14,7 @@ from datetime import datetime, timezone from typing import Literal +from sqlalchemy import select from sqlalchemy.orm import Session from openclear.bulk_import.results import BulkFileRowResult @@ -154,6 +155,13 @@ def _back_out_paid_item( note = "its linked issued item's status had already changed since and was left as-is" elif paid_item.match_status == PaidItemMatchStatus.EXCEPTION: note = _withdraw_exception_if_open(session, tenant_id, paid_item.id, actor_user_id=actor_user_id, channel=channel) + if paid_item.settlement_status == PaidItemSettlementStatus.PAID: + # Paid by a decision (REVIEW_1_33_PLAN.md RV-D1): the issued check it paid is outstanding again, as on a match. + issued_id = session.execute(select(ExceptionItem.related_reference_id).where( + ExceptionItem.source_item_id == paid_item.id, ExceptionItem.network_code == "check")).scalar_one_or_none() + issued_item = IssuedItemRepository(session, tenant_id, scoped_customer_id).get(issued_id) if issued_id else None + if issued_item is not None and issued_item.status == IssuedItemStatus.PAID: + issued_item.status = IssuedItemStatus.OUTSTANDING paid_item.settlement_status = PaidItemSettlementStatus.REVERSED session.flush() diff --git a/src/openclear/services/change_kinds.py b/src/openclear/services/change_kinds.py index d4bca7d3..f98ad442 100644 --- a/src/openclear/services/change_kinds.py +++ b/src/openclear/services/change_kinds.py @@ -17,10 +17,12 @@ from collections.abc import Callable from dataclasses import dataclass, field +from openclear.errors import UserFacingError + from sqlalchemy.orm import Session -class ChangeError(ValueError): +class ChangeError(UserFacingError): """The message is fit to show.""" @@ -43,6 +45,9 @@ class Kind: # Turning a second-person switch on: the permissions someone else must hold (DC-D16), by what turns on. switches: Callable[[Session, uuid.UUID | None, uuid.UUID | None, dict, dict], list[tuple[str, str]]] = field( default=lambda session, tenant_id, customer_id, before, after: []) + # A second person, always: with nobody else to approve, the request is refused rather than + # applied after the cooling-off, since what it does can't be undone (REVIEW_1_33_PLAN.md O3, RV-D4). + second_person: bool = False _KINDS: dict[str, Kind] = {} @@ -1476,7 +1481,7 @@ def _customer_offboard_apply(session, tenant_id, customer_id, payload, user_id) # Scheduling the end needs a second administrator; stopping it before the last day keeps the service, so one # administrator can; restoring a customer whose service already ended reopens access, so it takes two. loosens=lambda before, after: bool(after["last_service_day"]) or before["ended"], - apply=_customer_offboard_apply, setting=lambda payload: f"customer_offboard:{payload['customer_id']}", + apply=_customer_offboard_apply, setting=lambda payload: f"customer_offboard:{payload['customer_id']}", second_person=True, describe=lambda before, after: (f"End a customer's positive pay on {after['last_service_day']}" if after["last_service_day"] else "Keep a customer on positive pay"), )) @@ -1532,7 +1537,7 @@ def _offboard_apply(session, tenant_id, customer_id, payload, user_id) -> str: "platform.bank.offboard", "Offboarding a bank", "protection", _OPERATOR, _offboard_current, lambda payload: {"offboarding": bool(payload["offboarding"])}, loosens=lambda before, after: after["offboarding"], # stopping it keeps data: one operator - apply=_offboard_apply, scope="provider", setting=lambda payload: f"bank_offboard:{payload['bank_id']}", + apply=_offboard_apply, scope="provider", setting=lambda payload: f"bank_offboard:{payload['bank_id']}", second_person=True, describe=lambda before, after: "Start offboarding a bank" if after["offboarding"] else "Stop offboarding a bank", )) @@ -1555,7 +1560,7 @@ def _delete_apply(session, tenant_id, customer_id, payload, user_id) -> str: register(Kind( "platform.bank.delete", "Deleting a bank's data", "protection", _OPERATOR, _delete_current, lambda payload: {"deleted": True}, loosens=lambda before, after: True, # always two operators - apply=_delete_apply, scope="provider", setting=lambda payload: f"bank_delete:{payload['bank_id']}", + apply=_delete_apply, scope="provider", setting=lambda payload: f"bank_delete:{payload['bank_id']}", second_person=True, describe=lambda before, after: "Delete a bank's data", )) diff --git a/src/openclear/services/change_request_service.py b/src/openclear/services/change_request_service.py index d5c65ee3..c00ca80e 100644 --- a/src/openclear/services/change_request_service.py +++ b/src/openclear/services/change_request_service.py @@ -189,6 +189,9 @@ def request_or_apply(session: Session, kind_key: str, payload: dict, *, tenant_i raise ChangeError(f"A change to {kind.title.lower()} is already waiting for approval: approve, reject or withdraw it first.") now = _now() others = approvers(session, tenant_id, customer_id, kind.permission, excluding=user_id) + if not others and kind.second_person: + who = "operator" if tenant_id is None else "administrator" + raise ChangeError(f"{kind.title} needs a second {who} to approve it, and nobody else here can. Add one first.") change = ChangeRequest( tenant_id=tenant_id, customer_id=customer_id, scope="provider" if tenant_id is None else ("customer" if customer_id else "bank"), kind=kind.key, setting=setting, summary=summary, payload=payload, before=_jsonable(before), after=_jsonable(after), diff --git a/src/openclear/services/check_return_reason_service.py b/src/openclear/services/check_return_reason_service.py index c6f1906c..17b48699 100644 --- a/src/openclear/services/check_return_reason_service.py +++ b/src/openclear/services/check_return_reason_service.py @@ -8,6 +8,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.domain.check_return_reason import CheckReturnReason # Seeded for every bank: new ones at provisioning, existing ones by the migration that @@ -43,7 +44,7 @@ def get_check_return_reason(session: Session, tenant_id: uuid.UUID, reason_id: u return reason if reason is not None and reason.tenant_id == tenant_id else None -class InvalidCheckReturnReasonInput(ValueError): +class InvalidCheckReturnReasonInput(UserFacingError): """Bad input; the message is fit to show.""" diff --git a/src/openclear/services/control_checks.py b/src/openclear/services/control_checks.py index bf7e8062..b736f1d5 100644 --- a/src/openclear/services/control_checks.py +++ b/src/openclear/services/control_checks.py @@ -430,15 +430,16 @@ def incident_contacts(session: Session, tenant_id: uuid.UUID) -> Control: evidence = {"contacts": len(named), "ceo_and_cio": sorted(c.kind for c in rows if c.kind != "contact"), "confirmed_at": confirmed.isoformat() if confirmed else None, "unacknowledged_notices": len(waiting)} title = "Incident contacts named" - if not named: - return Control("incident_contacts", title, WARN, "No incident contacts are named, so the service provider's incident notices go " - "to your CEO and CIO, or your administrators (Settings → Incident contacts).", evidence) + if not rows: + return Control("incident_contacts", title, WARN, "No incident contacts are named, nor a CEO and CIO, so the service provider's " + "incident notices go to your administrators (Settings → Incident contacts).", evidence) if waiting: return Control("incident_contacts", title, WARN, f"{_plural(len(waiting), 'incident notice')} from the service provider " "not acknowledged yet.", evidence) if confirmed is None or confirmed < _now() - timedelta(days=365): return Control("incident_contacts", title, WARN, "The incident contacts haven't been confirmed in a year.", evidence) - return Control("incident_contacts", title, PASS, f"{_plural(len(named), 'incident contact')}, confirmed {confirmed:%Y-%m-%d}.", evidence) + who = _plural(len(named), "incident contact") if named else "The CEO and CIO as the fallback (304.24(a)(2))" + return Control("incident_contacts", title, PASS, f"{who}, confirmed {confirmed:%Y-%m-%d}.", evidence) def custom_fields(session: Session, tenant_id: uuid.UUID) -> Control: diff --git a/src/openclear/services/control_results_service.py b/src/openclear/services/control_results_service.py index 3d4aca56..e1e4fe78 100644 --- a/src/openclear/services/control_results_service.py +++ b/src/openclear/services/control_results_service.py @@ -23,13 +23,14 @@ from sqlalchemy import delete, select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.domain.control_result import ControlDeviation, ControlTestResult from openclear.domain.tenant import Tenant RESPONSE_LIMIT = 4000 -class DeviationError(ValueError): +class DeviationError(UserFacingError): """The message is fit to show.""" diff --git a/src/openclear/services/controls_service.py b/src/openclear/services/controls_service.py index a2a1e994..5ba32433 100644 --- a/src/openclear/services/controls_service.py +++ b/src/openclear/services/controls_service.py @@ -71,6 +71,23 @@ def _storage_encryption() -> Control: "then record how they're done in OPENCLEAR_STORAGE_ENCRYPTION_ATTESTATION.", {"attestation": None}) +def _rate_limit_backend() -> Control: + """Shared rate limits need Redis; while it can't be reached each instance limits on its own + (web/rate_limit.py). The page says which is happening (REVIEW_1_33_PLAN.md C2).""" + from openclear.web import rate_limit + + title = "Rate limits shared across instances" + backend, reachable = rate_limit.backend_status() + evidence = {"backend": backend, "reachable": reachable} + if backend != "redis": + return Control("rate_limit_backend", title, PASS, "Rate limits are kept in each process: one instance, or an edge that limits " + "for it (OPENCLEAR_RATE_LIMIT_BACKEND=redis shares them).", evidence) + if not reachable: + return Control("rate_limit_backend", title, WARN, "Redis at OPENCLEAR_RATE_LIMIT_REDIS_URL can't be reached: each instance is " + "limiting sign-in and API attempts on its own until it can.", evidence) + return Control("rate_limit_backend", title, PASS, "Rate limits are shared through Redis, which answers.", evidence) + + def _recovery_objectives() -> Control: settings = get_settings() statement, tested = settings.recovery_objectives_attestation, settings.last_restore_test diff --git a/src/openclear/services/custom_field_service.py b/src/openclear/services/custom_field_service.py index 51190387..5aadcd58 100644 --- a/src/openclear/services/custom_field_service.py +++ b/src/openclear/services/custom_field_service.py @@ -24,6 +24,7 @@ from sqlalchemy.exc import IntegrityError from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.domain.account import Account from openclear.domain.custom_field import ( CustomField, @@ -44,7 +45,7 @@ _TRUE, _FALSE = {"true", "yes", "1", "y"}, {"false", "no", "0", "n"} -class CustomFieldError(ValueError): +class CustomFieldError(UserFacingError): """Fit to show.""" @@ -106,12 +107,12 @@ def parse(definition, text: str | None) -> Any: raise CustomFieldError(f"{label}: at most {limit} characters.") return text if kind == CustomFieldType.INTEGER: - if not re.fullmatch(r"[+-]?\d{1,18}", text): + if not re.fullmatch(r"[+-]?[0-9]{1,18}", text): # ASCII digits only: what the message says raise CustomFieldError(f"{label}: a whole number.") return int(text) if kind == CustomFieldType.DECIMAL: try: - value = Decimal(text) + value = Decimal(text) if re.fullmatch(r"[+-]?[0-9]*\.?[0-9]+", text) else Decimal("x") except InvalidOperation: raise CustomFieldError(f"{label}: a number.") from None if not value.is_finite() or value.as_tuple().exponent < -6 or abs(value) >= Decimal("1e12"): @@ -119,6 +120,8 @@ def parse(definition, text: str | None) -> Any: return value if kind == CustomFieldType.DATE: try: + if not re.fullmatch(r"[0-9]{4}-[0-9]{2}-[0-9]{2}", text): + raise ValueError return date.fromisoformat(text) except ValueError: raise CustomFieldError(f"{label}: a date as YYYY-MM-DD.") from None @@ -296,6 +299,11 @@ def update_field(session: Session, tenant_id: uuid.UUID, field_id: uuid.UUID, da definition = get_field(session, tenant_id, field_id) values_exist = _values_exist(session, CustomFieldValue, definition.id) columns = _clean_definition(data, existing=definition, values_exist=values_exist) + if columns["options"]: + # A dropped option is retired only while a record holds it (CF-D11); a typo'd one simply goes (REVIEW_1_33_PLAN.md C9). + in_use = set(session.execute(select(CustomFieldValue.text_value).where(CustomFieldValue.field_id == definition.id, + CustomFieldValue.text_value.is_not(None)).distinct()).scalars()) + columns["options"] = [o for o in columns["options"] if not o["retired"] or o["value"] in in_use] if data.show_in_list and not definition.show_in_list: _check_list_limit(session, tenant_id, definition.owner, excluding=definition.id) turning_unique = columns["unique"] and not definition.unique @@ -309,8 +317,16 @@ def update_field(session: Session, tenant_id: uuid.UUID, field_id: uuid.UUID, da def archive_field(session: Session, tenant_id: uuid.UUID, field_id: uuid.UUID, *, archived: bool = True) -> CustomField: - """Archived: hidden from forms, still read everywhere, never written (CF-D11).""" + """Archived: hidden from forms, still read everywhere, never written (CF-D11). Restoring one + counts against the limits like adding one (REVIEW_1_33_PLAN.md C4).""" definition = get_field(session, tenant_id, field_id) + if not archived and definition.archived_at is not None: + active = session.execute(select(func.count()).select_from(CustomField).where( + CustomField.tenant_id == tenant_id, CustomField.owner == definition.owner, CustomField.archived_at.is_(None))).scalar_one() + if active >= MAX_FIELDS: + raise CustomFieldError(f"At most {MAX_FIELDS} fields on {definition.owner}s; archive one first.") + if definition.show_in_list: + _check_list_limit(session, tenant_id, definition.owner, excluding=definition.id) definition.archived_at = datetime.now(timezone.utc) if archived else None session.flush() return definition @@ -363,15 +379,16 @@ def values_for(session: Session, tenant_id: uuid.UUID, owner: str, record_ids, * CUSTOMER_PREFIX = "customer." -def output_keys(session: Session, tenant_id: uuid.UUID, text: str | None) -> list[str]: +def output_keys(session: Session, tenant_id: uuid.UUID, text: str | None, *, customer_id: uuid.UUID | None = None) -> list[str]: """The fields a destination asks for, in its order (CF-D7): comma-separated, an account's field by its key and its customer's as customer.. Raises CustomFieldError naming any that don't - exist.""" + exist; for a customer's reader, a field not shown to the customer doesn't exist (REVIEW_1_33_PLAN.md C3).""" keys = list(dict.fromkeys(k.strip() for k in (text or "").split(",") if k.strip())) if not keys: return [] - known = {f.key for f in list_fields(session, tenant_id, CustomFieldOwner.ACCOUNT, include_archived=True)} - known |= {CUSTOMER_PREFIX + f.key for f in list_fields(session, tenant_id, CustomFieldOwner.CUSTOMER, include_archived=True)} + visible = (lambda f: f.shown_to_customer) if customer_id is not None else (lambda f: True) + known = {f.key for f in list_fields(session, tenant_id, CustomFieldOwner.ACCOUNT, include_archived=True) if visible(f)} + known |= {CUSTOMER_PREFIX + f.key for f in list_fields(session, tenant_id, CustomFieldOwner.CUSTOMER, include_archived=True) if visible(f)} unknown = [k for k in keys if k not in known] if unknown: raise CustomFieldError(f"No such field: {', '.join(unknown)}. Name an account's field by its key and a customer's as " @@ -421,14 +438,27 @@ def set_values(session: Session, tenant_id: uuid.UUID, owner: str, record_id: uu changes = Changes() savepoint = session.begin_nested() try: + problems: list[str] = [] + parsed_values = {} + for key, text in values.items(): # every field's complaint at once, not the first (REVIEW_1_33_PLAN.md C10) + definition = fields[key] + # A retired option already on the record stays if it's sent back unchanged. + if definition.field_type == CustomFieldType.CHOICE and text is not None \ + and text.strip() == format_value(definition, existing.get(definition.id)): + continue + try: + parsed_values[key] = parse(definition, text) + except CustomFieldError as exc: + problems.append(str(exc)) + if problems: + raise CustomFieldError(" ".join(problems)) for key, text in values.items(): + if key not in parsed_values: + continue definition = fields[key] row = existing.get(definition.id) before = format_value(definition, row) - # A retired option already on the record stays if it's sent back unchanged. - if definition.field_type == CustomFieldType.CHOICE and text is not None and text.strip() == before: - continue - parsed = parse(definition, text) + parsed = parsed_values[key] after = None if parsed is None else format_value(definition, _preview(definition, parsed)) if before == after: continue diff --git a/src/openclear/services/customer_approval_service.py b/src/openclear/services/customer_approval_service.py index d999657e..90bdd0b1 100644 --- a/src/openclear/services/customer_approval_service.py +++ b/src/openclear/services/customer_approval_service.py @@ -22,6 +22,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.domain.customer import Customer from openclear.domain.security_group import SecurityGroup from openclear.domain.tenant import Tenant @@ -29,7 +30,7 @@ from openclear.domain.user import User -class ApprovalError(ValueError): +class ApprovalError(UserFacingError): """The message is fit to show.""" diff --git a/src/openclear/services/customer_offboarding_service.py b/src/openclear/services/customer_offboarding_service.py index b32037e3..5a6fb32b 100644 --- a/src/openclear/services/customer_offboarding_service.py +++ b/src/openclear/services/customer_offboarding_service.py @@ -24,6 +24,7 @@ from sqlalchemy import delete, select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.config import get_settings from openclear.db.base import Base from openclear.db.tenancy import bind_session_to_tenant @@ -31,15 +32,18 @@ from openclear.domain.customer import Customer from openclear.domain.tenant import Tenant -# Never deleted with a customer: the bank's own records, and what isn't the customer's. -_KEPT_TABLES = frozenset({"tenant", "user", "audit_log_entry", "security_event", "customer_deletion", "bank_deletion"}) +# Never deleted with a customer: the bank's own records, and what isn't the customer's. The +# bank's attestations and access reviews about the customer are the bank's compliance evidence +# and outlive it, unlinked from the customer (REVIEW_1_33_PLAN.md O8, RV-D6). +_KEPT_TABLES = frozenset({"tenant", "user", "audit_log_entry", "security_event", "customer_deletion", "bank_deletion", + "cuec_attestation", "access_review", "access_review_item"}) # Columns holding a stored file's key, by table: the file goes with the row. _FILE_COLUMNS = {"check_image": ("front_image_path", "back_image_path"), "bulk_upload_file": ("storage_path",), "data_export_job": ("archive_path",)} _CHUNK = 500 -class CustomerOffboardingError(ValueError): +class CustomerOffboardingError(UserFacingError): """Fit to show.""" @@ -172,18 +176,35 @@ def set_hold(session: Session, tenant_id: uuid.UUID, customer_id: uuid.UUID, hel # --- Ending the service ---------------------------------------------------------------------- -def ended(customer: Customer | None) -> bool: - return customer is not None and customer.service_ended_at is not None +def ended(customer: Customer | None, *, now: datetime | None = None) -> bool: + """Service has ended: the daily run has recorded it, or the last day is over and the run + hasn't come yet (REVIEW_1_33_PLAN.md O5, RV-D5). Every refusal reads this, so service ends at + midnight at the bank, not at the next run.""" + if customer is None: + return False + if customer.service_ended_at is not None: + return True + ends = _aware(customer.service_ends_at) + return ends is not None and ends <= (now or _now()) + + +def last_service_day(session: Session, customer: Customer) -> date | None: + """The last day, as the bank counts days: the one that ended when `service_ends_at` struck.""" + end = _aware(customer.service_ends_at) or _aware(customer.service_ended_at) + return (end.astimezone(_zone(session, customer.tenant_id)) - timedelta(seconds=1)).date() if end else None def end_service(session: Session, customer: Customer) -> None: """The last service day has passed: no sign-in, no sessions; files naming its accounts are - refused (services/account_service.py), and tellers are told it's not on positive pay.""" + refused (services/account_service.py), and tellers are told it's not on positive pay. The + end is recorded as the moment the last day ended, not when this ran, so the retention clock + and the pages agree with the day the customer was told.""" from openclear.services import session_service - customer.service_ended_at, customer.is_active = _now(), False - for user in _people(session, customer): - session_service.revoke_all_sessions(session, user) + customer.service_ended_at, customer.is_active = _aware(customer.service_ends_at) or _now(), False + # Their sessions at this bank only: someone who also works elsewhere stays signed in there (REVIEW_1_33_PLAN.md O11). + people = [u.id for u in _people(session, customer)] + session_service.end_sessions(session, session_service.active_sessions(session, customer.tenant_id, people)) _record(session, customer, "customer.offboard.ended", f"Positive pay for {customer.name} ended; its data is deleted on or after " f"{deletion_due(customer).date().isoformat()}", user_id=None) @@ -222,6 +243,24 @@ def _collect(session: Session, customer: Customer) -> dict[str, set]: return found +def _unlink_kept(session: Session, customer: Customer) -> dict[str, int]: + """The kept tables' references to the customer are cleared, so its row can go and the + records stay as the bank's own. Returns the rows unlinked per table.""" + from sqlalchemy import update + + unlinked: dict[str, int] = {} + for name in sorted(_KEPT_TABLES): + table = Base.metadata.tables.get(name) + if table is None: + continue + for fk in table.foreign_keys: + if fk.column.table.name == "customer" and fk.column.name == "id" and fk.parent.nullable: + count = session.execute(update(table).where(fk.parent == customer.id).values({fk.parent.name: None})).rowcount or 0 + if count: + unlinked[name] = unlinked.get(name, 0) + count + return unlinked + + def _files(session: Session, found: dict[str, set]) -> list[str]: paths: list[str] = [] for name, columns in _FILE_COLUMNS.items(): @@ -271,6 +310,7 @@ def delete_data(session: Session, customer: Customer) -> CustomerDeletion: _record(session, customer, "customer.offboard.delete", f"Deleting {customer.name}'s data: {sum(len(v) for v in found.values())} rows, {len(paths)} files", user_id=None) session.flush() + unlinked = _unlink_kept(session, customer) rows: dict[str, int] = {} for table in reversed(Base.metadata.sorted_tables): ids = found.get(table.name) @@ -307,8 +347,10 @@ def belongs_elsewhere(user_id: uuid.UUID) -> bool: "files_deleted": len(paths), "people_deleted": people_deleted, "people_kept_because_they_belong_elsewhere_or_are_named_in_records": len(people) - len(alone) + still_referenced, + "people_anonymised_because_they_are_named_in_records": still_referenced, + "rows_kept_unlinked_from_the_customer": dict(sorted(unlinked.items())), "kept": "The bank's action log and security events, which record what was done for the customer and this " - "deletion; this certificate.", + "deletion; the bank's attestations and access reviews about the customer, no longer naming it; this certificate.", "action_log_last_entry": {"sequence": last.sequence, "entry_hash": last.entry_hash} if last else None, "backups": BACKUPS_NOTE, } diff --git a/src/openclear/services/customer_people_service.py b/src/openclear/services/customer_people_service.py index 18df7be9..24141e9b 100644 --- a/src/openclear/services/customer_people_service.py +++ b/src/openclear/services/customer_people_service.py @@ -24,6 +24,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.auth.password_policy import context_words, effective_policy, validate_password from openclear.auth.security import hash_password, verify_password from openclear.db.tenancy import bind_session_to_tenant, memberships_elsewhere @@ -40,7 +41,7 @@ INVITATION_DAYS = 7 -class PeopleError(ValueError): +class PeopleError(UserFacingError): """Refused; the message is fit to show.""" diff --git a/src/openclear/services/customer_sign_on_service.py b/src/openclear/services/customer_sign_on_service.py index 8cfcd017..40460543 100644 --- a/src/openclear/services/customer_sign_on_service.py +++ b/src/openclear/services/customer_sign_on_service.py @@ -23,6 +23,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.domain.customer import Customer from openclear.domain.customer_admin import CustomerEmailDomain from openclear.domain.security_group import SecurityGroup @@ -40,7 +41,7 @@ }) -class SignOnError(ValueError): +class SignOnError(UserFacingError): """Refused; the message is fit to show.""" diff --git a/src/openclear/services/cutoff_service.py b/src/openclear/services/cutoff_service.py index c740f0a0..cbb7828b 100644 --- a/src/openclear/services/cutoff_service.py +++ b/src/openclear/services/cutoff_service.py @@ -20,6 +20,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.domain.bank_closure import BankClosure from openclear.domain.customer_disposition_setting import CustomerDispositionSetting from openclear.config import get_settings @@ -29,7 +30,7 @@ MAX_REVIEW_MINUTES = 24 * 60 -class CutoffError(ValueError): +class CutoffError(UserFacingError): """The message is fit to show.""" @@ -130,16 +131,22 @@ def time_zone_for(tenant: Tenant) -> str: # --- Deadlines ---------------------------------------------------------------------------------- -def next_cutoff(arrived: datetime, cutoff: str, zone: str, min_review_minutes: int, closed: frozenset[date] = frozenset()) -> datetime: - """The first `cutoff` (HH:MM in `zone`) on a business day at least `min_review_minutes` - after `arrived`, in UTC. The local time is kept across a daylight-saving change.""" +def _now() -> datetime: + return datetime.now(timezone.utc) + + +def next_cutoff(arrived: datetime, cutoff: str, zone: str, min_review_minutes: int, closed: frozenset[date] = frozenset(), *, + open_on_federal_holidays: bool = False) -> datetime: + """The first `cutoff` (HH:MM in `zone`) on a banking day at least `min_review_minutes` + after `arrived`, in UTC. The local time is kept across a daylight-saving change. A bank + that opens on the Federal Reserve's holidays has its cut-off on them too (REVIEW_1_33_PLAN.md G6).""" tz = ZoneInfo(zone) earliest = arrived.astimezone(tz) + timedelta(minutes=min_review_minutes) hours, minutes = (int(part) for part in cutoff.split(":")) day = earliest.date() for _ in range(400): # a year and more of closed days would be a configuration error candidate = datetime.combine(day, time(hours, minutes), tzinfo=tz) - if candidate >= earliest and is_business_day(day, closed): + if candidate >= earliest and is_banking_day(day, closed, open_on_federal_holidays=open_on_federal_holidays): return candidate.astimezone(timezone.utc) day += timedelta(days=1) raise CutoffError("No business day was found for the cut-off within a year; check the bank's closures.") @@ -160,7 +167,8 @@ def deadline_for(session: Session, tenant_id: uuid.UUID, customer_id: uuid.UUID except ZoneInfoNotFoundError: zone = ZoneInfo("UTC") closed = frozenset(c.closed_on for c in closures(session, tenant_id)) - return next_cutoff(arrived or datetime.now(timezone.utc), cutoff, zone.key, tenant.cutoff_min_review_minutes, closed) + return next_cutoff(arrived or _now(), cutoff, zone.key, tenant.cutoff_min_review_minutes, closed, + open_on_federal_holidays=tenant.open_on_federal_holidays) # --- A check's return deadline (REGULATORY_PLAN.md RG1) ------------------------------------------ @@ -210,8 +218,10 @@ def within_return_deadline(session: Session, tenant_id: uuid.UUID, customer_id: deadline: datetime | None, returns_by: datetime | None) -> datetime | None: """`deadline`, or earlier: never after the cut-off on the check's return day (or, with no cut-off, the return deadline itself), so the core can still send the return in time. A - check that arrives after that is due at once. No deadline (no cut-off anywhere: the tests) - stays none.""" + check that arrives after that cut-off, while its midnight deadline is still ahead, gets the + bank's minimum review time, never past midnight: a person sees it before a default decides + it (REVIEW_1_33_PLAN.md G1, RV-D7). One whose midnight deadline has passed is due at once. + No deadline (no cut-off anywhere: the tests) stays none.""" if returns_by is None or deadline is None: return deadline tenant = session.get(Tenant, tenant_id) @@ -225,6 +235,9 @@ def within_return_deadline(session: Session, tenant_id: uuid.UUID, customer_id: hours, minutes = (int(part) for part in cutoff.split(":")) last_day = (returns_by.astimezone(zone) - timedelta(days=1)).date() latest = datetime.combine(last_day, time(hours, minutes), tzinfo=zone).astimezone(timezone.utc) + now = _now() + if latest < now < returns_by: + latest = min(returns_by, now + timedelta(minutes=tenant.cutoff_min_review_minutes)) return latest if deadline is None or deadline > latest else deadline diff --git a/src/openclear/services/decision_feed_service.py b/src/openclear/services/decision_feed_service.py index 03430e1f..5209615d 100644 --- a/src/openclear/services/decision_feed_service.py +++ b/src/openclear/services/decision_feed_service.py @@ -17,6 +17,7 @@ import csv import hashlib import io +import re import uuid from datetime import datetime, timezone @@ -160,10 +161,14 @@ def custom_fields_setting(settings: dict | None) -> list[str]: return [k.strip() for k in str((settings or {}).get("custom_fields") or "").split(",") if k.strip()] +_PLAIN_NUMBER = re.compile(r"[+-]?\d+(\.\d+)?") + + def _csv_safe(value): # Originator names come from ACH files, custom-field values from people (CF-D13); don't let a - # spreadsheet run one. - if isinstance(value, str) and value[:1] in ("=", "+", "-", "@", "\t", "\r"): + # spreadsheet run one. A plain signed number (a decimal custom field's -5) is a number to a + # spreadsheet, not a formula, and must reach the core as one (REVIEW_1_33_PLAN.md C1). + if isinstance(value, str) and value[:1] in ("=", "+", "-", "@", "\t", "\r") and not _PLAIN_NUMBER.fullmatch(value): return "'" + value return value diff --git a/src/openclear/services/decision_outbox_service.py b/src/openclear/services/decision_outbox_service.py index 656257f4..b77459ef 100644 --- a/src/openclear/services/decision_outbox_service.py +++ b/src/openclear/services/decision_outbox_service.py @@ -26,6 +26,7 @@ from sqlalchemy import func, select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.domain.decision_outbox import DecisionBatch, DecisionOutbox logger = logging.getLogger(__name__) @@ -34,7 +35,7 @@ CONSUMER = "outbox" -class OutboxError(ValueError): +class OutboxError(UserFacingError): """Refused; the message is fit to show.""" diff --git a/src/openclear/services/decision_service.py b/src/openclear/services/decision_service.py index 47fc1e13..f8520766 100644 --- a/src/openclear/services/decision_service.py +++ b/src/openclear/services/decision_service.py @@ -14,6 +14,8 @@ from openclear.domain.ach_transaction import AchSettlementStatus, AchTransaction from openclear.domain.decision import Decision, DecisionOutcome, DecisionRevision, DecisionSource from openclear.domain.exception_item import ExceptionItem, ExceptionStatus +from openclear.domain.issued_item import IssuedItem, IssuedItemStatus +from openclear.domain.paid_item import PaidItem, PaidItemSettlementStatus from openclear.domain.tenant import Tenant from openclear.networks.registry import get_adapter from openclear.repositories.ach_return_reason_repo import AchReturnReasonRepository @@ -260,8 +262,8 @@ def after_return_deadline(decision: Decision, exception: ExceptionItem) -> bool: def lock(session: Session, decision: Decision, exception: ExceptionItem) -> None: """Makes a decision final and sends it out: its number in the decision feed (so the - bank's core and the decision files see it), the exception.decided webhook, and an ACH - return's settlement. Nothing leaves OpenClear before this.""" + bank's core and the decision files see it), the exception.decided webhook, and the item's + settlement (a check paid or returned, an ACH return). Nothing leaves OpenClear before this.""" if decision.locked_at is not None: return decision.locked_at = datetime.now(timezone.utc) @@ -282,12 +284,32 @@ def lock(session: Session, decision: Decision, exception: ExceptionItem) -> None source_item = get_adapter(exception.network_code).load_source_item(session, exception.source_item_id) if isinstance(source_item, AchTransaction): source_item.settlement_status = AchSettlementStatus.RETURNED + if exception.network_code == "check": + _settle_check(session, decision, exception) if exception.change_recommended_at is not None: # A change nobody approved by the cut-off: the decision already made is what locks. _restore_recommendation(session, exception, decision) session.flush() +def _settle_check(session: Session, decision: Decision, exception: ExceptionItem) -> None: + """A check paid by a decision is paid, and a returned one returned, from the moment the + decision is final; a paid check's issued item is paid too. Until 1.34.0 only a clean match + set these, so a check decided *pay* (on a reverse positive pay account, every check) could be + presented again and never be a duplicate (REVIEW_1_33_PLAN.md T1, RV-D1). A voided issued + item keeps the customer's void: a second presentment then shows as voided, which is enough.""" + paid_item = get_adapter("check").load_source_item(session, exception.source_item_id) + if not isinstance(paid_item, PaidItem) or paid_item.settlement_status == PaidItemSettlementStatus.REVERSED: + return + if decision.outcome == DecisionOutcome.PAY: + paid_item.settlement_status = PaidItemSettlementStatus.PAID + issued = session.get(IssuedItem, exception.related_reference_id) if exception.related_reference_id else None + if issued is not None and issued.status in (IssuedItemStatus.OUTSTANDING, IssuedItemStatus.STALE, IssuedItemStatus.STOPPED): + issued.status = IssuedItemStatus.PAID + else: + paid_item.settlement_status = PaidItemSettlementStatus.RETURNED + + def _restore_recommendation(session: Session, exception: ExceptionItem, decision: Decision) -> None: """Puts back what was recommended for the decision in place, over a change recommended for it.""" exception.change_recommended_at = None diff --git a/src/openclear/services/email_change_service.py b/src/openclear/services/email_change_service.py index ae8634c5..fc4757ee 100644 --- a/src/openclear/services/email_change_service.py +++ b/src/openclear/services/email_change_service.py @@ -17,6 +17,7 @@ from sqlalchemy import func, select, update from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.domain.email_change import EmailChange from openclear.domain.notification import NotificationType from openclear.domain.tenant import Tenant @@ -26,7 +27,7 @@ CONFIRM_HOURS = 24 -class EmailChangeError(ValueError): +class EmailChangeError(UserFacingError): pass diff --git a/src/openclear/services/embed_service.py b/src/openclear/services/embed_service.py index e8b97902..c7114e67 100644 --- a/src/openclear/services/embed_service.py +++ b/src/openclear/services/embed_service.py @@ -28,6 +28,7 @@ from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.domain.embed_launch import EmbedLaunch from openclear.domain.tenant import Tenant from openclear.domain.tenant_membership import TenantMembership @@ -38,7 +39,7 @@ MAX_ORIGINS = 20 -class EmbedError(ValueError): +class EmbedError(UserFacingError): """Refused; the message is fit to show the online banking developer.""" diff --git a/src/openclear/services/evidence_service.py b/src/openclear/services/evidence_service.py index 3a45ae02..752670dc 100644 --- a/src/openclear/services/evidence_service.py +++ b/src/openclear/services/evidence_service.py @@ -23,6 +23,8 @@ from datetime import date, datetime, time, timedelta, timezone from importlib.metadata import PackageNotFoundError, version +from openclear.errors import UserFacingError + from sqlalchemy import or_, select from sqlalchemy.orm import Session @@ -30,7 +32,7 @@ MAX_DAYS = 400 -class EvidenceError(ValueError): +class EvidenceError(UserFacingError): """The request can't be met; the message is fit to show.""" diff --git a/src/openclear/services/inactivity_service.py b/src/openclear/services/inactivity_service.py index 3b6a826e..45bad892 100644 --- a/src/openclear/services/inactivity_service.py +++ b/src/openclear/services/inactivity_service.py @@ -25,6 +25,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.domain.tenant import Tenant from openclear.domain.tenant_membership import MembershipAccount, TenantMembership @@ -33,7 +34,7 @@ USER_ADMIN = "user:manage" -class InactivityError(ValueError): +class InactivityError(UserFacingError): """The message is fit to show.""" diff --git a/src/openclear/services/incident_service.py b/src/openclear/services/incident_service.py index 216b0067..b9350f1b 100644 --- a/src/openclear/services/incident_service.py +++ b/src/openclear/services/incident_service.py @@ -19,6 +19,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.db.tenancy import bind_session_to_tenant from openclear.domain.incident import IncidentContact, IncidentNotice, IncidentNoticeDelivery from openclear.domain.tenant import Tenant @@ -26,11 +27,12 @@ KINDS = {"incident": "Incident", "maintenance": "Scheduled maintenance", "update": "Update", "closed": "Closed", "offboarding": "Offboarding"} # offboarding: services/offboarding_service.py, never chosen by hand +HAND_KINDS = {k: v for k, v in KINDS.items() if k != "offboarding"} # what the platform's form offers (REVIEW_1_33_PLAN.md G5) CONTACT_KINDS = ("contact", "ceo", "cio") MAX_CONTACTS = 10 -class IncidentError(ValueError): +class IncidentError(UserFacingError): """Fit to show.""" @@ -70,8 +72,9 @@ def set_contacts(session: Session, tenant_id: uuid.UUID, wanted: list[ContactInp """Replaces the bank's contacts and confirms them. Audited as incident_contact.changed (a high security event: whoever sets these decides where incident notices go), naming the old ones too.""" cleaned = [_clean(c) for c in wanted if (c.name or c.email)] - if not [c for c in cleaned if c.kind == "contact"]: - raise IncidentError("Name at least one incident contact.") + if not cleaned: + # The rule's own fallback (304.24(a)(2)): a bank may name only its CEO and CIO (REVIEW_1_33_PLAN.md G3). + raise IncidentError("Name at least one incident contact, or your CEO and CIO.") if len(cleaned) > MAX_CONTACTS: raise IncidentError(f"At most {MAX_CONTACTS} contacts.") before = [f"{c.kind}: {c.name} <{c.email}>" for c in contacts(session, tenant_id)] @@ -92,8 +95,8 @@ def set_contacts(session: Session, tenant_id: uuid.UUID, wanted: list[ContactInp def confirm_contacts(session: Session, tenant_id: uuid.UUID, *, user_id: uuid.UUID | None) -> None: """The yearly confirmation that the contacts are still right.""" - if not [c for c in contacts(session, tenant_id) if c.kind == "contact"]: - raise IncidentError("Name at least one incident contact first.") + if not contacts(session, tenant_id): + raise IncidentError("Name at least one incident contact, or your CEO and CIO, first.") session.get(Tenant, tenant_id).incident_contacts_confirmed_at = datetime.now(timezone.utc) audit_log_service.record_action(session, tenant_id, actor_user_id=user_id, channel="web", action="incident_contact.confirmed", summary="Confirmed the incident contacts", resource_type="tenant", resource_id=tenant_id) @@ -172,6 +175,20 @@ def _deliver(session: Session, notice: IncidentNotice, bank: Tenant, user_id: uu return delivery +def sent(session: Session, limit: int = 50) -> list[tuple[IncidentNotice, list[IncidentNoticeDelivery]]]: + """The notices issued, newest first, each with its deliveries, so the operator sees which banks + were told and which had nobody to tell (REVIEW_1_33_PLAN.md G4). Every scope: the caller is the provider.""" + notices = list(session.execute(select(IncidentNotice).order_by(IncidentNotice.issued_at.desc()).limit(limit)).scalars()) + if not notices: + return [] + deliveries = session.execute(select(IncidentNoticeDelivery).where( + IncidentNoticeDelivery.notice_id.in_([n.id for n in notices]))).scalars().all() + by_notice: dict[uuid.UUID, list[IncidentNoticeDelivery]] = {} + for d in deliveries: + by_notice.setdefault(d.notice_id, []).append(d) + return [(n, by_notice.get(n.id, [])) for n in notices] + + def unacknowledged(session: Session, tenant_id: uuid.UUID) -> list[tuple[IncidentNoticeDelivery, IncidentNotice]]: """A bank's notices nobody there has acknowledged yet: its administrators see them on every page.""" return list(session.execute(select(IncidentNoticeDelivery, IncidentNotice) diff --git a/src/openclear/services/issued_item_service.py b/src/openclear/services/issued_item_service.py index de48547e..2375d5ac 100644 --- a/src/openclear/services/issued_item_service.py +++ b/src/openclear/services/issued_item_service.py @@ -9,6 +9,7 @@ from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.bulk_import.fields import RowFieldError, optional_str, parse_date, parse_decimal, require_str from openclear.bulk_import.results import BulkFileRowResult from openclear.domain.issued_item import IssuedItem, IssuedItemSource, IssuedItemStatus @@ -95,7 +96,7 @@ def create_issued_items_bulk( return results -class CantVoid(ValueError): +class CantVoid(UserFacingError): """The message is fit to show.""" diff --git a/src/openclear/services/message_content.py b/src/openclear/services/message_content.py index 2163f6aa..96994df3 100644 --- a/src/openclear/services/message_content.py +++ b/src/openclear/services/message_content.py @@ -24,6 +24,8 @@ import io import re +from openclear.errors import UserFacingError + from PIL import Image, UnidentifiedImageError # No SVG or ICO here (unlike tenant_service.py's own _ALLOWED_IMAGE_CONTENT_TYPES, used @@ -41,7 +43,7 @@ _DATA_URI_RE = re.compile(r"data:image/([a-zA-Z0-9.+-]+);base64,([A-Za-z0-9+/]+=*)") -class InvalidMessageContent(ValueError): +class InvalidMessageContent(UserFacingError): """Raised for a message that's too long, or an embedded image that's too large, the wrong format, or not a genuinely decodable image -- callers turn this into a 422 form error, same as every other InvalidTenantSettingsInput-style validation in diff --git a/src/openclear/services/mrm_service.py b/src/openclear/services/mrm_service.py index 9c97c2f5..4dc6b6fd 100644 --- a/src/openclear/services/mrm_service.py +++ b/src/openclear/services/mrm_service.py @@ -26,6 +26,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.bulk_import.nacha import NachaParseError, parse_nacha_file from openclear.bulk_import.x937 import X937ParseError, parse_x937_file from openclear.domain.ach_transaction import AchMatchStatus, AchTransaction, AchTransactionSource, AchTransactionType @@ -39,7 +40,7 @@ MAX_REPLAY_ITEMS = 20_000 -class ReplayError(ValueError): +class ReplayError(UserFacingError): """The file can't be replayed; the message is fit to show.""" @@ -104,7 +105,7 @@ def _replay_check(session: Session, tenant_id: uuid.UUID, content: bytes) -> lis label = x937_item_label(item) account = None for number in (item.account_number, item.on_us.partition("/")[0].replace(" ", "")): - account = account_service.get_account_by_number(session, tenant_id, number) if number else None + account = account_service.find_account_by_number(session, tenant_id, number) if number else None # history, not a file if account: break if account is None or not item.check_number: diff --git a/src/openclear/services/offboarding_service.py b/src/openclear/services/offboarding_service.py index fcd325c9..33197861 100644 --- a/src/openclear/services/offboarding_service.py +++ b/src/openclear/services/offboarding_service.py @@ -25,6 +25,7 @@ from sqlalchemy import delete, func, select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.config import get_settings from openclear.db.base import Base from openclear.db.tenancy import acting_as_tenant, bind_session_to_tenant @@ -33,21 +34,20 @@ DELETE_JOB_KIND = "bank.delete" # The storage areas a bank's files live in (storage.AREAS), each under the bank's id. -FILE_AREAS = ("check_images", "bulk_uploads", "tenant_assets", "data_exports", "audit_archives") # Kept on the bank's row after deletion; every other column goes back to its default. _KEPT_COLUMNS = frozenset({"id", "slug", "name", "created_at", "is_demo", "is_sandbox", "offboarding_started_at", "data_deleted_at"}) # Each step's security event (services/security_events.py), for the installation's SIEM and the bank's. _EVENTS = { "bank.offboard.start": "bank.offboarding", "bank.offboard.cancel": "bank.offboarding", - "bank.delete.approve": "bank.offboarding", "bank.delete.done": "bank.offboarding", + "bank.delete.approve": "bank.offboarding", "bank.delete.done": "bank.offboarding", "bank.delete.failed": "bank.offboarding", "bank.legal_hold.on": "bank.legal_hold.changed", "bank.legal_hold.off": "bank.legal_hold.changed", } BACKUPS_NOTE = ("Copies remain in the host's backups until they expire on the host's backup retention schedule " "(CUEC H-CP-9-01); they aren't restored except to recover the whole installation.") -class OffboardingError(ValueError): +class OffboardingError(UserFacingError): """Fit to show.""" @@ -260,15 +260,27 @@ def queue_deletion(session: Session, bank_id: uuid.UUID, *, requested_by: uuid.U def _delete_files(bank_id: uuid.UUID, model_paths: list[str]) -> None: """Last, once the rows are gone: files can't be put back if the database step fails, but a - retried job deletes them again.""" + retried job deletes them again. Every storage area, from storage's own list, so one added + later can't be missed (REVIEW_1_33_PLAN.md O1, RV-D2).""" from openclear import storage for path in model_paths: storage.delete(path) - for area in FILE_AREAS: + for area in storage.AREAS: storage.delete_tenant(area, bank_id) +def _freeze(session: Session, bank: Tenant) -> None: + """Before the wipe, in a transaction of its own: the bank is inactive (sign-ins and tokens + refused from now) and its queued and running jobs are gone, so a worker mid-import loses its + lease at its next chunk and nothing is added while the rows go (REVIEW_1_33_PLAN.md O7).""" + from openclear.domain.job import Job + + bank.is_active = False + session.execute(delete(Job).where(Job.tenant_id == bank.id)) + session.commit() + + # What's a person's own, and goes with them: their sign-in material, messages and links to identity # providers. Every other reference to a person is a record of what they did (the action log, # decisions, approvals), which keeps them. @@ -292,23 +304,44 @@ def _forget(session: Session, user_ids: list[uuid.UUID]) -> tuple[int, int]: for user_id in user_ids: savepoint = session.begin_nested() try: - for name, column in _PERSONAL: - table = Base.metadata.tables.get(name) - if table is not None and column in table.columns: - session.execute(delete(table).where(table.c[column] == user_id)) + _delete_personal(session, user_id) delete_users(session, User.id == user_id) session.flush() savepoint.commit() deleted += 1 except IntegrityError: savepoint.rollback() + # Named in a record that stays (the action log, a decision), so the row stays: with nothing + # personal on it, and no way back in (REVIEW_1_33_PLAN.md O6, RV-D6). + _delete_personal(session, user_id) user = session.get(User, user_id) if user is not None: - user.is_active = False + _anonymise(user) + session.flush() kept += 1 return deleted, kept +def _delete_personal(session: Session, user_id: uuid.UUID) -> None: + for name, column in _PERSONAL: + table = Base.metadata.tables.get(name) + if table is not None and column in table.columns: + session.execute(delete(table).where(table.c[column] == user_id)) + + +def _anonymise(user) -> None: + """What's left of a person kept only as the actor on records: an id, and nothing that was theirs.""" + import secrets + + from openclear.auth.security import hash_password + + user.email = f"deleted-{user.id.hex[:16]}@deleted.invalid" + user.hashed_password = hash_password(secrets.token_urlsafe(32)) + user.phone = None + user.is_active = False + user.token_version = (user.token_version or 0) + 1 + + def _wipe(session: Session, bank: Tenant) -> dict[str, int]: """Every row with the bank's tenant_id, children before parents. Returns the rows deleted per table.""" deleted: dict[str, int] = {} @@ -326,7 +359,8 @@ def _strip(bank: Tenant) -> None: for column in Tenant.__table__.columns: if column.key in _KEPT_COLUMNS: continue - default = column.default.arg if column.default is not None and not callable(column.default.arg) else None + arg = column.default.arg if column.default is not None else None + default = arg() if arg in (dict, list) else (None if callable(arg) else arg) # an empty mapping, not JSON null setattr(bank, column.key, default) bank.is_active = False @@ -340,7 +374,8 @@ def _email(session: Session, user_id: uuid.UUID | None) -> str | None: def run(session: Session, deletion_id: uuid.UUID) -> BankDeletion | None: """The job: deletes the bank's data and signs the certificate. Every check is made again - here, so a hold put on, or offboarding cancelled, during the delay stops it. Commits nothing.""" + here, so a hold put on, or offboarding cancelled, during the delay stops it. Commits once, + after freezing the bank; the caller commits the rest.""" from openclear.domain.audit_log_entry import AuditLogEntry from openclear.domain.ml_model import MlModel from openclear.services import evidence_service @@ -354,6 +389,7 @@ def run(session: Session, deletion_id: uuid.UUID) -> BankDeletion | None: record.status, record.last_error = "refused", str(exc) return record bind_session_to_tenant(session, bank.id) + _freeze(session, bank) last = session.execute(select(AuditLogEntry).where(AuditLogEntry.tenant_id == bank.id) .order_by(AuditLogEntry.sequence.desc()).limit(1)).scalar_one_or_none() entries = session.execute(select(func.count()).select_from(AuditLogEntry).where(AuditLogEntry.tenant_id == bank.id)).scalar_one() @@ -381,9 +417,10 @@ def run(session: Session, deletion_id: uuid.UUID) -> BankDeletion | None: "last_entry_hash": getattr(last, "entry_hash", None) if last else None}, "rows_deleted": dict(sorted(rows.items())), "rows_deleted_total": sum(rows.values()), - "storage_areas_cleared": [*FILE_AREAS, "ml_models"], + "storage_areas_cleared": sorted(storage_areas()), "people_deleted": people_deleted, "people_kept_because_they_belong_elsewhere_or_are_named_in_records": len(kept) + still_referenced, + "people_anonymised_because_they_are_named_in_records": still_referenced, "kept": "The bank's name and slug, inactive, so the slug is never reused; this certificate; the service provider's " "own action log and incident notices; the provider's own custom-field values for the bank (its references, " "such as an ERP id).", @@ -403,6 +440,28 @@ def run(session: Session, deletion_id: uuid.UUID) -> BankDeletion | None: return record +def mark_failed(session: Session, deletion_id: uuid.UUID, error: str) -> BankDeletion | None: + """The job gave up (workers/job_handlers.py): the record says so, the page shows it, and the + operator hears of it as a critical event instead of a deletion that "runs at" a time long + past (REVIEW_1_33_PLAN.md O9). Requesting again queues a new run.""" + record = session.get(BankDeletion, deletion_id) + if record is None or record.status != "queued": + return record + bank = session.get(Tenant, record.bank_id) + record.status, record.last_error = "failed", error[:2000] + if bank is not None: + bank.is_active = True # frozen for a deletion that didn't happen + _event(session, bank, "bank.delete.failed", f"Deleting {bank.name}'s data failed and won't be retried: {error[:200]}", user_id=None) + session.flush() + return record + + +def storage_areas() -> list[str]: + from openclear import storage + + return list(storage.AREAS) + + def latest(session: Session, bank_id: uuid.UUID) -> BankDeletion | None: return session.execute(select(BankDeletion).where(BankDeletion.bank_id == bank_id) .order_by(BankDeletion.requested_at.desc()).limit(1)).scalar_one_or_none() diff --git a/src/openclear/services/operator_service.py b/src/openclear/services/operator_service.py index b4169e4b..4c515bff 100644 --- a/src/openclear/services/operator_service.py +++ b/src/openclear/services/operator_service.py @@ -15,6 +15,7 @@ from sqlalchemy import func, select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.db.tenancy import bind_session_to_tenant from openclear.domain.bulk_upload_file import BulkUploadFile from openclear.domain.customer import Customer @@ -26,7 +27,7 @@ SLUG = re.compile(r"^[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?$") -class ProvisioningError(ValueError): +class ProvisioningError(UserFacingError): """Bad input; the message is fit to show.""" diff --git a/src/openclear/services/outside_services.py b/src/openclear/services/outside_services.py index d118f6cd..f2b8e9ee 100644 --- a/src/openclear/services/outside_services.py +++ b/src/openclear/services/outside_services.py @@ -17,12 +17,13 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.config import get_settings REVIEW_EVERY_DAYS = 365 -class OutsideServiceError(ValueError): +class OutsideServiceError(UserFacingError): """A review the bank can't record; the message is written for the person who tried.""" diff --git a/src/openclear/services/password_reset_service.py b/src/openclear/services/password_reset_service.py index 20226852..7d99d9e6 100644 --- a/src/openclear/services/password_reset_service.py +++ b/src/openclear/services/password_reset_service.py @@ -17,6 +17,7 @@ from sqlalchemy import delete, select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.auth.password_policy import validate_password from openclear.auth.security import hash_password from openclear.config import get_settings @@ -26,7 +27,7 @@ from openclear.domain.user import User -class ResetError(ValueError): +class ResetError(UserFacingError): """Why a reset can't be started or finished, for the page to show.""" diff --git a/src/openclear/services/reminder_service.py b/src/openclear/services/reminder_service.py index ffc800b7..79d7fccd 100644 --- a/src/openclear/services/reminder_service.py +++ b/src/openclear/services/reminder_service.py @@ -23,6 +23,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.config import get_settings from openclear.db.tenancy import bind_session_to_tenant from openclear.domain.customer import Customer @@ -38,7 +39,7 @@ MAX_OFFSETS = 4 -class ReminderError(ValueError): +class ReminderError(UserFacingError): """Bad input on the settings form; the message is fit to show.""" diff --git a/src/openclear/services/sandbox_service.py b/src/openclear/services/sandbox_service.py index ffb6118e..7b45d743 100644 --- a/src/openclear/services/sandbox_service.py +++ b/src/openclear/services/sandbox_service.py @@ -21,12 +21,13 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.config import get_settings from openclear.domain.tenant import Tenant from openclear.domain.user import User -class SandboxError(ValueError): +class SandboxError(UserFacingError): pass diff --git a/src/openclear/services/security_events.py b/src/openclear/services/security_events.py index d1ef9280..8e9721f9 100644 --- a/src/openclear/services/security_events.py +++ b/src/openclear/services/security_events.py @@ -71,6 +71,8 @@ class EventType: # access "access.denied": EventType("access", "medium"), "rate_limit.exceeded": EventType("access", "medium"), + # Redis for shared rate limits can't be reached: each instance limits on its own until it can (REVIEW_1_33_PLAN.md C2). + "rate_limit.backend_down": EventType("integration", "high"), "sod.violation": EventType("access", "high"), # critical when enforce refused it (from_audit_entry) "edge.secret.refused": EventType("access", "high"), # identity @@ -138,6 +140,7 @@ class EventType: "incident.notice.sent": EventType("integrity", "critical"), "incident.notice.acknowledged": EventType("integrity", "medium"), "incident_contact.changed": EventType("configuration", "high"), + "incident_contact.confirmed": EventType("configuration", "low"), # the yearly attestation, for the SIEM too (REVIEW_1_33_PLAN.md G8) # Offboarding a bank: starting it, cancelling it, and approving, starting and finishing the deletion of its # data; and its legal hold (plans/OFFBOARDING_PLAN.md). "bank.offboarding": EventType("data", "critical"), @@ -210,6 +213,7 @@ class EventType: "incident.notice.sent": "incident.notice.sent", "incident.notice.acknowledged": "incident.notice.acknowledged", "incident_contact.changed": "incident_contact.changed", + "incident_contact.confirmed": "incident_contact.confirmed", "bank.offboard.start": "bank.offboarding", "bank.offboard.cancel": "bank.offboarding", "bank.delete.approve": "bank.offboarding", diff --git a/src/openclear/services/siem/delivery.py b/src/openclear/services/siem/delivery.py index 9fd90a5c..0409c3b8 100644 --- a/src/openclear/services/siem/delivery.py +++ b/src/openclear/services/siem/delivery.py @@ -17,6 +17,7 @@ from sqlalchemy import func, select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.auth.crypto import decrypt_secret, encrypt_secret from openclear.auth.outbound_http import UnsafeUrlError, check_url from openclear.config import Settings, get_settings @@ -38,7 +39,7 @@ STALE_AFTER = timedelta(hours=1) -class DestinationError(ValueError): +class DestinationError(UserFacingError): """Bad input on the settings form; the message is fit to show.""" diff --git a/src/openclear/services/support_access_service.py b/src/openclear/services/support_access_service.py index c53ac0cb..44ce2227 100644 --- a/src/openclear/services/support_access_service.py +++ b/src/openclear/services/support_access_service.py @@ -21,6 +21,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.auth.permissions import SUPPORT_READ_PERMISSIONS, SUPPORT_WRITE_PERMISSIONS from openclear.db.tenancy import bind_session_to_tenant from openclear.domain.support_access_grant import SupportAccessGrant @@ -32,7 +33,7 @@ PLATFORM_ROLES = ("operator", "support") -class SupportAccessError(ValueError): +class SupportAccessError(UserFacingError): """Not allowed, or bad input; the message is fit to show.""" diff --git a/src/openclear/services/teller_service.py b/src/openclear/services/teller_service.py index 6c9a52f7..9f243890 100644 --- a/src/openclear/services/teller_service.py +++ b/src/openclear/services/teller_service.py @@ -21,6 +21,7 @@ from sqlalchemy import func, select, update from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.domain.account import Account, CheckReviewMode from openclear.domain.paid_item import PaidItem from openclear.domain.teller_verification import TellerVerification @@ -49,19 +50,24 @@ # The issued check names a payee and the teller didn't say what's on the check: the payee must be # checked as written before cash is paid (the product owner, 2026-10-22). PAYEE_NEEDED = "payee_needed" -# The order the first problem is chosen in: the clearing rules', then a missing payee. -_ORDER = [*RESULTS.values(), PAYEE_NEEDED] -# Not-issued answers from one teller, branch or client within an hour before it looks like -# someone trying check numbers (TL-D6). +# Another teller verified this check a moment ago and hasn't recorded it cashed yet: the second +# is declined, so two branches can't both pay out (REVIEW_1_33_PLAN.md T5). +BEING_CASHED = "being_cashed" +# The order the first problem is chosen in: the clearing rules', then a check in another teller's hands, then a missing payee. +_ORDER = [*RESULTS.values(), BEING_CASHED, PAYEE_NEEDED] +# Declines to one teller, branch, client or person within an hour before it looks like someone +# trying check numbers or amounts (TL-D6; REVIEW_1_33_PLAN.md T3). PROBE_THRESHOLD = 10 -class TellerError(ValueError): - """Fit to show to the teller platform.""" +class TellerError(UserFacingError): + """Fit to show to the teller platform. `details` are facts the platform can act on beside + the sentence (what the first `cashed` call recorded, when its own answer was lost).""" - def __init__(self, message: str, status: int = 409): + def __init__(self, message: str, status: int = 409, **details): super().__init__(message) self.status = status + self.details = details @dataclass(frozen=True) @@ -123,13 +129,26 @@ def _rules(session: Session, tenant_id: uuid.UUID, account: Account, request: Ve def _enrolled(session: Session, account: Account) -> bool: """On positive pay (or reverse): a bank's own account, or a customer's still enrolled. A - customer deactivated isn't, and its checks are cashed by the bank's usual procedure.""" + customer deactivated, or whose service has ended, isn't, and its checks are cashed by the + bank's usual procedure.""" from openclear.domain.customer import Customer if account.customer_id is None: return True + from openclear.services import customer_offboarding_service as offboarding + customer = session.get(Customer, account.customer_id) - return customer is not None and customer.is_active + return customer is not None and customer.is_active and not offboarding.ended(customer) + + +def _in_another_tellers_hands(session: Session, tenant_id: uuid.UUID, account: Account, check_number: str) -> bool: + """A match for the same check, verified within the last 15 minutes and not yet recorded + cashed. Verifications of one check take turns on its issued item's lock, so the second sees + the first.""" + return session.execute(select(TellerVerification.id).where( + TellerVerification.tenant_id == tenant_id, TellerVerification.account_id == account.id, + TellerVerification.check_number == check_number, TellerVerification.result == MATCH, + TellerVerification.cashed_at.is_(None), TellerVerification.expires_at > _now()).limit(1)).first() is not None def _names_a_payee(issued) -> bool: @@ -146,12 +165,14 @@ def verify(session: Session, tenant_id: uuid.UUID, request: VerifyRequest, *, us account a check that isn't found is declined (`not_issued`, or `customer_review` on reverse positive pay), and when the issued check names a payee, the teller must send the payee as written on the check, or it's declined (`payee_needed`).""" - account = account_service.get_account_by_number(session, tenant_id, request.account_number) + account = account_service.find_account_by_number(session, tenant_id, request.account_number) if account is None or not _enrolled(session, account): result, problems, candidate = NOT_ON_POSITIVE_PAY, [NOT_ON_POSITIVE_PAY], None else: found, candidate = _rules(session, tenant_id, account, request) problems = [RESULTS[t] for t in found] + if not problems and _in_another_tellers_hands(session, tenant_id, account, request.check_number): + problems.append(BEING_CASHED) if candidate is not None and _names_a_payee(candidate) and not (request.payee_name or "").strip(): problems.append(PAYEE_NEEDED) result = next((code for code in _ORDER if code in problems), MATCH) @@ -161,7 +182,8 @@ def verify(session: Session, tenant_id: uuid.UUID, request: VerifyRequest, *, us routing_number=request.routing_number, account_number=request.account_number, check_number=request.check_number, amount=request.amount, payee_name=request.payee_name, result=result, problems=problems, issued_item_id=candidate.id if candidate is not None else None, teller_id=request.teller_id, branch=request.branch, - api_client_id=_client_id(session, user_id), created_at=now, expires_at=now + timedelta(minutes=VERIFICATION_MINUTES), + api_client_id=_client_id(session, user_id), user_id=user_id, created_at=now, + expires_at=now + timedelta(minutes=VERIFICATION_MINUTES), ) session.add(verification) session.flush() @@ -174,28 +196,30 @@ def verify(session: Session, tenant_id: uuid.UUID, request: VerifyRequest, *, us resource_type="teller_verification", resource_id=verification.id, metadata={"result": result, "problems": problems, "teller_id": request.teller_id, "branch": request.branch}, ) - if result in ("not_issued", NOT_ON_POSITIVE_PAY): + if result != MATCH: _watch_for_probing(session, tenant_id, verification) return verification def _watch_for_probing(session: Session, tenant_id: uuid.UUID, verification: TellerVerification) -> None: - """Many not-issued answers to one teller, branch or client within an hour looks like someone - trying check numbers (TL-D6): a high security event, at most one a minute for each, named - by the narrowest of them that crossed the line.""" - for column, value in (("teller_id", verification.teller_id), ("branch", verification.branch), - ("api_client_id", verification.api_client_id)): + """Many declines to one teller, branch, client or person within an hour looks like someone + trying check numbers or amounts (TL-D6): a high security event, at most one a minute for + each, named by the narrowest of them that crossed the line. Every decline counts: a wrong + amount answers with what was issued, so it reads the register as well as a miss does.""" + keys = [("teller_id", verification.teller_id), ("branch", verification.branch), ("api_client_id", verification.api_client_id)] + if verification.api_client_id is None: + keys.append(("user_id", verification.user_id)) # a person's token, not a platform's + for column, value in keys: if value is None: continue - misses = session.execute(select(func.count()).select_from(TellerVerification).where( + declines = session.execute(select(func.count()).select_from(TellerVerification).where( TellerVerification.tenant_id == tenant_id, getattr(TellerVerification, column) == value, - TellerVerification.result.in_(["not_issued", NOT_ON_POSITIVE_PAY]), - TellerVerification.created_at >= _now() - timedelta(hours=1))).scalar_one() - if misses >= PROBE_THRESHOLD: + TellerVerification.result != MATCH, TellerVerification.created_at >= _now() - timedelta(hours=1))).scalar_one() + if declines >= PROBE_THRESHOLD: security_events.emit_sampled( session, "teller.verify.probing", key=f"{tenant_id}:{column}:{value}", tenant_id=tenant_id, outcome="failure", - reason=f"{misses} checks not issued within an hour, by {column.replace('_', ' ')} {value}", - details={column: str(value), "not_issued_last_hour": misses}) + reason=f"{declines} checks declined within an hour, by {column.replace('_', ' ')} {value}", + details={column: str(value), "declined_last_hour": declines}) return # one alert for the call, by the narrowest of who asked @@ -221,7 +245,9 @@ def cash(session: Session, tenant_id: uuid.UUID, verification_id: uuid.UUID, amo if claimed != 1: session.refresh(verification) if verification.cashed_at is not None: - raise TellerError("That verification was already recorded as cashed.") + # A platform that lost the first answer can tell its own success from someone else's (REVIEW_1_33_PLAN.md T10). + raise TellerError("That verification was already recorded as cashed.", paid_item_id=verification.paid_item_id, + cashed_at=verification.cashed_at) raise TellerError(f"That verification expired ({VERIFICATION_MINUTES} minutes). Verify the check again.") session.refresh(verification) diff --git a/src/openclear/services/tenant_service.py b/src/openclear/services/tenant_service.py index 3da200e0..0b7634fe 100644 --- a/src/openclear/services/tenant_service.py +++ b/src/openclear/services/tenant_service.py @@ -11,6 +11,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear import colors from openclear.config import get_settings from openclear.domain.tenant import Tenant @@ -29,7 +30,7 @@ SERVABLE_IMAGE_CONTENT_TYPES = frozenset(_ALLOWED_IMAGE_FORMATS.values()) | {"image/vnd.microsoft.icon"} -class InvalidTenantSettingsInput(ValueError): +class InvalidTenantSettingsInput(UserFacingError): """Raised for any malformed tenant-settings form input — a bad accent color, a disallowed image content-type, or an invalid session timeout — callers turn this into a form error, never a 500.""" diff --git a/src/openclear/services/user_service.py b/src/openclear/services/user_service.py index 3756307f..0dfb4c72 100644 --- a/src/openclear/services/user_service.py +++ b/src/openclear/services/user_service.py @@ -9,6 +9,7 @@ from sqlalchemy import func, select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.auth.password_policy import PasswordPolicy, context_words, effective_policy, strictest, validate_password from openclear.auth.security import hash_password, verify_password from openclear.bulk_import.fields import RowFieldError, optional_str, require_str @@ -371,7 +372,7 @@ def admin_sign_out_everywhere( return user -class PasswordChangeError(ValueError): +class PasswordChangeError(UserFacingError): """A password change/reset was refused. The message is safe to show the user as-is (routers re-render the form with error=str(exc), this app's usual convention).""" diff --git a/src/openclear/services/webhook_service.py b/src/openclear/services/webhook_service.py index 84fc1a15..5a6965c8 100644 --- a/src/openclear/services/webhook_service.py +++ b/src/openclear/services/webhook_service.py @@ -34,6 +34,7 @@ from sqlalchemy import select from sqlalchemy.orm import Session +from openclear.errors import UserFacingError from openclear.auth.crypto import decrypt_secret, encrypt_secret from openclear.auth.outbound_http import UnsafeUrlError, check_url, public_only_client from openclear.config import get_settings @@ -62,7 +63,7 @@ _SESSION_CACHE_KEY = "webhook_endpoints_by_tenant" -class WebhookError(ValueError): +class WebhookError(UserFacingError): """Bad input; the message is fit to show.""" diff --git a/src/openclear/templates/base.html b/src/openclear/templates/base.html index 5ea296f9..39dd0b3c 100644 --- a/src/openclear/templates/base.html +++ b/src/openclear/templates/base.html @@ -200,7 +200,7 @@ {% if ctx and ctx.service_ends_at %} {# The bank is ending this customer's positive pay (services/customer_offboarding_service.py). #}
- Your positive pay service ends on . + Your positive pay service ends at the end of {{ ctx.service_last_day.isoformat() if ctx.service_last_day else ctx.service_ends_at.strftime("%Y-%m-%d") }}. {% if can(ctx, "data_export:run") %}Take a data export before then to keep with your records.{% else %}Ask your administrator to take a data export before then.{% endif %}
{% endif %} diff --git a/src/openclear/templates/custom_fields/upload.html b/src/openclear/templates/custom_fields/upload.html index 8a90d8b4..63c9d404 100644 --- a/src/openclear/templates/custom_fields/upload.html +++ b/src/openclear/templates/custom_fields/upload.html @@ -22,7 +22,7 @@

Update custom fields on {{ plural }}

{{ record_keys[0] }}Which record to update ({{ record_keys[1] }} works instead) {% for f in fields %} - cf_{{ f.key }}{{ f.label }} ({{ f.field_type }}{% if f.unique %}, unique{% endif %}) + cf_{{ f.key }}{{ f.label }} ({{ f.field_type | custom_field_type | lower }}{% if f.unique %}, unique{% endif %}) {% endfor %} diff --git a/src/openclear/templates/customers/deleted.html b/src/openclear/templates/customers/deleted.html index 86638af8..75498f81 100644 --- a/src/openclear/templates/customers/deleted.html +++ b/src/openclear/templates/customers/deleted.html @@ -14,8 +14,8 @@

Deleted customers

{% for d in deletions %} {{ d.customer_name }}
{{ d.customer_number }} - {{ d.service_ended_at.strftime("%Y-%m-%d") if d.service_ended_at else "—" }} - {{ d.completed_at.strftime("%Y-%m-%d") }} + {{ (d.service_ended_at | bank_day(bank_time_zone, last=True)) if d.service_ended_at else "—" }} + {{ d.completed_at | bank_day(bank_time_zone) }} Download {% endfor %} diff --git a/src/openclear/templates/customers/detail.html b/src/openclear/templates/customers/detail.html index 7c0311ee..d56584b2 100644 --- a/src/openclear/templates/customers/detail.html +++ b/src/openclear/templates/customers/detail.html @@ -214,7 +214,7 @@

Data export

Offboarding

{% if customer.service_ended_at %} -

Service ended on {{ customer.service_ended_at.strftime("%Y-%m-%d") }}. Its people +

Service ended after {{ offboarding_last_day or customer.service_ended_at.strftime("%Y-%m-%d") }}. Its people can't sign in and files naming its accounts are refused. Its data is deleted on or after {{ offboarding_deletion_due.strftime("%Y-%m-%d") }}{% if customer.legal_hold_reason %}, once it's off legal hold{% endif %}.

{% elif customer.service_ends_at %} diff --git a/src/openclear/templates/docs/admin/data-dictionary.html b/src/openclear/templates/docs/admin/data-dictionary.html index 7568b26f..6d19ba15 100644 --- a/src/openclear/templates/docs/admin/data-dictionary.html +++ b/src/openclear/templates/docs/admin/data-dictionary.html @@ -751,6 +751,7 @@

teller_verification

issued_item_idUUIDyesFK → issued_item.idThe issued check it was compared with. teller_id / branchString(64)yesWho asked, as the teller platform identifies them. api_client_idUUIDyesThe teller platform's API client. +user_idUUIDyesFK → user.idWhose token asked: the client's user, or a person given check:verify (1.34.0). created_at / expires_atTimestampnoA match can be recorded cashed for 15 minutes. cashed_atTimestampyesWhen it was recorded cashed; set once. paid_item_idUUIDyesFK → paid_item.idThe paid item the cash-out presented (source teller). diff --git a/src/openclear/templates/exceptions/list.html b/src/openclear/templates/exceptions/list.html index 406638de..97aff594 100644 --- a/src/openclear/templates/exceptions/list.html +++ b/src/openclear/templates/exceptions/list.html @@ -72,7 +72,7 @@

Exceptions

{% for v in branch_declines %}
  • Check {{ v.check_number }} for {{ "{:,.2f}".format(v.amount) }} on account {{ v.account_number }}{% if v.branch %} at branch {{ v.branch }}{% endif %}, : - {{ v.result | replace("_", " ") }}
    • + {{ v.result | teller_result }} {% endfor %} @@ -88,7 +88,7 @@

    Exceptions

    - + {% endif %} diff --git a/src/openclear/templates/platform/custom_fields.html b/src/openclear/templates/platform/custom_fields.html index 9483d92d..90b82271 100644 --- a/src/openclear/templates/platform/custom_fields.html +++ b/src/openclear/templates/platform/custom_fields.html @@ -18,7 +18,7 @@

    Your fields on banks

    {{ f.label }}{% if f.archived_at %} archived{% endif %} {{ f.key }} - {{ f.field_type }}{% if f.field_type == "choice" %}: {{ f.options | rejectattr("retired") | map(attribute="value") | join(", ") }}{% endif %} + {{ f.field_type | custom_field_type }}{% if f.field_type == "choice" %}: {{ f.options | rejectattr("retired") | map(attribute="value") | join(", ") }}{% endif %} {{ ["required" if f.required, "unique" if f.unique, "shown to the bank" if f.shown_to_bank] | select | join(", ") or "—" }} {% for b in banks %}{{ values.get(b.tenant.id, {}).get(f.key) or "—" }}{% endfor %} @@ -46,7 +46,7 @@

    {{ "Change " ~ editing.label if editing else "Add a field" }}

    A lowercase letter, then lowercase letters, digits and underscores. Never changes. diff --git a/src/openclear/templates/platform/incidents.html b/src/openclear/templates/platform/incidents.html index d252ed56..170885ee 100644 --- a/src/openclear/templates/platform/incidents.html +++ b/src/openclear/templates/platform/incidents.html @@ -15,7 +15,7 @@

    Incident and maintenance notices