Scam sentinel for small businesses, built during the Meta Global AI Developer Hackathon (Oct 27 – Nov 4, 2026).
VIGÍA is the WhatsApp number and email address a small business forwards anything suspicious to — a text, a screenshot, a voice note, a PDF, a link, an email — and gets back a verdict in plain language in seconds: scam / legitimate / doubtful, why, and what to do. If it's a scam, the rest of the team is warned too.
Measured by tools/measure.py over evals/casos.jsonl; the raw result is
evals/2026-10-31.resultado.json. make docs-check fails if this row drifts
from the newest dated run in evals/.
| Date | Cases | Accuracy | Scam recall | False alarms on legit | Doubtful | Cost | Median latency | Commit | Gate |
|---|---|---|---|---|---|---|---|---|---|
| 2026-10-31 | 231 | 93.5% | 98.5% | 2.1% | 6.5% | $0.879 | 3.22 s | a474a96 |
passed, 0 regressions |
In 73 of the 231 cases the deterministic rules were already conclusive, so VIGÍA
skipped the analyst on purpose (ADR-061): only the
gate's manipulation check called a model, at a median of $0.0015 and 1.5 s per case,
against $0.0047 and 4.0 s with the analyst. In 15 more, Muse Spark read the PDF or
screenshot and the rules decided on the extracted text. The analyst ran in the other 143.
The result file counts those 88 as degradados (no analyst); none of the 231 lost
the analyst to an unavailable model or a refusal.
Target (≥80 cases): recall_estafa ≥ 0.90, fp_legitimo ≤ 0.10,
dudosos ≤ 0.25.
Reproducibility. The same 231 cases were measured three times
(evals/2026-10-31.resultado.json, evals/2026-11-02-a.resultado.json,
evals/2026-11-02-b.resultado.json). Two results held in all three: scam recall
98.5%, and no real scam was ever labelled legitimate. The rest moves, because a
Meta model is in the loop — 6 of the 231 verdicts changed on each rerun, false
alarms landed at 0, 2 and 3 of the 95 legitimate messages, and accuracy between
92.6% and 93.5%. The row above is the run of 2026-10-31; rerunning make evals
will land near it, not exactly on it.
Needs Docker (with Compose) and uv.
git clone <this repo> && cd vigia
cp .env.example .env # MODEL_API_KEY is optional; without it the demo runs rules only
make demo
Starts Postgres and the API with Docker Compose, seeds a sample business with 3
members, sends 5 sample cases through the local channel, and opens the panel with the
verdicts at http://localhost:18000/panel?token=<VIGIA_PANEL_TOKEN> (it prints the full
link). There is no hosted instance: make demo on your machine is the way to try it.
The link sets an HttpOnly cookie and redirects to /panel, so the token leaves the address
bar. Escape the token if it has +, / or =. Without MODEL_API_KEY it says so
and runs rules only (no analyst). The .env.example keys are public demo keys:
VIGIA_MODO=prod refuses them.
The first make demo downloads and builds the Docker images and can take several
minutes; after that it takes seconds. Without MODEL_API_KEY VIGÍA never answers
"legitimate" (that call is the analyst's), so the legitimate sample comes out "unclear";
the panel says so. Postgres is published on 127.0.0.1:55432 and the API on
127.0.0.1:18000; if either port is taken, set VIGIA_DB_PORT or VIGIA_API_PORT in
.env (and the port in DATABASE_URL), run docker compose down and make demo again.
See DECLARACION.md for what was built during the event.
Everything else you can run. Only make demo and the two eval targets
need Meta API keys. Everything else runs without any configuration.
| Command | What it does | Needs keys |
|---|---|---|
make demo |
Brings up the API and walks sample cases through the local channel and the operator panel | yes |
make test |
The full test suite | no |
make check |
Everything at once: lint, architecture, duplication, security, docs, i18n, web and tests | no |
make lint |
ruff plus mypy --strict over the typed packages |
no |
make arch |
lint-imports: fails if a layer imports one it must not |
no |
make dup |
pylint duplicate-code detection, 8-line threshold |
no |
make security |
bandit, pip-audit and the OWASP tests |
no |
make docs-check |
Fails if docstring coverage drops below 90%, or this README, the repo map or the architecture doc drift from the code | no |
make evals |
Re-runs the 231 cases; spends up to $2.00 | yes |
make evals-rapido |
A faster subset; up to $1.00 | yes |
make db |
Postgres on its own | no |
make run |
Everything in the foreground, with build output | no |
At runtime (VIGIA_MODO=eval|prod) every model call is a Meta model: Muse Spark 1.3
(analyst loop, attachment reading, and the gate's fallback judge), SAM 3.1 (crops), Muse
Voice Transcribe (voice notes). Everything else behind a "model" name is deterministic
code: the country-pack signals, the manipulation floor (ADR-051), fusion and policy.
An external non-Meta judge was used during development only (judge of steps) and was
removed from the codebase, so it can never be called at runtime (ADR-085). The gate
(guard/gate.py) always runs the deterministic floor first; its model fallback is Spark in
eval/prod and the deterministic mock in dev.
| Model | Does |
|---|---|
| Muse Spark 1.3 | The analyst's tool-using loop (web search, pre-resolved link lookup), reading screenshots/PDFs when rules alone aren't enough, and the gate's fallback judge for manipulation scoring at runtime |
| SAM 3.1 | Crops logos and QR codes out of a screenshot; the QR is then decoded in code, not by the model |
| Muse Voice Transcribe | Turns a WhatsApp voice note into text, in code, before any signal or model ever sees it |
Where each judging criterion can be checked in this repo or in the demo:
| Criterion | Weight | Where to see it |
|---|---|---|
| Impact | 30% | A small business with no security team gets a verdict on WhatsApp with no app to install; the whole team is warned on a scam (src/vigia/services/whatsapp.py, make demo) |
| Meta tooling | 25% | Muse Spark as a tool-using analyst (src/vigia/analyst/loop.py), SAM 3.1 crops with the QR decoded in code, Muse Voice Transcribe for voice notes (see "What each Meta model does" above) |
| Demo | 25% | make demo: sample cases through the local channel and the operator panel with their verdicts |
| Technical | 20% | Measured accuracy with a regression gate (tools/measure.py, the metrics row above), adversarial and blind evals (evals/adversarios.jsonl, evals/ciego-01.md), row-level security and encryption (docs/ARCHITECTURE.md, docs/THREAT_MODEL.md), every decision as an ADR (docs/decisions/) |
OWASP categories this repo tests for, mapped to the test that covers them:
| OWASP category | Covered by |
|---|---|
| A01 — Broken access control | tests/test_owasp.py (test_a01_business_sees_only_its_own_rows, test_a01_without_business_context_no_rows_are_visible, test_a01_cannot_write_rows_for_another_business) |
| A03 — Injection | tests/test_owasp.py (test_a03_detector_flags_runtime_built_queries, test_a03_no_runtime_built_sql_in_src, test_a03_business_id_is_bound_not_interpolated) |
| A05 — Security misconfiguration | tests/test_owasp.py (test_a05_security_headers_are_set, test_a05_no_api_docs_in_prod, test_a05_public_texts_never_name_internal_signals) |
| A09 — Logging and monitoring failures | tests/test_owasp.py (test_a09_case_logs_carry_ids_but_no_pii, test_a09_uvicorn_errors_are_json_without_pii) |
| A10 — Server-side request forgery | tests/test_net.py, over src/vigia/net/unroll.py (ADR-099): short-link unrolling refuses non-public addresses, caps hops and time, never forwards cookies |
make check runs pip-audit as part of security, and it's blocking: any real vulnerability fails the build. It needs network access to fetch the vulnerability database; without it, the check detects the missing connection and skips with "pip-audit: omitido, sin conexión" instead of failing for an unrelated reason (D-124).
Full write-up: docs/THREAT_MODEL.md and
docs/PRIVACY.md; encryption design in
ADR-020 and ADR-025;
how to report a vulnerability in
SECURITY.md.
Not goals of this hackathon build: monetization, full multi-country coverage, an MCP server for third parties, auditing third-party chatbots, zero-click attacks, and a published benchmark.
MIT — see LICENSE.
docs/MAP.md— one line per module, its layer and responsibility.docs/ARCHITECTURE.md— component and flow diagrams, import contracts, multi-tenancy.- API reference: generated locally with
make docs(pdoc, intodocs/api/). docs/decisions/— every technical decision (ADR-001 through ADR-112), migrated to English from the working decision log.

