A retrieval-grounded compliance auditor for Corporate Tax. Point it at a company document — financials, contracts, policy memos — and it returns, for every problem it finds: the offending text, the exact Article it violates, and the exact penalty.
The hard part isn't finding issues; it's not inventing them. LexVecta is built so that every claim is traceable to a verbatim quote from a fixed corpus of official tax legislation. It cannot cite a law that does not exist, and it cannot quote a penalty that isn't in the binding penalties table.
⚠️ Not legal advice. Decision-support only;needs_reviewitems require a human.
A single unit of output, from a real run (vanguard_v3.33), lightly trimmed:
Every field above is grounded: the quote is a substring of an indexed chunk, the citation is
resolved from that chunk's metadata (never from model output), and source_authority
distinguishes binding law from explanatory guidance so findings never overstate their
own weight.
Measured on three multi-page documents with known defects. MIC is the one that matters most:
a 20-page transfer-pricing agreement audited blind, with no answer key, whose defects were
identified only afterwards by manual review.
| Document | Planted / real defects | Caught | False positives |
|---|---|---|---|
| MIC (blind, no answer key) | 4 | 4 / 4 | — |
| VANGUARD (20p) | 3 | 3 / 3 | 0 |
| APEX (5 planted mistakes) | 5 | 5 / 5 surfaced (4 hard-flagged, 1 at needs_review) |
0 |
Acceptance eval (held-out gold set): recall 1.00, 0 invented citations, 0 false positives on a clean document.
Honest caveats. APEX and VANGUARD are dev-set numbers — the same documents drove both diagnosis and validation. MIC is the only genuinely blind result, and it is the one that exposed two real misses (a shredded penalty table and a missing checklist backstop) that the dev set never could. Next milestone is a sealed test set authored blind and scored once.
flowchart LR
subgraph A["A · Ingestion (one-time)"]
A1[PDF corpus] --> A2[Chapter/Article/Clause tree]
A2 --> A3[Chunk + embed]
A3 --> A4[(Chroma + BM25)]
A2 --> A5[Penalty catalog<br/>hand-verified]
A2 --> A6[Obligation checklist]
end
subgraph B["B · User doc"]
B1[PDF/DOCX/XLSX/CSV] --> B2[Segment into units]
B2 --> B3[Extract numeric/date facts]
end
subgraph C["C · Engine"]
C1[pass 0 · deterministic rules] --> C2[pass i · grounded assessment]
C2 --> C3[pass ii · obligation/omission]
C3 --> C4[penalty mapping]
C4 --> C5[independent verifier]
end
A4 --> C2
A5 --> C4
A6 --> C3
B3 --> C1
C5 --> D[["D · Report<br/>SQLite · JSON/HTML · Streamlit"]]
The accuracy machinery, which is most of the engineering:
- Hybrid retrieval — dense (bge-large) + BM25, fused with Reciprocal Rank Fusion, then cross-encoder reranked. Plus LLM query expansion: the raw text of a violation rarely shares vocabulary with the rule that governs it, so each unit is also retrieved by generated legal-concept queries.
- Quote grounding — a finding is discarded unless its quote is verifiably present in a retrieved chunk. Citations are read from chunk metadata, never parsed from the LLM's prose.
- Two-pass coverage — pass i catches what the document says (misstatements); pass ii catches what it omits, by judging 368 corpus-derived obligations against the document.
- Independent verifier — a second LLM pass, given the full grounded chunk text, must
confirm each finding or downgrade it to
needs_review. - Determinism guards — canonical dedup keys, re-ask on unstable verdicts, seed passthrough,
double-confirmed penalty selection, and per-run
trace.jsondiagnostics. - Provenance — every run records base commit, prompt version, and a code fingerprint, so any result can be traced to the exact code that produced it.
Full design rationale in ARCHITECTURE.md; day-by-day decisions in
CHANGELOG.md.
| Layer | Choice | Why |
|---|---|---|
| Parsing | PyMuPDF, python-docx, openpyxl | Digital-native PDFs; no OCR needed |
| Embeddings | BAAI/bge-large-en-v1.5 |
Strong retrieval quality on CPU |
| Reranking | BAAI/bge-reranker-v2-m3 |
Cross-encoder; precision at small k |
| Vector store | Chroma (persistent) | Local, zero-ops, no external service |
| Lexical | rank-bm25 |
Legal text is citation- and term-heavy |
| LLM | Provider-agnostic — Ollama, Gemini, Anthropic, any OpenAI-compatible | Swap via config; no code changes |
| Schemas | Pydantic v2 | Structured LLM output, validated at the boundary |
| Storage / UI | SQLite, Streamlit | Single-file persistence, local-first UI |
| Quality | pytest (61), ruff, GitHub Actions | Lint + matrix tests on 3.12/3.13, Linux + Windows |
~5.4k lines of Python across ingestion, retrieval, engine, storage, and CLI/UI.
Requires Python 3.12 or 3.13.
git clone https://github.com/Jaden-Varkey/LexVecta.git && cd LexVecta
python -m venv .venv && .venv/Scripts/activate # Windows
# source .venv/bin/activate # macOS/Linux
pip install -r requirements.txt && pip install -e .
cp .env.example .env # add a key only for cloud LLMslexvecta ingest --force # 1. build the knowledge base (one-time)
lexvecta audit path/to/doc.pdf # 2. audit → report.json + report.html
lexvecta report <job_id> # 3. re-print a stored report
lexvecta eval # 4. precision/recall on the gold set
streamlit run app/streamlit_app.pyChoosing a provider (config.yaml → llm.provider):
| Provider | Notes |
|---|---|
ollama |
Local and private — no document text leaves your machine |
gemini |
Strong free tier accuracy; ~20 req/day unbilled |
anthropic |
Paid accuracy upgrade (claude-opus-4-8) |
groq / cerebras / openrouter / openai |
OpenAI-compatible endpoints |
Cloud providers receive your document text; the UI shows a banner when one is active.
Six official government documents: an explanatory corporate tax guide (107p), the tax procedures decree-law, the binding administrative-penalties decision (the only penalty source), and three ministerial decisions covering small business relief, accounting standards, and related definitions. Retrieval is closed-book over exactly these files, which is what makes "it cannot invent a law" a structural property rather than a prompt instruction.
pip install -e ".[test]" # lean: offline suite in ~20s, no ML stack needed
pytest -q
ruff check src tests app scriptsHeavy ML dependencies are imported lazily, so the unit suite runs without torch, Chroma, or
sentence-transformers; index-backed retrieval tests self-skip when those are absent. CI runs
lint plus the suite on Python 3.12/3.13 on Linux and 3.13 on Windows, then builds and
smoke-tests the wheel on main.
src/lexvecta/
├── ingestion/ extract · structure · chunk · embed_index · penalties · checklist
├── retrieval/ hybrid search · RRF · rerank · cross-ref expansion
├── engine/ rules · assess · penalty_map · verify · checklist_pass · consolidate
├── userdocs/ load · segment · fact extraction
├── llm/ provider adapters · prompts · schemas
└── storage/ SQLite DAO · report builders
{ "status": "non_compliant", "location": "page 13 · 13.2 Client Acquisition Hospitality Protocols", "user_excerpt": "The corporate finance desk allows a full 100% tax deduction for all client-facing catering, luxury hotel bookings, and airline transportation…", "issue_summary": "Asserts a full 100% deduction for client-facing meals, hotels and transport. These are entertainment expenditure, capped at 50%.", "violated_provisions": [{ "citation": "CT Guide — Chapter Nine — Article 32: Entertainment Expenditure", "quote": "…50% of any entertainment, amusement, or recreation expenditure incurred during a Tax Period by a Taxable Person may be deducted…", "source_authority": "explanatory" }], "penalty": { "citation": "Cabinet Decision No. 75 of 2023 — Penalties Table, Item 9", "amount_text": "500, unless the Person corrects his Tax Return before the deadline…" }, "confidence": 0.99, "verified": true }