Skip to content
estherdev03Public

About

AI assistant for Canada's Express Entry: NLP profile extraction, RAG-based NOC classification, and rule-based CRS/eligibility scoring

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

MaplePath AI

An AI-powered Express Entry assistant for Canada’s skilled immigration system.

Live demo: maplepath.net

Watch: a one-minute walkthrough of how MaplePath is built, from profile intake to scoring.

maplepath-walkthrough.mp4

MaplePath helps prospective immigrants understand where they stand: it extracts a structured profile from natural language, classifies the occupation under NOC 2021, calculates a CRS score, and assesses eligibility for Federal Skilled Worker, Canadian Experience Class, and Federal Skilled Trades.

Built as a production-oriented Python backend: LangGraph orchestration, pgvector semantic retrieval over PostgreSQL (chosen by benchmark against keyword, RRF, and reranked hybrid search), retrieval-augmented occupation classification, and rule-based scoring engines that stay explainable. A Next.js frontend walks applicants through intake, confirmation, and results against that API.


Recent changes

  • MaplePath now runs on AWS at maplepath.net. The frontend, backend, and database moved from Vercel and Railway to a single EC2 instance running Docker Compose, with Caddy serving HTTPS. Only Caddy is public, the frontend calls the API over Docker's private network, so the backend and database are no longer reachable from the internet.

  • The score simulator now uses the backend's CRS engine for every change. It used to recalculate in the browser with a single-applicant copy of the rules, so married applicants saw a different score there than on the results page. Each move now sends the edited profile to POST /crs/simulate, so the simulator and the results page can no longer disagree. The simulator also gained a marital status toggle and spouse controls (education, language, Canadian work experience), so it covers every CRS factor. To support it, /crs/simulate accepts CLB/NCLC levels directly when no raw test scores are sent.

  • CEC work experience now counts only when the occupation is TEER 0–3. Applicants aren't asked the TEER of past work, so the classified occupation stands in for it.

  • Clearer confirm form. Required fields are marked, the work experience section is grouped by inside and outside Canada, and each field has a hint explaining what counts (fulltime vs part-time, and work that doesn't count for Express Entry).

  • CRS and eligibility rules checked against IRCC's current pages, and corrected. Fixes include:

    • FSW no longer requires a second-language test.
    • Age points stop at 46.
    • The skill-transferability groups are capped at 50 each.
    • Spouse points use the spouse's first official language.
    • Canadian study points follow IRCC's 1–2 year and 3+ year bands.
    • Proof of funds now scales with family size (spouse and dependent children) and is waived only with a job offer and work authorization.

    The profile, AI parser, and confirm form gained the fields these rules need.

  • Occupation classification now uses pgvector semantic search instead of the reranked hybrid pipeline. On the expanded labelled set, vector search scored higher NDCG@10 than hybrid (0.609 vs 0.583) with a 99% top-10 hit rate, so /profile/complete now retrieves NOC candidates with noc_semantic_search. BM25, RRF, and hybrid remain in noc/ for comparison in the benchmark.

  • Labelled evaluation set grew from 9 to 300 job descriptions (data/noc_eval_labels.csv), and hit rate is now strictly top-10. A bug had counted a match anywhere in the result list.

  • Frontend styling moved from inline styles to Tailwind CSS. The design-system rules in frontend/app/globals.css now sit in CSS layers so utility classes can override them.

  • 100% test coverage on the CRS engine (crs/) and REST API (api/app.py). The new tests also led to fixes for crashes on missing language scores and on profiles without an occupation. Run make testcov for a coverage report.


Why it exists

Express Entry decisions depend on occupation codes, language conversions, and program-specific rules. Applicants often receive generic advice that cannot be audited. MaplePath separates AI where it is useful (parsing and occupation matching) from deterministic policy (CRS and eligibility), so every result can be inspected category by category.


Capabilities

Area What it delivers
Profile intelligence Converts unstructured text into a validated applicant profile, flagging missing or ambiguous fields instead of inventing data
Occupation matching pgvector semantic search retrieves official NOC 2021 profiles, an LLM then picks a unit group, TEER, and confidence score from those candidates only
CRS scoring Full Comprehensive Ranking System breakdown: core human capital, spouse factors, skill transferability, and additional points
Score simulator Adjust age, languages, work experience, education, marital status, spouse factors and additional points, and see the CRS rescored by the same engine as the results
Program eligibility Requirement-level results for FSW, CEC, and FST, including the FSW 67-point selection grid and settlement-fund checks by family size
Personalized advice An LLM turns the computed CRS breakdown and eligibility results into ranked, profile-specific recommendations, without recalculating scores

Language support: IELTS, CELPIP, and PTE (English), TEF and TCF (French), converted to CLB / NCLC before scoring. CLB / NCLC levels can also be sent directly, which is how the simulator expresses language changes.


Engineering highlights

Skills this project is designed to demonstrate:

  • Agentic workflow design with LangGraph (event routing, typed state, multi-step profile → score → eligibility pipeline)
  • Four retrieval pipelines implemented side by side: BM25 full-text, pgvector semantic search, Reciprocal Rank Fusion, and RRF + Cohere reranking (hybrid). The shipped method is picked by measured results, not assumed
  • Retrieval-augmented generation (RAG) for NOC classification: retrieved occupation records are passed as context, and the model must choose from that list rather than invent a code
  • LLM structured extraction with LangChain / OpenAI, constrained to stated facts, with missing-field and warning lists
  • Domain rule engines for CRS and Express Entry, implemented in Python with Pydantic models (auditable point tables, not prompt-only scoring)
  • Retrieval evaluation using NDCG@10 and hit rate across BM25, vector, RRF, and hybrid pipelines. On the 300-query labelled set, the shipped vector pipeline scores 0.609 NDCG@10 (+0.026 over Hybrid) with a 99% top-10 hit rate. Results are shown on the frontend's benchmark page, with a paginated per-example report
  • API and data layer: FastAPI, SQLAlchemy, PostgreSQL, Docker Compose for local and production infrastructure
  • Cloud deployment on AWS EC2: multi-container Docker Compose stack behind a Caddy reverse proxy with automatic HTTPS, private-network-only backend and database, keyless access through Systems Manager, and encrypted, snapshotted storage
  • End-to-end product surface: a Next.js/TypeScript frontend (intake, confirmation, results, CRS simulator, NOC match, eligibility, advice, and retrieval benchmark views) calling the FastAPI backend
  • Automated test suite with pytest covering the graph, services, and API layer, with 100% line coverage on the CRS engine (crs/) and the REST API (api/app.py)

Architecture

flowchart TD
    applicant(("Applicant"))

    subgraph web["Frontend (Next.js)"]
        intake["Profile intake<br/>IntakeForm.tsx"]
        confirm["Profile confirmation<br/>ConfirmForm.tsx"]
        context["Confirmed profile in the browser<br/>profile-context.tsx"]
        views["Results, NOC match,<br/>eligibility and advice pages"]
        simulator["CRS simulator<br/>SimulatorView.tsx"]
        benchmark["Retrieval benchmark<br/>benchmark/page.tsx"]
        client["Server actions and API client<br/>actions.ts, api.ts"]
    end

    subgraph backend["Backend (FastAPI)"]
        api["REST API<br/>api/app.py"]

        subgraph workflow["LangGraph workflow"]
            lg["Graph and routers<br/>construct.py, routers.py"]
            nodes["Nodes<br/>parse, create profile, CRS,<br/>eligibility, advice"]
            profile["Profile service<br/>user_profile/service.py"]
        end

        subgraph scoring["Scoring and eligibility (rule-based)"]
            crs["CRS scoring<br/>crs/service.py"]
            lang["Language conversion<br/>english.py, french.py"]
            transfer["Skill transferability<br/>transferability.py"]
            elig["Program eligibility<br/>eligibility/service.py"]
        end

        subgraph noc["NOC classification"]
            nocsvc["NOC search<br/>noc/service.py"]
            nocrepo["NOC repository<br/>noc/repository.py"]
            evaluate["Retrieval evaluation<br/>noc/evaluate.py"]
        end
    end

    subgraph external["External services"]
        openai["OpenAI<br/>chat model and embeddings"]
        cohere["Cohere Rerank"]
    end

    db[("PostgreSQL + pgvector<br/>NOC records, benchmark cache")]

    applicant --> intake
    intake -->|"POST /profile/parse"| client
    confirm -->|"POST /profile/complete"| client
    simulator -->|"POST /crs/simulate on each change"| client
    benchmark -->|"GET /evaluate"| client
    client -->|"stores confirmed profile"| context
    context --> views
    context --> simulator
    client -->|"HTTP, X-API-Key"| api

    api -->|"parse and complete"| lg
    lg --> nodes
    nodes --> profile
    profile -->|"LLM calls"| openai
    profile -->|"NOC candidates"| nocsvc
    profile --> crs
    profile --> elig

    api -->|"simulate, no LLM"| crs
    api -->|"simulate"| elig
    crs --> lang
    crs --> transfer

    api -->|"evaluate"| evaluate
    evaluate -->|"BM25, vector, RRF, hybrid"| nocsvc
    nocsvc -->|"embeddings"| openai
    nocsvc -->|"hybrid rerank"| cohere
    nocsvc --> nocrepo
    nocrepo --> db
    evaluate -->|"caches results"| db
Loading
Module Responsibility
graph/ Workflow graph, routers, and shared application state
user_profile/ Profile parse, occupation classification, score and eligibility orchestration
noc/ NOC ingest, keyword / semantic / hybrid search, retrieval evaluation
crs/ CRS points, language conversion, skill transferability
eligibility/ FSW, CEC, and FST rule evaluation
db/ SQLAlchemy models and persistence
api/ REST API
frontend/ Next.js application (intake, confirmation, results, simulator, NOC match, eligibility, advice, benchmark)
tests/ pytest suite mirroring the module layout above

Tech stack

Languages & runtime: Python 3.12, uv, Node.js

AI & retrieval: LangGraph, LangChain, OpenAI (gpt-5.4-mini chat, text-embedding-3-large embeddings), Cohere Rerank (rerank-english-v3.0, used by the hybrid method in the benchmark)

Backend: FastAPI, Pydantic, SQLAlchemy

Frontend: Next.js (App Router), React, TypeScript, Tailwind CSS

Data: PostgreSQL, pgvector, BM25 (pg_textsearch)

Ops: Docker Compose, Caddy, AWS (EC2, Elastic IP, EBS snapshots, Systems Manager), Makefile

Testing: pytest


Getting started

Prerequisites: Python 3.12+, uv, Node.js 20+, Docker, OpenAI and Cohere API keys.

Backend:

uv sync
cp example.env .env   # set OPENAI_API_KEY, COHERE_API_KEY, POSTGRES_*, DB_URL
make pgup
uv run python init_db.py          # first run: load NOC 2021 from ESDC and generate embeddings
uv run fastapi dev api/app.py

Frontend:

cd frontend
npm install
npm run dev   # http://localhost:3000, expects the API at http://localhost:8000 (override with API_URL)
Method Endpoint Description
POST /profile/parse Extract a structured draft from free-text
POST /profile/complete Confirm a profile; return occupation, CRS, and eligibility
POST /crs/simulate Recalculate CRS score and eligibility for an edited profile (used by the simulator on every change)
GET /evaluate Run the NOC retrieval benchmark and return its report

NOC retrieval benchmark (labelled set in data/noc_eval_labels.csv): call GET /evaluate, open the frontend's /benchmark page, or run it directly:

uv run python -c "from noc.evaluate import noc_retrieval_method_evaluate as run; print(run('data/noc_eval_labels.csv')[1])"

Results are cached in the database by the labels file's hash, so the benchmark only re-runs (and calls Cohere) when the labelled set changes.

Run the test suite with uv run pytest, or make testcov for a coverage report.

Stop the database with make pgdown (add make pgdownvol to drop the volume). Initial init_db.py scrapes official NOC profiles and may take a substantial amount of time.


Deployment

Live at https://maplepath.net.

The whole stack runs on one AWS EC2 instance with Docker Compose (docker-compose.prod.yaml):

Browser ──HTTPS──▶ EC2 instance
                    ├─ caddy     ports 80/443, automatic HTTPS (Let's Encrypt), the only public service
                    ├─ frontend  Next.js standalone server, port 3000, internal only
                    ├─ backend   FastAPI, port 8000, internal only
                    └─ postgres  timescaledb-ha:pg18 (pgvector + pg_textsearch), internal only, encrypted EBS
  • Images: the backend builds from the root Dockerfile; the frontend builds from frontend/Dockerfile (multi-stage, output: "standalone", runs as a non-root user).
  • Routing: Caddy (Caddyfile) terminates TLS for the domain in API_DOMAIN and proxies to frontend:3000. The frontend's server components and actions call the API at http://backend:8000 with the X-API-Key header.
  • Network: the security group allows only 80 and 443. There's no SSH: the instance is administered through AWS Systems Manager Session Manager.
  • Data: the instance's EBS volume is encrypted and snapshotted daily by a Data Lifecycle Manager policy.

Deploy or update:

git pull
docker build -t maplepath-backend .
docker build -t maplepath-frontend frontend
docker compose -f docker-compose.prod.yaml up -d postgres
docker compose -f docker-compose.prod.yaml run --rm backend python init_db.py   # first deploy only
docker compose -f docker-compose.prod.yaml up -d

The server's .env holds OPENAI_API_KEY, COHERE_API_KEY, POSTGRES_PASSWORD, POSTGRES_DB, API_KEY, and API_DOMAIN (see example.env). DB_URL is built by the compose file. When API_KEY is set, the backend rejects any request without a matching X-API-Key header; leave it unset locally.


Roadmap

Current (v1)
AI profile extraction · benchmarked semantic NOC search · RAG occupation classification · CRS calculator · Express Entry eligibility · retrieval evaluation · Next.js frontend · automated test suite · containerized deployment (AWS EC2 + Docker Compose + Caddy)

Next
IRCC's November 3, 2026 CRS changes (one combined factor for Canadian work experience and job offers, with high-wage occupation points) · Policy Q&A with cited IRCC sources · Alberta (AAIP) pathways · journey planning · policy monitoring and notifications


MaplePath is an engineering project for educational and demonstration purposes. It is not legal advice and does not replace IRCC tools or a licensed immigration representative. Scoring tables and program rules should be verified against current Government of Canada publications.

About

AI assistant for Canada's Express Entry: NLP profile extraction, RAG-based NOC classification, and rule-based CRS/eligibility scoring

Topics

Resources

Stars

0 stars

Watchers

0 watching

Forks

Contributors

Languages