Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .env.example
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
COMPOSE_FILE=docker-compose.yml:docker-compose.prod.yml

DB_URL=jdbc:mariadb://mariadb:3306/jobtracker
DB_URL=jdbc:postgresql://localhost:5432/jobtracker
DB_NAME=jobtracker
DB_USERNAME=jobtracker
DB_PASSWORD=sua_senha_forte_aqui
Expand Down
29 changes: 29 additions & 0 deletions .github/workflows/data-migration-tests.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
name: MariaDB to PostgreSQL migrator tests

on:
pull_request:
paths:
- "scripts/migrate_mariadb_to_postgresql.py"
- "scripts/requirements-migrate-postgresql.txt"
- "tests/test_migrate_mariadb_to_postgresql.py"
- ".github/workflows/data-migration-tests.yml"
push:
branches: [main]
paths:
- "scripts/migrate_mariadb_to_postgresql.py"
- "tests/test_migrate_mariadb_to_postgresql.py"
- ".github/workflows/data-migration-tests.yml"
workflow_dispatch:

jobs:
unit-tests:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/setup-python@v6
with:
python-version: "3.13"
- name: Compile and run isolated unit tests
run: |
python -m compileall -q scripts/migrate_mariadb_to_postgresql.py
python -m unittest discover -s tests -p 'test_migrate_mariadb_to_postgresql.py' -v
23 changes: 11 additions & 12 deletions .github/workflows/load-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -21,28 +21,27 @@ jobs:
- name: Build API image
run: docker build -t job-tracker:loadtest .

- name: Start MariaDB (test)
- name: Start PostgreSQL (test)
run: |
docker run -d \
--name loadtest-db \
--network loadtest-net \
-e MARIADB_ROOT_PASSWORD=root \
-e MARIADB_DATABASE=jobtracker \
-e MARIADB_USER=jobtracker \
-e MARIADB_PASSWORD=jobtracker \
mariadb:11
-e POSTGRES_DB=jobtracker \
-e POSTGRES_USER=jobtracker \
-e POSTGRES_PASSWORD=jobtracker \
postgres:17

- name: Wait for MariaDB
- name: Wait for PostgreSQL
run: |
for i in {1..30}; do
if docker exec loadtest-db mariadb-admin ping -h 127.0.0.1 -uroot -proot --silent; then
echo "MariaDB is ready"
if docker exec loadtest-db pg_isready -h 127.0.0.1 -U jobtracker -d jobtracker; then
echo "PostgreSQL is ready"
exit 0
fi
echo "Waiting for MariaDB... ($i/30)"
echo "Waiting for PostgreSQL... ($i/30)"
sleep 2
done
echo "MariaDB did not become ready in time"
echo "PostgreSQL did not become ready in time"
exit 1

- name: Start API with server-like limits (1 CPU / 4GB)
Expand All @@ -55,7 +54,7 @@ jobs:
--memory-swap=4g \
-p 8080:8080 \
-p 8081:8081 \
-e DB_URL="jdbc:mariadb://loadtest-db:3306/jobtracker?createDatabaseIfNotExist=true&characterEncoding=UTF-8&useUnicode=true" \
-e DB_URL="jdbc:postgresql://loadtest-db:5432/jobtracker" \
-e DB_USERNAME="jobtracker" \
-e DB_PASSWORD="jobtracker" \
-e JWT_SECRET="LoadTestSecretKeyThatIsAtLeast256BitsLongAndOnlyForGithubActions" \
Expand Down
63 changes: 57 additions & 6 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

[Frontend - React-JobApplyTracker](https://github.com/vitorhugo-java/React-JobApplyTracker)

A production-ready Spring Boot REST API for tracking job applications, built with Java 21, Spring Security JWT authentication, MariaDB, and comprehensive test coverage. Exposes all domain services via a **Model Context Protocol (MCP) server** so AI assistants (Claude Desktop, Claude.ai, and any MCP-compatible client) can manage applications on behalf of authenticated users.
A production-ready Spring Boot REST API for tracking job applications, built with Java 21, Spring Security JWT authentication, PostgreSQL, and comprehensive test coverage. Exposes all domain services via a **Model Context Protocol (MCP) server** so AI assistants (Claude Desktop, Claude.ai, and any MCP-compatible client) can manage applications on behalf of authenticated users.

> **Alterações no MCP podem exigir atualização manual dos clientes.**
>
Expand All @@ -27,7 +27,7 @@ A production-ready Spring Boot REST API for tracking job applications, built wit
- **Spring Security** with stateless JWT authentication + role-based authorization (`USER`, `BETA`, `ADMIN`)
- **JWT + Refresh Tokens** (access: 15 min, refresh: 7 days with rotation)
- **Resilience4j Rate Limiting** on auth endpoints
- **MariaDB** (production) / **Testcontainers** (tests)
- **PostgreSQL** (production and local development) / **H2** (unit tests)
- **Flyway** for DB migrations
- **JUnit 5 + Mockito** (unit tests)
- **Testcontainers + MockMvc** (integration tests)
Expand Down Expand Up @@ -331,10 +331,10 @@ By default this compose file pulls image: `ghcr.io/vitorhugo-java/springboot-job

The API will be available at `http://localhost:8080`.

### With Maven (requires a running MariaDB)
### With Maven (requires a running PostgreSQL)

```bash
export DB_URL=jdbc:mariadb://localhost:3306/jobtracker?createDatabaseIfNotExist=true
export DB_URL=jdbc:postgresql://localhost:5432/jobtracker
export DB_USERNAME=jobtracker
export DB_PASSWORD=jobtracker
export JWT_SECRET=your-secret-key-at-least-256-bits-long
Expand Down Expand Up @@ -726,7 +726,7 @@ mvn test -Dtest="com.jobtracker.integration.*"
mvn test -Dtest="com.jobtracker.e2e.*"
```

> **Note:** Integration and E2E tests require Docker to be running (Testcontainers pulls a MariaDB image automatically).
> **Note:** Integration and E2E tests require Docker to be running.

## Seed Fake Data

Expand Down Expand Up @@ -764,7 +764,7 @@ If `APP_SEED_ENABLED=true` and `APP_SEED_USER_EMAIL` is not provided (or the use

| Variable | Default | Description |
|----------|---------|-------------|
| `DB_URL` | `jdbc:mariadb://localhost:3306/jobtracker` | JDBC URL |
| `DB_URL` | `jdbc:postgresql://localhost:5432/jobtracker` | JDBC URL |
| `DB_USERNAME` | `jobtracker` | DB username |
| `DB_PASSWORD` | `jobtracker` | DB password |
| `JWT_SECRET` | *(dev default)* | JWT signing secret (min 256 bits) |
Expand Down Expand Up @@ -865,3 +865,54 @@ funnel status, including moving between interview stages. Yes adds one to the fo
without adding; closing the popup cancels saving. The total can also be corrected manually in the form.
Dashboard and MCP analytics sum stored completed counts. Analytics `interviewRate` is the percentage
of applications with at least one completed interview, rather than interviews divided by applications.

## PostgreSQL migration and rollback

The PostgreSQL backend uses native UUID columns, PostgreSQL Flyway migrations V1-V36 and a generated full-text index. **Flyway migrates the schema, not the existing MariaDB records.** For a one-time data cutover, use [scripts/migrate_mariadb_to_postgresql.py](scripts/migrate_mariadb_to_postgresql.py). It uses PyMySQL + psycopg 3 (streaming PostgreSQL COPY), without pgloader.

### Prerequisites

1. Verify the MariaDB backup can be restored, and restore it into an **isolated, read-only MariaDB source**. The Python tool connects to a running MariaDB instance; it does **not** parse a .sql/.gz dump directly.
2. Provision a **new, dedicated, empty PostgreSQL database**. Start the new backend **once** with this database to let Flyway apply V1-V36, then **stop it**. Disable app seeding/scheduled jobs while preparing the target. Only the nine Flyway-seeded `application_statuses` rows are allowed before import.
3. Keep the application stopped so nothing writes to PostgreSQL during the copy. Confirm the MariaDB backup and database version represent the latest application data. Maintain the old MariaDB backup and old backend image for rollback.
4. Use least-privilege credentials: read-only MariaDB user and a PostgreSQL user allowed to COPY/DELETE and reset the identity sequence. Do not commit credentials to the repository.

### Install and run

From repository root on a host that can reach **both** database instances:

```bash
python3 -m venv .venv
. .venv/bin/activate
python -m pip install -r scripts/requirements-migrate-postgresql.txt

export MIGRATE_MYSQL_HOST=127.0.0.1
export MIGRATE_MYSQL_PORT=3306
export MIGRATE_MYSQL_DATABASE=jobtracker
export MIGRATE_MYSQL_USER=jobtracker
read -rsp 'MariaDB backup password: ' MIGRATE_MYSQL_PASSWORD; echo; export MIGRATE_MYSQL_PASSWORD

export MIGRATE_PG_HOST=127.0.0.1
export MIGRATE_PG_PORT=5432
export MIGRATE_PG_DATABASE=jobtracker
export MIGRATE_PG_USER=jobtracker
read -rsp 'PostgreSQL password: ' MIGRATE_PG_PASSWORD; echo; export MIGRATE_PG_PASSWORD

# Default: validate schema, Flyway history and destination emptiness; no writes.
python scripts/migrate_mariadb_to_postgresql.py

# Only after preflight succeeds and backup restore is verified:
python scripts/migrate_mariadb_to_postgresql.py --execute
```

The tool validates exact source/target table and writable-column sets, refuses unexpected destination records, excludes Flyway history and generated search vectors, copies data in FK dependency order, converts MariaDB BINARY(16) UUIDs to native UUIDs and TINYINT(1) booleans, and compares order-independent SHA-256 content fingerprints for all copied columns in every table. It restores PostgreSQL identity sequences after verification, then commits. Errors roll back data modifications; **sequence changes are not transactional**, so the empty destination should be discarded and rebuilt after a failed attempt. It will not overwrite an existing populated ApplyWell PostgreSQL database.

**Important:** This is a one-time cutover tool. Its unit tests cover pure conversion and validation logic, but **it has not been tested against your production backup**. Test the entire procedure on isolated instances before production.

### Cutover / rollback

1. For the final cutover, stop writes to the old MariaDB and restore a **final** consistent backup into the isolated MariaDB source. Recreate an empty PostgreSQL target and allow Flyway V1-V36 to initialize it before running the Python migrator.
2. Confirm counts, UUID samples, referential integrity, login, Google Drive, OAuth, MCP, assistant search and export using the new backend and migrated data.
3. Switch traffic to the PostgreSQL-backed new backend only after validation; do not run both backends writing in parallel.
4. For rollback, **stop the new backend, restore the old MariaDB-compatible backend image and its DB_URL**, and reconcile any writes accepted after cutover. Repointing the new PostgreSQL-only application at MariaDB is not supported. Keep MariaDB and the backups until acceptance.

2 changes: 1 addition & 1 deletion docker-compose.prod.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ services:
image: ghcr.io/vitorhugo-dotnet/jobapplytracker-backend:latest
environment:
SPRING_PROFILES_ACTIVE: prod
DB_URL: "jdbc:mariadb://mariadb:3306/${DB_NAME}?createDatabaseIfNotExist=true&characterEncoding=UTF-8&useUnicode=true"
DB_URL: "jdbc:postgresql://postgres:5432/${DB_NAME}"
deploy:
resources:
limits:
Expand Down
20 changes: 19 additions & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
@@ -1,4 +1,19 @@
services:
postgres:
image: postgres:17
environment:
POSTGRES_DB: ${DB_NAME:-jobtracker}
POSTGRES_USER: ${DB_USERNAME:-jobtracker}
POSTGRES_PASSWORD: ${DB_PASSWORD:-jobtracker}
ports:
- "5432:5432"
volumes:
- postgres_data:/var/lib/postgresql/data
healthcheck:
test: ["CMD-SHELL", "pg_isready -U ${DB_USERNAME:-jobtracker} -d ${DB_NAME:-jobtracker}"]
interval: 5s
timeout: 5s
retries: 10
app:
build: .
container_name: job-tracker-app
Expand All @@ -8,7 +23,7 @@ services:
- .env
environment:
SPRING_PROFILES_ACTIVE: dev
DB_URL: ${DB_URL}
DB_URL: jdbc:postgresql://postgres:5432/${DB_NAME:-jobtracker}
DB_USERNAME: ${DB_USERNAME}
DB_PASSWORD: ${DB_PASSWORD}
JWT_SECRET: ${JWT_SECRET}
Expand Down Expand Up @@ -38,3 +53,6 @@ services:
timeout: 10s
retries: 5
start_period: 60s

volumes:
postgres_data:
13 changes: 4 additions & 9 deletions pom.xml
Original file line number Diff line number Diff line change
Expand Up @@ -75,13 +75,8 @@

<!-- JDBC Driver -->
<dependency>
<groupId>org.mariadb.jdbc</groupId>
<artifactId>mariadb-java-client</artifactId>
<scope>runtime</scope>
</dependency>
<dependency>
<groupId>com.mysql</groupId>
<artifactId>mysql-connector-j</artifactId>
<groupId>org.postgresql</groupId>
<artifactId>postgresql</artifactId>
<scope>runtime</scope>
</dependency>

Expand All @@ -92,7 +87,7 @@
</dependency>
<dependency>
<groupId>org.flywaydb</groupId>
<artifactId>flyway-mysql</artifactId>
<artifactId>flyway-database-postgresql</artifactId>
</dependency>

<!-- JWT -->
Expand Down Expand Up @@ -293,7 +288,7 @@
</dependency>
<dependency>
<groupId>org.testcontainers</groupId>
<artifactId>mariadb</artifactId>
<artifactId>postgresql</artifactId>
<version>${testcontainers.version}</version>
<scope>test</scope>
</dependency>
Expand Down
Loading
Loading