R1999 Code Bot

繁體中文

R1999 Code Bot watches public Reverse: 1999 redemption-code sources, extracts likely codes, deduplicates them in SQLite, and publishes new-code messages to Bluesky. It is built for long-running automated operation, but it can also run in dry-run mode for local verification.

If you only want regular code notifications, you do not need to host this bot. Follow the long-running Bluesky account instead: https://bsky.app/profile/r1999-code-kanban.bsky.social

How it works

public sources
  ├─ Nitter RSS: GL / JP active source
  ├─ Facebook FetchRSS: TW active source
  ├─ Bahamut fixed redemption-code thread: TW / JP / GL fallback
  ├─ Wikiru JP redemption-code page: JP supplement
  ├─ Fandom Promotional Code page: GL supplement
  └─ Pocket Tactics codes page: GL attributable-guide supplement
        ↓
normalization → coarse eligibility → source-batched LLM extraction
        ↓
SQLite deduplication and resume state
        ↓
Bluesky formatter and publisher

Source health is source-aware. A Wikiru, Fandom, or Pocket Tactics outage is recorded as supplemental-source health, not as an official server failure. A server is considered failed only when both its active source and the relevant Bahamut fallback path are unavailable.

Quick start

Requirements: Python 3.12+, SQLite through Python sqlite3, and the packages in requirements.txt.

python3.12 -m venv .venv
source .venv/bin/activate
python -m pip install --upgrade pip
python -m pip install -r requirements.txt
cp .env.example .env
python -m pytest -q
PUBLISH_DRY_RUN=true python main.py

The app reads process environment variables. It does not automatically load .env; load it with your shell, process manager, Docker, or another supervisor before starting Python.

Configuration

All deployment-specific values belong in environment variables. Do not commit local .env files, databases, logs, or real credentials.

Common settings:

Variable	Purpose
DB_PATH	SQLite database path; default is data/r1999_codes.db.
NITTER_PRIMARY_BASE_URL / NITTER_FALLBACK_BASE_URLS	Nitter RSS base URLs for GL / JP collection.
FACEBOOK_FETCHRSS_FEED_URL	FetchRSS feed URL for the TW Facebook source.
BAHAMUT_THREAD_URL	Fixed Bahamut redemption-code thread URL (display thread only; the Jina Reader path targets the canonical Co single-floor route).
BAHAMUT_FETCH_PROVIDER	jina_reader (default, raw HTML) or desktop_html (legacy rollback).
BAHAMUT_JINA_MAX_BODY_BYTES	Bounded Jina response cap; bodies above the cap fail closed (default 1048576).
BAHAMUT_JINA_READER_BASE_URL	Jina Reader base URL (default https://r.jina.ai).
WIKIRU_PAGE_URL	JP Wikiru redemption-code page.
FANDOM_GL_PAGE_URL / FANDOM_GL_API_URL	GL Fandom page / MediaWiki API source.
POCKETTACTICS_URL / ENABLE_POCKETTACTICS	GL Pocket Tactics attributable-guide source and enable flag.
LLM_PRIMARY_* / LLM_FALLBACK_*	Required primary OpenAI-compatible extraction endpoint; fallback is optional.
PUBLISH_DRY_RUN	Keep true until live posting is intentionally enabled.
BLUESKY_HANDLE / BLUESKY_APP_PASSWORD	Required only for live Bluesky publishing.
CYCLE_INTERVAL_HOURS	Scheduler interval; default is 6 hours.

At process startup, the bot sends a bounded capability probe to each configured primary and fallback model. It logs and caches each model's json_schema, json_object, or plain-JSON mode for that process; no credentials or response content are logged.

For live posting, set PUBLISH_DRY_RUN=false only after the database, logs, and formatted output have been checked.

Transport-specific checkpoints

Fandom, Pocket Tactics, and Bahamut keep independent jina and direct transport slots inside the existing SQLite checkpoint row. Only the validated transport used by the current cycle updates its slot; the bot does not fetch both transports routinely. Switching transports may intentionally replay once if that transport has never been completed before. Legacy flat hashes migrate on the first validated observation without being copied into both slots; malformed v2 checkpoint JSON fails closed and preserves the stored bytes.

Bahamut Jina transport

Bahamut is fetched credential-free through Jina Reader using the canonical Co.php?bsn=75028&sn=13937 first-floor route. The bot validates the expected DOM identity and parses only that floor.

• Trust boundary: Jina Reader is the upstream trust root. The bot does not verify an origin signature and cannot detect content injected inside the accepted article.
• Fail-closed: Invalid DOM identity, challenge pages, transport failures, malformed checkpoint state, and responses above BAHAMUT_JINA_MAX_BODY_BYTES do not advance the checkpoint.
• Upgrade: Keep the existing database. Evergreen-page cold-start baselines remain suppressed from publishing; extraction completion is recorded only after durable persistence, and existing LLM attempt rows are backfilled once during DB initialization.
• Verification and rollback: Review one dry-run checkpoint after deployment. Use BAHAMUT_FETCH_PROVIDER=desktop_html for degraded rollback or ENABLE_BAHAMUT=false to disable the source.

Useful commands

Run the offline default test suite. Live external integration tests are excluded and no network request is made:

python -m pytest -q

Run the two real LLM integration tests only when the required credentials and network access are intentionally available:

python -m pytest tests/test_integration.py -m integration -v

Run one scheduled bot process in dry-run mode:

PUBLISH_DRY_RUN=true python main.py

Docker

docker build -t r1999-code-bot .
mkdir -p ./data
docker run --rm \
  --env-file .env \
  -e DB_PATH=/app/data/r1999_codes.db \
  -v "$(pwd)/data:/app/data" \
  r1999-code-bot

Clean release artifacts include release_identity.json, so ordinary Docker builds need no identity build arguments. A development checkout without a generated manifest reports metadata as unknown.

Keep PUBLISH_DRY_RUN=true for first-run validation. Use a persistent mounted data directory for the SQLite database.

Repository map

Path	Purpose
main.py, main_policy.py	Scheduler, pipeline orchestration, health and checkpoint policy.
collectors/	Nitter, Facebook, Bahamut, Wikiru, Fandom, and Pocket Tactics collection and source normalization.
filters/	Multilingual coarse-filter signals.
llm/	Required-primary OpenAI-compatible extraction and batching; fallback endpoint optional.
publisher/	Bluesky formatting and publishing.
storage/	SQLite schema and database access.
docs/	Deployment notes.

Security and privacy

• Keep .env, local databases, logs, caches, generated support bundles, and real credentials out of git.
• Use Bluesky App Passwords for live publishing; do not store the account password.
• Treat FetchRSS feed URLs, LLM endpoints, model names, API keys, and Bluesky credentials as deployment-specific values.
• .env.example must stay placeholder-only and safe to publish.

Acknowledgments

Thanks to the public Bahamut thread maintainers, wiki contributors, Pocket Tactics, and the Nitter project for public source availability that makes cross-source validation possible.

License

MIT. See LICENSE.