LinguaScreen Translator is a real-time, privacy-first screen translator for Android. It reads the text that is on your screen — from any app — and overlays the translation in place, using a hybrid OCR pipeline (on-device Tesseract + an optional local Python engine) and a multi-layer, zero-cost translation stack. No paid APIs. No accounts. No telemetry.
الأداء السريع، الخصوصية الكاملة، والتكلفة الصفرية — مترجم فوري لكل ما يظهر على شاشتك.
| ⚡ 5–10× faster | End-to-end latency of 50–500 ms on cache hits / warm engines vs. 500–2000 ms typical cloud flows |
| 🔒 100% private | Screen content never leaves your device unless you point the app at your own local engine or explicitly enable a public fallback |
| 💸 Zero cost | 100% open-source stack: Tesseract, OPUS-MT (Helsinki-NLP), Argos Translate, LibreTranslate |
| 🧠 Context-aware | Domain detection, glossaries, and a translation-memory cache keep repeated UI strings instant and consistent |
| 🌍 100+ languages | ISO-639-1 codes, RTL-aware UI, Arabic-first design (واجهة عربية أولاً) |
| 🎨 Material You | Jetpack Compose, dynamic theming, dark mode, TalkBack and 48 dp touch targets |
v2 upgrades the algorithmic core while keeping the stack 100 % free:
- 🧬 Fuzzy translation memory — a screen re-read with a single OCR glyph wrong no longer pays for a full translation; near-identical sources reuse the previous translation at ~zero latency (Levenshtein/Damerau + token-set + char-n-gram cosine, two-stage prefilter/verify).
- 🧩 Layout analysis — recursive XY-cut block grouping, same-row fragment merging and RTL-aware reading order turn fragmented OCR lines into coherent paragraphs before translation (a big MT quality win on menus and multi-column screens).
- 🌓 Adaptive preprocessing — every frame is measured (dark mode? low contrast? small text?) and the pipeline re-tunes itself: auto-invert, CLAHE boost, smart upscale, noise-aware denoise.
- ⚔️ OCR ensemble voting — when the fast pass is weak, every available engine runs and the calibrated best result wins.
- 🎯 Calibrated quality scores — text-sanity analysis is fused with raw engine confidence so "confident garbage" can no longer masquerade as a good read;
/ocrexposes it asquality. - 🛡️ Translation adequacy checks — dropped numbers/URLs, wrong-script output and untranslated echoes are detected and reported on every
/translate. - 📱 Android v2 — dual-fingerprint screen-change detection (dHash + histogram) with adaptive scan cadence for even better battery life, a full Material 3 shape/typography design system, animated components, and share + listen actions on results.
- Screen text detection and extraction via MediaProjection screen capture
- Floating bubble overlay — drag anywhere, tap to translate, auto-translate mode
- Hybrid OCR: on-device Tesseract4Android always available; optional local Python engine (FastAPI) for heavy lifting with lazy-loaded EasyOCR / PaddleOCR and graceful degradation
- Layered translation: memory cache → Python engine (Helsinki-NLP OPUS-MT → Argos) → LibreTranslate fallback
- Offline translation with downloaded models; online fallback only when you allow it
- Translation history, favorites, clipboard translation, camera OCR, voice input, text-to-speech
- Per-app settings, notification controls, auto-start on boot, battery optimizations
- Full accessibility support (TalkBack, RTL, WCAG AA contrast)
flowchart LR
subgraph Android["Android app (Kotlin · Compose · Clean Architecture)"]
CAP[ScreenCaptureService<br/>MediaProjection FGS] --> OCR
BUBBLE[FloatingWindowService<br/>bubble overlay] --> UI[Translation overlay UI]
ACC[AccessibilityService] --> CAP
OCR[HybridOcrEngine<br/>Tesseract4Android on-device] --> PIPE[TranslationPipeline]
PIPE --> L1[L1 · Translation memory<br/>Room cache 0 ms]
PIPE --> CLIENT[EngineClient<br/>Retrofit / OkHttp]
PIPE --> L3[L3 · LibreTranslate<br/>opt-in fallback]
L1 --> UI
CLIENT --> UI
L3 --> UI
end
subgraph Python["Python engine (optional, local)"]
API[FastAPI · /ocr /translate /translate/batch] --> HEL[Helsinki OPUS-MT]
API --> ARG[Argos Translate]
end
CLIENT -- "http://host:8600 (LAN/Termux)" --> API
Details: docs/architecture/ARCHITECTURE.md · API contract: docs/api/PYTHON_ENGINE_API.md
Prerequisites: JDK 17, Android Studio Ladybug (2024.2.1)+ or Android SDK 35. See docs/SETUP.md for the full compatibility matrix.
git clone https://github.com/linguascreen/linguascreen.git
cd linguascreen
# Debug build (uses the Gradle wrapper 8.9 pinned in gradle/wrapper)
./gradlew :app:assembleDebug
# Install on a connected device / emulator (API 26+)
adb install -r app/build/outputs/apk/debug/app-debug.apkFirst launch walks you through onboarding: overlay permission → screen-capture consent → accessibility service → (optional) battery-optimization exemption. See the user manual.
The Android app works out of the box with on-device Tesseract OCR. For maximum accuracy and speed, run the FastAPI engine on the same device via Termux or on any machine in your local network:
cd app/python
python3 -m venv .venv
source .venv/bin/activate # on Termux: source .venv/bin/activate
pip install -r requirements.txt
# exact entry point & Termux notes: see app/python/README.mdThen in the app: Settings → Translation engine → Host/Port and point it at http://<engine-host>:8600. The app pings /health and degrades gracefully (on-device OCR + memory cache + opt-in LibreTranslate) when the engine is unreachable.
Model binaries are never committed to git. Use the helper script:
python3 models/download_models.py tessdata --langs ara,eng --variant fast
python3 models/download_models.py --helpSee models/README.md for the full model inventory, sizes, and licenses.
linguascreen/
├── app/
│ ├── src/ # Android app — Kotlin, Compose, Clean Architecture (MVVM)
│ └── python/ # FastAPI OCR/translation engine (Termux / local server)
├── docs/ # Documentation suite: architecture, APIs, guides, ADRs
│ ├── architecture/ # ARCHITECTURE.md + ADR/ decision records
│ ├── api/ # Python engine REST contract + Android internal API
│ └── user-guides/ # User manual
├── models/ # Model inventory + download script (no binaries committed)
├── tests/ # Python engine test suite (unit / integration / performance)
├── gradle/ # Version catalog (libs.versions.toml) + wrapper
└── .github/workflows/ci.yml # CI: Python tests, Android build/tests, docs lint
The project is spec'd against hard performance budgets — tracked in docs/PERFORMANCE.md:
| Metric | Target |
|---|---|
| Translation latency (p95) | < 500 ms |
| OCR accuracy | > 95% standard text · > 90% complex text |
| Memory (normal operation) | < 300 MB |
| Battery (active use) | < 5% / hour |
| App size | < 100 MB base · < 500 MB with all models |
| Startup / service start | < 2 s / < 500 ms |
| Cache hit rate | > 70% |
| Doc | What's inside |
|---|---|
| docs/README.md | Documentation index & reading order |
| docs/SETUP.md | Environment setup, version compatibility matrix, troubleshooting |
| docs/DEPLOYMENT.md | Release builds, signing, Play Store / F-Droid, engine deployment (Termux, systemd, Docker) |
| docs/architecture/ARCHITECTURE.md | System design, data flow, caching layers, threading, error strategy |
| docs/architecture/adr/ | Architecture Decision Records (ADRs) |
| docs/api/PYTHON_ENGINE_API.md | Locked REST contract: /health, /ocr, /translate, /translate/batch, /languages, /glossary, /cache/clear, /stats, /shutdown |
| docs/api/ANDROID_INTERNAL_API.md | Internal service/repo contracts (informative) |
| docs/user-guides/USER_MANUAL.md | Full user manual (English + Arabic quick guide) |
| docs/PERFORMANCE.md | Benchmarks, methodology, tuning knobs |
| docs/TESTING.md | Test strategy & how to run unit / integration / UI tests |
| docs/SECURITY_PRIVACY.md | Threat model, permission rationale, data-safety mapping |
| docs/LIMITATIONS.md | Known limitations & future improvements |
| docs/CHANGELOG.md | Release notes |
- No telemetry, no analytics, no ads, no accounts.
- Screen captures are processed in memory and never persisted as images.
- Translation memory and history live in an app-private Room database; sensitive values are encrypted with AndroidX Security-Crypto.
- Network egress happens only if you enable a remote fallback (LibreTranslate) or point the app at your own engine; all calls are HTTPS except to explicit LAN addresses you configure.
- No hardcoded keys or secrets anywhere in the codebase. Details and the threat model: docs/SECURITY_PRIVACY.md.
Current known limitations and the forward plan (on-device neural MT via LiteRT, iOS client, GPU acceleration, more OCR language packs) are tracked in docs/LIMITATIONS.md.
- Read the architecture overview and the relevant ADRs — decisions marked LOCKED (API contract, language codes, fallback chain) must not be changed without a superseding ADR.
- Keep the version catalog (
gradle/libs.versions.toml) as the single source of dependency versions; update the matrix in docs/SETUP.md together with it. - Run the full test suites before opening a PR:
./gradlew testfor Android,pytest tests/for the engine (see docs/TESTING.md). - Every PR is validated by CI (.github/workflows/ci.yml): Python tests, Android build + unit tests, and markdown lint.
This project is licensed under the Apache License 2.0 — see LICENSE.
Key third-party components (full inventory in models/README.md and NOTICE files):
| Component | License | Used for |
|---|---|---|
| Tesseract & Tesseract4Android | Apache-2.0 | On-device OCR |
| OPUS-MT / Helsinki-NLP models | CC-BY 4.0 / Apache-2.0 (per model) | Neural machine translation |
| Argos Translate | MIT | Offline MT fallback |
| LibreTranslate | AGPL-3.0 | Optional network fallback (used as a service, not linked) |
| FastAPI, spaCy, OpenCV | MIT / Apache-2.0 | Engine service, NLP, image preprocessing |
Google Play and Android are trademarks of Google LLC. This project is not affiliated with Google.