Skip to content
MenuuTUXPublic

About

Native macOS image analysis on Apple Silicon: an on-device vision-language model plus computer-vision measurements, with every module labelled by how far it can be trusted.

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

Tribe Studio

Native macOS creative suite for AI image analysis on Apple Silicon.

An on-device vision-language model reads your image alongside computer-vision measurements and writes an actual report — what's depicted, where the eye lands, what it makes a viewer feel, and what it appears designed to do.

Swift only. No Python. No Homebrew. Single folder.

Target: macOS 14+, MacBook Air M3 16GB (and other Apple Silicon).

⚠️ Important: Methodology Transparency

Tribe Studio mixes real computer vision with hand-tuned heuristics. Every module declares a TransparencyTier, and that tier drives the UI badge, the JSON export and the CLI exit code. See Transparency Tiers and Known Biases.

Tier Meaning Modules
✅ measured Validated CV/ML model attention, face (+ aesthetic & safety with a Core ML model installed)
🔵 derived Objective computation against a standard accessibility (WCAG), platform (aspect fit)
🟠 heuristic Model-backed but unvalidated taxonomy affect, scene, aesthetic, brand, social
🔴 illustrative Unvalidated — do not use for decisions safety (without an NSFW model)

Seven modules were deleted rather than relabelled: brain-proxy, virality, emotion-proxy, audience-resonance, trend-alignment, fine-grained-emotion and appeal. They computed hand-authored coefficients and measured nothing. Affect is now a CLIP probe; the narrative is written by a VLM.

Not pseudo-science — all limitations are declared in code and surfaced to the user. But not all outputs are measurements; most are research-informed estimates that have not been empirically validated against real engagement data.


Quick Start

cd /path/to/tribe
xcodebuild -downloadComponent MetalToolchain   # one time, ~690 MB
chmod +x Scripts/*.sh
./Scripts/build_metallib.sh                    # one time after a clean checkout
./Scripts/build_app.sh
open "Build/Tribe Studio.app"

The first analysis downloads a ~2 GB vision-language model from Hugging Face (cached afterwards). Everything then runs offline on the Neural Engine / GPU.

Or run the binary directly:

swift build -c release
TRIBE_ROOT="$(pwd)" .build/release/TribeApp

Headless analysis:

TRIBE_ROOT="$(pwd)" .build/release/TribeApp --analyze photo.jpg

Batch a folder, with ranking and aggregate stats:

TRIBE_ROOT="$(pwd)" .build/release/TribeApp --batch ./my-photos

Exit codes: 0 everything measured/derived · 2 report contains estimates (expected without Core ML models) · 1 error · 3 timeout.

Tests

swift test

99 tests across 22 suites covering EXIF orientation (all 8 cases), saliency coordinate spaces, fixation paths, pipeline bounds/NaN safety, export artifacts, quality metrics, VLM engine parsing, explicitness verdicts, analysis progress, validation store, accessibility, batch analysis, creator history, frame geometry, the fusion engine, grounding checks, model download, path config, platform optimisation and suggestion prioritisation.

Contributing

Read HANDOFF.md first — it documents the coordinate-space and concurrency invariants that are easy to break silently.

What It Does

  1. Drop in an image
  2. Click Analyze
  3. See scores and overlays for:
    • Attention / saliency heatmap (Vision)
    • Face detection + landmarks (Vision)
    • Aesthetic quality (heuristics ± Core ML)
    • Affect / scene understanding (CLIP zero-shot probes)
    • Safety / platform risk (heuristic ± Core ML NSFW ± VLM explicitness)
    • Accessibility (WCAG contrast, colour-blind simulation)
    • Brand consistency (palette adherence vs. Assets/Presets)
    • Social readiness fusion score
    • AI-generated narrative report (on-device VLM)
  4. Export JSON, heatmap PNG, overlay PNG, summary under Artifacts/Reports/

Documentation

Docs/ is an Obsidian vault — 44 interlinked notes covering every module, formula, bias and open research question. Open the folder in Obsidian, or read Docs/00-INDEX.md as plain markdown.

🗺️ Index entry point, start here
🏗️ Architecture system map and data flow
🤖 AI Layer perception → reasoning → grounding
🧩 Modules one note per analysis module
⚠️ Known Biases enumerated failure modes with reproductions
🔬 Open Research Questions the agenda
🛠️ CLI Reference headless usage
📜 Audit 2026-08-05 what was broken and fixed
🤝 HANDOFF.md invariants — read before contributing

Folder Layout

tribe/
  Package.swift              # Swift package manifest
  README.md                  # This file
  DEVELOPER_GUIDE.md         # Developer onboarding guide
  HANDOFF.md                 # Invariants + audit log — read before contributing
  Docs/                      # Obsidian vault — research & background documents
  Sources/TribeApp/          # SwiftUI app + pipeline modules
    AI/                      # Perception layer, CLIP probes, VLM, grounding
    Models/                  # Data structures
    Modules/                 # Analysis module implementations (10 modules)
    Pipeline/                # Core analysis infrastructure
    Services/                # Model downloader, batch analysis service
    Managers/                # Creator history manager
    Views/                   # SwiftUI views
    Utils/                   # Helper utilities
  Assets/Models/             # Optional .mlmodel / .mlpackage files
  Assets/Presets/            # Configuration presets
  Artifacts/                 # Reports, heatmaps, overlays, cache
  Build/                     # Packaged .app (after build_app.sh)
  Scripts/                   # build_app.sh, build_metallib.sh
  Tests/TribeAppTests/       # 46 tests across 9 suites (swift test)

Everything stays inside tribe/. Nothing installs system-wide.

Optional Core ML Models

No models ship with this project and none are downloaded by default. Drop compiled models into Assets/Models/ yourself:

Filename contains Used by Tier upgrade
aesthetic, nima, laion AestheticModule heuristic → measured
nsfw, safety, nude, racy SafetyModule illustrative → measured

If missing, those modules fall back to labelled heuristics and stay functional — but safety's fallback is a skin-ratio rule that is biased across skin tones and is reported as illustrative. Installing a real NSFW classifier is the highest-value addition you can make.

Optional automatic download is opt-in via Assets/Models/sources.json. See Assets/Models/README.md.

Architecture

Modular pipeline (AnalysisModule protocol):

Module Role Tier
AttentionModule Vision saliency + first-look metrics measured
FaceModule Faces, landmarks, pose-derived framing metrics measured
AccessibilityModule WCAG contrast, colour-blind simulation derived
PlatformOptimizerModule Per-platform aspect/crop fit derived
AestheticModule Quality + composition heuristic (measured with model)
AffectModule CLIP zero-shot affect probes (12 labels) heuristic
SceneUnderstandingModule CLIP zero-shot scene/style classification heuristic
BrandConsistencyModule Palette adherence vs. Assets/Presets heuristic
SocialScoreModule Feeds fusion narrative heuristic
SafetyModule Suggestiveness / shadowban risk illustrative (measured with model + VLM)

An on-device vision-language model (Qwen2.5-VL via MLX Swift) writes the narrative report, grounded against measured evidence by GroundingChecker. See AI Layer for the full design.

Overall score (see FusionEngine.fuse for the authoritative weights):

0.30×Attention + 0.24×Aesthetic + 0.20×Safety
+ 0.14×Platform + 0.12×Accessibility

Social readiness:

0.34×Attention + 0.22×Aesthetic + 0.22×Safety
+ 0.14×Platform + 0.08×Composition

Models load lazily and unload after analysis to respect 16GB unified memory.

Constraints (Non-Negotiable)

  • ✅ No Python runtime in the app
  • ✅ No pip / conda / venv / PyTorch
  • ✅ No Electron / Node / web wrapper
  • ✅ No system-wide app install required
  • ✅ Final bundle: Build/Tribe Studio.app

Development

See DEVELOPER_GUIDE.md for:

  • Complete project structure
  • How to add new analysis modules
  • Testing guidelines
  • Debugging tips
  • Common tasks

Research Notes

Earlier design docs in Docs/ (Python MLX / Swift-vs-Python research) informed the product goals. Implementation language is Swift.

Brain / fMRI-grade prediction is intentionally a proxy until Core ML weights are added.

License

Local project — use as you wish.

About

Native macOS image analysis on Apple Silicon: an on-device vision-language model plus computer-vision measurements, with every module labelled by how far it can be trusted.

Topics

Resources

Contributing

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages