representation-geometry

Trajectory Geometry of Transformer Representations Across Layers

[EMNLP 2026 Findings] Decomposing Transformer updates into parallel & perpendicular subspaces for geometric probing, compression, and training dynamics.

Mechanistic interpretability of multilingual reasoning in transformers. 170+ causal intervention experiments across 4 model families.

Corpus operators recover different structures, and task-aligned conditionals predict held-out model behaviour — with musical keys as the instrument. Code, results and audit trail for the paper.

Code, results and paper for "The geometry of single-cell foundation models: what they inherit, what they add, and what shapes it"

Prosthetic cognition architecture for AI agents. Deterministic scaffolding over probabilistic reasoning.

Effective rank, RankMe, E1, CKA and anisotropy on transformer hidden states are determined by one direction. The exact identity, and the attention sink behind it.

Mechanistic interpretability of transformer hallucinations via attention flow, residual stream geometry, and head-level attribution analysis.

A 2-D map of GPT-2's token embeddings you can poke at. Pick two axes, project the vocabulary, and see where analogies, cyclic features, and hubness show up (and where the 2-D view is lying to you). Companion to a writeup on which "cyclic" concepts actually form circles.

Visualizing Modern LLM Mechanics, Loss Landscapes & HPC Topologies

Code for 'Exploring the Impact of a Transformer's Latent Space Geometry on Downstream Task Performance' (arXiv:2406.12159)

Personal learning notebook on latent space engineering, representation geometry, and exploratory research notes.

Reproducibility package for "Context Is King: How In-Context Specification Shapes the Geometry of Concepts" — code, cached data, and interactive 3D explorers.

An empirical and geometric analysis of Neural Collapse under different optimizers on CIFAR-10

Com la tokenització fractura la morfologia catalana i si una segmentació conscient dels morfemes recupera la geometria. Provat en 3 llengües indoeuropees (català, castellà, anglès): el català es fragmenta ~1,7× més que l'anglès; forçar el tall morfèmic recupera la composicionalitat (robust a portadora i replicat en castellà).

Code for 'Reliable Measures of Spread in High Dimensional Latent Spaces' (ICML 2023)