Lockstep

Detects coordinated inauthentic behaviour on social media — groups of accounts acting together to manufacture the appearance of organic consensus — and shows the evidence behind every account it flags.

Evaluated on ten real state-linked influence operations from the Twitter/X Information Operations archive: 25,684,794 posts, 10,844 attributed accounts and 208,877 organic controls.

The detector beats chance on all ten operations. On six of them it is beaten by a predictor that ranks accounts on post count alone. That result is first because it is the number that decides whether anything else here means much.

---

Quickstart

Requires Python 3.10 or newer.

git clone https://github.com/YOUR-USERNAME/lockstep
cd lockstep
pip install -e ".[dev]"
make demo

make demo runs the full pipeline on a real 18 MB corpus committed to this repository — two attributed operations, no download needed. About seven seconds.

Armenia
  94,604 posts   578 accounts   31 attributed   base rate 0.054
  159 edges survived thresholding   9 clusters

  detector     AP 0.511  ██████████████··············    9.5x base rate
  post count   AP 0.215  ██████······················    4.0x base rate
  -> detector wins by 0.296 AP

Ghana_Nigeria
  52,918 posts   405 accounts   60 attributed   base rate 0.148
  19 edges survived thresholding   3 clusters

  detector     AP 0.308  █████████···················    2.1x base rate
  post count   AP 0.750  █████████████████████·······    5.1x base rate
  -> POST COUNT WINS by 0.443 AP

Two operations that disagree, on purpose. Showing only Armenia would misrepresent the result.

---

How it works

Six behavioural signals, fused into one weighted graph, thresholded against a permutation null model, then clustered.

1. Signals. Each answers one question about a pair of accounts and produces a sparse account-similarity matrix.

Signal	Question
coshare	Do they retweet the same posts?
hashtag	Do they use the same uncommon hashtags?
url	Do they share the same links?
text_reuse	Do they post near-identical text?
temporal	Do they post in the same time windows?
rapid_share	Do they react to the same content within seconds of each other?

Five reduce to the same underlying question — which accounts share artefacts that are rare? — and share one TF-IDF engine. Traits held by almost everyone carry no information and are filtered out before the similarity step, which is also what keeps it tractable. rapid_share works on reaction latency instead.

2. Fusion and thresholding. Raw cosine scales are not comparable across signals, so each layer is rank-normalised before being combined by weight. The fused graph is then cut twice: once at a percentile of its own edge-weight distribution, and once against an absolute floor from a degree-preserving permutation null model. The higher wins.

The null model matters. A percentile threshold is relative — it retains the top 2% of edges whether or not any coordination exists, so it will invent clusters in a corpus containing none. The null floor is what makes "nothing here" a reachable answer. See docs/DESIGN-NOTES.md.

3. Clustering. Louvain community detection over the surviving graph, with communities below min_cluster_size discarded.

4. Evidence. Every flagged account carries the artefacts that implicated it — which URLs it co-shared, with whom, how many minutes apart. The output is built for human review, not enforcement.

---

Results

Full detail in docs/RESULTS.md.

Each release is scored twice: by the detector, and by a predictor using post count alone. Lift over base rate is the comparable column, since base rates span 0.002 to 0.393 and some large releases are sampled to fit in memory.

Release	Base rate	AP	Lift	Volume lift
Armenia	0.017	0.484	28.1×	9.6×
Cuba	0.016	0.415	25.6×	42.5×
Catalonia	0.028	0.251	8.9×	4.2×
Spain	0.114	0.660	5.8×	5.5×
Qatar	0.002	0.008	5.3×	272.7×
Ghana/Nigeria	0.049	0.225	4.6×	14.2×
Venezuela	0.055	0.079	1.5×	14.4×
Ecuador	0.197	0.281	1.4×	2.3×
Egypt/UAE	0.393	0.548	1.4×	2.2×
Thailand	0.151	0.211	1.4×	1.3×

The detector loses wherever attributed accounts are unusually prolific, which in this archive is most of the time. Precision at the top of the ranking is much better than average precision suggests — Spain 1.000, Ecuador 0.830, Egypt/UAE 0.750 at P@100 — and an analyst with fixed review capacity only ever sees the top.

Clustering

Pooling nine operations into one corpus of 985,601 posts:

No cluster contains accounts from two different operations. Purity over operation identity is exactly 1.000, and each operation lands in between one and nine clusters. Post count cannot produce this at all — a ranking is not a partition.

But of 336 clusters, 314 contain no attributed account. Cluster output hands an analyst 94% false positives, and the clusters that do contain an operation still miss most of it: 78% of Spain's accounts recovered, 1% of Thailand's.

Signals do not transfer between operations

Leave-one-signal-out across all ten operations. Only coshare has a consistent sign — removing it hurts every one. Every other signal helps some campaigns and hurts others, and rapid_share splits five to five: removing it costs Catalonia 0.194 average precision and gains Egypt/UAE 0.240.

An argument for keeping six signals, since no single one would have caught all ten. Also an argument against the fixed global weights in config.py, which are tuned on synthetic data. Per-corpus calibration is not implemented.

---

Using it on your own data

Command reference

make demo                       # bundled corpus, ~7 seconds
make test                       # test suite
make lint                       # static checks
make check                      # verify docs match the repository
make smoke                      # verify the API and dashboard
make serve                      # API + dashboard on :8000
make eval                       # synthetic evaluation suite
make bench                      # scaling benchmark

make real-eval ROOT=data/raw    # screen and evaluate every release
make figures   ROOT=data/raw    # regenerate result figures
make sample    ROOT=data/raw    # rebuild the bundled demo corpus

As a library

from lockstep.detector import CoordinationDetector
from lockstep.models import Post

posts = [
    Post(post_id="1", account_id="a", created_at="2024-01-01T00:00:00+00:00",
         text="...", hashtags=["vote"], urls=["https://example.com/x"]),
    # ...
]

result = CoordinationDetector().detect(posts)

for cluster in result.clusters:
    print(cluster.cluster_id, cluster.size, cluster.dominant_signals)
    print(cluster.top_shared_traits)

for account in result.accounts[:10]:
    print(account.account_id, account.score, account.contributions)

Configuration lives in src/lockstep/config.py and is overridable by environment variables prefixed LOCKSTEP_:

LOCKSTEP_THRESHOLD_PERCENTILE=99 LOCKSTEP_MIN_CLUSTER_SIZE=5 make demo

Working with the IO archive

The ground truth is the Twitter/X Information Operations archive. The labelled Parquet form used here — attributed accounts packaged alongside organic controls — is the release described in Seckin et al., Labeled Datasets for Research on Information Operations (ICWSM 2025). It is roughly 3.6 GB and is not redistributed here.

Arrange one directory per release:

data/raw/
  Armenia/Armenia_part_1.gzip.parquet
  Catalonia/Catalonia_part_1.gzip.parquet
  ...

Nesting is inconsistent between releases and several ship a duplicate copy of every shard inside an inner zip. discover_datasets handles both, deduplicating by (filename, size) so a repeated copy cannot silently double a campaign.

python scripts/screen_releases.py data/raw reports/screening.json   # always first
python scripts/run_real_eval.py       --root data/raw
python scripts/run_pooled_analysis.py --root data/raw

Screen before evaluating. screen_releases.py reads only account, label and timestamp, so it sweeps 25M posts quickly, and it rejects releases whose two classes cannot support a comparison. One release I worked with had its control accounts sampled almost entirely after the operation was suspended — the classes barely overlapped in time, and a naive run would have separated them beautifully while measuring nothing but when each account was observed. No metric in the harness catches that.

Very large releases are sampled by account, never by post — thinning posts would strip each account's traits and destroy the co-occurrences being measured. --sample-controls keeps every attributed account and thins only controls, which raises the base rate. --sample-accounts thins both and preserves it.

---

Project structure

src/lockstep/
  detector.py            Orchestration: signals -> fusion -> clusters -> evidence
  fusion.py              Rank normalisation, weighted combination, null-model floor
  config.py              Settings, overridable with LOCKSTEP_ env vars
  models.py              Post, Cluster, AccountScore, DetectionResult
  signals/               The six behavioural layers and their shared TF-IDF engine
  data/
    synthetic.py         Labelled corpus generator with tunable stealth
    loaders.py           CSV and JSONL ingestion
    io_archive.py        Parquet IO releases, labels, diagnostics, volume baseline
    _parquet/            Vendored pure-Python Parquet reader (see pqlite)
  evaluation/            Metrics and the evaluation harness
  api/main.py            FastAPI service

scripts/
  demo.py                    Full pipeline on the bundled corpus
  screen_releases.py         Reject releases that cannot support evaluation
  run_real_eval.py           Per-release: detector vs post-count baseline
  run_pooled_analysis.py     Pool operations, recover the partition, cluster anatomy
  run_full_ablation.py       Leave-one-signal-out across every release
  check_null_model_real.py   Which threshold term binds; organic-only controls
  diagnose_volume_bias.py    Is the score a volume proxy?
  sweep_sparsity_filter.py   What min_account_posts costs
  make_sample_corpus.py      Rebuild the bundled demo corpus
  make_real_figures.py       Result figures
  make_figures.py            Synthetic figures
  run_eval.py                Synthetic evaluation suite
  benchmark.py               Scaling benchmark
  smoke_test_api.py          Every API endpoint and the fields the dashboard reads
  check_consistency.py       Assertions tying the docs to the repository
  lint.py                    Static checks

data/sample/           18 MB demo corpus (Armenia + Ghana/Nigeria)
reports/               Every result quoted in the documentation
docs/                  Methodology, results, limitations, figures
dashboard/index.html   Analyst review console
tests/                 Test suite

---

API and dashboard

make smoke     # verify every endpoint and the fields the dashboard reads
make serve     # http://localhost:8000

Endpoint	Purpose
GET /health	Liveness and configured signals
POST /detect	Run detection over a batch of posts
POST /demo	Generate a synthetic corpus and detect on it
GET /runs/{id}	Retrieve a completed run
GET /runs/{id}/clusters/{cid}	One cluster and its accounts
GET /runs/{id}/accounts/{aid}	Why an account was flagged
GET /runs/{id}/graph	Cytoscape-shaped graph for the dashboard

Runs are held in memory; there is no database.

---

Development

pip install -e ".[dev]"
make test
make lint
make check

make check runs 80 assertions tying the documentation to the repository: every figure, report, script and doc link referenced must exist, every figure on disk must be shown somewhere, every report must be cited and non-empty, and the corpus totals in the text must match reports/screening.json. Documentation drifts from code silently and nothing else in the toolchain notices.

make test and make lint prefer pytest and ruff and fall back to bundled runners where those are not installed.

CI runs the tests, the consistency check, the API smoke test, the demo, and the synthetic evaluation suite behind a quality floor.

---

Performance

Single core, all six signal layers, two null-model permutation rounds.

Corpus	Posts	Accounts	Seconds
Ghana/Nigeria	83,409	1,226	6.0
Armenia	142,078	1,798	10.0
Egypt/UAE	229,610	610	18.1
Pooled, nine operations	985,601	12,309	88

Roughly 10,000 posts per second. Memory binds before time does: releases above about a million posts need account sampling on a 4 GB machine.

---

Limitations

Detailed in docs/LIMITATIONS.md. The ones that matter most:

• Coordination is not inauthenticity. Newsrooms, fan communities and activist networks coordinate legitimately. On Thailand this detector ranks organic control accounts above attributed ones — those controls are dense communities of heavy retweeters, and they are genuinely coordinated. This ranks accounts for human review; it is not an enforcement mechanism.
• A trivial baseline beats it on six of ten operations. Post count alone.
• The false-positive floor is high. 94% of clusters contain no attributed account, and organic-only input never returns zero clusters.
• Signal weights do not transfer between operations, and per-corpus calibration is not implemented.
• Two evaluation metrics need care. mean_purity treats all organic accounts as one campaign, so an all-organic cluster scores 1.0. best_f1_threshold fits its threshold on the evaluation labels, making the derived precision, recall and F1 upper bounds rather than field estimates.
• Attribution is the platform's judgement, and the control accounts are not part of the public archive — how they were sampled is unverified.

---

Licence

MIT.