Skip to content

Add HandPenned detector submission - #201

Closed
russellsailors-hub wants to merge 2 commits into
liamdugan:mainfrom
russellsailors-hub:handpenned-submission
Closed

russellsailors-hub wants to merge 2 commits into
liamdugan:mainfrom
russellsailors-hub:handpenned-submission

Conversation

@russellsailors-hub

Copy link
Copy Markdown

Adds leaderboard/submissions/HandPenned/ with predictions.json (all 672,000 test ids, score in [0,1], higher = more likely machine) and metadata.json.

Detector: HandPenned (https://handpenned.com/), two fine-tuned DeBERTa-v3-large classifiers voting; trained on ~29k human and ~29k machine paragraphs across 23 genres, including a stratified slice of the RAID train split (disjoint from any evaluation set we use). Contact: support@handpenned.com.

🤖 Generated with Claude Code

Two DeBERTa-v3-large judges trained on 29k human / 29k machine paragraphs across 23 kinds of writing plus a slice of RAID train; score is the mean of the two. Predictions for all 672,000 test rows.
@github-actions

github-actions Bot commented Sep 6, 2026

Copy link
Copy Markdown

Eval run succeeded! Link to run: link

Here are the results of the submission(s):

HandPenned

Release date: 2026-09-05

I've committed detailed results of this detector's performance on the test set to this PR.

On the RAID dataset as a whole (aggregated across all generation models, domains, decoding strategies, repetition penalties, and adversarial attacks), it achieved an AUROC of 84.18 and a TPR of 91.33% at FPR=5% and 73.49% at FPR=1%.
Without adversarial attacks, it achieved AUROC of 83.13 and a TPR of 89.56% at FPR=5% and 72.80% at FPR=1%.

If all looks well, a maintainer will come by soon to merge this PR and your entry/entries will appear on the leaderboard. If you need to make any changes, feel free to push new commits to this PR. Thanks for submitting to RAID!

@russellsailors-hub

Copy link
Copy Markdown
Author

Closing this — HandPenned has been discontinued and the project is no longer maintained, so the submission's links are no longer live. Apologies for the noise, and thanks for the work on RAID.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant