Repository navigation
Closed
Labels
codingImplementation task (DARC: C)Implementation task (DARC: C)good first issueGood for newcomers; no prior SKaiNET codebase knowledge assumedGood for newcomers; no prior SKaiNET codebase knowledge assumedsize:sA few hoursA few hoursskill:kotlin-coreKotlin implementation in commonMainKotlin implementation in commonMainsub-issueSub-issue of a tracking issueSub-issue of a tracking issuetrainingTraining pipeline, optimizers, lossTraining pipeline, optimizers, loss
Description
Sub-issue of #1222 (Precision, Recall and F1Score metrics).
Lane: 2 · Kotlin core
Skill needed: Kotlin. Read Precision.kt / Recall.kt from #1227 / #1228 first — F1Score composes their formulas over one accumulator.
Size: s (~2–3 h)
Blocked by: #1227 and #1228 (or at least #1226 — F1 only needs the accumulator, but its tests cross-check against Precision/Recall).
What to do
- Add
public class F1Score(averaging: Averaging = Averaging.MACRO, dim: Int = -1, threshold: Float? = null) : Metric
inskainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/nn/metrics/F1Score.kt:override val name = "f1";- one private
ConfusionMatrixAccumulator— do not hold aPrecisionand a
Recalland callupdate()on both; that walks every batch twice; compute()=reduce(acc.counts(), averaging) { tp, fp, fn -> per-class F1 }where
per-class F1 =2·tp / (2·tp + fp + fn)(algebraically2PR/(P+R), but this form has
a single zero-division check:0.0when2·tp + fp + fn == 0).- Note that for
MACRO, F1 is the mean of per-class F1s — not the F1 of macro-P and
macro-R. Say so in the KDoc; it is the most common F1 implementation mistake.
- Factories:
public fun f1Score(averaging: Averaging = Averaging.MACRO, dim: Int = -1): Metric
andpublic fun binaryF1Score(threshold: Float = 0.5f): Metric. - Tests in
skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/metrics/F1ScoreTest.kt,
reference fixturepred [2,0,1,2],target [2,1,1,0]:- macro F1 → 0.4444 (per-class
[0, 0.667, 0.667]), and assert it is not
2·0.5·0.5/(0.5+0.5) = 0.5— the mean-of-F1s vs. F1-of-means trap; - micro F1 → 0.5, equal to micro precision, micro recall and accuracy;
- binary with threshold on 2-class logits, hand-computed;
- a class with TP+FP+FN = 0 contributes
0.0; fresh metriccompute()is0.0; - accumulation across two
update()calls equals one concatenated call; computeForBatch(fromMetric.kt) round-trips throughreset().
- macro F1 → 0.4444 (per-class
./gradlew :skainet-lang:skainet-lang-core:apiDumpand
./gradlew :skainet-backends:skainet-backend-cpu:jvmTest --tests "sk.ainet.exec.metrics.*".
Acceptance
-
F1Score+f1Score()+binaryF1Score()public and in the API dump - All test cases above pass on JVM
- Batches are walked once per
update() - PR body states "unit tests only; ground-truth wiring tracked in SKEEP-005 ([SKEEP-005]: Ground-truth validation for stateful metrics #1223)"
Notes
Reference: sklearn f1_score docs — https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html ;
torchmetrics F1Score — https://lightning.ai/docs/torchmetrics/stable/classification/f1_score.html .
Closing this issue is what unblocks the platform-parity lane (#1230–#1233) and review (#1235).
Metadata
Metadata
Assignees
Labels
codingImplementation task (DARC: C)Implementation task (DARC: C)good first issueGood for newcomers; no prior SKaiNET codebase knowledge assumedGood for newcomers; no prior SKaiNET codebase knowledge assumedsize:sA few hoursA few hoursskill:kotlin-coreKotlin implementation in commonMainKotlin implementation in commonMainsub-issueSub-issue of a tracking issueSub-issue of a tracking issuetrainingTraining pipeline, optimizers, lossTraining pipeline, optimizers, loss
Activity
EdinEd commentedon Oct 29, 2026
Hi @michalharakal , I would like to take on this issue.
michalharakal commentedon Oct 29, 2026
Cool thanks. 👍
feat(metrics): add F1Score metric and factory functions (#1229)
Merge pull request #1462 from EdinEd/feature/1229-f1-metric