Skip to content

[Lane 2 · kotlin-core] F1Score metric #1229

Closed

Description

@michalharakal

Sub-issue of #1222 (Precision, Recall and F1Score metrics).

Lane: 2 · Kotlin core
Skill needed: Kotlin. Read Precision.kt / Recall.kt from #1227 / #1228 first — F1Score composes their formulas over one accumulator.
Size: s (~2–3 h)
Blocked by: #1227 and #1228 (or at least #1226 — F1 only needs the accumulator, but its tests cross-check against Precision/Recall).

What to do

  1. Add public class F1Score(averaging: Averaging = Averaging.MACRO, dim: Int = -1, threshold: Float? = null) : Metric
    in skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/nn/metrics/F1Score.kt:
    • override val name = "f1";
    • one private ConfusionMatrixAccumulator — do not hold a Precision and a
      Recall and call update() on both; that walks every batch twice;
    • compute() = reduce(acc.counts(), averaging) { tp, fp, fn -> per-class F1 } where
      per-class F1 = 2·tp / (2·tp + fp + fn) (algebraically 2PR/(P+R), but this form has
      a single zero-division check: 0.0 when 2·tp + fp + fn == 0).
    • Note that for MACRO, F1 is the mean of per-class F1s — not the F1 of macro-P and
      macro-R. Say so in the KDoc; it is the most common F1 implementation mistake.
  2. Factories: public fun f1Score(averaging: Averaging = Averaging.MACRO, dim: Int = -1): Metric
    and public fun binaryF1Score(threshold: Float = 0.5f): Metric.
  3. Tests in skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/metrics/F1ScoreTest.kt,
    reference fixture pred [2,0,1,2], target [2,1,1,0]:
    • macro F1 → 0.4444 (per-class [0, 0.667, 0.667]), and assert it is not
      2·0.5·0.5/(0.5+0.5) = 0.5 — the mean-of-F1s vs. F1-of-means trap;
    • micro F1 → 0.5, equal to micro precision, micro recall and accuracy;
    • binary with threshold on 2-class logits, hand-computed;
    • a class with TP+FP+FN = 0 contributes 0.0; fresh metric compute() is 0.0;
    • accumulation across two update() calls equals one concatenated call;
    • computeForBatch (from Metric.kt) round-trips through reset().
  4. ./gradlew :skainet-lang:skainet-lang-core:apiDump and
    ./gradlew :skainet-backends:skainet-backend-cpu:jvmTest --tests "sk.ainet.exec.metrics.*".

Acceptance

Notes

Reference: sklearn f1_score docs — https://scikit-learn.org/stable/modules/generated/sklearn.metrics.f1_score.html ;
torchmetrics F1Score — https://lightning.ai/docs/torchmetrics/stable/classification/f1_score.html .
Closing this issue is what unblocks the platform-parity lane (#1230–#1233) and review (#1235).

Activity

  1. added
    good first issueGood for newcomers; no prior SKaiNET codebase knowledge assumed
    trainingTraining pipeline, optimizers, loss
    sub-issueSub-issue of a tracking issue
    codingImplementation task (DARC: C)
    skill:kotlin-coreKotlin implementation in commonMain
    on Aug 31, 2026
  2. EdinEd commented on Oct 29, 2026

    @EdinEd
    Contributor

    Hi @michalharakal , I would like to take on this issue.

  3. michalharakal commented on Oct 29, 2026

    @michalharakal
    ContributorAuthor

    Cool thanks. 👍

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    codingImplementation task (DARC: C)good first issueGood for newcomers; no prior SKaiNET codebase knowledge assumedsize:sA few hoursskill:kotlin-coreKotlin implementation in commonMainsub-issueSub-issue of a tracking issuetrainingTraining pipeline, optimizers, loss

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions