Skip to content

Fix: Ornith 35B A3B OV — quant is int8-asym (community-verified artifact audit) - #146

Open
twilson63 wants to merge 1 commit into
labscommunity:mainfrom
twilson63:results/ornith-35b-int8-label
Open

twilson63 wants to merge 1 commit into
labscommunity:mainfrom
twilson63:results/ornith-35b-int8-label

Conversation

@twilson63

Copy link
Copy Markdown
Contributor

Fix: Ornith 1.5-35B-A3B OV results — quant is INT8 asymmetric, not int4

Community reviewer @Ratgor flagged the HF artifact
(twilson63/Ornith-1.5-35B-A3B-INT4-ov, discussion #1):
"...the model file size 34 GB seems to be int8..".

Verified against the artifact immediately (openvino_language_model.xml):

  • 1,082 tensors with element_type="u8" + per-tensor zero_point constants
  • zero i4/u4 tensors anywhere in the graph
  • 541 f16 tensors (embeddings/scales/router-side unquantized halves)
  • .bin math: 35B params → 31.9 GB ≈ 0.91 bytes/param

Root cause: same as the Mellum int4→int8-asym fallback fixed in #119. The export requested
int4-sym, group_size 128 (optimum-intel 2.2.0 + NNCF 3.4), but the data-aware fallback
reassigned every weight tensor to U8 + zero-points on this custom multi-component export
(VLM + MTP sidecars). The bench numbers themselves are unaffected — the pipeline measured what
the artifact actually was; only the label was wrong.

Changes here: quant: int4 → int8 on both Ornith 35B OV rows (openvino-genai VLMPipeline

  • cascadia bundle), evidence note added to notes (kept runtimeFlags under the 200-char
    limit). Related: HF model card + repo rename (-INT4-ov → -INT8-ov) being handled on the
    HF side.

Also cross-referenced with the quant audit of all 7 twilson63 OV repos: four are true INT4
(granite-4.2-8b, LFM2.5-8B-A1B, Qwen3.5-9B, Moonlight-16B-A3B — verified hundreds of i4/u4
tensors each); three were int8-asym (Mellum — fixed in #119, Humanizer, Ornith — this PR).

cc @SergiioB — relates to the data-model honesty items in #130 (this is exactly the
"conditions/labels must match the artifact" class; happy to help audit the existing catalog
against shipped artifacts if useful).

Community reviewer @Ratgor flagged the HF artifact (Ornith-1.5-35B-A3B-INT4-ov discussion #1): the language model XML carries 1082 U8 tensors + zero_points and zero i4/u4 tensors; 31.9 GB bin for 35B params = 0.91 B/param. Same optimum-intel 2.2.0 + NNCF 3.4 data-aware fallback as the Mellum fix in labscommunity#119. quant field corrected; evidence note added to notes. Flags text kept under the 200-char limit.
@vercel

vercel Bot commented Nov 18, 2026

Copy link
Copy Markdown

@twilson63 is attempting to deploy a commit to the Community Labs Team on Vercel.

A member of the Team first needs to authorize it.

@github-actions

Copy link
Copy Markdown
Result submission failed: results/twilson63/ornith-35b-b70-cascadia.json: Only new files can submit runs. Use a new filename; edit existing live results on the site.
Fix the problem, then push an update or ask a maintainer to rerun this workflow. No partial batch is committed.

Site / sign up · Workflow details and retry

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant