Repository navigation
Conversation
Community reviewer @Ratgor flagged the HF artifact (Ornith-1.5-35B-A3B-INT4-ov discussion #1): the language model XML carries 1082 U8 tensors + zero_points and zero i4/u4 tensors; 31.9 GB bin for 35B params = 0.91 B/param. Same optimum-intel 2.2.0 + NNCF 3.4 data-aware fallback as the Mellum fix in labscommunity#119. quant field corrected; evidence note added to notes. Flags text kept under the 200-char limit.
|
@twilson63 is attempting to deploy a commit to the Community Labs Team on Vercel. A member of the Team first needs to authorize it. |
Result submission failed: results/twilson63/ornith-35b-b70-cascadia.json: Only new files can submit runs. Use a new filename; edit existing live results on the site. Fix the problem, then push an update or ask a maintainer to rerun this workflow. No partial batch is committed. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fix: Ornith 1.5-35B-A3B OV results — quant is INT8 asymmetric, not int4
Community reviewer @Ratgor flagged the HF artifact
(
twilson63/Ornith-1.5-35B-A3B-INT4-ov, discussion #1):"...the model file size 34 GB seems to be int8..".
Verified against the artifact immediately (
openvino_language_model.xml):element_type="u8"+ per-tensorzero_pointconstantsi4/u4tensors anywhere in the graphf16tensors (embeddings/scales/router-side unquantized halves).binmath: 35B params → 31.9 GB ≈ 0.91 bytes/paramRoot cause: same as the Mellum int4→int8-asym fallback fixed in #119. The export requested
int4-sym, group_size 128(optimum-intel 2.2.0 + NNCF 3.4), but the data-aware fallbackreassigned every weight tensor to U8 + zero-points on this custom multi-component export
(VLM + MTP sidecars). The bench numbers themselves are unaffected — the pipeline measured what
the artifact actually was; only the label was wrong.
Changes here:
quant: int4 → int8on both Ornith 35B OV rows (openvino-genai VLMPipelinenotes(keptruntimeFlagsunder the 200-charlimit). Related: HF model card + repo rename (
-INT4-ov→-INT8-ov) being handled on theHF side.
Also cross-referenced with the quant audit of all 7 twilson63 OV repos: four are true INT4
(granite-4.2-8b, LFM2.5-8B-A1B, Qwen3.5-9B, Moonlight-16B-A3B — verified hundreds of
i4/u4tensors each); three were int8-asym (Mellum — fixed in #119, Humanizer, Ornith — this PR).
cc @SergiioB — relates to the data-model honesty items in #130 (this is exactly the
"conditions/labels must match the artifact" class; happy to help audit the existing catalog
against shipped artifacts if useful).