Repository navigation
Restructure into a multi-engine TPC-DS reference repository - #1
Merged
Merged
Conversation
Extend the Oracle-only scripts to cover Oracle, PostgreSQL, Vertica,
ClickHouse and StarRocks behind one common set of commands.
Layout
- engines/<name>/{ddl,load,queries,tuning}/ per engine
- Oracle assets moved with git mv, so their history is intact
- Oracle .ctl files renamed by table name and parameterised with
@DATA_DIR@ / @LOG_DIR@, since SQL*Loader does not expand env vars
- Queries normalised to query01..query99 with _1/_2 for the four
queries that have two formulations, giving 103 streams per engine
Provenance and licensing
- tools/sync-upstream.sh imports from pinned upstream commits and
writes a header on every file naming source, commit, licence and
the adaptation applied; re-running it reproduces all 412 files
- ClickHouse and StarRocks imported verbatim (both Apache-2.0)
- PostgreSQL and Vertica derived from the standard query text:
date_add(d, n) -> (d +/- n), and the ORDER BY alias in q36/q70/q86
expanded to its defining expression, which standard SQL requires
- The TPC-DS toolkit is TPC EULA material and is never vendored;
datagen/fetch-toolkit.sh fetches it into the git-ignored .toolkit/
- NOTICE.md records per-source licensing, the sources deliberately
rejected, and the non-comparability disclaimer
Tooling
- bin/{ddl,load,run}.sh with engine dispatch in bin/lib/common.sh
- Per-engine bulk loaders using each engine's native tool
- datagen/ scripts, docker/ profiles, config/*.env.example
- docs/ as a bilingual GitHub Pages site, published by a workflow
Verified against postgres:16: schema applies, 103/103 queries plan,
102/103 execute on an empty schema (q90 divides by a count that is
zero without data), and ddl/load/run work end to end.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
tools/verify.sh starts an engine from docker-compose, applies the schema,
loads a synthetic fixture, runs all 103 queries, and writes a report.
No TPC EULA acceptance and no local DB client needed: the engines and
their clients both run in containers and the data is our own.
postgres schema ok, 24/24 tables, 103/103 queries
clickhouse schema ok, 24/24 tables, 100/103 queries
starrocks schema ok, 24/24 tables, 103/103 queries
Oracle and Vertica stay manual: neither has an anonymously pullable
image. The vertica compose profile now reads VERTICA_IMAGE instead of
naming vertica/vertica-ce, which no longer exists on Docker Hub.
New tooling
- tools/make-fixture.py ~140k rows over 24 tables in dsdgen format.
Column lists come from the DDL, NOT NULL is
honoured, and the 6 fact tables' composite
primary keys are unique by construction.
- tools/compare-schemas.py reports where the 5 schemas disagree.
- .github/workflows/verify.yml runs all of it per push, plus shellcheck,
a generator-idempotency check and a TPC-EULA-material guard.
Bugs the harness found, all fixed
- The "<table>_*.dat" chunk glob also matched other tables: store_sales
loaded into store, customer_address into customer. Now matched
strictly as _<digits>_<digits> via data_files().
- StarRocks reorders columns in all 6 fact tables, so a positional load
wrote ss_ticket_number into ss_item_sk. The loader now sends an
explicit columns: header built from the canonical order.
- StarRocks Stream Load needs PUT; --data-binary sent POST and the FE
answered {"status":"FAILED","msg":"Not implemented"}.
- The StarRocks loader never stripped dsdgen's trailing "|", so every
row was rejected as "too many filtered rows".
- Upstream StarRocks call_center partitions cc_rec_start_date over
2023-06-01..2023-07-01, which no TPC-DS row can satisfy. Removed as a
documented adaptation.
- The StarRocks healthcheck only queried the FE and reported healthy
before any BE registered, so CREATE TABLE failed with "Current alive
backend is []". It now requires an alive backend.
- A ";" inside this repo's own bilingual SQL header made the mysql
client split there and send a comment-only fragment. ddl.sh now
strips comments, and the headers no longer contain one.
Also wires in the ClickHouse settings from upstream's settings.json,
which were imported as reference but never applied. Three prevent
outright failures (q14_1, q14_2, q49, q75) and three change results
silently; that took ClickHouse from 96/103 to 100/103.
Remaining ClickHouse failures are not defects here: q61 divides by a
count the small fixture leaves at 0, and q30/q81 hit a 25.11 limitation
resolving an outer CTE alias inside a subquery over the same CTE, which
upstream's known-issues file does not list.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Switch the site from just-the-docs to Material for MkDocs, because the
requested navigation shape is not something just-the-docs can express:
- navigation.tabs top-level sections as tabs in the header
- navigation.sections their children in the left sidebar
- toc.integrate per-page ToC folded into the left sidebar, so
there is no right-hand column at all
Verified in the built HTML: 5 header tabs, 0 md-sidebar--secondary,
md-nav--integrated present.
Structure, so the tabs have something to hold
- Getting started: quickstart, commands, data generation
- Engines: overview + the five engine guides
- Verification: how to verify, schema divergence (split out)
- Reference: TPC-DS schema (new), methodology, licensing (new)
The new schema page documents the 24 tables, the star layout as a mermaid
diagram, the composite fact keys, the load order and the per-engine type
mapping. The licensing page is a readable summary of NOTICE.md.
Presentation
- Custom stylesheet naming Korean faces. Every page here is bilingual, so
Korean is half the content, and Material's default stack names no
Korean face; word-break: keep-all is applied to prose but never to code.
- Status pills for the verification tables, cards on the landing page,
light/dark toggle, and an SVG favicon drawn as a star schema.
- Jekyll callouts converted to admonitions. Under just-the-docs the
"{: .warning }" markers were rendering as plain paragraphs anyway,
because the theme needs a callouts: block in config that was missing.
Pinned dependency versions, including Markdown itself
pymdown-extensions 10.14 against Markdown 3.10 makes superfences stop
processing fenced code blocks *silently*: every ``` block renders as
inline code, the build succeeds, and even --strict reports nothing. Found
it by grepping the built HTML for <pre> and finding zero. The working
combination is mkdocs-material 9.7.7 with pymdown-extensions 11.0.1, and
requirements-docs.txt records why the pins matter.
Also
- pages.yml builds with mkdocs --strict, so a broken internal link fails
CI rather than shipping; pull requests build but do not deploy.
- README links the published site and lists the pages per section.
- site/ is git-ignored.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Re-ran the engine verification and added a browser-based check for the documentation site. Everything runs in containers — MkDocs, nginx and headless Chromium — so nothing is installed on the host. tools/verify.sh --all engines tools/verify-docs.sh docs site, 20 checks + screenshots Engine verification, re-run and unchanged: postgres 24/24 tables, 103/103 queries clickhouse 24/24 tables, 100/103 queries starrocks 24/24 tables, 103/103 queries Why a browser was needed Grepping the built HTML proves the markup exists. It cannot prove that mermaid draws, that copy buttons appear, that the Korean font resolves, or that dark mode works — all of that is client-side. tools/verify-docs.js asserts against the live DOM and computed styles instead, and writes screenshots to results/docs-screenshots/. Two of my own earlier claims were wrong, and the browser showed it - I said copy buttons were "added client-side" after misreading a grep. They are not in the bundle at all under that name: Material renamed .md-clipboard to .md-code__button[data-md-type=copy] in 9.7. The check now accepts either so a theme upgrade cannot silently pass or fail. - The mermaid check looked for '.mermaid svg' and reported no diagram. Material renders into a CLOSED shadow root, which that selector can never see. It measures the host's laid-out size instead. Real defects the browser found - Landing-page cards were rendering as plain text. Markdown inside a <div markdown> wraps each link in a <p>, so '.card-grid > a' matched nothing. Retargeted, and a check now asserts the computed border. - The star-schema diagram rendered 938x79px: every dimension against every fact is an unreadable hairball. Replaced with one representative channel plus a compact channel overview, now 938x906 and 938x482. The check also fails on an extreme aspect ratio, which is what a squashed diagram looks like to an assertion. - Card borders used --md-default-fg-color--lightest, ~7% black, which reads as no border. Set per scheme instead. pages.yml gains a browser-check job that must pass before deploy, and uploads the screenshots as an artifact. The 404 from api.github.com/repos/.../releases/latest is Material's header badge on a repo with no releases: a console network message, not an error, not ours. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Extends the Oracle-only scripts to cover Oracle, PostgreSQL, Vertica, ClickHouse and StarRocks behind one common set of commands, with per-file provenance and a verification harness anyone can run.
Oracle 전용 스크립트를 Oracle, PostgreSQL, Vertica, ClickHouse, StarRocks 로 확장합니다. 공통 명령 세트, 파일별 출처 표기, 누구나 실행할 수 있는 검증 하네스를 포함합니다.
What changed / 변경 내용
engines/<name>/{ddl,load,queries,tuning}/per engine — 515 query files, 103 streams eachgit mv, so their history is intact;.ctlfiles renamed by table and parameterised with@DATA_DIR@/@LOG_DIR@bin/{ddl,load,run}.shwith engine dispatch inbin/lib/common.shdatagen/,docker/profiles,config/*.env.exampledocs/as a bilingual Material for MkDocs site (16 pages)Provenance and licensing / 출처 및 라이선스
tools/sync-upstream.shimports from pinned upstream commits and writes a header on every file naming source, commit, licence and the adaptation applied. Re-running it reproduces all 515 files, so provenance is checkable rather than claimed.date_add(d, n)→(d ± n), and theORDER BYalias in q36/q70/q86 expanded to its defining expressiondatagen/fetch-toolkit.shfetches it into the git-ignored.toolkit/NOTICE.mdrecords per-source licensing, the three sources deliberately rejected, and the non-comparability disclaimerVerification / 검증
Everything runs in containers — nothing installed on the host, no TPC EULA acceptance needed (the fixture is synthetic).
모든 것이 컨테이너에서 실행됩니다. 호스트 설치도, TPC EULA 동의도 필요 없습니다(픽스처가 합성 데이터).
Oracle and Vertica are unverified because neither has an anonymously pullable image — not because anything is known to be wrong. The three ClickHouse failures are not defects here: q61 divides by a
count(*)the small fixture leaves at 0, and q30/q81 hit a 25.11 limitation resolving an outer CTE alias inside a subquery over the same CTE.Docs site: 20/20 browser checks — header tabs, no right-hand column, mermaid draws, copy buttons, Korean font resolves, dark mode, every page 200.
Bugs the harness found / 하네스가 찾은 버그
Ten, all fixed. The ones worth knowing:
<table>_*.datchunk glob also matched other tables —store_salesloaded intostore,customer_addressintocustomerss_ticket_numberintoss_item_skPUT;--data-binarysendsPOSTand the FE answers{"status":"FAILED","msg":"Not implemented"}call_centerpartitionscc_rec_start_dateover 2023-06-01..2023-07-01, which no TPC-DS row can satisfypymdown-extensions10.14 againstMarkdown3.10 makessuperfencesstop processing fenced code blocks silently — every ``` block rendered as inline code and--strictreported nothingNotes for review / 리뷰 참고
s_state = 'SD'/SR_FEEvs'TN'/SR_RETURN_AMT), so its output is not row-for-row comparable even at the same scale factor. Documented, not changed.tools/compare-schemas.pyreports where the five schemas disagree — currentlycustomer.c_last_review_date, thestore.s_tax_precentagespelling, and the six reordered StarRocks fact tables.Merging enables the Pages deploy and the CI verification, neither of which can run from a non-default branch.
머지하면 Pages 배포와 CI 검증이 활성화됩니다. 둘 다 기본 브랜치가 아닌 곳에서는 실행될 수 없습니다.
🤖 Generated with Claude Code