JOTE Parsers

Monorepo for a suite of parsers used in the Journal of Trial and Error.

The goal is to automate the process of converting a manuscript from a word processor to a JATS XML file, which can then be used to generate a PDF and HTML version of the manuscript. Ideally this would allow authors to work in Word/Google Docs but still have the benefits of a modern publishing workflow.

The only current implementation is found at convert.centeroftrialanderror.com (very shodyy!).

Contains 2 suites of parsers:

• ooxast/reoff: Tools to parse, convert from, and create OOXML (.docx) XML. Contains a parser and converters to JATS (jast), Markdown (mdast) and LaTeX (unified-latex), plus some tools.
• jats/rejour: Tools to parse, convert from, and create JATS XML. Contains a parser and stringifier, plus some tools.

Additionally, there are a few other tools:

• citations: Tools to parse and convert citations, and types for the OJS API.
• xast: Utilities for working with xast trees.
• unified-latex: Utilities for working with unified-latex trees.
• book-converter: a CLI to turn a .docx book into LaTeX.

apps/converter is the web converter (Next.js) that uses the parsers to turn .docx manuscripts into LaTeX.

See below for more info.

Development

A pnpm workspace run with Turborepo. Needs Node 24+ and pnpm 11.

pnpm install
pnpm build          # turbo run build: every lib to <lib>/dist with tsc (TypeScript 7)
pnpm typecheck      # turbo run typecheck, against workspace sources (no build needed)
pnpm test           # turbo run test (vitest)
pnpm lint           # oxlint
pnpm format         # oxfmt
pnpm dev            # turbo run dev (the converter on http://localhost:3100)
pnpm --filter <package> <script>   # one package, e.g. pnpm --filter reoff-parse test
pnpm gen:lib        # scaffold a new lib (turbo gen)
pnpm readme <dir>   # regenerate a lib README (or --all)

How packages see each other

Each lib's package.json declares its own dependencies; workspace deps are workspace:^. Its exports carry a custom @jote/source condition pointing at src/index.ts:

".": { "@jote/source": "./src/index.ts", "types": "./dist/index.d.ts", "default": "./dist/index.js" }

tsconfig.base.json sets customConditions: ["@jote/source"], so the editor, typecheck, vitest and the converter (webpack conditionNames) all resolve workspace packages straight to source: go-to-definition lands in src, and nothing needs building first. Each lib's tsconfig.lib.json turns the condition off, so builds compile against dependencies' dist types (turbo builds dependencies first). Published packages only ship dist.

Releasing

Through GitHub Actions with Changesets and npm trusted publishing (OIDC). Nothing is published from a local machine. See docs/releasing.md.

Packages

Packages marked private are not published.

other

Package	Description
@trialanderror/converter-cli	convert books
unified-ast-stringify

citations

Package	Description
crossref-json	Defined in: [index.ts:38](https://github.com/TrialAndErrorOrg/parsers/blob/main/libs/citations/crossref-json/src/index.ts
crossref-to-csl	crossrefToCsl(item: CrossrefJSON): CSL;
csl-consolidate	Try to resolve a list of CSL data with crossref metadata
csl-to-biblatex	Somewhat jank CSL-JSON to biblatex converter
ojs-api-types	Some typescript types for OJS api responses
parse-text-cite	Small tool that parses a string of text containing APA style in text citations, e.g.

jast

Package	Description
jast-types	jast (journal article/abstract syntax tree) is a syntax for abstract syntax trees representing JATS XML, specifically the "Green" publishing tag set.
jast-util-from-csl	cslToFront(data: Data): void;
jast-util-to-csl	Convert JATS XML bibliography to CSL JSON

ooxast

Package	Description
ooxast	Type definitions for ooxast (Open Office XML abstract syntax tree), a syntax for abstract syntax trees representing Open Office XML documents in the unist format.
ooxast-util-citation-plugin	Small ooxast utility which scans the text to identify the citation plugin used, either Mendely, Zotero, EndNote, Citavi, native word citations or none at all.
ooxast-util-citations	This package is ESM only.
ooxast-util-get-style	Get style from a w:p or w:r element.
ooxast-util-markup-to-style	Find certain markup in an ooxast tree and turn it into styles.
ooxast-util-parse-bib	Find and convert raw references to CSL-JSON using anystyle.
ooxast-util-properties	Return the properties of an ooxast node as a JSON object
ooxast-util-remove-rsid	Cleans all the rsid tags from an ooxast tree, and merges w:r elements if they only differ by rsid values.
ooxast-util-to-hast private	Convert docx to html (Not working)
ooxast-util-to-jast	Util to convert ooxast syntax tree to jast syntax tree, allowing for .docx to JATS XML conversion.
ooxast-util-to-mdast	Convert ooxast syntax tree to mdast syntax tree.
ooxast-util-to-unified-latex	Convert ooxast syntax tree to unified-latex syntax tree.

rejour

Package	Description
rejour-frontmatter private	rejourFrontmatter(): Function;
rejour-meta private	Doesn't do anything atm
rejour-move-abstract private	Really simple plugin for rejour that moves the abstract from the body to the front of a JATS document.
rejour-parse	Parser for rejour that parses the JATS document to a jast tree.
rejour-stringify	Plugin for rejour that stringifies a jast syntax tree to a JATS XML document.

reoff

Package	Description
docx-to-vfile	Reads a .docx file and stores its components in vfile format to be processed by other tools, like reoff-parse.
reoff-cite	default(options: Options = ...): Function;
reoff-clean	Plugin for [reoff][reoff] to clean the ooxast tree.
reoff-compile private	Compile a reoff-compatible VFile or bare ooxast syntax tree to a .docx document
reoff-infer-headings	Plugin for reoff that turns a single bolded or emphasized line into a heading
reoff-markup-to-style	Plugin for reoff that is able to change the styles of paragraphs based on the markup of the underlying text
reoff-parse	Plugin for [reoff][reoff] to parse a .docx XML file into an ooxast AST.
reoff-parse-references	default(options: Options = {}): Function;
reoff-rejour	Plugin for reoff that transforms an ooxast syntax tree into a jats syntax tree, i.e.
reoff-remark	Plugin for reoff that takes an ooxast tree and turns it into a remark tree, allowing for .docx to .tex conversion
reoff-unified-latex	Plugin for reoff that takes an ooxast tree and turns it into a unified-latex tree, allowing for .docx to .tex conversion

unified-latex

Package	Description
unified-latex-stringify	Plugin for unified-latex that takes an unified-latex tree and turns it into LaTeX

xast

Package	Description
xast-util-has-attribute	Port of hast-util-has-property for xast
xast-util-is-element	Port of hast-util-is-element for xast
xast-util-minify-whitespace	Minify whitespace between xast elements: a port of rehype-minify-whitespace 5 for xast
xast-util-select	Port of (hast-util-select)[https://github.com/syntax-tree/hast-util-select] for use with xast nodes.

Deprecated packages that are still on npm live in deprecated/.

Typst

Typst output for JOTE articles, next to the LaTeX route: Word or LaTeX in, a main.typ for the jote-article Typst template out. Plus retypst, a unified ecosystem for Typst built on Typst's own parser.

typast and retypst

typast

A unist syntax tree for Typst that mirrors Typst's own (typst_syntax::ast): its node names, parbreaks instead of paragraph nodes, code as expressions.

typast-util-from-string

Parse Typst into typast with Typst's parser (typst-syntax 0.15.1): a native addon in Node.js, WebAssembly in the browser.

typast-syntax-node / typast-syntax-wasm

The parser bindings (napi-rs and wasm-bindgen) around the Rust crate crates/typast-syntax.

typast-util-to-typst

Serialize a typast tree to Typst source, escaping text so it renders literally.

typast-util-visit

Walk every node of a typast tree, code (call arguments, operands) included.

typast-util-to-string

The text content of a typast tree, including footnotes, captions and other content in calls.

typast-builder

Node constructors, and the document model (paragraphs, lists, calls by name) the converters build and lower() turns into typast.

retypst, retypst-parse, retypst-stringify

unified processor and plugins for Typst, like rehype for HTML.

Converters

ooxast-util-to-typast

Convert an ooxast (Word document) tree to typast for the jote-article template, including the title page, figures, tables, boxes, footnotes and citation fields.

reoff-typast

reoff plugin to go from Word (ooxast) to Typst (typast).

unified-latex-util-to-typast

Convert a unified-latex tree (a main.tex written against jote-new-article.cls) to typast.

unified-latex-typast

unified plugin to go from unified-latex (LaTeX) to typast (Typst).

docx-to-typst

Processor and CLI: Word manuscript (.docx) → main.typ + media/ + bibliography.bib.

tex-to-typst

Processor and CLI: JOTE LaTeX article (main.tex) → main.typ, with its images and .bib files.

templates/typst/jote-article

The Typst port of jote-new-article.cls. Converters emit #import "jote-article/lib.typ": * and #show: jote-article.with(…).

Rust

crates/typast-syntax (with the binding crates in libs/typast/typast-syntax-{node,wasm}) is a Cargo workspace (Cargo.toml, toolchain pinned in rust-toolchain.toml). Nothing else needs Rust: the WebAssembly build is checked in, and without a native binary the Node.js package falls back to it.

cargo test                                       # the crate's tests
pnpm --filter typast-syntax-node build:native    # the Node.js addon for this machine
pnpm --filter typast-syntax-wasm build:wasm      # rebuild the checked-in WebAssembly (needs wasm-bindgen-cli 0.2.129)

To follow a new Typst version, bump typst-syntax in Cargo.toml (and TYPST_VERSION in the crate and typast-syntax-wasm's api.ts), rebuild both bindings and run the tests.

How to produce a Typst article

From a Word manuscript (--meta takes template arguments or a converter metadata form as JSON):

npx tsx --conditions=@jote/source libs/processors/docx-to-typst/src/cli.ts \
  Manuscript.docx out/article [--meta meta.json]

From a LaTeX article:

npx tsx --conditions=@jote/source libs/processors/tex-to-typst/src/cli.ts \
  "path/to/article/main.tex" out/article

Both write out/article/main.typ and media/, and link the template as out/article/jote-article (--copy-template copies it instead). Then compile with the JOTE fonts:

typst compile \
  --font-path ~/JOTE/archive/fonts \
  --font-path ~/JOTE/archive/OverpassFonts \
  --font-path ~/JOTE/archive \
  out/article/main.typ

Or pass --compile --font-path … to either CLI to do it in one go. The converter app (apps/converter) has a Typst tab that downloads the same output, template included, as a zip.

License

GPL-3.0+ © Thomas F. K. Jorna