Skip to content

Weave highlighted, language-aware documentation by default - #74

Open
dbosk wants to merge 21 commits into
masterfrom
tominted-default-weave
Open

dbosk wants to merge 21 commits into
masterfrom
tominted-default-weave

Conversation

@dbosk

@dbosk dbosk commented Jun 14, 2026

Copy link
Copy Markdown
Owner

Makes noweb.mk's default weave produce syntax-highlighted documentation with a language-aware identifier index, using the dbosk noweb fork's autolang/tominted/autodefs filters.

What's here

  • NOWEAVE.tex default now weaves -autolang -autodefs python3 -autodefs sh -autodefs make -index -filter 'tominted -lexer noweb_lexer.py': each chunk syntax-highlighted with minted, the index gated by language so foreign chunks don't pollute it. noweb.mk provides a rule copying noweb_lexer.py from noweb's lib dir; documents declare a dependency on it (as makefiles.pdf now does). NOWEAVE.pdf stays classic (its generated preamble can't load minted).
  • Build fixes the new defaults exposed: corrected noweb quoting of bracket-notation examples (the doc was unbuildable at HEAD), and kept indexed-identifier links out of section headings (hyperref dies on links in moving arguments).
  • Accurate filter-slot budget note (-index consumes two slots; the default fills all seven).

Requires

The dbosk noweb fork (autolang/tominted + autodefs filters), minted, and a one-time latexminted whitelist for the custom lexer. See the corresponding noweb PRs (dbosk/noweb#3, #4).

🤖 Generated with Claude Code

dbosk and others added 21 commits June 12, 2026 09:04
The prose examples like [[<<module_name.py>>=]] made noweave parse the
inner <<...>> as a real chunk reference: the raw underscore in the
chunk name reached LaTeX unescaped (breaking compilation with "Missing
$ inserted") and the reference linked to a never-defined chunk.
Examples quoting the [[...]] notation itself, like [[[[...]]]],
truncated at the first ]] and left stray bracket text in the PDF.

Literal chunk syntax is now written with noweb's @<<...@>> escape
inside [[...]], and literal double brackets with \texttt and split
brackets, so the document compiles again and the examples render as
intended.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
NOWEAVE.tex now weaves through noweb's autolang and tominted filters
(dbosk noweb fork): each chunk is syntax-highlighted with minted using
a per-chunk lexer, and -autodefs python3 fills the identifier index
with -autolang gating the autodefs filters so chunks in other
languages stay out of the index.  The chunk-name-equals-filename
convention our tangling rules already enforce is what drives the
language inference.

The bundled Pygments lexer keeps chunk references hyperlinked inside
Python strings; noweb.mk provides a rule copying it from noweb's
library directory (read from the LIB= line of the noweave script),
and documents declare a dependency on it, as makefiles.pdf now does.
Documents must load minted and compile with -shell-escape, which our
own build already did.

NOWEAVE.pdf keeps the classic rendering: its preamble is generated by
noweave and cannot load minted.  Everything is set with ?=, so
projects without the fork override NOWEAVEFLAGS.tex.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The autodefs filters now index make targets such as upload and
autocommit, which turns every [[...]] quote of those names into a
hyperlinked identifier use --- including the ones in section
headings, where hyperref cannot survive a link inside the moving
argument (the PDF bookmark breaks with "Undefined control sequence
\hyper@@link").  Headings now write \texttt where body prose keeps
the quoted, linked form.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
noweb now ships autodefs filters for shell and make (and Haskell,
Rust and Java), gated by the same @language annotations -autolang
provides, so stacking them indexes every chunk by exactly the filter
that understands it.  The default stack --- Python, shell, make ---
covers what our projects mix into one document: program modules,
helper scripts and their build files; our own woven documentation
now indexes its make variables and targets.

The stack stays at three because noweave has seven filter slots and
-autolang, each -autodefs and tominted occupy one each; the default
leaves two slots free for project filters, and other languages
remain a one-line NOWEAVEFLAGS.tex override.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The caution claimed the default stack leaves two of noweave's seven
filter slots free.  It missed that -index inserts two filters
(finduses + noidx); with -autolang and tominted also taking one each,
the three-autodefs default fills all seven exactly.  Adding a fourth
-autodefs or a project -filter overflows unless something is dropped.

Prose-only change in a documentation chunk; the tangled noweb.mk is
unchanged.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- Replace the seven-filter-slot caution with the dbosk fork's unbounded
  pipeline; keep the old ceiling only as a note for pre-fork noweave.
- Note that the preamble loads minted through the noweb package
  (\usepackage[minted]{noweb}) rather than a hand-added \usepackage{minted}.
- Record that noweave -minted can load minted into the generated preamble,
  so the standalone .pdf weave stays classic by choice, not necessity.

Documentation only; the tangled noweb.mk is unchanged.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Replace the hand-added \usepackage{minted} (plus a separate
\usepackage{noweb}) with \usepackage[minted]{noweb}, so the minted
dependency rides the noweb package the way tominted weaves expect.
\setminted{autogobble} still applies, as minted is loaded (via
\AtEndOfPackage) before it runs.

Requires the noweb fork whose noweb.sty declares the minted option;
to pass minted options such as outputdir, load minted directly before
\usepackage{noweb} instead.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The dbosk fork's noweave, with -index, runs -autolang's inference as a
pre-pass to discover which languages a document contains and stacks the
matching autodefs filter for each. The default weave therefore no
longer lists -autodefs python3/sh/make: discovery covers them, and an
explicit -autodefs is still merged without duplication for chunks whose
language the name does not reveal.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Test chunks follow the convention <<test [[foo.py]]>>, labelling the
file rather than being it, and autolang deliberately does not infer a
language from such a labelled name. Left unclassified, a test chunk is
scanned by every autodefs filter and salts the index with
cross-language false matches. Add one -langrule per supported language
(python, shell, make) so each test chunk is classified, and hence
indexed, by the same filter as the module it exercises.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The cus_dep hardcoded its PythonTeX invocation, so a project that points
PythonTeX at a virtualenv interpreter -- the usual case when the document
executes the very package it documents -- was ignored whenever latexmk drove
the build, since latexmk reaches PythonTeX through the cus_dep rather than
through tex.mk's rule. 53e54b0 had aligned the cus_dep with tex.mk's default
invocation, but copying a default is not the same as sharing a configuration
point: the two paths still diverge the moment anyone overrides it.

The failure is silent and remote from its cause. LaTeX succeeds, latexmk
reports the cus_dep as run, and the only symptom is a Python traceback about a
missing module several build stages later.

So export PYTHONTEX and PYTHONTEXFLAGS from tex.mk -- the environment is the
only channel that reaches a latexmkrc, which Perl reads in a separate process
-- and have the cus_dep read them back, keeping the literal defaults as Perl
fallbacks so a hand-run latexmk with no make in the picture behaves as before.
Unset is distinguished from empty so a project can deliberately pass no flags.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QiBv9BVhRKCyYJJNDhiowW
…er itself

tominted now defaults to the lexer next to its own script and writes that
absolute path into the woven TeX, so the file no longer has to be copied
next to the LaTeX sources. Remove NOWEB_LEXER/NOWEB_LIB, the copy rule,
and the makefiles.pdf dependency, clean and .gitignore entries. Rewrite the
prose to explain the default, the latexminted hash whitelist and the
-nolexer escape hatch.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01JAtMqxVU6NPFdf3Xp9iU8S
Three related fixes so biblatex documents build cleanly and
non-interactively:

- Add -interaction=nonstopmode to LATEX/PDFLATEX so TeX completes the
  round (writing the .bcf that biber needs) instead of stalling for
  terminal input on an error.
- Add the TEX_BIB selector (default biber).  Both the %.bbl:%.aux
  (bibtex) and %.bbl:%.bcf (biber) rules matched the same target, and
  the bibtex one always wins, so make wrote a malformed .bbl for
  biblatex documents.  Set TEX_BIB=bibtex for classical bibtex.
- Make COMPILE.tex succeed based on the output PDF existing rather than
  on latexmk's exit status, so a recoverable warning (e.g. a lost
  margin float) no longer aborts make, while a genuine error that
  leaves no PDF still fails.  This matches the file's stated philosophy
  that LaTeX errors are caught by reading the log, not the exit code.

Also sync tex.mk.nw with the bounded rerun loop that had been applied
only to the generated tex.mk.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The PythonTeX interpreter fix (f8306fc) lived only on that branch and
was lost when this branch was checked out, regressing doc builds that
point PythonTeX at a virtualenv interpreter. The generated tex.mk
conflicted; resolved by retangling from the merged tex.mk.nw.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MeDZNHCkeqh7u8C6SuhMr9
Non-stop mode buries errors in the log while the build ploughs on; for
the same reason we already reject -halt-on-error, we prefer TeX to stop
at the error where we can see it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MeDZNHCkeqh7u8C6SuhMr9
TEX_PYTHONTEX was wired in with a recipe-less pattern rule,

    ${TEX_OUTDIR}/%.pdf: ${TEX_OUTDIR}/%.pytxmcr

which GNU make reads as cancelling a rule, not as adding a prerequisite
(manual, "Canceling Implicit Rules").  So the switch never did anything;
PythonTeX output only appeared when `latexmk -use-make` asked make for a
missing .pytxmcr on a clean build, and every later edit left it stale.

Collect the switches' prerequisites in TEX_DOC_PREREQS and put that
variable inside the %.pdf/%.dvi pattern rules themselves.  TEX_PYTHONTEX
now adds `latexmkrc`, so latexmk's custom dependency runs pythontex
whenever the .pytxcode changes.  TEX_BBL/TEX_IND have the same dead
construction; they are documented as such but left alone, since latexmk
runs bibtex/biber/makeindex itself and making them effective would pick
the bibtex .bbl rule also for biblatex documents.

Also drop the symlink "fudge" from the CTAN pythontex-latexmkrc example.
Recent latexmk (tested 4.87) finds the cus_dep in the output directory by
itself; with the fudge it registers a second, root-anchored rule that runs
pythontex in the project root, where \setpythontexworkingdir{..} points
outside the project, the run fails (taking the build with it) and leaves
.pytxmcr/.pytxpyg/pythontex_data.pkl in the root.

Fixes #92

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VBn5a9DLhspciVa4Rxecbz
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant