feat: add wizard subcommand (unstructured input → markdown) - #27
Open
m-messer wants to merge 5 commits into
Open
feat: add wizard subcommand (unstructured input → markdown)#27m-messer wants to merge 5 commits into
m-messer wants to merge 5 commits into
Conversation
`in2lambda wizard INPUT -o draft.md` turns a PDF/docx/tex/md document into the #/## markdown the Markdown filter reads, for a human to review before `in2lambda convert draft.md Markdown`. - wizard/run.py: routes input through Mathpix (PDF) / pandoc (docx) / raw read, runs one LLM extraction pass, renders #/## markdown, echoes check_markdown() warnings, writes the file. - wizard/extract.py: pydantic WizardSet/WizardQuestion/WizardPart + extract_set() via the OpenAI structured-output parse helper against OpenRouter, with a system prompt and one few-shot example. to_markdown() renders the contract Step 3's filter consumes. - main.py: `wizard` command; run.py imported lazily so the rest of the CLI works without the llm extra. - Root conftest.py skips the pydantic-dependent wizard modules from --doctest-modules on a bare install (CI runs --all-extras). - docs/source/wizard.md + toctree, quickstart + README pointers. Slimmed from conversion2025/converter.py on Summer2025: the line-number extraction and the trim/dedupe/evaluate passes are left out of v1; prompt shape informed by wizard/to_question.py on wxyang_hackathon. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_017VXb8aZqgFBjoeuuddjW6r
- Extend `runner()` and `convert` CLI to accept an optional `--name` (`-n`) parameter for naming question sets. - Sanitize `set_name` into a filesystem-safe "slug" for output paths. - Update JSON generation to use the sanitized name in filenames (`set_<name>.json`) and directories. - Add tests for named sets and update documentation to reflect the new functionality.
- Add `_demangle` function to restore LaTeX control words whose backslashes were lost to JSON un-escaping. - Adjust `extract_set` to apply `_demangle` to question titles, text, solutions, and parts. - Add test to verify proper handling of mangled LaTeX commands and preservation of newlines.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
in2lambda wizard INPUT -o draft.mdturns a PDF/docx/tex/md document into the#/##markdown the Markdown filter reads, for a human to review beforein2lambda convert draft.md Markdown.Commits on this branch
wizard/run.pyroutes input through Mathpix (PDF) / pandoc (docx) / raw read, runs one LLM extraction pass, renders#/##markdown, echoescheck_markdown()warnings, writes the file.wizard/extract.py: pydanticWizardSet/WizardQuestion/WizardPart+extract_set()via the OpenAI structured-output parse against OpenRouter (system prompt + one few-shot).to_markdown()renders the filter's contract.run.pyimported lazily so the rest of the CLI works without thellmextra.docs/source/wizard.md+ toctree, quickstart + README pointers.--name/-nforconvert— optional set name; sanitised into a filesystem-safe slug forset_<slug>.json/<slug>//<slug>.zip, raw name kept in the JSONnamefield (what Lambda Feedback shows on import). Default unchanged (set).\text/\frac/\betawith a single backslash;\t/\f/\b/\rare valid JSON escapes so the backslash is swallowed and the field ends up with a bare control char glued to the command.extract_setnow re-escapes any TAB/CR/FF/BS immediately followed by a letter. Newlines left as-is..gitignore(docs/_bt/,/e2e/); CI test matrix reduced to Python 3.11 (.github/workflows/test.yml; the commit is mislabelled "Updated gitignore").Testing done
black --check .,isort,pydocstyle, fullpytest --cov— green (93 passed, 92%;wizard/extract.py100%).openai/gpt-4o-mini) for.tex,.mdand.docxinputs:wizard→ review →convert→ validset.zip. The\text→TAB corruption is fixed and verified on the real model output.Known gaps / notes
pyproject.tomlkeepspython = "^3.10"while CI now tests only 3.11..texruns) — hence the mandatorydraft.mdreview step.Stack (top): … ← mathpix ← wizard-command
Base:
mathpix— #26. Merges after the whole stack (#21–#26).🤖 Generated with Claude Code