Preprocess Jinja YAML before validation

[nonprofittechy]

Files whose first line is exactly # use jinja now render before the existing YAML validation pass instead of being skipped. Validation and fixing share the directive matcher (LF, CRLF, or EOF). The sandboxed renderer supports expressions, loops, macros, and local includes, including partial blocks.

Compilation and rendering run in an isolated worker with a 5-second wall timeout, 2-second CPU limit, and 4 MiB source/output limits. A 256 MiB address-space limit applies on Linux and other supported Unix platforms; macOS skips it because Darwin rejects a limit below the process's existing virtual address space, and the CPU, response-size, and parent wall-time bounds still apply there. A bounded worker response protects parent memory. Limit failures produce EG106. Unix resource-limit support is required for Jinja rendering; unsupported platforms report an error instead of rendering without limits. Ordinary YAML validation is unaffected.

Missing or unavailable package-qualified {% include %} files are skipped by default with warning WG106. A placeholder identifies affected YAML documents, which are blanked while the remaining documents are checked. The warning explains that findings are partial because missing includes may supply document boundaries or definitions. Warnings do not fail CI unless a warning limit is set. Include fallback lists try every candidate, and repeated execution of one include site is reported once.

Values the server supplies

This is an offline check, so jinja data, docassemble's context variables, and server configuration are unavailable. Rather than failing a file that is valid on a real server, unknown values read as empty throughout: in arithmetic and comparisons, through every built-in filter and test, and through the Python protocols (abs(), round(), range(), format(), tojson) and the implicit else of an inline if, which reach an undefined value without passing through Jinja's own operator table. A branch that tests such a value is taken as if it were empty, and only that branch is checked.

An expression that renders to one of these values is replaced with a distinct placeholder such as dayc_unknown_1 and reported as warning WG107 against the original template line. Rendering it to nothing instead would leave an empty YAML value, an invalid Python line, or a block with no id, which hid every real problem around it: across the checks that touch a substituted value, the placeholder removes five false positives (EG420 on a field label, EG122 in a code block, EG103/EG414 on a block id, EG132 on mandatory) and introduces none. Distinct placeholders keep two unknown block ids from looking like duplicates of each other. Checks that resolve a name against the rest of the interview treat a placeholder as unresolvable rather than reporting the substitution as an undefined variable. Nothing that depends on the real value is checked, so WG107 suppresses from the template source like WG106, with # no-dayc: WG107 or # no-dayc-block: WG107.

Sandbox violations are never turned into a placeholder, and remain errors.

Errors

Missing imports, parent templates, and rendering failures produce EG106. This code was EG105 in earlier revisions of this branch, which collided with the pre-existing test-module pre-load check; suppression matches on the code string alone, so # no-dayc-block: EG105 for a Jinja dependency also silenced that unrelated check. Anyone who adopted EG105 from an earlier revision of this branch needs to update those suppressions to EG106. Validation errors in the remaining YAML retain their normal severity. Jinja syntax and runtime diagnostics preserve original source locations, including nested local templates, across the worker boundary.

Findings from rendered output

Findings produced from rendered YAML name the original file and carry a rendered_jinja marker rather than a synthetic path (rendered Jinja) filename, which GitHub could not resolve — it dropped every inline annotation for these files. Text output is unchanged; --format github annotates the file as a whole and reports the generated line in the message, since generated line numbers are not lines of the source file. Automatic fixes skip templates. Server context, package-qualified file resolution, and template-aware formatting remain outside this offline implementation.

Also fixed

MakoTemplate raises rather than reporting when handed a non-string, so question: with no text ended a run with an unhandled exception instead of findings. This predates the branch, but rendering a server value to an empty one made it easy to reach.

Validation

499 tests and 56 subtests pass; mypy, Black, and git diff --check pass on src and tests. Regression coverage includes oversized output, large allocations, worker timeout/cleanup, exact directives, nested runtime locations and suppressions, CI exit behavior, partial blocks, and include fallback lists. Offline-value coverage is verified against the installed Jinja rather than by example: a guard test enumerates every attribute Jinja routes to an undefined error and fails if a future release adds one that is not handled, and sweeps exercise all 54 built-in filters, every built-in test, and 39 language constructs. Fixtures include an example adapted from docassemble's documented include pattern.

Known residual: x is in "a string" still fails, because CPython requires a str left operand for string containment.

Closes #61.

🤖 Generated with Claude Code

[Copilot]

Copilot review overview

🟡 Changes recommended

Rendering is not resource-bounded, and directive matching and nested source-location handling remain unresolved.

Get a fresh assessment by requesting another Copilot review.

Review effort: Lite
Findings: 1 · 3

Open (4)

• Unbounded template rendering can hang or exhaust memory · New
• Prefix check misidentifies YAML files as Jinja templates · New
• Incomplete Jinja directive match renders ordinary files · New
• Runtime errors lose nested template source locations · New

What changed in this PR

This pull request adds opt-in Jinja preprocessing before YAML validation, including local includes, partial validation, diagnostics, and fixer bypasses.

Changes:

• Added sandboxed Jinja rendering and missing-include handling.
• Integrated validation, suppressions, and fixer behavior.
• Added tests, fixtures, documentation, and dependency metadata.

File	Summary	Review notes
tests/​test_jinja.py	Jinja behavior coverage	No final comments.
tests/​fixtures/​jinja/​interview.yml	Jinja fixture	No final comments.
tests/​fixtures/​jinja/​fields.yml	Include fixture	No final comments.
src/​dayamlchecker/​yaml_structure.py	Rendering integration and diagnostics	Two moderate findings (2 votes each): match the complete directive line and preserve nested-template source locations for runtime errors.
src/​dayamlchecker/​messages.py	Jinja findings	No final comments.
src/​dayamlchecker/​fixer.py	Skips Jinja templates	Moderate finding (2 votes): match the complete directive line to avoid skipping ordinary YAML files.
src/​dayamlchecker/​_jinja.py	Jinja renderer and include handling	Critical finding (2 votes): bound rendering execution time and memory usage.
requirements.txt	Adds Jinja2 dependency	No final comments.
README.md	Documents preprocessing	No final comments.
pyproject.toml	Adds Jinja2 dependency	No final comments.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[nonprofittechy]

@jpagh if you have time, would love your eyes on this/run it against your interviews that use Jinja2. I don't know others who use this feature

[jpagh]

When that bug is fixed, this is the output:

ERROR [EG105] .../x_documents.yml:60
  Could not render Jinja YAML: 'jinja_config_automatedpleading_document_categories' is undefined

It doesn't seem to like that I refer to a config variable via jinja (https://docassemble.org/docs/config.html#jinja%20data).

If it's helpful, you/agent can look at my docassemble-yaml/lsp to see how I handle the jinja.

---

Jinja rendering broken on macOS: setrlimit(RLIMIT_AS, 256 MiB) fails with "current limit exceeds maximum limit"

Environment

• dayamlchecker 1.6.0 (main @ 1ebb3ee)
• CPython 3.14.6 (uv-managed), aarch64
• macOS 27.0 (26A428, Apple Silicon)
• Jinja2 3.1.6
• Invoked via uv run dayamlchecker ...

Summary

On macOS, any YAML file with the # use jinja directive produces:

ERROR [EG105] <file>
  Could not render Jinja YAML: current limit exceeds maximum limit

The error occurs before the template is compiled or rendered. It is caused by the resource-limit setup added in 1ebb3ee ("Bound Jinja rendering and preserve nested error locations"): the worker
tries to cap address space with resource.setrlimit(RLIMIT_AS, 256 MiB), which macOS rejects with EINVAL because Darwin does not allow an address-space limit below the process's current virtual
size. CPython 3.14 maps any EINVAL from setrlimit to ValueError("current limit exceeds maximum limit"), and the worker reports that as a Jinja render error.

Steps to reproduce

Minimal case — content is irrelevant:

printf '# use jinja\nfoo: "{{ 1 + 1 }}"\n' > /tmp/tiny_jinja.yml
dayamlchecker /tmp/tiny_jinja.yml

Real-world case:

dayamlchecker /path/to/docassemble/.../data/questions/x_documents.yml

Both produce the same EG105 finding.

Expected

The trivial template renders and reports no findings.

Root cause

1. render_yaml() spawns the isolated worker; _worker() calls _set_resource_limits() before reading or rendering the template (src/dayamlchecker/_jinja.py:190; definition at :168).
2. _set_resource_limits() sets RLIMIT_AS to 256 MiB (_jinja.py:178).
3. On Darwin, setrlimit(RLIMIT_AS, <below current VA>) returns EINVAL. Every Python process on macOS already reserves an enormous virtual address space (measured: ~466 GiB via ps -o vsz), so
   256 MiB is always below current usage.
4. CPython maps EINVAL from setrlimit to ValueError("current limit exceeds maximum limit") — see CPython 3.14 Modules/resource.c lines 287–290:
   https://github.com/python/cpython/blob/3.14/Modules/resource.c#L287-L290
5. _worker() catches Exception and returns str(exc) as the render error; messages.py:318 wraps it as Could not render Jinja YAML: {error}.

Evidence

Raw syscall via ctypes confirms errno 22 (EINVAL); the requested struct is unchanged:

rc: -1 errno: 22 Invalid argument
current AS limit: 9223372036854775807 9223372036854775807

Clean-subprocess threshold test on a fresh venv interpreter:

RLIMIT_AS 1 GiB … 64 GiB   -> ValueError: current limit exceeds maximum limit
RLIMIT_AS 1 TiB            -> OK  (above current ~466 GiB VA)
RLIMIT_CPU / RLIMIT_FSIZE  -> OK

Fresh process virtual size:

$ ps -o vsz= -p <pid>     # freshly launched `uv run python`
488807152                 # ~466 GiB

The repo's own test reproduces it on macOS:

$ uv run --with pytest python -m pytest \
    tests/test_jinja.py::test_large_allocation_is_confined_to_worker -q
...
E  AssertionError: assert 'memory' in 'Could not render Jinja YAML: current limit exceeds maximum limit'
FAILED tests/test_jinja.py::test_large_allocation_is_confined_to_worker

This is a regression from 1ebb3ee; before that commit _jinja.py had no setrlimit calls.

Impact

All Jinja rendering is broken on macOS: every # use jinja file emits a false-positive EG105 regardless of content. There is no CLI flag to disable Jinja rendering, so the only workarounds are
downgrading before 1ebb3ee, patching _set_resource_limits() locally, or suppressing EG105 (which also hides real Jinja errors).

Suggested fix

Options, in order of preference:

1. Platform-gate the address-space limit. Keep RLIMIT_CPU and RLIMIT_FSIZE on all platforms but skip RLIMIT_AS on Darwin. The worker remains bounded by CPU time, output size, and the
   parent's 5-second wall timeout. Memory would not be hard-capped on macOS; if that is unacceptable, see option 2.
2. Adaptive RLIMIT_AS on Darwin. Query the current virtual size and set the limit to current_va + 256 MiB. I verified this succeeds and that a subsequent 1 GiB allocation raises
   MemoryError, so it does bound new mappings. It requires reading VA via ps or a mach call, plus a retry if VA grows between measurement and setrlimit.
3. Separate sandbox-setup failures from render failures. Whatever policy is chosen, a _set_resource_limits() failure should not surface as "Could not render Jinja YAML"; naming the missing
   OS limit would have made this trivially diagnosable.

A macOS-aware test (skip or assert platform-specific behavior) would prevent regressions. Linux is unaffected: setrlimit(RLIMIT_AS, 256 MiB) succeeds there and only constrains future mappings,
so CI stays green.

[nonprofittechy]

I did look at your implementation but wanted to focus on preprocessing, so it's a pretty different approach. Do you have any sample Jinja2 YAMLs you can share for validation?

[jpagh]

x_documents_anonymized.yml
x_review_anonymized.yml

Here are two that show what I use jinja for. Document loads options from the server config, and Review uses jinja macros to make repeatable questions for generics/list items.

Sorry, I didn't look at your implementation before I said anything.

[nonprofittechy]

@jpagh when you have a chance, one more check? the latest version should be safe for the kind of file you described, and I added your examples as new test fixtures.

[jpagh]

Looks good to me. It worked on both of my Jinja files and reported accurate errors (missing id). Ran it over my whole project and it seemed to work properly as well.