Python: a spliced template's context reaches the file it lands in

[knutwannheden]

A Python template's context parsed the template and nothing else, so a recipe splicing a name the target file did not import emitted code that raises NameError when it runs. It parses, it prints, and it reads as correct in a diff: two recipes in a downstream package shipped that way, their docstrings showing an import the code never emitted and their tests asserting the broken output as expected. Both were found by reading.

A template's context is now also what it needs bound where it lands:

_arg = capture('arg')
_tmpl = template(f"subprocess.run({_arg}, shell=True)", context=["import subprocess"])

class Visitor(PythonVisitor[ExecutionContext]):
    def visit_method_invocation(self, method, p):
        match = _pat.match(method, self.cursor)
        return _tmpl.apply(self.cursor, visitor=self, values=match) if match else method

Applied to out = os.popen('ls'), what the file already binds decides what it gets:

the file binds	the splice reads	imports added
nothing	subprocess.run('ls', shell=True)	import subprocess
import subprocess as sp	sp.run('ls', shell=True)	none
from subprocess import run	subprocess.run('ls', shell=True)	import subprocess, a member import not binding the module
if TYPE_CHECKING: import subprocess	subprocess.run('ls', shell=True)	import subprocess, a conditional import binding nothing at runtime
import subprocess inside the enclosing function	subprocess.run('ls', shell=True)	none, the lazy import binds it where the splice lands
the name, to anything else, in a scope the splice sits in	—	refused, no import reaching a shadowed name

A context binding counts only where the template's own code reads it, judged per occurrence, so a name the template binds for itself is neither renamed nor imported, and context that exists only to type a capture stays out of the file. What does get imported joins the file's existing import of the same module, from subprocess import Popen becoming Popen, run as r, under whichever name the context or the file aliases it to. A relative context import stays relative.

visitor= is what reaches the file's imports; the cursor is where the result lands, and the two differ for a recipe splicing somewhere other than where it stands. Omitting the visitor still works for a template whose context binds no module, and raises otherwise, naming the modules and the call to make.

Where the bindings come from

The engine already parses the context statements ahead of the template, so import_bindings reads them there, as it does for the file being edited. One model answers for both sides, which is what lets a binding be compared rather than merely matched by name: whether it is guarded, whether it is the same member, whether two dotted modules share a root.

Why automatic

TypeScript treats this as the template's job. Template.resolveBindings(visitor) binds each module the context declares and returns the local name it actually bound; apply({bindings}) renames the template's references to those names; rewrite(...).tryOn(cursor, node, {visitor: this}) does the whole thing once a pattern matches, throwing if given neither a visitor nor bindings (templating/rewrite.ts:65). Python has no tryOn — apply() is the splice path — so the wiring and the refusal both land there.

Where it refuses rather than deconflicts

TS's maybeBind picks another name when the one it wants is taken. This has no such machinery, so it raises where emitting the reference would read the wrong object: a scope at the splice site binding the name, a module-scope import binding it to another module, or a rename landing on a name the context binds elsewhere. Nothing is registered until every binding holds, so a refusal leaves the file's imports alone. from subprocess import run as _run in the context is the way out of the second.

Tests

tests/python/template/test_template_imports.py, one per decision: the table's six rows, context the template never references, a name the template binds for itself, an aliased member merging into the file's import, a splice inside a function whose import still lands at module scope, a splice landing where the visitor does not stand, and the call that omits the visitor.

Reconciling the two import styles, so a spliced subprocess.run(...) collapses to run(...) where the file already imports the member, is #8791: it belongs to a normalizer that sees the whole file, since binding is per member and a template reading two members of a partly imported module can be collapsed for neither use or for one.

The Java peer's isReferenced also took a dotted module's last segment, so import os.path was only ever added to a file already using a name path. import a.b.c binds a. The Python half of that was fixed separately in #8831.

[jkschneider]

@copilot resolve the merge conflicts in this pull request

[Copilot]

@copilot resolve the merge conflicts in this pull request

Resolved and pushed in 43106a4.