Repository navigation
Scratch/riel/preproc variants - #99
Merged
rikvanriel merged 19 commits intoOct 7, 2026
Merged
rikvanriel merged 19 commits into
rikvanriel merged 19 commits into
Conversation
_cond_resched() is defined four times in sched.h, once per configuration, and the index kept whichever the name-keyed collapse preferred, so queries reported a clean dead end. This adds the guard (the arm's condition chain, "" at file scope) to FunctionInfo/MacroParams, populates it at all three analyzer sites, stores it as the last column of functions and object_macros, and extends the merge key and both Rust pre-merge dedups to include it, with SCHEMA_VERSION 9->10. Resolution is unchanged: the column is written and nothing reads it yet (the extractor dedup still keys on name alone, so one arm per name still arrives here). Tests: a two-arm insert_batch coexists with distinct guards while file scope round-trips to None; the extractor collapse pins the survivor carrying its arm's guard. 385 pass, clippy clean.
Before the within-file dedup is rekeyed on the guard, record what it chooses now, so the change of behaviour shows up in the diff instead of being asserted afterwards. On the real sched.h text _cond_resched() keeps its "return 0;" arm and drops the three that reach the scheduler; on the real dev_printk.h text dev_dbg() keeps the dev_no_printk() arm, which a CONFIG_DYNAMIC_DEBUG build never uses.
guard_of() joins the conditions of nested arms with " && ", but wrote each condition bare. sched.h's outer arm is "!defined(CONFIG_PREEMPTION) || defined(CONFIG_PREEMPT_DYNAMIC)", so the static-call arm under it was stored as !defined(CONFIG_PREEMPTION) || defined(CONFIG_PREEMPT_DYNAMIC) && ... which by C precedence means "not preemptible, or dynamic with the call", a configuration no arm states. Parenthesize a condition with a top-level || or ?: when it is one of several conjuncts; a lone condition is kept exactly as written.
The within-file dedup kept one row per name, so the four configurations of _cond_resched() in sched.h collapsed to "return 0;" and the three arms of dev_dbg() collapsed to the dev_no_printk() one. Key both the function and the macro dedup on (name, guard) instead. The storage key has carried the guard since the guard column was added, so every arm now reaches the index as its own row. Within one arm nothing changes: definitions still beat declarations and the longer definition still wins, so a macro redefined under the same arm (pr_fmt after each #undef in bugs.c) stays one row. A name the pre-heal parse already read still gains no arm from the healed parse, since the two parses do not see the same conditionals and their guards are not comparable. The recovered-macro count for the logging-header fixture goes from 3 to 5 because dev_dbg's three arms are no longer one.
The guard column was added to both table definitions and to their merge keys, but create_all_tables() only creates missing tables, so a version 9 database keeps its old functions and object_macros. Every insert then fails with "Merge insert key column 'guard' does not exist in schema", and re-indexing, the repair the version check asks for, can never succeed. Recreate both tables when the column is missing, the same way dispatch_sites and registrations are handled; their rows are derived from files that the version bump makes the indexer read again.
Two places still decided by name alone once the dedup was keyed on the arm: - A declaration capture never recorded a node, so its guard was None even under an #ifdef. A declaration and the definition it announces under one arm then read as two configurations and both survived, the bodiless one included. Take the guard from the declaration node. - A function a macro opens (SYSCALL_DEFINE*) was skipped when any function of that name had already been read, so a second arm of it was dropped before the dedup could keep it. Skip only when the same name was read under the same arm.
The previous fix decided which conditions needed parentheses by scanning
for a top-level || or ?:, which miscounts as soon as a condition holds a
'(' character constant or a comment. Wrap every conjunct that is not a
single name, a negated name, or one wholly parenthesized group instead:
a redundant pair costs nothing, a missing one changes the configuration.
negated() had the same shape of bug: "!(A) && (B)" starts with "!(" and
ends with ")", and stripping those gave "A) && (B". Unwrap only a group
whose opening parenthesis closes at the last character.
ObjectMacroStore::all() reduced the table to one expansion per name, whichever row came back first. With every arm of a macro now stored, that picks between "#define __tag __attribute__((x))" and the empty "#define __tag" of its #else arm by table order, so whether a member behind __tag is flattened away changes from run to run. Return every distinct expansion of a name and treat it as an attribute when any of them reaches __attribute__, directly or through aliases.
The condition text kept comments and joined a continuation only when the
backslash stood apart from its neighbours. A parenthesis in a comment,
as in "(A /* ( */) || (B /* ) */)", then made two groups look like one,
and "A\<newline>|| B" kept its backslash. Remove splices and comments
before collapsing whitespace, and skip character constants when deciding
whether a condition is one parenthesized group, so "(A == '(') || (B ==
')')" is read as two.
…cros Recreating functions for a version 9 index dropped every branch's functions but left indexed_branches saying each branch was current, so a later run skipped the branches whose tips had not moved and they stayed empty. Clear indexed_branches when functions is started again. clear_all_data() left object_macros behind. Now that every expansion of a macro decides whether it names an attribute, a row from a definition that no longer exists keeps deciding it. Clear it with the other tables.
find_by_name_file_and_hash() kept the first row matching (name, file, blob), so once each preprocessor arm became its own row, every git-aware lookup answered with whichever arm the table returned first and hid the rest: on a kernel index, 'func _cond_resched' showed one definition and said nothing about the other three. Return every row, ordered by line, and let both callers collect all of them, so the existing choose-and-report machinery sees the arms as other definitions. The search readers also hard-coded guard: None; read the column instead.
Two guards could not be compared as stored. A lone condition was kept as written while the same condition joined to another was parenthesized, so one arm of a #if and its sibling under an outer #if spelled the shared condition two ways. And every definition in a header carried the header's include guard, so on a kernel index every guard began with "!defined(_LINUX_SCHED_H) &&", which told no two definitions apart. Move the term handling into src/guard.rs. A guard is now always terms joined by " && ", each a name, a negated name, a group or a negated group, and negating a term gives the sibling arm's term back exactly. That lets excludes() say that two definitions sit in different arms of one #if (one holds a term and the other its negation), which is something two independent #ifs never show. An #ifndef X at file scope whose first line is #define X is the include guard and is left out.
With every arm of a name coming back, the reader still could not tell
them apart: four definitions of _cond_resched in sched.h were listed
with identical headers. Carry the guard on DefinitionSite and
CalleeDefinition and show it wherever a definition is named: "Under:"
in func output (REPL and MCP), " under <guard>" in the callee listing
per definition, in the "[1 of N definitions]" marker, and in the
ambiguity note's list of sites.
When every definition of a name sits in one file and every two of them
are in different arms of one #if, the general note ("depends on the
file it is written in and on the configuration") is wrong about the
file. A new note says the name has one definition per configuration,
lists each arm with its line and guard, and says that no build compiles
more than one. Distinct guards alone do not qualify: two independent
#ifs can both hold.
choose_definition counted languages per row; with arms as rows, one
header's #if could outvote the rest of the tree. Count per file.
…s it A chain walked the callees of the one definition choose_definition picked, so cond_resched -> _cond_resched -> __cond_resched -> rcu_all_qs was reachable only if the ranking happened to prefer that arm; on a kernel index it preferred the "return 0;" arm and the chain ended there. Walk every definition of the name in the file the chosen definition is in. Never other files: definitions elsewhere belong to other architectures or programs, and merging their callees is how a chain rooted in x86 grew sparc leaves. The name-level callee lists (get_function_callees_git_aware and get_function_callees_in) take the union over those arms, so callchain collection and find-paths see every route. The tree view makes each arm its own node under its guard, and the REPL callchain lists each arm of a callee with what it calls. Along a path, the tree carries the Kconfig terms that the guards above it asserted. An arm whose guard negates one of them cannot run there: it is shown with "[cannot run here: <term> holds above]" and not walked. Only Kconfig symbols count, because any other macro can differ between translation units. Disjunctions are never split, so they never prune.
Three problems in the previous guard commits: is_include_guard() accepted any top-level "#ifndef X" whose first line is "#define X", including one with an #else. "#ifndef MODE / #define MODE / A / #else / B / #endif" then lost both guards, and the (name, guard) dedup merged A and B. Accept only an #else-free wrapper that is the file's only top-level construct. Also recognise the "#if !defined(X)" spelling, which kernel headers use too. IS_ENABLED(CONFIG_X), and any other one-argument test of a name, was not an atom, so it was stored as "(IS_ENABLED(CONFIG_X))" and its sibling arm as "!(IS_ENABLED(CONFIG_X))". Treat it as an atom. config_facts() missed facts in a parenthesized conjunction "(defined(CONFIG_A) && defined(CONFIG_B))", and did not match "defined CONFIG_X" against "defined(CONFIG_X)". Flatten a group that holds only &&, and give each atom one spelling. Groups with a top-level || or ?: still assert nothing.
Once every arm of a file reached choose_definition, the arm with the
best row features spoke for its file. arch/um/include/shared/user.h
defines printk() as a macro under IS_ENABLED(CONFIG_PRINTK) and as
"static inline int printk(...) { return 0; }" in the #else. The stub has
a body and include/linux/printk.h's macro does not, so on a kernel index
6,895 calls to printk resolved to user-mode Linux's stub.
Rank each file by one row: the first one, by line, that defines the
name. That is the #if side, which kernel headers conventionally write as
the configured implementation, with the stub in #else. The file's other
arms are still reported as other definitions, and chains still walk all
of them. A rung preferring unguarded definitions was tried first and
dropped: it moved hundreds of unrelated answers across architectures.
The recursive tree walker that marks contradictions is not what either shipped callchain uses: the REPL and MCP each print their own two-level listing, and the MCP one showed a single definition with no guard. Move the arm listing into callchain::write_callee_arms and use it from both. Each arm of a callee its file defines more than once is listed with its guard and what it calls. An arm whose guard negates a Kconfig term of the chain root's guard is marked "[cannot run here: ...]" and its callees are not listed. The per-configuration note also claimed more than the text can prove: a macro redefined between two conditionals defeats a textual contradiction. Say "as written", and name that exception.
Within an architecture, get_function_callees_in() took the first admitted definition after sorting the architecture's own ahead of generic ones, while the definition a chain names comes from choose_definition_in(), which had no such preference. A chain could name one definition and list another's calls. On a kernel index, 'callchain cond_resched' named lib/test_maple_tree.c:56 and listed calls from include/linux/sched.h. Now that the listing walks every arm of the chosen file, the mismatch decides which arms are walked. Make choose_definition_in() rank the architecture's own definitions first, keeping the others as reported alternatives, and let get_function_callees_in() take its arms from that choice.
Two ways the contradiction marks could hide a reachable callee: - The root's callees are the union over the root's arms, but the facts came from the chosen arm alone, so a callee another root arm reaches could be marked impossible. Use only the facts every root arm asserts (guard::shared_facts). - IS_REACHABLE(CONFIG_X) was a fact, but it depends on whether the file asking is built as a module, so a caller and a callee can disagree. Facts are now limited to bare symbols, defined(), IS_ENABLED(), IS_BUILTIN() and IS_MODULE().
meta-cla
Bot
added
the
CLA Signed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
More prep work to get closer to real object oriented language support, so we can do Rust call chains correctly.