+ The Windows half of the fields Linux already answers - #85
Merged
Conversation
…n guessed at An ELF says what it is in a header this program already reads. A PE says it in a version resource three levels down a tree of self-relative offsets, and nothing here could read one — which is why five §14 boxes stood unticked with nobody having written the Windows half. The reader takes bytes and carries no platform attribute, so it runs on every CI leg. That is not a formality here: every assembly this repository builds is a real PE image with a version resource in it, written by a compiler nobody here controls, and the tests read those. The parser is therefore held against files it did not produce, on Linux, where it was developed. The strongest check is that a PE writes its file version twice — once as the four numbers of VS_FIXEDFILEINFO and once as a string in a string table, reached by a different path through the same tree — and a walk that lands in the wrong place cannot make the two agree. Held against 1617 real PE images on the machine this was written on: 478 carry a version resource, 472 agree exactly, and the six that do not are publishers who wrote "8.00" where the binary field says 8.0. The header fields were separately checked against objdump -x, which is an implementation of this format that has nothing to do with this one. Both bitnesses are exercised: PE32 by the managed assemblies, PE32+ by the 64-bit images in the sweep.
…nst the confident zero they would otherwise be §14, §20 and §21 between them had two dozen boxes marked Windows-only. Several were marked that way because Linux genuinely cannot answer them; all of them were unticked because nobody had written the Windows half. This is the catalogue half: the descriptors, the rendering and the filtering, none of which needs a Windows machine to be correct or to be tested. The mitigation policies are held as the raw flags word each PROCESS_MITIGATION_* structure is a union over, and decoded where the column is rendered. That is what puts the part most likely to be wrong — a bit position off by one, which reads as a mitigation that is off and looks exactly like a mitigation that is off — into portable code that every CI leg exercises. Two constants are worth naming because getting either wrong is invisible on a screen. PROTECTION_LEVEL_NONE is 0xFFFFFFFE, not the -1 a sentinel usually is; 0xFFFFFFFF is PROTECTION_LEVEL_SAME. And nought is a real protection level — WINTCB_LIGHT — so a record nobody filled must not read as the most protected process on the machine. Which is the other half of this commit. default(Counter) is a confident nought and this program has already shipped one: default(SystemCounters) reported machines as having 0 B free. Seventeen new counters would have done the same on Linux — every process reading as unprotected, unsandboxed and running with no mitigations, which is a finding rather than an absence. The existing arrangement has each probe refuse the other platforms' readings one by one, and that works only until somebody adds a field without editing all three probes. So a record now starts out saying it knows nothing about these, once, where records are handed out, and a probe that can answer overwrites it.
… is paying for Four readings, four costs, four switches — none on by default (§5.4). The token half is free in the sense that matters. The owner lookup already opens every new process once and caches the answer for its lifetime; the protection level, the AppContainer flag, the translated machine and the image path all come off that same handle with the same PROCESS_QUERY_LIMITED_INFORMATION, so seven answers now come out of the open that used to yield three. The image path is the one that unblocked the rest: SYSTEM_PROCESS_INFORMATION carries the file name alone, so until now the Windows probe filled no path at all and nothing could be keyed on the image. The mitigation policies are the exception and have their own switch, because GetProcessMitigationPolicy documents PROCESS_QUERY_INFORMATION rather than the limited form — a stronger right, a second open, six calls. For another user's process it will usually fail, and it says so per policy rather than reporting six mitigations as absent. The object tallies come out of one walk of the machine's whole handle table, because Windows has no per-process handle query at all. The type indices are the running kernel's — handed out in the order its object types were created at boot — so they are discovered by duplicating one handle per distinct index and asking what it is called, a few dozen duplications rather than one per row of a table with a million rows in it. An index nothing could name leaves that column missing rather than nought. GetGuiResources returns nought both for a process with no such objects and for a call that failed, and says so in its own documentation, so the last error is cleared first and asked afterwards. A console service really does hold no USER objects, and that is a measurement. Where each layout and constant came from is on the declaration. The two that are not from Microsoft's reference pages say so: the PROTECTION_LEVEL_* values appear on no page and are the ones winbase.h defines, and the mitigation-policy and token-information ordinals are derived from the order of the members, because neither enumeration prints a number.
The API calls are unreachable from here and stay untested until the windows-latest leg runs them. Everything that decides what a value *means* is portable, and that is where the mistakes live that would survive a demo — so it is all held against the structures and the constants that define it, on every leg. The bit positions are transcribed one test case at a time, which is deliberate: a table of expected words is a thing a reader can check against winnt.h without running anything. Three of them encode judgements rather than layout. Audit is not a weaker "on" for dynamic code — under it nothing is prevented, only watched, and reporting it as on would claim a protection that is not in force. Strict mode upgrades the shadow-stack word rather than being listed beside it. And a policy nobody was allowed to read is not a policy that is off, which is the single most dangerous cell in the section. The tally tests pin the thing that no test on a Windows machine would catch: the object type indices differ between boots, so a build that hard-coded them would report a plausible tally of entirely the wrong objects. They also pin that the kernel calls a mutex a "Mutant" — matching on "Mutex" would leave that column permanently empty on every Windows there is. The last test is the one that already found something. It runs the Linux probe over a recorded /proc tree and asserts that none of the seventeen new counters carries a value, which before the previous commit's guard they all did.
…thing ever would §5.4 says a reading nobody named is a reading nobody pays for, and the four new switches obey it. The consequence is that the interop behind them would never run anywhere: --self-test names no columns, and it is the only thing in the whole pipeline that executes the Windows probe against a real kernel. So the self-test names all of them. That turned out to matter immediately, because the win-x64 build runs under wine on this machine and the run found two things. The first is the object type indices. They are the running kernel's, learnt by duplicating one handle of each index and asking what it is called — and a type is only discoverable while some process is holding a handle of it. Doing that pass once left any type absent from the first sample permanently unknown: two runs a minute apart, one of which learnt the semaphore index and one of which did not. It now repeats until all five are known, asking about each index once ever, and stops entirely after that. The second is that "the call failed" was being reported as "you may not look". An information class this Windows does not implement and a process this user may not open are opposite findings, and only one of them is fixed by running as an administrator. They are told apart by the error now. What the checks assert is deliberately narrow. The value a mitigation policy holds is the runtime's business and changes between versions; that the call succeeded is not, and a wrong structure size or a wrong policy ordinal makes it fail rather than crash — which is invisible on a screen and is the whole of what HasValue is guarding. Where an independent answer exists it is used instead: FileVersionInfo reads the same version resource out of the same file through an entirely separate implementation, MainModule.FileName is a different call for the image path, and RuntimeInformation's two architectures say whether a process is being translated without asking IsWow64Process2. Under wine the version resource, the image path, the emulation state, the subsystem and the object tallies all agree. Four checks fail there — the protection level, the DEP flag and the two desktop quotas — and all four are wine stubbing the call rather than this reading it wrongly. The wine leg is advisory and was already red on the elevation check, which has been there since #8.
…der wine The naming pass kept one candidate handle per object type index — the first in the table — and asked the process holding it to duplicate it. On a real kernel it learnt nothing at all: the table is ordered by process and the first process in it is the System process, so the first handle of very nearly every type belongs to something that will not open for PROCESS_DUP_HANDLE at any privilege. All five columns came back missing, which was at least the honest answer for a discovery pass that had discovered nothing. Under wine there is no System process to speak of and the same code worked perfectly, which is exactly the kind of difference the windows-latest leg exists to find. It now keeps several candidates of each type and tries the caller's own handles first, a process being able to duplicate out of itself unconditionally. An index is recorded as asked only when something actually answered, so a type whose every candidate refused is tried again from a later sample instead of being written off. Also stops closing the pseudo-handle from GetCurrentProcess, which is not a real handle, now that our own process is one of the owners the loop opens.
…s waiting on Somebody counting unticked boxes could not tell "nobody has written this" from "this operating system has no such thing", and the two want completely different responses. Sixteen ticks move, and every line that stays open now names its platform and its reason on its own line. What the remainder turns out to be, once it is written down: eight lines waiting on macOS, which is a stub rather than eight separate gaps; five waiting on a signature verdict, which is the one kind of thing that genuinely cannot be written honestly from here; three that no interface on any platform publishes per process; and one — the AppContainer capability list — that is written off rather than unwritten, being a per-row allocation of unbounded size that belongs in §36's on-demand view rather than in a column. The ticked lines say how far the verification actually goes, which for most of them is further than expected and for none of them is "it compiles". The PE walk is held against 1617 real images on the Linux leg and against FileVersionInfo — Microsoft's own reader of the same bytes — on the Windows one. The mitigation bits are decoded in portable code with a test case each. The two constants that came from a header rather than from documentation say so on the line that uses them, and so does the one place where an ordinal is derived from the order of an enumeration because Microsoft prints no numbers for it.
A real Windows passes every one of the new checks — the windows-latest self-test is green. Wine is not Windows: it implements a great deal of Win32 and stubs the rest, and its stubs answer honestly, so an unimplemented call returns "not supported" and this program reports that faithfully. Asserting that a protection level reads PROTECTION_LEVEL_NONE, that a 64-bit process has DEP on, and that the two desktop object counts came back is asserting what wine cannot say. Those four now report themselves as skipped rather than passed. A check that quietly succeeds where it was never run is worse than one that fails, because it is counted. The leg keeps its value: it still catches a call that crashes, one that returns nonsense, and every reading wine does implement — which is most of them. Told rather than detected. Sniffing for an export in ntdll would mean interop in a file compiled for every platform, to answer a question the thing running the test already knows, so the leg says so in its own environment instead.
…cation says to By the counts in the directory header and the rule that named entries come first, which is what the PE format document actually states, rather than only by the high bit of the entry's first word — a convention every real image follows and that the document describes only for the second word of the pair. Both are checked now; they have never disagreed across the 1617 images this was held against, and the one that is written down is the one that decides.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
§14, §20 and §21 between them carried two dozen boxes marked Windows-only. Some are marked that way because Linux genuinely cannot answer them; every one of them was unticked because nobody had written the Windows half. This writes it.
What is now answered
§14 —
description,company,product,product.version,file.versionout of the PE version resource;subsystemout of the same file's optional header;emulationfromIsWow64Process2.image.pathandexe.namefill in on Windows for the first time as a side effect, since the bulk query carries only the file name and everything else here is keyed on the path.§20 —
event.count,semaphore.count,mutex.count,section.count,regkey.countfrom one walk of the machine's handle table;user.objectsandgdi.objectsfromGetGuiResources.§21 —
protected,protection.level,appcontainer(whichsandboxis an alias of, Windows having one answer to both questions), and the six mitigation policiesdep,aslr,cfg,cet,acg,cig.What the Windows leg says
--self-testnow names every one of these columns, because §5.4's opt-in rule would otherwise mean the interop never executed anywhere at all. Onwindows-latest, all 51 checks pass:Where an independent answer exists it is used rather than invented:
FileVersionInfois Microsoft's own reader of the same version resource out of the same file,MainModule.FileNameis a different call for the path, andRuntimeInformation's two architectures say whether a process is being translated without askingIsWow64Process2the same question twice.gdi objects 0is the case that matters most —GetGuiResourcesreturns nought both for a process with no such objects and for a call that failed, and this is the reading that proves the two are told apart.How the rest is defensible without a Windows machine
The PE reader takes bytes and carries no platform attribute, so it runs on every leg — and it is not exercised against a buffer synthesised from its own struct definitions, which is the weakness the §9.4 replay tests admit to. Every assembly this repository builds is a real PE image with a version resource in it, so the tests read those. A PE writes its file version twice, in two encodings reached by two different paths through the resource tree, and a walk that lands in the wrong place cannot make the two agree. Held against 1617 real PE images: 478 carry a version resource, 472 agree exactly, and the six that do not are publishers who wrote
8.00where the binary field says8.0. Header fields cross-checked againstobjdump -x; both bitnesses covered.The mitigation policies are carried as the raw flags word each
PROCESS_MITIGATION_*structure is a union over and decoded where the column is rendered, so the part most likely to be wrong — a bit position off by one, which reads as a mitigation that is off and looks exactly like one — is portable code with a test case per bit. The handle tally is a span walk with the type indices passed in.Where the layouts and constants came from
On the declaration, every one of them. Two are not from Microsoft's reference pages and say so:
PROTECTION_LEVEL_*values appear on no documentation page at all — the struct page prints the constant names against an empty value column — so they are the oneswinbase.hdefines.PROTECTION_LEVEL_NONEis0xFFFFFFFE, not-1;0xFFFFFFFFisPROTECTION_LEVEL_SAME; and nought is a real level,WINTCB_LIGHT.PROCESS_MITIGATION_POLICYandTOKEN_INFORMATION_CLASSordinals are derived from the order of the members, because neither enumeration prints a number.ProcessUserShadowStackPolicy = 15is derived that way, and the Windows leg reportingcet: on, IP validationis what confirms it. Note that theGetProcessMitigationPolicypage lists the same members in a different order and omits six of them, so it must not be used to derive them.Two defects the runs found
The confident zero, again.
default(Counter)is a confident nought and this program has already shipped one. Seventeen new counters would have done it again on Linux — every process reading as unprotected, unsandboxed and running with no mitigations. The existing arrangement has each probe refuse the other platforms' readings one by one, which works until somebody adds a field without editing all three probes. A record now starts out saying it knows nothing about these, once, where records are handed out. The test that asserts this over a recorded/proctree is what found it.The tally learnt nothing on a real Windows and everything under wine. The type indices are the running kernel's, learnt by duplicating one handle of each and asking what it is called — and the table is ordered by process, so the first handle of nearly every type belongs to the System process, which will not open for
PROCESS_DUP_HANDLEat any privilege. It now keeps several candidates per type and tries the caller's own handles first, a process being able to duplicate out of itself unconditionally. Only thewindows-latestleg could have caught this.Verification
dotnet build -c Release: 0 warnings.dotnet test: 1805 passing, up from 1716. The five field-registry invariants pass with all 26 new descriptors. Every CI job green, wine leg included.