Repository navigation
Conversation
added 7 commits
October 22, 2026 22:26
…ss logs - resolveAll now resolves the whole candidate list in one dnsx -a -aaaa -resp pass through the operator/public resolver file (the same list s.resolver round-robins), then screens answers through the unchanged wildcard-aware admission gate. Fallback to the legacy per-name Go path stays for tool-free builds, missing dnsx, or a failed tool run; both paths share one classify closure, so admit/reject sets are name-for-name comparable. - dnsx -cname -resp prefilter now captures the CNAME target too, removing the second in-process lookup for unresolved candidates. - dnsxCNAMECandidates gains the target map (third return); only caller updated. - New parseDNSXResponseLine helper (bracket-group -resp parsing, IPv6 + multi- answer safe, unit-tested in dnsx_parse_test.go). - Progress visibility: batch-pass finish timing plus a 30s heartbeat on the fallback path (checked/verified/wildcard/CNAME/unresolved) instead of a silent window before the final gate line. No schema, API, or report changes; admission counts unchanged by design.
An operator-cancelled monitor_watch pass never stamped targets.monitor_last_run, so runDueMonitors (1-minute tick) kept re-creating the scan the operator had just stopped: cancel -> <=30-minute creation guard -> re-run -> cancel again, forever. A long-pass cancel could resurface within a minute. Stamp monitor_last_run for finished AND operator-cancelled watch passes, so the next automatic check waits the full configured interval (MAX(1, monitor_interval_hours || 12h default)). Semantics preserved: graceful-shutdown passes keep status 'interrupted' and are NOT stamped (auto-resume owns their restart window); watchdog timeouts land in 'failed' with the existing short failure backoff; manual scans are unaffected. New tests (monitor_cancel_test.go): a cancelled watch is stamped and is NOT re-created by runDueMonitors even after the 30-minute guard expires; an interrupted watch is NOT stamped and stays guarded.
The recursion depth knob existed, but recursion itself could not be turned off without dropping the remembered depth. Add a subdomain_recursion_enabled config field (default true, upgrade-safe: Load() unmarshals onto defaultConfig(), so an old config.json without the key keeps recursion on), gate the recursiveEnum call site on it next to the deep-work toggle, and send it through the scan-config GET/PATCH payloads. System -> Scanning panel: checkbox 'Enable recursive enumeration' beside the depth input (matches the codebase accent-accent checkbox style); the depth input grays out when off and one Save persists both values. Disabled -> apex only: no resolvingSubdomainsUnder frontier, no per-level passive discovery or resolution. Brute/permutation deep-work behavior is unchanged. Depth stays stored and inert until re-enabled. Tests: TestSubdomainRecursionEnabledDefault (nil-cfg defaults to enabled, mirroring subdomainRecursionDepth; explicit true/false honored).
runDueMonitors hardcoded the periodic watch pipeline. It now builds the task from config.monitor_modules (System -> Scanning page, live pickup via the shared in-memory config — no restart). Names are validated against AllModules: unknown tokens are logged and skipped, and an empty/invalid list falls back to the built-in set (subdomain_enum, monitor, http_probe, takeover, cors, backup_discovery) so a stale config can never murder the schedule. Order invariant preserved server-side, not by the form: the snapshot module (monitor) is hoisted ahead of the first refresh-capable module (http_probe/js_analysis) whenever the operator order puts it after, while staying after subdomain_enum so newly-appearing names stay inside the diffed baseline. Legacy default order is unchanged. PATCH /system/scan-config accepts monitor_modules (trimmed/deduped; name validation stays scheduler-side so the API never needs the catalog); GET returns it too. UI: Default monitor modules checkbox card on System -> Scanning with per-module hints, fed by the same save button. Tests: configured set honored + unknown skipped; monitor-before-refresh hoist; monitor stays after subdomain_enum; empty config -> built-in set; all-unknown -> fallback; full scheduler/api/config suites green; frontend builds.
… diff viewer
- css_files table (migration) + first-party and external stylesheet harvest
during the monitor pass, with 256 KB body capture on detection and change
- css_change rows on confirmed content change (double-fetch verified), old
body rolled into old_body; js_change rows now capture bodies the same way
- external <link> churn from ?ver= cache-busting no longer creates
add/remove pairs: stylesheet attr rows key on path without query, and a
query-only bump with unchanged bytes stays silent; a new/removed path
still alerts at info severity and no longer raises vuln findings
- notification titles: 'Stylesheet content changed', 'External stylesheet
added/removed' (was mislabeled as JavaScript changes)
- GET /targets/{id}/css-files/body and /js-files/body (url + side params,
target-scoped) expose the captured snapshots
- Changes tab: filter chips (All/Security/Scripts/Stylesheets/HTTP/Tech/
Headers) with row counts, severity-driven badge colors, and a Diff
drawer rendering a line-level +/− diff from the stored captures
…egacy-snapshot diff keys - escalateIfChanged ignored change severity and treated any monitoring row as escalation fuel: a css_change-only redeploy (or a security: stylesheet_* attr row) fell through to the default heavy set and fired probe + backup discovery + nuclei. Stylesheet signals are observational now — they count toward Changes-tab history and the bell, never toward a watch escalation (same policy as the js_change JS-refresh set, which has no CSS equivalent). A js_change beside css churn still escalates. - diffSecuritySnapshots keyed stylesheet attrs on the raw href, so every snapshot recorded before the ?ver= stripping change emitted one spurious add+remove pair per stylesheet on the first pass after upgrade. The key strips the query for stylesheet kind on both sides; other kinds keep exact-match keys.
ItsNilDev
changed the title
added 8 commits
October 24, 2026 12:31
The old random-label gate probed 5 fixed names, treated every DNS error like NXDOMAIN, and only looked at the apex. Three holes admitted junk fleets wholesale: - A flaky answer (SERVFAIL/rate-limit) looked like NXDOMAIN, so <3 resolving probes meant no wildcard evidence and the whole junk namespace was admitted (the 'sometimes' in the bug report). - Rotating CDN pools and rotating CNAME targets need more samples than one batch of 5 can give. - Branch wildcards (*.git.example.com) were never probed at all — asdf-garbage.git.realestate.com.au style junk was admitted even with a perfectly flipped apex verdict. Fixes, all in the DNS gate so junk stops before HTTP probing: - Probe verdicts now distinguish definitive NXDOMAIN (net.DNSError.IsNotFound) from ambiguous errors, and ambiguous answers retry on DIFFERENT upstream resolvers (pinned per attempt) — one rate-limited resolver can no longer blind the gate. - When neither agreement threshold (<3 resolved AND <3 NX) is reached, a second batch of 5 fresh random labels runs before deciding. - Branch wildcards: clustered candidate suffixes (>=5 names sharing a strictly-deeper suffix, e.g. git.example.com) get probed; confirmed branches become per-name zones the admission gate, brute skip, deep resolution rechecks and vhost classification all respect. 8 branches per apex scan, 3 per recursion sub-apex, bounded DNS traffic. - wildcardIPs/wildcardCNAMEs maps replaced by wildcardProfile (apex + branch zones with label-boundary suffix matching: evil-git.name must never match *.git.name). nil profile behaves as no-evidence. - Inconclusive probe batches now log their histogram instead of silently deciding. Unit tests cover suffix clustering, zone matching (label boundaries), nil-profile safety and the decision/summary logic without network.
Mark subdomain rows that serve identical wildcard content: a fleet of junk hosts that all redirect to the apex (>=3 members) or clone the apex homepage (>=5 members, apex itself included in the group). Reads only ALREADY-stored http_services rows - no extra network requests - and runs at the end of http_probe when enabled. Rows are marked via the new subdomains.catchall column and hidden from the Hosts tab, reports, graph, stats and wordlist learning; their http_services rows are deleted so JS/params/DAST/nuclei never see them. Seeds and vhost rows are never touched. DNS rows keep the mark across rescans; the probe loader skips them. Default OFF; enable via System -> Toolchain -> Scanning configuration.
Found during the post-implementation validation pass: - Retry probes dialed BARE resolver IPs (the dnsx resolver list strips :53), so every cross-resolver retry failed with 'missing port in address' and was counted as ambiguous. resolverServerAddress now normalizes entries to host:port (IPv4/IPv6/bracketed/non-standard port); a nil resolver falls back to the system one. - Catch-all marking now defense-in-depth rejects hosts that are not strictly under the target apex, and dedupes hosts across http/https service rows before logging/marking. - Remaining consumer surfaces now respect the catchall mark: takeover checks, asset priority/planning, recursive-enumeration frontier, admitted-name/wordlist seeds, global search, dashboard target stats and scan-done notifications. Tests: retry-address normalization + second-pass idempotency. Suites: scanner/api/scheduler/database green; frontend build green.
…ged)
Adds a tri-state targets.catchall_detection column: NULL inherits the
global http_catchall_detection config (code default OFF), 0 forces the
catch-all pass off for the target, 1 forces it on.
- scanner: catchallEnabledForTarget resolves override > global; the
http_probe hook uses it instead of the raw config bool.
- api: GET /targets/{id} returns catchall_detection (inherit|on|off) and
catchall_effective; PUT /targets/{id} accepts the tri-state string and
rejects anything else.
- ui: TargetDetail gets an 'HTTP catch-all detection' select next to the
monitoring settings, showing the effective state; applies from the
next scan.
- tests: resolver matrix (scanner) + handler round-trip set/clear/invalid
(api).
…-only assets - subdomains: new miss_count column; pruneRejectedSubdomains deletes only after two CONSECUTIVE missed resolve passes and resolveAll's upsert resets the counter. One dnsx/resolver flake no longer wipes a previously verified DNS row (and its recursion-frontier seed) - subdomain recursion + vhost scan escape LIKE metacharacters in the apex and use ESCAPE '\', so a malformed stored domain cannot widen the pattern - monitoring-changes API takes offset (500/page, id tiebreaker for stable paging); Changes tab gains 'Load older changes' past the newest 500 rows - project-asset monitor only queries kind='web' — ip/cidr assets no longer turn into bogus https://<range> GETs - tests: two-miss prune, miss reset on resolve, escapeLike, network assets skipped while web assets stay monitored
- List/create target responses omit catchall_detection/catchall_effective instead of emitting empty values; the detail endpoint now always sends an explicit effective bool via a pointer field. - Move the runCatchallPass doc comment back above its function. - Surface a toast when saving the per-target catch-all override fails. - Record the audit findings and status in TODO-minor-findings.md.
TargetDetail shares one data state across tabs and replaces it only after the next tab's fetch resolves. Clicking Changes with rows loaded in the previous tab therefore rendered those rows under the monitor branch for one frame; the filter chips and severity badge dereferenced change_type on rows that do not have it: TypeError: Cannot read properties of undefined (reading 'includes') The inline throw is outside the page's own ErrorBoundary, so the route unmounted instead of showing a tab-level error. - tab click now clears data and shows the skeleton before switching, so a tab never renders another tab's row shape (silent refresh paths keep the existing rows) - monitorRowKind/changeBadgeVariant tolerate a missing change_type - inner ErrorBoundary keys on the active tab so a caught error cannot stick after a switch - regression test for foreign-row classification
Conflict resolution in internal/scanner/subdomain.go: - Port upstream's definitive-NXDOMAIN rule into the shared classify closure. The per-name fallback uses lookupIPs(): only an authoritative NXDOMAIN rejects/prunes, transient misses are kept. The batched dnsx -resp fast path cannot distinguish NXDOMAIN from a throttled query, so absent names are treated as transient and never pruned there. - Keep the dnsx fast path, batched writes, new-only broadcasts and the 30s progress heartbeat. - Recursion stays opt-in per scan (recursive_subdomain token). The System -> Scanning toggle now supplies the deployment default, and the code default flips to false (apex-only) to match upstream's fail-closed intent. Also merged: cancel re-probe guard, notification z-index, Changes dedup and the v4.8.0 version bump.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
fixing the notification ui bug.