Skip to content

Commit bdae3fd

Browse files
loong10kloong10k
committed
接通 CFQuery 结构候选、嵌入身份与修复指引并纠正方言误报提示
1 parent 7ac7c22 commit bdae3fd

39 files changed

Lines changed: 3864 additions & 42 deletions

‎.github/workflows/ci.yml‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -49,7 +49,7 @@ jobs:
4949
- name: Verify Linux WASM worker resource boundary
5050
run: |
5151
# 帮助目录须保留当前构建特性；默认 suite 无法证明 WASM 入口状态。
52-
cargo test --locked -p codeguard-cli --features wasm-precheck --test help_command_contract --test rules_list_cli --test version_cli --test rust_lint_cli --test ruby_lint_cli --test check_all_ruby --test ruby_native_hook --test ruby_project_version --test shell_lint_cli --test shell_workbench --test check_all_shell --test shell_native_hook --test hook_failed_write_options --test go_native_hook --test cfquery_postgres_dialect
52+
cargo test --locked -p codeguard-cli --features wasm-precheck --test help_command_contract --test rules_list_cli --test version_cli --test rust_lint_cli --test ruby_lint_cli --test check_all_ruby --test ruby_native_hook --test ruby_project_version --test shell_lint_cli --test shell_workbench --test check_all_shell --test shell_native_hook --test hook_failed_write_options --test go_native_hook --test cfquery_postgres_dialect --test cfquery_structure
5353
cargo test --locked -p codeguard-cli --features wasm-precheck --test workspace_msrv --test crate_boundaries
5454
cargo test -p codeguard-runtime --features wasm-precheck --test process_contract
5555
cargo test -p codeguard-runtime --features wasm-precheck --test wasm_grammar_load --test wasm_recovery_scan --test wasm_empty_block_scan --test wasm_root_child_scan --test dart_upstream_corpus

‎README.md‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -768,3 +768,35 @@ Edit feedback uses outer0.22/inner0.12 and native-first observations0.10; histor
768768
Go native-first task guidance uses `repair_brief_preview` 0.18 with a `syntax-confirm-` observation reference. Actual task-recheck guidance retains 0.14 with a `syntax-native-` reference; existing schemas are unchanged. Final affected regressions: WASM 78 passed / 5 conditionally ignored, default 15 passed / 0 ignored. The earlier 1,461-pass full default run predates this final protocol correction.
769769

770770
Go's fixed 1.23.4 syntax SDK now statically checks the nearest `go.mod` and nearest `go.work` before editing or original-tool task verification. A newer minimum/suggested toolchain, ambiguous or unreadable declarations produce an environment observation without running that SDK or falling back to WASM. Changes during observation withdraw diagnostics; incompatible current declarations withdraw saved repair positions. This is a bounded compatibility guard, not general project toolchain or language-version acceptance. Evidence: [Go project version](tests/acceptance/go-project-version.md).
771+
772+
### CFQuery static DISTINCT projection candidate
773+
774+
The fixed CFQuery grammar tokenizes SQL without validating every SQL clause. Codeguard now reports adjacent AST `SELECT DISTINCT FROM` keywords as an independent candidate, preserving raw ERROR/MISSING observations. String/identifier literals and CFML interpolation interrupt the sequence; comments can be ignored. `SELECT FROM users` remains unflagged by this rule because PostgreSQL permits an empty projection without DISTINCT.
775+
776+
```mermaid
777+
flowchart LR
778+
A[CFQuery SQL fragment] --> B[Fixed WASM AST]
779+
B --> C[Raw ERROR / MISSING]
780+
B --> D[Adjacent SELECT DISTINCT FROM keywords]
781+
D --> E[Independent unqualified candidate]
782+
E --> F[Whole-file identity + fragment identity + file positions]
783+
F --> G[One stable confirmation task]
784+
G --> H[Resolve datasource, dialect, version and template context]
785+
H --> I[Applicable native SQL confirmation required]
786+
```
787+
788+
The worker uses 1.3, explicit probe 0.4, project feedback 0.53, editing feedback 0.23/0.13, and saved candidate observation 0.11. Historical schemas and grammar assets remain unchanged. Embedded observations carry whole-file `source_sha256`, separate `fragment_source_sha256`, and restored file positions. Repeat scans share a task; candidate disappearance does not close it. `next` explicitly asks for datasource, dialect/version and dynamic-template/schema context: the CFQuery native task-verification adapter is still unavailable, and no project database is contacted automatically.
789+
790+
```bash
791+
codeguard grammar probe cfquery query.sql --format=json
792+
codeguard check all . --format=json
793+
codeguard next . --format=json
794+
```
795+
796+
Report example (selected fields, not a complete schema):
797+
798+
```json
799+
{"language":"cfquery","recoveries":[],"structural_observations":[{"basis":"codeguard_structure_rule","rule_id":"codeguard.cfquery.distinct_projection","rule_version":"1.0.0","parent_syntax_kind":"program"}],"grammar_qualified":false,"status":"incomplete","delivery_decision":"not_evaluated","next_action":"confirm_candidate_structure_with_applicable_native_tool"}
800+
```
801+
802+
This addresses one fixed native-counterexample at the candidate layer. It does not qualify the grammar, measure an independent holdout, infer every SQL dialect, validate injection safety or prove native adapter/release acceptance. Evidence: [CFQuery candidate acceptance](tests/acceptance/cfquery-structure.md).

‎README.zh-CN.md‎

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -751,3 +751,35 @@ flowchart LR
751751
Go首次原生观察的任务指引使用 `repair_brief_preview` 0.18和 `syntax-confirm-` 引用；实际任务复检继续使用0.14和 `syntax-native-` 引用，历史schema保持不变。最终受影响回归：WASM 78通过/5条件忽略，默认15通过/0忽略。此前1461通过的默认全量结果早于这次末尾协议修正。
752752

753753
Go固定1.23.4语法SDK现于编辑及原工具任务复检前静态检查最近 `go.mod` 和最近 `go.work`。最低版本或建议工具链超出支持范围、声明歧义或不可读时返回环境观察，不运行不适用SDK、不降级WASM；运行期间变化撤回诊断，当前声明不适用时撤回旧修复位置。该边界不代表通用工具链选择或语言版本验收。证据：[Go项目版本](tests/acceptance/go-project-version.md)。
754+
755+
### CFQuery 静态 DISTINCT 投影候选
756+
757+
固定 CFQuery grammar 能识别 SQL token，不能完整验证 SQL 子句。Codeguard 对直接相邻的 AST 关键词 `SELECT DISTINCT FROM` 增加独立候选，原始 ERROR/MISSING 保持不变。字符串、引号标识符和 CFML 插值会打断匹配；注释可跳过。该规则不标记 `SELECT FROM users`，因为 PostgreSQL 允许未使用 DISTINCT 的空投影。
758+
759+
```mermaid
760+
flowchart LR
761+
A[CFQuery SQL fragment] --> B[Fixed WASM AST]
762+
B --> C[Raw ERROR / MISSING]
763+
B --> D[Adjacent SELECT DISTINCT FROM keywords]
764+
D --> E[Independent unqualified candidate]
765+
E --> F[Whole-file identity + fragment identity + file positions]
766+
F --> G[One stable confirmation task]
767+
G --> H[Resolve datasource, dialect, version and template context]
768+
H --> I[Applicable native SQL confirmation required]
769+
```
770+
771+
worker 使用1.3、显式probe 0.4、项目反馈0.53、编辑反馈0.23/0.13、持久候选0.11；历史schema及grammar资产不改。嵌入观察同时记录完整文件 `source_sha256` 和独立 `fragment_source_sha256`，坐标还原到文件。重复扫描复用同一任务；候选消失不关闭任务。`next` 明确要求确认datasource、数据库方言/版本、动态模板和schema上下文：CFQuery原生task verify adapter仍未接入，不自动连接项目数据库。
772+
773+
```bash
774+
codeguard grammar probe cfquery query.sql --format=json
775+
codeguard check all . --format=json
776+
codeguard next . --format=json
777+
```
778+
779+
报告示例（字段节选，不是完整schema）：
780+
781+
```json
782+
{"language":"cfquery","recoveries":[],"structural_observations":[{"basis":"codeguard_structure_rule","rule_id":"codeguard.cfquery.distinct_projection","rule_version":"1.0.0","parent_syntax_kind":"program"}],"grammar_qualified":false,"status":"incomplete","delivery_decision":"not_evaluated","next_action":"confirm_candidate_structure_with_applicable_native_tool"}
783+
```
784+
785+
本轮只在候选层纠正一个固定原生反例，不代表grammar取得资格、独立holdout精度、全部SQL方言、SQL注入安全或原生adapter/发行验收完成。证据：[CFQuery候选验收](tests/acceptance/cfquery-structure.md)。

‎crates/codeguard-adapters/src/cfquery_projection_rule.rs‎

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
use sha2::{Digest, Sha256};
2+
const RULE: &[u8] = include_bytes!("../../../rulepacks/cfquery/distinct_projection.json");
3+
4+
/// 返回独立 CFQuery 结构候选规则的固定字节摘要；不授予数据库方言或规则资格。
5+
#[must_use]
6+
pub fn cfquery_projection_rule_sha256() -> String {
7+
format!("{:x}", Sha256::digest(RULE))
8+
}
9+
10+
/// 核对关键词锚点只包含 SELECT、DISTINCT、FROM 及 SQL 空白/注释；拒绝伪造坐标。
11+
#[must_use]
12+
pub fn cfquery_projection_span_valid(source: &[u8]) -> bool {
13+
let mut cursor = 0;
14+
for (index, word) in [b"SELECT".as_slice(), b"DISTINCT", b"FROM"]
15+
.iter()
16+
.enumerate()
17+
{
18+
if index > 0 {
19+
let before = cursor;
20+
loop {
21+
if source.get(cursor).is_some_and(u8::is_ascii_whitespace) {
22+
cursor += 1;
23+
continue;
24+
}
25+
if source.get(cursor..).is_some_and(|b| b.starts_with(b"--")) {
26+
cursor += 2;
27+
while source.get(cursor).is_some_and(|b| *b != b'\n') {
28+
cursor += 1;
29+
}
30+
continue;
31+
}
32+
if source.get(cursor..).is_some_and(|b| b.starts_with(b"/*")) {
33+
cursor += 2;
34+
let mut depth = 1;
35+
while depth > 0 {
36+
let Some(tail) = source.get(cursor..).filter(|b| !b.is_empty()) else {
37+
return false;
38+
};
39+
if tail.starts_with(b"/*") {
40+
depth += 1;
41+
cursor += 2;
42+
} else if tail.starts_with(b"*/") {
43+
depth -= 1;
44+
cursor += 2;
45+
} else {
46+
cursor += 1;
47+
}
48+
if depth > 64 {
49+
return false;
50+
}
51+
}
52+
continue;
53+
}
54+
break;
55+
}
56+
if before == cursor {
57+
return false;
58+
}
59+
}
60+
if !source
61+
.get(cursor..cursor + word.len())
62+
.is_some_and(|s| s.eq_ignore_ascii_case(word))
63+
{
64+
return false;
65+
}
66+
cursor += word.len();
67+
}
68+
cursor == source.len()
69+
}
70+
71+
#[cfg(test)]
72+
mod tests {
73+
use super::cfquery_projection_span_valid;
74+
#[test]
75+
fn span_does_not_accept_injected_literals_interpolation_or_unclosed_comments() {
76+
for valid in [
77+
"SELECT DISTINCT FROM",
78+
"select\ndistinct\tfrom",
79+
"SELECT DISTINCT /* nested /* x */ y */ FROM",
80+
"SELECT DISTINCT -- x\nFROM",
81+
] {
82+
assert!(cfquery_projection_span_valid(valid.as_bytes()), "{valid}");
83+
}
84+
for invalid in [
85+
"SELECTFROM",
86+
"SELECT DISTINCT 'x' FROM",
87+
"SELECT DISTINCT #x# FROM",
88+
"SELECT DISTINCT /* unterminated FROM",
89+
"SELECT DISTINCT FROM users",
90+
"SELECT DISTINCT \"from\" FROM",
91+
"SELECT DISTINCT\nFROManything",
92+
] {
93+
assert!(
94+
!cfquery_projection_span_valid(invalid.as_bytes()),
95+
"{invalid}"
96+
);
97+
}
98+
}
99+
}

‎crates/codeguard-adapters/src/lib.rs‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -415,3 +415,6 @@ mod shellcheck_json;
415415
pub use shellcheck_diagnostic::ShellCheckDiagnostic;
416416
pub use shellcheck_parsed::ShellCheckParsed;
417417
pub use shellcheck_json::parse_shellcheck_json1;
418+
419+
mod cfquery_projection_rule;
420+
pub use cfquery_projection_rule::{cfquery_projection_rule_sha256, cfquery_projection_span_valid};

‎crates/codeguard-cli/src/check_command.rs‎

Lines changed: 12 additions & 13 deletions
Original file line numberDiff line numberDiff line change
@@ -7,9 +7,7 @@ use std::time::{Duration, Instant};
77

88
use codeguard_adapters::{capability_row, legacy_registry};
99
use codeguard_core::{CHECK_CATEGORIES, TaskGraph, TaskNode};
10-
use codeguard_runtime::{
11-
NativeObservation, SourceSnapshot, TaskExecution, TaskOutcome, run_task_graph,
12-
};
10+
use codeguard_runtime::{NativeObservation, SourceSnapshot, TaskExecution, TaskOutcome, run_task_graph};
1311
use serde_json::{Value, json};
1412
use std::sync::Mutex;
1513
use std::sync::atomic::{AtomicBool, Ordering};
@@ -23,9 +21,7 @@ use crate::discovery::{DiscoveryReport, discover};
2321
use crate::go_lint_command::observe_for_check as observe_go_vet;
2422
use crate::java_checker_config_status::{checker_for_category, summarize};
2523
use crate::java_cve_attribution::attach_candidates;
26-
use crate::java_cve_scan::{
27-
NativeContext as CveNativeContext, observe_project as observe_cve_project,
28-
};
24+
use crate::java_cve_scan::{NativeContext as CveNativeContext, observe_project as observe_cve_project};
2925
use crate::java_dependency_scan::{
3026
NativeContext as DependencyNativeContext, observe_project as observe_dependency_project,
3127
};
@@ -35,9 +31,7 @@ use crate::java_javadoc_scan::{
3531
use crate::java_p3c_scan::{NativeContext, observe_project};
3632
use crate::next_command::{read_local_brief, read_local_brief_for_checker};
3733
use crate::partial_sarif_feedback::partial_check_sarif;
38-
use crate::python_lint_command::{
39-
annotate_conversation_budget, scan_and_sync_report_with_deadline,
40-
};
34+
use crate::python_lint_command::{annotate_conversation_budget, scan_and_sync_report_with_deadline};
4135
use crate::report_export::export_report;
4236
use crate::rust_lint_scan::observe_cargo_clippy_with_coverage;
4337
use crate::work_sync::{save_local_report, sync_local_workspace};
@@ -2014,7 +2008,7 @@ pub fn run(args: &[String]) -> ExitCode {
20142008
crate::check_shell_scan::refresh(&root, &mut shell_lint, deadline);
20152009
}
20162010
let mut report = json!({
2017-
"schema_version":if shell_lint.is_object() || next["schema_version"]=="0.17.0" {"0.52.0"}else if ruby_lint.is_object() || next["schema_version"] == "0.15.0" {"0.51.0"} else if next["schema_version"] == "0.14.0" {"0.50.0"} else if next["schema_version"] == "0.13.0" || syntax_candidates["observations"].as_array().is_some_and(|rows| rows.iter().any(|row| row["language"] == "go" && row.get("structural_observations").is_some())) {"0.49.0"} else if syntax_candidates["observations"].as_array().is_some_and(|rows| rows.iter().any(|row| row.get("structural_observations").is_some())) {"0.48.0"} else if zig_lint["schema_version"] == "0.2.0" || next["schema_version"] == "0.12.0" {"0.47.0"}else if zig_lint.is_object() {"0.46.0"} else if matches!(parsed.selection, Selection::Language(_)) {"0.45.0"} else if swift_lint["schema_version"] == "0.2.0" || next["schema_version"] == "0.11.0" {"0.44.0"} else if swift_lint.is_object() {"0.43.0"} else if kotlin_lint.is_object() || next["schema_version"] == "0.10.0" {"0.42.0"} else if matches!(next["schema_version"].as_str(), Some("0.8.0" | "0.9.0")) {"0.41.0"} else if next["schema_version"] == "0.7.0" {"0.40.0"} else if next["schema_version"] == "0.6.0" {"0.39.0"} else {"0.38.0"}, "report_type":"check_feedback",
2011+
"schema_version":if syntax_candidates["observations"].as_array().is_some_and(|rows|rows.iter().any(|row|row["language"]=="cfquery")) {"0.53.0"}else if shell_lint.is_object() || next["schema_version"]=="0.17.0" {"0.52.0"}else if ruby_lint.is_object() || next["schema_version"] == "0.15.0" {"0.51.0"} else if next["schema_version"] == "0.14.0" {"0.50.0"} else if next["schema_version"] == "0.13.0" || syntax_candidates["observations"].as_array().is_some_and(|rows| rows.iter().any(|row| row["language"] == "go" && row.get("structural_observations").is_some())) {"0.49.0"} else if syntax_candidates["observations"].as_array().is_some_and(|rows| rows.iter().any(|row| row.get("structural_observations").is_some())) {"0.48.0"} else if zig_lint["schema_version"] == "0.2.0" || next["schema_version"] == "0.12.0" {"0.47.0"}else if zig_lint.is_object() {"0.46.0"} else if matches!(parsed.selection, Selection::Language(_)) {"0.45.0"} else if swift_lint["schema_version"] == "0.2.0" || next["schema_version"] == "0.11.0" {"0.44.0"} else if swift_lint.is_object() {"0.43.0"} else if kotlin_lint.is_object() || next["schema_version"] == "0.10.0" {"0.42.0"} else if matches!(next["schema_version"].as_str(), Some("0.8.0" | "0.9.0")) {"0.41.0"} else if next["schema_version"] == "0.7.0" {"0.40.0"} else if next["schema_version"] == "0.6.0" {"0.39.0"} else {"0.38.0"}, "report_type":"check_feedback",
20182012
"operation":"check", "selection":parsed.selection.as_str(), "command_status":if request_cancelled { "cancelled" } else { "incomplete" },
20192013
"exit_code":if request_cancelled { 130 } else { 3 }, "delivery_decision":if parsed.selection == Selection::All { "incomplete" } else { "not_evaluated" }, "authority":"local_unverified",
20202014
"reason":if request_cancelled { "request_cancelled" } else if parsed.selection == Selection::All { "full_project_obligations_and_trusted_policy_unavailable" } else if parsed.selection == Selection::Java { "java_selection_obligations_and_trusted_policy_unavailable" } else { "language_selection_obligations_and_trusted_policy_unavailable" },
@@ -2034,7 +2028,7 @@ pub fn run(args: &[String]) -> ExitCode {
20342028
if zig_lint.is_object()
20352029
|| matches!(
20362030
report["schema_version"].as_str(),
2037-
Some("0.47.0" | "0.48.0" | "0.49.0" | "0.50.0" | "0.51.0" | "0.52.0")
2031+
Some("0.47.0" | "0.48.0" | "0.49.0" | "0.50.0" | "0.51.0" | "0.52.0" | "0.53.0")
20382032
)
20392033
{
20402034
report["native_results"]["zig_lint"] = zig_lint.clone();
@@ -2054,6 +2048,7 @@ pub fn run(args: &[String]) -> ExitCode {
20542048
| "0.50.0"
20552049
| "0.51.0"
20562050
| "0.52.0"
2051+
| "0.53.0"
20572052
)
20582053
) {
20592054
if let Some(native) = report["native_results"].as_object_mut() {
@@ -2073,16 +2068,20 @@ pub fn run(args: &[String]) -> ExitCode {
20732068
| "0.50.0"
20742069
| "0.51.0"
20752070
| "0.52.0"
2071+
| "0.53.0"
20762072
)
20772073
) {
20782074
if let Some(native) = report["native_results"].as_object_mut() {
20792075
native.remove("swift_lint");
20802076
}
20812077
}
2082-
if matches!(report["schema_version"].as_str(), Some("0.51.0" | "0.52.0")) {
2078+
if matches!(
2079+
report["schema_version"].as_str(),
2080+
Some("0.51.0" | "0.52.0" | "0.53.0")
2081+
) {
20832082
report["native_results"]["ruby_lint"] = ruby_lint.clone();
20842083
}
2085-
if report["schema_version"] == "0.52.0" {
2084+
if matches!(report["schema_version"].as_str(), Some("0.52.0" | "0.53.0")) {
20862085
report["native_results"]["shell_lint"] = shell_lint.clone();
20872086
}
20882087
if parsed.format != OutputFormat::Human {

0 commit comments

Comments
 (0)