Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,9 +94,9 @@ cargo build --locked -p codeguard-cli

Version output includes `cli_version`, `target`, and protocol information. Discovery and init preview return observations and unknowns. Dry-run creates neither `AGENTS.md` nor a project data directory. Add Cargo `--offline` only when dependencies are cached.

The workspace requires `std`. The opt-in `wasm-precheck` build feature uses a bounded Rust worker with seven pinned grammar candidates: Java, Objective-C, Python, Solidity, TypeScript, TSX, and Zig. Java and TypeScript/TSX have narrow single-file fallback paths; `lint python` can provide bounded suspected syntax positions when Ruff is unavailable or no project Ruff configuration is declared. Native Ruff results take priority. It is absent from the published npm `0.1.2` package. No `no_std`, completed project-wide WASM fallback, zero-unsafe, or fastest-runtime claim is made. Runtime OS calls require safety review; a comprehensive security audit is not claimed.
The workspace requires `std`. The opt-in `wasm-precheck` build feature uses a bounded Rust worker with eleven pinned grammar candidates: C, Go, Java, JavaScript, Objective-C, Python, Rust, Solidity, TypeScript, TSX, and Zig. Java and TypeScript/TSX have narrow single-file fallback paths; `lint python` can provide bounded suspected syntax positions when Ruff is unavailable or no project Ruff configuration is declared. Native Ruff results take priority. It is absent from the published npm `0.1.2` package. No `no_std`, completed project-wide WASM fallback, zero-unsafe, or fastest-runtime claim is made. Runtime OS calls require safety review; a comprehensive security audit is not claimed.

The current source-built CLI's `codeguard grammar status --format=json` reports the coverage gap without loading grammars or running lint: the pinned CodeGraph source has 30 vendored WASM files plus two distinct grammars resolved through its dependency; CodeGuard has seven unqualified asset candidates (Java, Objective-C, Python, Solidity, TypeScript, TSX, Zig) and zero released syntax capabilities. The two dependency grammars retain their source bytes and use a byte-pinned legacy `dylink` metadata conversion for Rust loading; they have no public lint route or release qualification. Zig uses a byte-pinned import adaptation so the corrected CodeGraph grammar can load in Rust; it has no public `lint zig` route or release qualification yet. [Zig's limited acceptance record](tests/acceptance/zig-grammar-candidate.md) distinguishes this from released syntax support. Python has a source-build-only, unqualified CLI fallback path; its report remains incomplete and cannot approve delivery. The separate [coverage inventory](grammars/codegraph-coverage.json) is not an approved asset manifest. COBOL's 16.4 MB source asset also exceeds the current Rust loader's 8 MiB limit.
The current source-built CLI's `codeguard grammar status --format=json` reports the coverage gap without loading grammars or running lint: the pinned CodeGraph source has 30 vendored WASM files plus two distinct grammars resolved through its dependency; CodeGuard has eleven unqualified asset candidates (C, Go, Java, JavaScript, Objective-C, Python, Rust, Solidity, TypeScript, TSX, Zig) and zero released syntax capabilities. C, Go, JavaScript and Rust are byte-pinned CodeGraph assets that load in the Rust worker but have no public lint route or release qualification. The two dependency grammars retain their source bytes and use a byte-pinned legacy `dylink` metadata conversion for Rust loading; they have no public lint route or release qualification. Zig uses a byte-pinned import adaptation so the corrected CodeGraph grammar can load in Rust; it has no public `lint zig` route or release qualification yet. [Zig's limited acceptance record](tests/acceptance/zig-grammar-candidate.md) distinguishes this from released syntax support. Python has a source-build-only, unqualified CLI fallback path; its report remains incomplete and cannot approve delivery. The separate [coverage inventory](grammars/codegraph-coverage.json) is not an approved asset manifest. COBOL's 16.4 MB source asset also exceeds the current Rust loader's 8 MiB limit.

### npm installation and one-off use

Expand Down
2 changes: 1 addition & 1 deletion README.zh-CN.md
Original file line number Diff line number Diff line change
Expand Up @@ -96,7 +96,7 @@ cargo build --locked -p codeguard-cli

工作区要求 `std`。可选 `wasm-precheck` 构建特性已用固定的 Java、Python、TypeScript、TSX grammar 候选资产与有界 Rust worker。Java 和 TypeScript/TSX 有局部单文件兜底;`lint python` 在 Ruff 不可用或项目未声明 Ruff 配置时可报告有界疑似语法位置,原生 Ruff 结果优先。公开 npm `0.1.2` 不含该路径。不声称 `no_std`、已完成项目级 WASM 兜底、零 unsafe 或性能最快。运行层系统调用需要安全审查,当前不宣称已完成整体安全审计。

当前源码构建的 CLI 可用 `codeguard grammar status --format=json` 只读展示来源覆盖缺口,不加载 grammar 或运行 lint:固定 CodeGraph 来源含 30 份随仓 WASM,另有两种独立 grammar 从其依赖取得;CodeGuard 当前有 Java、Objective-C、Python、Solidity、TypeScript、TSX、Zig 七份未验收资产候选,已发行语法能力为零。Objective-C 与 Solidity 的依赖包原始字节、许可证和固定散列的 `dylink` 元数据转换已入库,可作为 Rust 隔离 worker 的候选观察,但没有公开 lint 路由或发行验收;Zig 也仍是未验收候选。Python 已有仅限源码可选构建的 CLI 候选兜底,结果保持不完整,不能批准交付。[覆盖库存](grammars/codegraph-coverage.json)不是已批准资产清单。COBOL 来源资产约 16.4 MB,还超过当前 Rust 加载器的 8 MiB 限额。
当前源码构建的 CLI 可用 `codeguard grammar status --format=json` 只读展示来源覆盖缺口,不加载 grammar 或运行 lint:固定 CodeGraph 来源含 30 份随仓 WASM,另有两种独立 grammar 从其依赖取得;CodeGuard 当前有 C、Go、Java、JavaScript、Objective-C、Python、Rust、Solidity、TypeScript、TSX、Zig 十一份未验收资产候选,已发行语法能力为零。C、Go、JavaScript、Rust 已作为固定字节的 Rust worker 候选接入,但没有公开 lint 路由或发行验收。Objective-C 与 Solidity 的依赖包原始字节、许可证和固定散列的 `dylink` 元数据转换已入库,可作为 Rust 隔离 worker 的候选观察,但没有公开 lint 路由或发行验收;Zig 也仍是未验收候选。Python 已有仅限源码可选构建的 CLI 候选兜底,结果保持不完整,不能批准交付。[覆盖库存](grammars/codegraph-coverage.json)不是已批准资产清单。COBOL 来源资产约 16.4 MB,还超过当前 Rust 加载器的 8 MiB 限额。

### npm 安装与一次性调用

Expand Down
73 changes: 71 additions & 2 deletions crates/codeguard-adapters/src/grammar_asset_manifest.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,10 @@ const JAVA_COMMIT: &str = "94703d5a6bed02b98e438d7cad1136c01a60ba2c";
const TYPESCRIPT_COMMIT: &str = "f975a621f4e7f532fe322e13c4f79495e0a7b2e7";
const PYTHON_COMMIT: &str = "bffb65a8cfe4e46290331dfef0dbf0ef3679de11";
const ZIG_COMMIT: &str = "b670c8df85a1568f498aa5c8cae42f51a90473c0";
const C_COMMIT: &str = "b780e47fc780ddc8da13afa35a3f4ed5c157823d";
const GO_COMMIT: &str = "3c3775faa968158a8b4ac190a7fda867fd5fb748";
const JAVASCRIPT_COMMIT: &str = "44c892e0be055ac465d5eeddae6d3e194424e7de";
const RUST_COMMIT: &str = "77a3747266f4d621d0757825e6b11edcbf991ca5";

/// 代码来源、许可和每份候选 grammar 的固定身份。
#[derive(Clone, Debug, Deserialize, Eq, PartialEq)]
Expand Down Expand Up @@ -98,6 +102,18 @@ pub fn bundled_grammar_candidates() -> Result<GrammarAssetManifest, String> {
}
for asset in &manifest.assets {
let (wasm, license) = match asset.language.as_str() {
"c" => (
include_bytes!("../../../grammars/c/parser.wasm").as_slice(),
include_bytes!("../../../grammars/c/LICENSE").as_slice(),
),
"go" => (
include_bytes!("../../../grammars/go/parser.wasm").as_slice(),
include_bytes!("../../../grammars/go/LICENSE").as_slice(),
),
"javascript" => (
include_bytes!("../../../grammars/javascript/parser.wasm").as_slice(),
include_bytes!("../../../grammars/javascript/LICENSE").as_slice(),
),
"java" => (
include_bytes!("../../../grammars/java/parser.wasm").as_slice(),
include_bytes!("../../../grammars/java/LICENSE").as_slice(),
Expand Down Expand Up @@ -126,6 +142,10 @@ pub fn bundled_grammar_candidates() -> Result<GrammarAssetManifest, String> {
include_bytes!("../../../grammars/solidity/parser.wasm").as_slice(),
include_bytes!("../../../grammars/solidity/LICENSE").as_slice(),
),
"rust" => (
include_bytes!("../../../grammars/rust/parser.wasm").as_slice(),
include_bytes!("../../../grammars/rust/LICENSE").as_slice(),
),
_ => return Err("候选语言未知".into()),
};
verify_grammar_asset(asset, wasm, license)?;
Expand Down Expand Up @@ -172,7 +192,7 @@ pub fn parse_grammar_asset_manifest(raw: &[u8]) -> Result<GrammarAssetManifest,
|| manifest.dependency_package_license.as_deref() != Some("LICENSE.tree-sitter-wasms")
|| manifest.dependency_package_license_sha256.as_deref()
!= Some("6b0382b16279f26ff69014300541967a356a666eb0b91b422f6862f6b7dad17e")
|| manifest.assets.len() != 7
|| manifest.assets.len() != 11
{
return Err("grammar 清单版本、来源或资产数量不符".into());
}
Expand All @@ -183,9 +203,13 @@ pub fn parse_grammar_asset_manifest(raw: &[u8]) -> Result<GrammarAssetManifest,
.collect();
if languages
!= BTreeSet::from([
"c",
"go",
"java",
"javascript",
"objc",
"python",
"rust",
"solidity",
"typescript",
"tsx",
Expand All @@ -197,6 +221,26 @@ pub fn parse_grammar_asset_manifest(raw: &[u8]) -> Result<GrammarAssetManifest,
for asset in &manifest.assets {
let (repo, commit, version, wasm_path, wasm_sha, wasm_bytes, license_path, license_sha) =
match asset.language.as_str() {
"c" => (
"https://github.com/tree-sitter/tree-sitter-c",
C_COMMIT,
"v0.24.2",
"c/parser.wasm",
"a271e584616c7c3c0ac663f01cd05dd5f1a6c2ce6d4cd23096548985a95d0ccb",
625716,
"c/LICENSE",
"2e0110e07abef7c2548b26ec9d6969775617ca539a0dc8dbeeb14d6452c711d1",
),
"go" => (
"https://github.com/tree-sitter/tree-sitter-go",
GO_COMMIT,
"v0.23.4",
"go/parser.wasm",
"4eda5d91c99ca981e88bc7d3d33f0db166b4bab0a84d0021a9abf39b364c78ef",
210013,
"go/LICENSE",
"2e0110e07abef7c2548b26ec9d6969775617ca539a0dc8dbeeb14d6452c711d1",
),
"java" => (
"https://github.com/tree-sitter/tree-sitter-java",
JAVA_COMMIT,
Expand All @@ -217,6 +261,26 @@ pub fn parse_grammar_asset_manifest(raw: &[u8]) -> Result<GrammarAssetManifest,
"python/LICENSE",
"d724405ce238a22c0d35769c5a36b386ad5958192efe8bbb304fb2896254575f",
),
"javascript" => (
"https://github.com/tree-sitter/tree-sitter-javascript",
JAVASCRIPT_COMMIT,
"v0.25.0",
"javascript/parser.wasm",
"7978e62bcc851ab1d1f6dcd4678f9eda79df2b3b3490e75f81dd819d9bccccfa",
411832,
"javascript/LICENSE",
"2e0110e07abef7c2548b26ec9d6969775617ca539a0dc8dbeeb14d6452c711d1",
),
"rust" => (
"https://github.com/tree-sitter/tree-sitter-rust",
RUST_COMMIT,
"v0.24.2",
"rust/parser.wasm",
"206031e0f67fb41ecae505868ca3bb917df7375031aebafd2f97314a849713fe",
1114303,
"rust/LICENSE",
"31d5b6f4243d5c7c6e1c4ebbbb9f6407bd1457a08bcc4f706521710341acba36",
),
"typescript" => (
"https://github.com/tree-sitter/tree-sitter-typescript",
TYPESCRIPT_COMMIT,
Expand Down Expand Up @@ -295,7 +359,12 @@ pub fn parse_grammar_asset_manifest(raw: &[u8]) -> Result<GrammarAssetManifest,
|| asset.bytes != wasm_bytes
|| asset.license != license_path
|| asset.license_sha256 != license_sha
|| asset.abi_version != if asset.language == "zig" { 15 } else { 14 }
|| asset.abi_version
!= if matches!(asset.language.as_str(), "zig" | "c" | "javascript" | "rust") {
15
} else {
14
}
|| asset.codegraph_runtime != "web-tree-sitter 0.25.3"
|| asset.codeguard_runtime_validation != "rust_loader_smoke_passed"
|| !asset.language_versions.is_empty()
Expand Down
25 changes: 23 additions & 2 deletions crates/codeguard-adapters/tests/grammar_asset_manifest.rs
Original file line number Diff line number Diff line change
Expand Up @@ -6,9 +6,21 @@ use codeguard_adapters::{
#[test]
fn bundled_candidates_pin_source_bytes_license_and_abi_without_claiming_support() {
let manifest = bundled_grammar_candidates().expect("bundled manifest");
assert_eq!(manifest.assets.len(), 7);
assert_eq!(manifest.assets.len(), 11);
for asset in &manifest.assets {
let (wasm, license) = match asset.language.as_str() {
"c" => (
include_bytes!("../../../grammars/c/parser.wasm").as_slice(),
include_bytes!("../../../grammars/c/LICENSE").as_slice(),
),
"go" => (
include_bytes!("../../../grammars/go/parser.wasm").as_slice(),
include_bytes!("../../../grammars/go/LICENSE").as_slice(),
),
"javascript" => (
include_bytes!("../../../grammars/javascript/parser.wasm").as_slice(),
include_bytes!("../../../grammars/javascript/LICENSE").as_slice(),
),
"java" => (
include_bytes!("../../../grammars/java/parser.wasm").as_slice(),
include_bytes!("../../../grammars/java/LICENSE").as_slice(),
Expand Down Expand Up @@ -37,6 +49,10 @@ fn bundled_candidates_pin_source_bytes_license_and_abi_without_claiming_support(
include_bytes!("../../../grammars/solidity/parser.wasm").as_slice(),
include_bytes!("../../../grammars/solidity/LICENSE").as_slice(),
),
"rust" => (
include_bytes!("../../../grammars/rust/parser.wasm").as_slice(),
include_bytes!("../../../grammars/rust/LICENSE").as_slice(),
),
other => panic!("unexpected language {other}"),
};
verify_grammar_asset(asset, wasm, license).expect("pinned candidate bytes");
Expand Down Expand Up @@ -98,7 +114,12 @@ fn missing_provenance_duplicate_fields_and_unknown_languages_are_rejected() {
#[test]
fn changed_wasm_license_or_abi_never_qualifies_a_candidate() {
let manifest = bundled_grammar_candidates().unwrap();
let mut asset = manifest.assets[0].clone();
let mut asset = manifest
.assets
.iter()
.find(|asset| asset.language == "java")
.expect("Java candidate")
.clone();
let wasm = include_bytes!("../../../grammars/java/parser.wasm");
let license = include_bytes!("../../../grammars/java/LICENSE");
let mut changed = wasm.to_vec();
Expand Down
4 changes: 4 additions & 0 deletions crates/codeguard-cli/src/syntax_worker_command.rs
Original file line number Diff line number Diff line change
Expand Up @@ -64,13 +64,17 @@ fn observe(language: &str, source: &[u8]) -> Result<SyntaxWorkerEnvelope, String
return Err("grammar 与当前 Rust WASM 运行时不兼容".into());
}
let wasm = match language {
"c" => include_bytes!("../../../grammars/c/parser.wasm").as_slice(),
"go" => include_bytes!("../../../grammars/go/parser.wasm").as_slice(),
"javascript" => include_bytes!("../../../grammars/javascript/parser.wasm").as_slice(),
"java" => include_bytes!("../../../grammars/java/parser.wasm").as_slice(),
"python" => include_bytes!("../../../grammars/python/parser.wasm").as_slice(),
"typescript" => include_bytes!("../../../grammars/typescript/parser.wasm").as_slice(),
"tsx" => include_bytes!("../../../grammars/tsx/parser.wasm").as_slice(),
"zig" => include_bytes!("../../../grammars/zig/parser.wasm").as_slice(),
"objc" => include_bytes!("../../../grammars/objc/parser.wasm").as_slice(),
"solidity" => include_bytes!("../../../grammars/solidity/parser.wasm").as_slice(),
"rust" => include_bytes!("../../../grammars/rust/parser.wasm").as_slice(),
_ => return Err("不支持的 grammar 语种".into()),
};
let mut grammar = WasmGrammar::load(language, wasm, &asset.sha256, asset.abi_version as usize)?;
Expand Down
6 changes: 5 additions & 1 deletion crates/codeguard-cli/tests/grammar_status_cli.rs
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ fn codegraph_coverage_is_explicit_and_never_claims_parser_or_gate_completion() {
assert_eq!(report["report_type"], "grammar_coverage_inventory");
assert_eq!(report["codegraph_grammar_count"], 32);
assert_eq!(report["codegraph_vendored_count"], 30);
assert_eq!(report["candidate_count"], 7);
assert_eq!(report["candidate_count"], 11);
assert_eq!(report["released_count"], 0);
assert_eq!(report["authority"], "source_inventory_only");
assert_eq!(report["parser_capability"], "unverified");
Expand All @@ -26,9 +26,13 @@ fn codegraph_coverage_is_explicit_and_never_claims_parser_or_gate_completion() {
assert!(!asset["gap"].as_str().unwrap().is_empty());
}
for language in [
"c",
"go",
"java",
"javascript",
"objc",
"python",
"rust",
"solidity",
"typescript",
"tsx",
Expand Down
35 changes: 35 additions & 0 deletions crates/codeguard-cli/tests/syntax_worker_candidate.rs
Original file line number Diff line number Diff line change
Expand Up @@ -98,6 +98,41 @@ fn dependency_grammars_run_in_worker_without_claiming_clean() {
}
}

#[test]
fn mainstream_grammars_run_in_worker_without_claiming_clean() {
for (language, path, source) in [
(
"c",
"src/main.c",
b"int main(void) { return 0; }".as_slice(),
),
(
"go",
"main.go",
b"package main\nfunc main() {}\n".as_slice(),
),
("javascript", "src/main.js", b"const value = 1;".as_slice()),
(
"rust",
"src/main.rs",
b"fn main() { let x = 1; }".as_slice(),
),
] {
let result = run_syntax_worker_candidate(
env!("CARGO_BIN_EXE_codeguard").as_ref(),
language,
path,
source,
Instant::now() + Duration::from_secs(60),
&AtomicBool::new(false),
)
.expect("mainstream grammar worker candidate");
assert!(result.recoveries.is_empty(), "{language}");
assert_eq!(result.precheck.status, SyntaxPrecheckStatus::Incomplete);
assert!(!result.grammar_qualified);
}
}

#[test]
fn invalid_input_and_cancellation_cannot_yield_observations() {
let exe = env!("CARGO_BIN_EXE_codeguard").as_ref();
Expand Down
Loading
Loading