diff --git a/.claude/skills/java-codegen-from-db/SKILL.md b/.claude/skills/java-codegen-from-db/SKILL.md index 1d32ffb..291dde2 100644 --- a/.claude/skills/java-codegen-from-db/SKILL.md +++ b/.claude/skills/java-codegen-from-db/SKILL.md @@ -216,6 +216,11 @@ LLM **必须**把 schema 概述、命名推断和模板推荐呈现给用户， 如果目标文件已存在,生成 `*.codegen.bak` 备份,告诉用户怎么回滚(`mv x.bak x`)。 +### 5.4 释放连接 + +工作流结束或用户决定中止时,调用 **`db_disconnect`** with +`{"connection_id": ""}`。释放后的 ID 不应继续用于查询或生成。 + --- ## 错误处理与重试规则 @@ -252,6 +257,7 @@ LLM **必须**把 schema 概述、命名推断和模板推荐呈现给用户， codegen_render_dto (×1 per 表, 仅 sb35-java21) [写盘] (内置于上述工具,Phase 3 后拆出 file_write_to_project) +[收尾] db_disconnect (会话结束时释放连接) ``` ## 设计原则 diff --git a/.github/workflows/metadata.yml b/.github/workflows/metadata.yml new file mode 100644 index 0000000..abc8d47 --- /dev/null +++ b/.github/workflows/metadata.yml @@ -0,0 +1,21 @@ +name: Repository metadata + +on: + issues: + types: [opened, edited, reopened] + +permissions: + contents: read + +jobs: + issue-policy: + name: Issue title and body policy + runs-on: ubuntu-latest + steps: + - name: Checkout validation script + uses: actions/checkout@v4 + + - name: Validate Issue metadata + env: + ISSUE_EVENT: ${{ github.event_path }} + run: python scripts/validate_commit_title.py --issue-event "$ISSUE_EVENT" diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 0a99cc0..8bd2b69 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -8,9 +8,8 @@ 2. 从 `main` 创建带 Issue 编号的短分支。 3. 小步提交，遵循统一的 Gitmoji + Conventional Commits 标题和正文格式。 4. 本地运行相关测试和 lint/format 检查。 -5. 创建 PR，使用 `Closes #123` 或 `Refs #123` 关联 Issue，并填写 PR 模板的七个固定章节。 -6. 在实现完成和验证完成两个里程碑发布事实回帖，记录输入、环境、命令、原始结果、设计取舍、边界、风险和回滚。 -7. 等待 CI 和审查通过后合并；不要直接向 `main` 推送功能代码。 +5. 创建 PR，关联 Issue，并填写 PR 模板中的测试、实验结果、风险和回滚信息。 +6. 等待 CI 和审查通过后合并；不要直接向 `main` 推送功能代码。 Issue 和 PR 的标题、正文描述尽量使用中文，便于项目协作者审查和追踪。Gitmoji、commit 类型、代码标识、命令和 API 名称保留原文。 @@ -37,17 +36,8 @@ PYTHONPATH=src uv run python scripts/verify_java_compile.py :test_tube: test(generator): 覆盖可空枚举字段 ``` -Issue 正文必须填写对应表单要求的环境/问题、方案、验收标准、测试计划和风险章节。PR 正文固定使用： -`关联 Issue`、`背景（Situation）`、`任务（Task）`、`行动（Action）`、`验证（Verification）`、 -`实验与证据（Evidence）`、`兼容性、风险与回滚`。每个章节都要有实际内容；不要留下 `<...>`、`TODO`、 -连续 `??`、U+FFFD、字面量 `\\n` 或 `\\r`。命令、原始结果和未执行检查必须如实记录，且先脱敏再粘贴。 - -校验器可在本地复用： - -```bash -python scripts/validate_commit_title.py --title ":books: docs(governance): 更新协作规范" -python scripts/validate_commit_title.py --pr-event event.json -python scripts/validate_commit_title.py --issue-event event.json -``` - -提交后不需要轮询 CI；只有准备 squash merge 前，才检查最新提交的全部 required checks。 +Bug Issue 依次写 `版本与环境`、`问题与预期行为`、`最小复现`、`验收标准`、`非目标、风险与安全`； +功能或架构 Issue 使用 `问题与用户价值`、`建议方案与替代方案`、`验收标准`、`架构、兼容性与测试计划`、 +`非目标与风险`。PR 必须按模板中的七个固定章节填写，且每节有实际内容；实现完成和验证完成各发一条 +包含结论、取舍、命令/结果、风险和下一步的回帖。Issue 编辑会触发轻量元数据检查，不能使用空章节、 +连续 `??`、替代字符或字面量 `\\r\\n` 代替 Markdown 换行。 diff --git a/README.es-ES.md b/README.es-ES.md index df3a693..bc514bd 100644 --- a/README.es-ES.md +++ b/README.es-ES.md @@ -86,6 +86,7 @@ En el cliente LLM, di "**Genera código Spring Boot a partir de las tres tablas 5. Llamará a `ai_recommend_template` para recomendar → detectará RBAC y recomendará `MybatisPlus-Mixed` 6. Usará `codegen_build_context` + 5 `codegen_render_*` para generar por capas, devolviendo code-diff en cada capa 7. Tras la confirmación del usuario, escribirá en disco +8. Al finalizar la sesión, llamará a `db_disconnect(connection_id)` para liberar la conexión ## Capacidades principales (Fase 1 → 5) @@ -126,11 +127,11 @@ En el cliente LLM, di "**Genera código Spring Boot a partir de las tres tablas - Logs estructurados: `DBJAVAGENIX_LOG_FORMAT=json` permite salida de JSON en una sola línea, ideal para Loki/ELK - [Manual de despliegue](docs/deployment.md): 3 modos de despliegue + 6 escenarios de troubleshooting -## Resumen de herramientas (33 en total) +## Resumen de herramientas (34 en total) | Categoría | Herramienta | |------|------| -| Conexión / Consulta | db_connect_test / db_query_databases / db_query_tables / db_query_table_exists / db_query_execute | +| Conexión / Consulta | db_connect_test / db_disconnect / db_query_databases / db_query_tables / db_query_table_exists / db_query_execute | | Estructura de tabla | db_table_describe / db_table_columns / db_table_primary_keys / db_table_foreign_keys / db_table_indexes | | Algoritmos de grafo de schema | schema_topo_order / schema_cluster_tables / schema_check_cycles | | Generación de código (atómica) | codegen_build_context / codegen_render_entity / codegen_render_dao / codegen_render_service / codegen_render_controller / codegen_render_dto / codegen_render_mapper | @@ -162,7 +163,7 @@ Consulta [`iteration-plan/01-target-architecture.md`](iteration-plan/01-target-a ``` [ Capa Skills ] Define "cómo hacerlo" — .claude/skills/*.md Flujo de 5 fases explícito ↓ -[ Capa MCP ] Proporciona "qué se puede hacer" — 33 herramientas Contexto transferido explícitamente +[ Capa MCP ] Proporciona "qué se puede hacer" — 34 herramientas Contexto transferido explícitamente ↓ [ Capa Apps ] Hace los resultados "visibles" — 4 componentes de UI (mermaid/dashboard/code-diff/tree) ``` diff --git a/README.ja-JP.md b/README.ja-JP.md index 00c9c9e..62f3482 100644 --- a/README.ja-JP.md +++ b/README.ja-JP.md @@ -85,6 +85,7 @@ LLM クライアントで、例えば **「myapp データベースの sys_user 5. `ai_recommend_template` でテンプレートを推薦し、RBAC を検出して `MybatisPlus-Mixed` を提案する 6. `codegen_build_context` + 5 つの `codegen_render_*` でレイヤーごとに生成し、各レイヤーの code-diff を返す 7. ユーザーの確認後にファイルへ書き込む +8. セッション終了時に `db_disconnect(connection_id)` を呼び出して接続を解放する ## 主な機能（Phase 1 → 5） @@ -130,11 +131,11 @@ LLM クライアントで、例えば **「myapp データベースの sys_user - 構造化ログ: `DBJAVAGENIX_LOG_FORMAT=json` で Loki / ELK に適した 1 行 JSON を出力 - [デプロイガイド](docs/deployment.md): 3 つのデプロイ方式 + 6 つのトラブルシューティング事例 -## ツール一覧（33 個） +## ツール一覧（34 個） | カテゴリ | ツール | |------|------| -| 接続 / クエリ | db_connect_test / db_query_databases / db_query_tables / db_query_table_exists / db_query_execute | +| 接続 / クエリ | db_connect_test / db_disconnect / db_query_databases / db_query_tables / db_query_table_exists / db_query_execute | | テーブル構造 | db_table_describe / db_table_columns / db_table_primary_keys / db_table_foreign_keys / db_table_indexes | | スキーマグラフアルゴリズム | schema_topo_order / schema_cluster_tables / schema_check_cycles | | コード生成（アトミック） | codegen_build_context / codegen_render_entity / codegen_render_dao / codegen_render_service / codegen_render_controller / codegen_render_dto / codegen_render_mapper | @@ -166,7 +167,7 @@ LLM クライアントで、例えば **「myapp データベースの sys_user ``` [ Skills 層 ] 「方法」を定義 — .claude/skills/*.md 明示的な 5 段階ワークフロー ↓ -[ MCP 層 ] 「できること」を提供 — 33 ツール context を明示的に受け渡し +[ MCP 層 ] 「できること」を提供 — 34 ツール context を明示的に受け渡し ↓ [ Apps 層 ] 結果を「見える化」 — 4 つの UI コンポーネント（mermaid/dashboard/code-diff/tree） ``` diff --git a/README.md b/README.md index 33ae042..44d85ae 100644 --- a/README.md +++ b/README.md @@ -13,7 +13,7 @@ graph LR Skills[".claude/skills/
java-codegen-from-db
springboot-migration"] Skills -->|按需调用| MCP - subgraph MCP[MCP Server 33 工具] + subgraph MCP[MCP Server 34 工具] direction TB DB[db_* 连接 / 查询 / 描述] Atom[codegen_build_context
codegen_render_entity/dao/service/
controller/dto/mapper] @@ -90,6 +90,7 @@ SQL Server 的类型映射保留为后续扩展准备，但尚未实现运行时 5. 调用 `ai_recommend_template` 推荐 → 检测到 RBAC,推 `MybatisPlus-Mixed` 6. 用 `codegen_build_context` + 6 个 `codegen_render_*` 分层生成,每层返回 code-diff 7. 用户确认后写盘 +8. 会话结束时调用 `db_disconnect(connection_id)` 释放连接 ## 核心能力 (Phase 1 → 5) @@ -130,11 +131,11 @@ SQL Server 的类型映射保留为后续扩展准备，但尚未实现运行时 - 结构化日志: `DBJAVAGENIX_LOG_FORMAT=json` 可输出单行 JSON,适合 Loki/ELK - [部署手册](docs/deployment.md): 3 种部署模式 + 6 个排障场景 -## 工具总览 (33 个) +## 工具总览 (34 个) | 类别 | 工具 | |------|------| -| 连接 / 查询 | db_connect_test / db_query_databases / db_query_tables / db_query_table_exists / db_query_execute | +| 连接 / 查询 | db_connect_test / db_disconnect / db_query_databases / db_query_tables / db_query_table_exists / db_query_execute | | 表结构 | db_table_describe / db_table_columns / db_table_primary_keys / db_table_foreign_keys / db_table_indexes | | Schema 图算法 | schema_topo_order / schema_cluster_tables / schema_check_cycles | | 代码生成 (atomic) | codegen_build_context / codegen_render_entity / codegen_render_dao / codegen_render_service / codegen_render_controller / codegen_render_dto / codegen_render_mapper | @@ -166,7 +167,7 @@ SQL Server 的类型映射保留为后续扩展准备，但尚未实现运行时 ``` [ Skills 层 ] 定义"怎么做" — .claude/skills/*.md 显式 5 阶段工作流 ↓ -[ MCP 层 ] 提供"能做什么" — 33 个原子工具 context 显式传递 +[ MCP 层 ] 提供"能做什么" — 34 个原子工具 context 显式传递 ↓ [ Apps 层 ] 让结果"看得见" — 4 个 UI 组件 (mermaid/dashboard/code-diff/tree) ``` @@ -187,7 +188,7 @@ SQL Server 的类型映射保留为后续扩展准备，但尚未实现运行时 | [docs/screenshots/README.md](docs/screenshots/README.md) | MCP Apps 4 组件客户端兼容性 | | [docs/algorithms-overview.md](docs/algorithms-overview.md) | v0.2.1 schema 图算法 (topo / cluster / cycle) | | [docs/design-patterns-catalog.md](docs/design-patterns-catalog.md) | 生成器与生成代码中的设计模式 | -| [docs/adr/](docs/adr/) | 14 个 ADR (架构 / 原子 / 渐进 / 规则 / 不引依赖 / schema 算法 / 规范配置 / MCP v3 / 1h 缓存 / agentic / 多方言 / SDK 契约 / 工具契约 / 元数据契约) | +| [docs/adr/](docs/adr/) | 15 个 ADR (架构 / 原子 / 渐进 / 规则 / 不引依赖 / schema 算法 / 规范配置 / MCP v3 / 1h 缓存 / agentic / 多方言 / SDK 契约 / 工具契约 / 元数据契约 / 连接生命周期) | | [.claude/skills/java-codegen-from-db/SKILL.md](.claude/skills/java-codegen-from-db/SKILL.md) | 主 Skill: 代码生成 5 阶段工作流 | | [.claude/skills/springboot-migration/SKILL.md](.claude/skills/springboot-migration/SKILL.md) | 第二 Skill: Spring Boot 2.7→3.x 迁移 | diff --git a/docs/adr/011-multi-dialect-strategy.md b/docs/adr/011-multi-dialect-strategy.md index af7f42d..5ffd997 100644 --- a/docs/adr/011-multi-dialect-strategy.md +++ b/docs/adr/011-multi-dialect-strategy.md @@ -137,13 +137,14 @@ postgresql: - 36 个 unit test 覆盖 mysql/postgres 两套映射,跨方言隔离测试防串 - D3 用真实 PG container 验证 information_schema 上报的字符串确实命中 我们的 key (这是配置驱动方案做不到的) -- `template_context.py` 之后会渐进迁移到调用 `get_dialect(...).java_type_for()`, - 本 ADR 不强制一次性切换 +- `template_context.py`、MCP `db_table_describe` 已统一调用 + `get_dialect(...).java_type_for()`；未注册运行时方言仍保留旧 YAML 回退，避免把 + SQL Server/Oracle 的配置映射误报为连接能力 **坏**: - `dialect.py` 文件略大 (~300 行,主要是两张映射表),但都是数据 -- `template_context.py` 现在有重复的 MySQL 映射,**v0.3.1 计划重构** — 不在这个 - ADR 范围内,先把 PG 跑起来,old code 保留向后兼容 +- MCP 描述工具对已注册方言不再维护独立的 Java 类型表；Java 类型对应 imports + 也由 `DialectAdapter.java_imports_for()` 统一提供 **实测**: - D3 PG 16 实测 23 个 PG 类型全部命中预期 Java 类型 diff --git a/docs/adr/015-connection-lifecycle.md b/docs/adr/015-connection-lifecycle.md new file mode 100644 index 0000000..ef0374b --- /dev/null +++ b/docs/adr/015-connection-lifecycle.md @@ -0,0 +1,44 @@ +# ADR-015: MCP 连接生命周期与显式释放 + +- **状态**: Accepted +- **日期**: 2026-09-09 +- **关联 Issue**: #209 +- **关联实现**: `src/dbjavagenix/database/mcp_tools.py` + +## 背景 + +`ConnectionManager` 已经能够按 connection ID 关闭连接，但公共 MCP 合同只 +暴露 `db_connect_test`。长生命周期客户端无法在探索完成、重试失败或切换数据库 +时主动释放连接，连接对象和脱敏配置会一直保留到 server 进程退出。 + +## 决定 + +新增 `db_disconnect(connection_id)` 作为显式连接生命周期工具： + +1. handler 只负责校验参数、调用 `ConnectionManager.close_connection()` 和序列化 + 响应，不复制连接状态或驱动逻辑。 +2. 成功关闭返回 `success=true`；未知、空值和重复关闭统一返回 + `success=false`、`error=connection_not_found`。 +3. 关闭异常返回 `disconnect_failed`，异常文本经过现有凭据脱敏边界后再返回。 +4. 工具加入 canonical MCP 列表和 progressive discovery 元数据；默认 progressive + 可见集合不变，客户端可通过 `search_tools("disconnect")` 发现它。 +5. 客户端在会话结束时调用该工具；不引入连接池、TTL、后台回收或自动关闭策略。 + +## 备选方案 + +- 仅依赖进程退出时的 `__del__`：无法覆盖长会话中的提前释放和异常重试，已拒绝。 +- 增加后台 TTL：可能误杀仍在使用的连接，并引入不可预测的时序，已拒绝。 +- 在 handler 内直接操作驱动连接：会重复生命周期逻辑，绕过 `ConnectionManager`，已拒绝。 + +## 影响 + +- 连接建立和查询 API 保持兼容，旧客户端无需立即改造。 +- 释放后的 connection ID 不可继续使用；后续查询收到既有的连接不存在错误。 +- 显式释放降低长生命周期会话的资源占用，但不承诺性能收益或自动回收。 + +## 验证 + +- 单元测试覆盖工具 schema、成功关闭后的连接与配置清理、未知/空值/重复 ID、 + 异常脱敏以及 canonical server dispatch。 +- 使用 SQLite in-memory manager 和 monkeypatch 验证，不伪造真实 MySQL/PostgreSQL + 连接释放结果。 diff --git a/docs/adr/016-async-database-boundary.md b/docs/adr/016-async-database-boundary.md new file mode 100644 index 0000000..d8a44be --- /dev/null +++ b/docs/adr/016-async-database-boundary.md @@ -0,0 +1,45 @@ +# ADR-016: 将同步数据库调用移出 MCP 事件循环 + +- **状态**: Accepted +- **日期**: 2026-09-09 +- **关联 Issue**: #211 +- **关联实现**: `src/dbjavagenix/database/connection_manager.py` + +## 背景 + +MCP handler 使用 `async def`，但数据库驱动和元数据 introspector 仍是同步实现。 +网络等待或大型 SQLite 查询会占住事件循环，使同一会话的健康检查、取消和其他连接请求 +延迟。SQLite 默认还会拒绝跨线程使用连接；多个 worker 直接共享一个连接也可能交叉使用 +cursor。 + +## 决定 + +1. 在 MCP 数据库边界统一使用 `asyncio.to_thread`；保留同步 `ConnectionManager` API， + 供 CLI、测试和外部调用方继续使用。 +2. `ConnectionManager` 为每个 connection ID 建立可重入锁，锁覆盖健康探测、cursor + 生命周期、SQL 执行和关闭；不同连接不共享这把锁。 +3. SQLite 连接启用 `check_same_thread=False`，但只有在上述连接锁保护下交给 worker。 +4. 异步 analyzer 通过 worker 中的短生命周期 event loop 执行，输入输出合同保持不变。 +5. 取消异步 handler 不强杀底层驱动调用；worker 返回后由 context manager 关闭 cursor， + 连接仍可被显式 `db_disconnect` 释放。 + +## 备选方案 + +- 替换原生异步 MySQL/PostgreSQL 驱动：迁移成本高且会改变方言、事务和依赖边界，暂不采用。 +- 只在 handler 外包线程、不加连接锁：会留下 SQLite thread-affinity 和 cursor 交叉风险， + 不采用。 +- 为每个请求建立新连接：增加连接开销并破坏现有 connection ID 生命周期，不采用。 + +## 影响 + +- 同步公共 API、SQL、事务提交/回滚、权限和 MCP 文本/Raw Response 保持不变。 +- 同一连接的数据库阶段串行，不同连接可以并行等待；不宣称生产吞吐提升。 +- worker 调度带来少量开销；驱动额外的线程绑定限制需要在真实 MySQL/PostgreSQL 环境 + 单独验证。 + +## 验证 + +- 受控 60ms 阻塞 driver + heartbeat 证明事件循环仍获调度。 +- fake cursor 实验覆盖同连接最大并发为 1、跨连接同时进入和 close 等待 SQL body 完成。 +- SQLite in-memory worker 查询和现有 SQLite MCP/codegen 合同继续通过；真实网络数据库 + 线程模型不在本地实验范围内。 diff --git a/docs/adr/README.md b/docs/adr/README.md index 68c5458..03e9c46 100644 --- a/docs/adr/README.md +++ b/docs/adr/README.md @@ -25,3 +25,5 @@ ### v0.3 (数据库元数据契约) - [ADR-014: 统一数据库元数据描述契约](014-metadata-introspection-contract.md) +- [ADR-015: MCP 连接生命周期与显式释放](015-connection-lifecycle.md) +- [ADR-016: 将同步数据库调用移出 MCP 事件循环](016-async-database-boundary.md) diff --git a/docs/algorithms-overview.md b/docs/algorithms-overview.md index 09b7c7a..1d283ae 100644 --- a/docs/algorithms-overview.md +++ b/docs/algorithms-overview.md @@ -138,6 +138,13 @@ CycleResult( - 多解时按字典序选择 - 这让单测可以严格断言 +### 输入规范化 + +MCP 包装和三个纯算法共享同一输入边界：只接受数组中的非空字符串，表名和 +`[child, parent]` 外键边按首次出现去重；非法容器、空值和非二元外键被忽略。 +因此 `users, users` 只会在结果中出现一次，重复外键也不会影响入度、聚类或环 +检测。外部表引用和自引用仍按各算法原有规则处理。 + ### 无外部依赖 只用 `collections` 和 `dataclasses` (Python stdlib)。 对比:如果用 networkx,镜像大小 +10MB,启动延迟 +1-2 秒。 diff --git a/docs/demo-script.md b/docs/demo-script.md index f4a65cf..4fbe7ec 100644 --- a/docs/demo-script.md +++ b/docs/demo-script.md @@ -49,7 +49,7 @@ ``` **说**: -> "33 个工具就绪,modules 全部 OK。注意 progressive 模式 — 初始只暴露 6 个 always-visible 工具,启动 token 从 3300 降到 985。" +> "34 个工具就绪,modules 全部 OK。注意 progressive 模式 — 初始只暴露 6 个 always-visible 工具,启动 token 从 3300 降到 985。" ### [0:45] 任务输入 (10 秒) @@ -99,7 +99,7 @@ **做**: `server_metrics` 看一下累积调用。 **说**: -> "整个流程透明可观测,33 个工具,5 阶段 Skill 编排,4 个 MCP App 组件。代码在 [github.com/ZhaoXingPeng/DBJavaGenix](https://github.com/ZhaoXingPeng/DBJavaGenix),完整迭代方案在 iteration-plan/ 目录。" +> "整个流程透明可观测,34 个工具,5 阶段 Skill 编排,4 个 MCP App 组件。会话结束后调用 db_disconnect 释放连接。代码在 [github.com/ZhaoXingPeng/DBJavaGenix](https://github.com/ZhaoXingPeng/DBJavaGenix),完整迭代方案在 iteration-plan/ 目录。" --- @@ -116,7 +116,7 @@ 打开 README.md 的 mermaid 架构图,讲三层: - **Skills**: `.claude/skills/java-codegen-from-db/SKILL.md` — 看一眼"5 阶段工作流" -- **MCP 33 工具**: 强调 atomic 拆分 (db_codegen_generate → 7 个原子工具) 与 schema 图算法 +- **MCP 34 工具**: 强调 atomic 拆分 (db_codegen_generate → 7 个原子工具) 与 schema 图算法,以及显式 db_disconnect 生命周期 - **Apps**: 4 个 UI 组件,客户端按 _meta 渲染 提一句: diff --git a/docs/dependency-management.md b/docs/dependency-management.md index b0a6e98..d5a8e1e 100644 --- a/docs/dependency-management.md +++ b/docs/dependency-management.md @@ -98,6 +98,7 @@ dbjavagenix migration-guide /path/to/project - **用户优先**: 优先使用用户现有配置风格 - **版本兼容**: 确保依赖版本间的兼容性 - **最小干预**: 只在必要时添加或修改依赖 +- **调用隔离**: 每次分析从默认依赖目录开始，Spring Boot 版本调整不会泄漏到后续项目 ### 3. 自动修复机制 - Maven 项目: 自动修改 pom.xml 文件 diff --git a/docs/engineering-standards.md b/docs/engineering-standards.md index ca0e4d4..597c322 100644 --- a/docs/engineering-standards.md +++ b/docs/engineering-standards.md @@ -65,9 +65,11 @@ Issue 可以先描述问题，但同样必须使用标题语法，不能使用 ` - 推送前修正本分支中错误的未合并提交；已经合并的历史不强制重写，以后续、单独的维护提交 覆盖其文件元数据。 -CI 校验 PR 标题、PR 内全部提交标题和 PR 正文的必备章节；`scripts/validate_commit_title.py` -还提供 Issue 正文校验入口，供本地审计或后续 Issue workflow 复用。校验器只检查结构和编码信号， -不会把测试或性能结论当成已经证明的事实；审查者仍须检查命令、输出和声明是否匹配。 +CI 校验 PR 标题、PR 内全部提交标题和 PR 正文的必备章节、顺序和非空内容。独立的 +`Repository metadata` workflow 会在 Issue 创建、编辑或重新打开时校验标题和对应正文模板； +它不触发数据库、Docker 或 Java 构建。两个门禁都会拒绝连续 `??`、Unicode 替代字符、控制 +字符和将换行错误写成字面量转义的正文。它们只检查结构和编码信号，不会把测试或性能结论当成 +已经证明的事实；审查者仍须检查命令、输出和声明是否匹配。 ## 3. Issue 要求 @@ -94,10 +96,14 @@ Bug Issue 正文按以下顺序填写： Issue 在关闭前应由关联 PR 的 `Closes #` 自动关闭；仅作研究、路线图或尚未完成的任务用 `Refs #`，并在评论中说明状态而不是提前关闭。 +Issue 创建、编辑或重新打开后会触发轻量元数据门禁。失败时先修复标题、补齐相应固定章节，并将 +实际换行替换掉字面量 `\\r\\n`/`\\n`；不要通过删除中文、把内容改成 `?` 或关闭 Issue 绕过校验。 +Issue 表单会生成三级标题，手工 Markdown 使用二级标题，两种格式都受同一章节合同约束。 + ## 4. PR 要求 -PR 标题必须与提交使用同一套格式。PR 正文必须按下列固定标题填写，CI 会检查这些章节和 -`Closes #` 或 `Refs #` 关联： +PR 标题必须与提交使用同一套格式。PR 正文必须按下列固定标题填写，CI 会检查这些章节的精确名称、 +顺序、非空内容，以及 `Closes #` 或 `Refs #` 关联： 1. `关联 Issue`：关闭或引用的编号及其验收目标。 2. `背景（Situation）`：现有行为、受影响用户和问题根因。 diff --git a/scripts/validate_commit_title.py b/scripts/validate_commit_title.py index c81eacf..8bccec1 100644 --- a/scripts/validate_commit_title.py +++ b/scripts/validate_commit_title.py @@ -30,55 +30,30 @@ ) ISSUE_REFERENCE_PATTERN = re.compile(r"\b(?:Closes|Refs)\s+#\d+\b", re.IGNORECASE) REQUIRED_PR_SECTIONS = ( - "关联 Issue", - "背景（Situation）", - "任务（Task）", - "行动（Action）", - "验证（Verification）", - "实验与证据（Evidence）", - "兼容性、风险与回滚", + "## 关联 Issue", + "## 背景（Situation）", + "## 任务（Task）", + "## 行动（Action）", + "## 验证（Verification）", + "## 实验与证据（Evidence）", + "## 兼容性、风险与回滚", ) -ISSUE_SECTION_SETS = ( - ( - "版本与环境", - "问题与预期行为", - "最小复现", - "验收标准", - "非目标、风险与安全", - ), - ( - "问题与用户价值", - "建议方案与替代方案", - "验收标准", - "架构、兼容性与测试计划", - "非目标与风险", - ), - ( - "问题与用户价值", - "建议方案与非目标", - "验收标准", - "架构、兼容性与测试计划", - "非目标与风险", - ), - ( - "环境信息", - "问题与预期行为", - "复现步骤", - "验收标准", - "非目标、风险与安全信息", - ), +ISSUE_BUG_SECTIONS = ( + "版本与环境", + "问题与预期行为", + "最小复现", + "验收标准", + "非目标、风险与安全", ) -HEADING_PATTERN = re.compile(r"(?m)^#{2,6}\s+(?P[^\r\n#]+?)\s*$") -PLACEHOLDER_PATTERN = re.compile( - r"<\s*(?:[^>\r\n]{1,80})\s*>|\b(?:TODO|TBD|FIXME)\b|请填写|待填写|按模块列出", - re.IGNORECASE, -) -CONTROL_CHARACTER_PATTERN = re.compile(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]") -PSEUDO_NEWLINE_PATTERN = re.compile(r"\\[nr]") -GENERIC_TITLE_PATTERN = re.compile( - r"^(?:config|init|\.gitignore|initial commit|请填写(?:标题|摘要)?)$", - re.IGNORECASE, +ISSUE_FEATURE_SECTIONS = ( + "问题与用户价值", + "建议方案与替代方案", + "验收标准", + "架构、兼容性与测试计划", + "非目标与风险", ) +MARKDOWN_HEADING_PATTERN = re.compile(r"(?m)^#{2,3}\s+(?P<title>[^\r\n]+?)\s*$") +LITERAL_ESCAPE_PATTERN = re.compile(r"\\(?:r|t)(?![A-Za-z0-9_])|\\n\s*#{2,3}\s+") def validate_title(title: str) -> list[str]: @@ -87,18 +62,8 @@ def validate_title(title: str) -> list[str]: normalized = title.rstrip("\r\n") if len(normalized) > 72: errors.append("title exceeds 72 characters") - if "\ufffd" in normalized: - errors.append("title contains the Unicode replacement character; check UTF-8 encoding") if "??" in normalized: errors.append("title contains consecutive '?' characters; check UTF-8 encoding") - if PSEUDO_NEWLINE_PATTERN.search(normalized): - errors.append("title contains a literal \\n or \\r escape; use real line structure") - if CONTROL_CHARACTER_PATTERN.search(normalized): - errors.append("title contains a control character") - if GENERIC_TITLE_PATTERN.fullmatch(normalized.strip()) or PLACEHOLDER_PATTERN.search( - normalized - ): - errors.append("title contains a generic or unreplaced placeholder subject") match = TITLE_PATTERN.fullmatch(normalized) if not match: errors.append("expected ':gitmoji: type(scope): imperative subject'") @@ -108,74 +73,78 @@ def validate_title(title: str) -> list[str]: errors.append(f"unsupported Gitmoji {match.group('emoji')}") elif match.group("type") != expected_type: errors.append(f"Gitmoji {match.group('emoji')} must use type {expected_type}") - if normalized.endswith(("。", ".")): - errors.append("title must not end with sentence punctuation") return errors -def validate_pr_body(body: str | None) -> list[str]: - """Return structural and encoding-policy violations for a PR body.""" - normalized = body or "" +def _validate_encoding(text: str, label: str) -> list[str]: errors: list[str] = [] - errors.extend(_validate_text_quality(normalized, "PR body")) - content = _without_comments(normalized) - if not ISSUE_REFERENCE_PATTERN.search(content): - errors.append("PR body must contain 'Closes #<number>' or 'Refs #<number>'") - _validate_sections(content, REQUIRED_PR_SECTIONS, "PR body", errors) + if "??" in text: + errors.append(f"{label} contains consecutive '?' characters; check UTF-8 encoding") + if "\ufffd" in text: + errors.append(f"{label} contains Unicode replacement characters; check UTF-8 encoding") + controls = sorted({ord(char) for char in text if ord(char) < 32 and char not in "\r\n\t"}) + if controls: + codes = ", ".join(f"U+{code:04X}" for code in controls) + errors.append(f"{label} contains control characters: {codes}") + if LITERAL_ESCAPE_PATTERN.search(text): + errors.append(f"{label} contains literal escape sequences; use real Markdown line breaks") return errors -def validate_issue_body(body: str | None) -> list[str]: - """Return structural and encoding-policy violations for an Issue body.""" - normalized = body or "" - errors = _validate_text_quality(normalized, "Issue body") - content = _without_comments(normalized) - headings = {match.group("title").strip() for match in HEADING_PATTERN.finditer(content)} - matching_schema = next( - (schema for schema in ISSUE_SECTION_SETS if set(schema) <= headings), None - ) - if matching_schema is None: - errors.append("Issue body does not contain a complete bug or feature section set") - return errors - _validate_sections(content, matching_schema, "Issue body", errors) +def _validate_sections(text: str, sections: tuple[str, ...], label: str) -> list[str]: + matches = list(MARKDOWN_HEADING_PATTERN.finditer(text)) + positions: list[tuple[int, str, re.Match[str]]] = [] + errors: list[str] = [] + for section in sections: + match = next( + (item for item in matches if item.group("title") == section), + None, + ) + if match is None: + errors.append(f"{label} is missing required section: {section}") + continue + positions.append((match.start(), section, match)) + + if len(positions) == len(sections): + ordered = [section for _, section, _ in sorted(positions)] + if ordered != list(sections): + errors.append(f"{label} sections are out of order") + + for index, (_, section, match) in enumerate(sorted(positions)): + next_start = ( + sorted(positions)[index + 1][0] if index + 1 < len(positions) else len(text) + ) + content = text[match.end() : next_start] + content = re.sub(r"<!--.*?-->", "", content, flags=re.DOTALL).strip() + if not content: + errors.append(f"{label} section is empty: {section}") return errors -def _without_comments(text: str) -> str: - return re.sub(r"<!--.*?-->", "", text, flags=re.DOTALL) - - -def _validate_text_quality(text: str, label: str) -> list[str]: - errors: list[str] = [] - if "\ufffd" in text: - errors.append(f"{label} contains the Unicode replacement character; check UTF-8 encoding") - if "??" in text: - errors.append(f"{label} contains consecutive '?' characters; check UTF-8 encoding") - if PSEUDO_NEWLINE_PATTERN.search(text): - errors.append(f"{label} contains literal \\n or \\r escapes; use real line breaks") - if CONTROL_CHARACTER_PATTERN.search(text): - errors.append(f"{label} contains a control character") - if PLACEHOLDER_PATTERN.search(_without_comments(text)): - errors.append(f"{label} contains an unreplaced template placeholder") +def validate_pr_body(body: str | None) -> list[str]: + """Return structural and encoding-policy violations for a PR body.""" + normalized = body or "" + errors = _validate_encoding(normalized, "PR body") + if not ISSUE_REFERENCE_PATTERN.search(normalized): + errors.append("PR body must contain 'Closes #<number>' or 'Refs #<number>'") + errors.extend( + _validate_sections( + normalized, + tuple(section.removeprefix("## ") for section in REQUIRED_PR_SECTIONS), + "PR body", + ) + ) return errors -def _validate_sections(text: str, sections: Iterable[str], label: str, errors: list[str]) -> None: - headings = list(HEADING_PATTERN.finditer(text)) - positions = {match.group("title").strip(): match for match in headings} - previous_position = -1 - for section in sections: - match = positions.get(section) - if match is None: - errors.append(f"{label} is missing required section: ## {section}") - continue - if match.start() < previous_position: - errors.append(f"{label} sections are out of order: ## {section}") - previous_position = match.start() - next_heading = next((item for item in headings if item.start() > match.start()), None) - section_body = text[match.end() : next_heading.start() if next_heading else None].strip() - if not section_body or PLACEHOLDER_PATTERN.search(section_body): - errors.append(f"{label} section is empty: ## {section}") +def validate_issue_body(body: str | None, title: str | None = None) -> list[str]: + """Return structural and encoding-policy violations for one Issue body.""" + normalized = body or "" + errors = _validate_encoding(normalized, "Issue body") + is_bug = bool(re.search(r"\bfix\([a-z0-9][a-z0-9._/-]*\)", title or "")) + sections = ISSUE_BUG_SECTIONS if is_bug else ISSUE_FEATURE_SECTIONS + errors.extend(_validate_sections(normalized, sections, "Issue body")) + return errors def _titles_from_range(rev_range: str) -> list[str]: @@ -213,7 +182,7 @@ def main(argv: list[str] | None = None) -> int: ) parser.add_argument( "--issue-event", - help="GitHub issues event payload used to validate the Issue body.", + help="GitHub issues event payload used to validate the Issue title and body.", ) args = parser.parse_args(argv) titles = list(_iter_titles(args)) @@ -244,7 +213,17 @@ def main(argv: list[str] | None = None) -> int: if args.issue_event: with open(args.issue_event, encoding="utf-8") as event_file: event = json.load(event_file) - body_errors = validate_issue_body(event.get("issue", {}).get("body")) + issue = event.get("issue", {}) + issue_title = issue.get("title", "") + title_errors = validate_title(issue_title) + if title_errors: + failures += 1 + print("INVALID: issue title", file=sys.stderr) + for error in title_errors: + print(f" - {error}", file=sys.stderr) + else: + print("OK: issue title") + body_errors = validate_issue_body(issue.get("body"), issue_title) if body_errors: failures += 1 print("INVALID: issue body", file=sys.stderr) diff --git a/src/dbjavagenix/algorithms/__init__.py b/src/dbjavagenix/algorithms/__init__.py index 6a64fcd..31e78de 100644 --- a/src/dbjavagenix/algorithms/__init__.py +++ b/src/dbjavagenix/algorithms/__init__.py @@ -9,6 +9,7 @@ from .schema_topo import TopoResult, topological_sort from .schema_cluster import ClusterResult, cluster_tables from .schema_cycle_check import CycleResult, find_cycles +from .graph_input import normalize_graph_input __all__ = [ "TopoResult", @@ -17,4 +18,5 @@ "cluster_tables", "CycleResult", "find_cycles", + "normalize_graph_input", ] diff --git a/src/dbjavagenix/algorithms/graph_input.py b/src/dbjavagenix/algorithms/graph_input.py new file mode 100644 index 0000000..e14756c --- /dev/null +++ b/src/dbjavagenix/algorithms/graph_input.py @@ -0,0 +1,45 @@ +"""Normalize schema graph inputs shared by all graph algorithms.""" + +from __future__ import annotations + +from typing import Any + + +def normalize_graph_input(tables: Any, fks: Any) -> tuple[list[str], list[tuple[str, str]]]: + """Return a deterministic, duplicate-free graph input. + + The MCP schema describes both values as arrays of strings, but direct + callers and LLM payloads can still provide malformed values. Keep the + existing tolerant contract by ignoring invalid entries while preventing + duplicate nodes and edges from skewing algorithm results. + """ + normalized_tables: list[str] = [] + seen_tables: set[str] = set() + if isinstance(tables, (list, tuple)): + for table in tables: + if not isinstance(table, str) or not table.strip() or table in seen_tables: + continue + seen_tables.add(table) + normalized_tables.append(table) + + normalized_fks: list[tuple[str, str]] = [] + seen_fks: set[tuple[str, str]] = set() + if isinstance(fks, (list, tuple)): + for fk in fks: + if not isinstance(fk, (list, tuple)) or len(fk) != 2: + continue + child, parent = fk + if ( + not isinstance(child, str) + or not isinstance(parent, str) + or not child.strip() + or not parent.strip() + ): + continue + edge = (child, parent) + if edge in seen_fks: + continue + seen_fks.add(edge) + normalized_fks.append(edge) + + return normalized_tables, normalized_fks diff --git a/src/dbjavagenix/algorithms/schema_cluster.py b/src/dbjavagenix/algorithms/schema_cluster.py index 4e3fd95..2d0620b 100644 --- a/src/dbjavagenix/algorithms/schema_cluster.py +++ b/src/dbjavagenix/algorithms/schema_cluster.py @@ -22,6 +22,8 @@ from collections import defaultdict from dataclasses import dataclass, field +from .graph_input import normalize_graph_input + @dataclass class ClusterResult: @@ -67,10 +69,9 @@ def union(self, x: str, y: str) -> None: self._rank[px] += 1 -def cluster_tables( - tables: list[str], fks: list[tuple[str, str]] -) -> ClusterResult: +def cluster_tables(tables: list[str], fks: list[tuple[str, str]]) -> ClusterResult: """Cluster tables by FK connectivity (FK graph treated as undirected).""" + tables, fks = normalize_graph_input(tables, fks) uf = _UnionFind(tables) table_set = set(tables) for child, parent in fks: diff --git a/src/dbjavagenix/algorithms/schema_cycle_check.py b/src/dbjavagenix/algorithms/schema_cycle_check.py index 9038279..b375294 100644 --- a/src/dbjavagenix/algorithms/schema_cycle_check.py +++ b/src/dbjavagenix/algorithms/schema_cycle_check.py @@ -22,6 +22,8 @@ from dataclasses import dataclass, field +from .graph_input import normalize_graph_input + _WHITE, _GRAY, _BLACK = 0, 1, 2 @@ -44,9 +46,7 @@ def safe(self) -> bool: return len(self.cycles) == 0 -def find_cycles( - tables: list[str], fks: list[tuple[str, str]] -) -> CycleResult: +def find_cycles(tables: list[str], fks: list[tuple[str, str]]) -> CycleResult: """Find FK cycles using iterative DFS. Args: @@ -64,6 +64,7 @@ def find_cycles( - Duplicate cycle representations are deduplicated by canonical form (lexicographically smallest rotation). """ + tables, fks = normalize_graph_input(tables, fks) table_set = set(tables) adj: dict[str, list[str]] = {t: [] for t in tables} for child, parent in fks: diff --git a/src/dbjavagenix/algorithms/schema_topo.py b/src/dbjavagenix/algorithms/schema_topo.py index 7e51822..0f232c8 100644 --- a/src/dbjavagenix/algorithms/schema_topo.py +++ b/src/dbjavagenix/algorithms/schema_topo.py @@ -16,6 +16,8 @@ from collections import defaultdict, deque from dataclasses import dataclass, field +from .graph_input import normalize_graph_input + @dataclass class TopoResult: @@ -36,9 +38,7 @@ def has_cycle(self) -> bool: return bool(self.unresolved) -def topological_sort( - tables: list[str], fks: list[tuple[str, str]] -) -> TopoResult: +def topological_sort(tables: list[str], fks: list[tuple[str, str]]) -> TopoResult: """Topologically sort tables by FK dependencies using Kahn's algorithm. Args: @@ -55,6 +55,7 @@ def topological_sort( - FKs pointing to tables outside the `tables` list are ignored (the dependency is treated as already satisfied / external). """ + tables, fks = normalize_graph_input(tables, fks) table_set = set(tables) in_degree = {t: 0 for t in tables} children: dict[str, list[str]] = defaultdict(list) diff --git a/src/dbjavagenix/cli_helpers.py b/src/dbjavagenix/cli_helpers.py index 736d53c..de5a38f 100644 --- a/src/dbjavagenix/cli_helpers.py +++ b/src/dbjavagenix/cli_helpers.py @@ -1,8 +1,10 @@ -""" -Synchronous wrapper functions for MCP tools to be used in CLI -""" +"""Synchronous adapters for MCP handlers used by the command-line interface.""" + import asyncio -from typing import Dict, Any, List, Optional +import json +import re +from typing import Any, Awaitable, Callable, Dict, List, Optional + from .database.mcp_tools import ( handle_db_connect_test as async_handle_db_connect_test, handle_db_query_databases as async_handle_db_query_databases, @@ -13,167 +15,114 @@ from .database.mcp_tools import ( handle_springboot_read_config as async_handle_springboot_read_config ) +from .utils.security import redact_sensitive_text -def handle_db_connect_test(arguments: Dict[str, Any]) -> Dict[str, Any]: - """ - Synchronous wrapper for db_connect_test +_AsyncHandler = Callable[[Dict[str, Any]], Awaitable[List[Any]]] + + +def _parse_json_object(response_text: str) -> Optional[Dict[str, Any]]: + """Extract the first JSON object from an MCP text response. + + MCP handlers normally append a JSON object after ``Raw Response:``. A + direct JSON response is also accepted for compatibility with lightweight + adapters and test doubles. """ + normalized = response_text.strip() + candidates = [] + if "Raw Response:" in normalized: + candidates.append(normalized.rsplit("Raw Response:", 1)[1].strip()) + candidates.append(normalized) + + for candidate in candidates: + try: + payload = json.loads(candidate) + except (TypeError, json.JSONDecodeError): + continue + if isinstance(payload, dict): + return payload + return None + + +def _parse_connection_text(response_text: str) -> Optional[Dict[str, Any]]: + """Recover a successful connection result from human-readable MCP text.""" + if not re.search(r"database\s+connection\s+successful", response_text, re.IGNORECASE): + return None + + connection_match = re.search( + r"connection\s+id\s*:\s*(?P<value>[^\s]+)", response_text, re.IGNORECASE + ) + if connection_match is None: + return None + + server_match = re.search(r"server(?:\s+info)?\s*:\s*(?P<value>[^\r\n]+)", response_text, re.IGNORECASE) + return { + "success": True, + "connection_id": connection_match.group("value"), + "server_info": server_match.group("value").strip() if server_match else None, + } + + +def _run_mcp_sync( + handler: _AsyncHandler, + arguments: Dict[str, Any], + *, + text_fallback: Optional[Callable[[str], Optional[Dict[str, Any]]]] = None, +) -> Dict[str, Any]: + """Run one async MCP handler and normalize its response for the CLI.""" try: - result = asyncio.run(async_handle_db_connect_test(arguments)) - if result and len(result) > 0: - # Parse the response from TextContent - response_text = result[0].text - if "Raw Response:" in response_text: - # Extract the JSON part after "Raw Response:" - import json - try: - json_part = response_text.split("Raw Response:")[-1].strip() - return json.loads(json_part) - except: - # If parsing fails, create a response based on content - if "Connection successful" in response_text: - # Extract connection_id from the text - lines = response_text.split('\n') - connection_id = None - server_info = None - - for line in lines: - if "Connection ID:" in line: - connection_id = line.split("Connection ID:")[-1].strip() - elif "Server info:" in line: - server_info = line.split("Server info:")[-1].strip() - - return { - "success": True, - "connection_id": connection_id, - "server_info": server_info - } - else: - return {"success": False, "error": "Connection failed"} - else: - # Direct response - return {"success": False, "error": response_text} + result = asyncio.run(handler(arguments)) + except Exception as exc: + return {"success": False, "error": redact_sensitive_text(exc)} + + if not result: + return {"success": False, "error": "No response"} + + response_text = getattr(result[0], "text", None) + if not isinstance(response_text, str) or not response_text.strip(): return {"success": False, "error": "No response"} - except Exception as e: - return {"success": False, "error": str(e)} + + payload = _parse_json_object(response_text) + if payload is not None: + return payload + + if text_fallback is not None: + fallback = text_fallback(response_text) + if fallback is not None: + return fallback + + return {"success": False, "error": "Failed to parse response"} + + +def handle_db_connect_test(arguments: Dict[str, Any]) -> Dict[str, Any]: + """Synchronous wrapper for ``db_connect_test``.""" + return _run_mcp_sync( + async_handle_db_connect_test, + arguments, + text_fallback=_parse_connection_text, + ) def handle_db_query_databases(arguments: Dict[str, Any]) -> Dict[str, Any]: - """ - Synchronous wrapper for db_query_databases - """ - try: - result = asyncio.run(async_handle_db_query_databases(arguments)) - if result and len(result) > 0: - response_text = result[0].text - if "Raw Response:" in response_text: - import json - try: - json_part = response_text.split("Raw Response:")[-1].strip() - return json.loads(json_part) - except: - return {"success": False, "error": "Failed to parse response"} - else: - return {"success": False, "error": response_text} - return {"success": False, "error": "No response"} - except Exception as e: - return {"success": False, "error": str(e)} + """Synchronous wrapper for ``db_query_databases``.""" + return _run_mcp_sync(async_handle_db_query_databases, arguments) def handle_db_query_tables(arguments: Dict[str, Any]) -> Dict[str, Any]: - """ - Synchronous wrapper for db_query_tables - """ - try: - result = asyncio.run(async_handle_db_query_tables(arguments)) - if result and len(result) > 0: - response_text = result[0].text - if "Raw Response:" in response_text: - import json - try: - json_part = response_text.split("Raw Response:")[-1].strip() - return json.loads(json_part) - except: - return {"success": False, "error": "Failed to parse response"} - else: - return {"success": False, "error": response_text} - return {"success": False, "error": "No response"} - except Exception as e: - return {"success": False, "error": str(e)} + """Synchronous wrapper for ``db_query_tables``.""" + return _run_mcp_sync(async_handle_db_query_tables, arguments) def handle_db_codegen_analyze(arguments: Dict[str, Any]) -> Dict[str, Any]: - """ - Synchronous wrapper for db_codegen_analyze - """ - try: - result = asyncio.run(async_handle_db_codegen_analyze(arguments)) - if result and len(result) > 0: - response_text = result[0].text - if "Raw Response:" in response_text: - import json - try: - json_part = response_text.split("Raw Response:")[-1].strip() - return json.loads(json_part) - except: - return {"success": False, "error": "Failed to parse response"} - else: - # Try to parse as JSON directly - import json - try: - return json.loads(response_text) - except: - return {"success": False, "error": response_text} - return {"success": False, "error": "No response"} - except Exception as e: - return {"success": False, "error": str(e)} + """Synchronous wrapper for ``db_codegen_analyze``.""" + return _run_mcp_sync(async_handle_db_codegen_analyze, arguments) def handle_db_codegen_generate(arguments: Dict[str, Any]) -> Dict[str, Any]: - """ - Synchronous wrapper for db_codegen_generate - """ - try: - result = asyncio.run(async_handle_db_codegen_generate(arguments)) - if result and len(result) > 0: - response_text = result[0].text - if "Raw Response:" in response_text: - import json - try: - json_part = response_text.split("Raw Response:")[-1].strip() - return json.loads(json_part) - except: - return {"success": False, "error": "Failed to parse response"} - else: - # Try to parse as JSON directly - import json - try: - return json.loads(response_text) - except: - return {"success": False, "error": response_text} - return {"success": False, "error": "No response"} - except Exception as e: - return {"success": False, "error": str(e)} + """Synchronous wrapper for ``db_codegen_generate``.""" + return _run_mcp_sync(async_handle_db_codegen_generate, arguments) def handle_springboot_read_config(arguments: Dict[str, Any]) -> Dict[str, Any]: - """ - Synchronous wrapper for springboot_read_config - """ - try: - result = asyncio.run(async_handle_springboot_read_config(arguments)) - if result and len(result) > 0: - response_text = result[0].text - if "Raw Response:" in response_text: - import json - try: - json_part = response_text.split("Raw Response:")[-1].strip() - return json.loads(json_part) - except Exception: - return {"success": False, "error": "Failed to parse response"} - else: - return {"success": False, "error": response_text} - return {"success": False, "error": "No response"} - except Exception as e: - return {"success": False, "error": str(e)} + """Synchronous wrapper for ``springboot_read_config``.""" + return _run_mcp_sync(async_handle_springboot_read_config, arguments) diff --git a/src/dbjavagenix/config/default_type_mapping.yaml b/src/dbjavagenix/config/default_type_mapping.yaml index 19c6d07..4b9f0f7 100644 --- a/src/dbjavagenix/config/default_type_mapping.yaml +++ b/src/dbjavagenix/config/default_type_mapping.yaml @@ -280,15 +280,24 @@ postgresql: TIME: java_type: "LocalTime" imports: ["java.time.LocalTime"] + TIME_WITHOUT_TIME_ZONE: + java_type: "LocalTime" + imports: ["java.time.LocalTime"] TIME_WITH_TIME_ZONE: java_type: "OffsetTime" imports: ["java.time.OffsetTime"] TIMESTAMP: java_type: "LocalDateTime" imports: ["java.time.LocalDateTime"] + TIMESTAMP_WITHOUT_TIME_ZONE: + java_type: "LocalDateTime" + imports: ["java.time.LocalDateTime"] TIMESTAMP_WITH_TIME_ZONE: java_type: "OffsetDateTime" imports: ["java.time.OffsetDateTime"] + TIMESTAMPTZ: + java_type: "OffsetDateTime" + imports: ["java.time.OffsetDateTime"] INTERVAL: java_type: "String" imports: [] diff --git a/src/dbjavagenix/database/ai_tools.py b/src/dbjavagenix/database/ai_tools.py index 274e656..f5b3d33 100644 --- a/src/dbjavagenix/database/ai_tools.py +++ b/src/dbjavagenix/database/ai_tools.py @@ -12,7 +12,6 @@ - 所有响应附带 metrics (源/token 使用),便于 P4.4 监控 """ -import json import logging from typing import Any, Dict, List @@ -31,6 +30,7 @@ ) from ..ai.schema_summary import summarize_schema from ..ai.template_recommender import recommend_template +from ..utils.json_serialization import dumps as _json_dumps logger = logging.getLogger(__name__) @@ -173,10 +173,12 @@ async def handle_ai_infer_business_names(arguments: Dict[str, Any]) -> List[Text model = arguments.get("model", "claude-sonnet-4-6") if not tables: - return [TextContent( - type="text", - text=json.dumps({"error": "tables is required and non-empty"}, ensure_ascii=False), - )] + return [ + TextContent( + type="text", + text=_json_dumps({"error": "tables is required and non-empty"}), + ) + ] inferences_out: List[Dict[str, Any]] = [] source_used = "rule" @@ -213,10 +215,12 @@ async def handle_ai_infer_business_names(arguments: Dict[str, Any]) -> List[Text "llm_metrics": llm_metrics, "llm_available": is_llm_available(), } - return [TextContent( - type="text", - text=json.dumps(response, ensure_ascii=False, indent=2), - )] + return [ + TextContent( + type="text", + text=_json_dumps(response, indent=2), + ) + ] def _rule_to_dict(ri: NamingInference) -> Dict[str, Any]: @@ -242,16 +246,16 @@ def _metrics_to_dict(m: LLMMetrics) -> Dict[str, Any]: } -def _merge_inference( - llm_item: Dict[str, Any], rule_fallback: NamingInference -) -> Dict[str, Any]: +def _merge_inference(llm_item: Dict[str, Any], rule_fallback: NamingInference) -> Dict[str, Any]: """LLM 输出补全缺失字段 (从规则推断里取)""" merged: Dict[str, Any] = { "table": llm_item.get("table", ""), - "class_name": llm_item.get("class_name") or (rule_fallback.class_name if rule_fallback else ""), + "class_name": llm_item.get("class_name") + or (rule_fallback.class_name if rule_fallback else ""), "field_naming": llm_item.get("field_naming") or {}, "reason": llm_item.get("reason") or (rule_fallback.reason if rule_fallback else ""), - "table_kind": llm_item.get("table_kind") or (rule_fallback.table_kind if rule_fallback else "unknown"), + "table_kind": llm_item.get("table_kind") + or (rule_fallback.table_kind if rule_fallback else "unknown"), "source": "llm", "confidence": 0.9, } @@ -262,6 +266,7 @@ def _merge_inference( # P4.2 / P4.3 handlers # ============================================================ + async def handle_ai_recommend_template(arguments: Dict[str, Any]) -> List[TextContent]: """根据整库 schema 推荐模板分类 + 生成选项""" table_names = arguments.get("table_names", []) @@ -269,10 +274,12 @@ async def handle_ai_recommend_template(arguments: Dict[str, Any]) -> List[TextCo hint_modern = bool(arguments.get("hint_modern_stack", False)) if not table_names: - return [TextContent( - type="text", - text=json.dumps({"error": "table_names is required and non-empty"}, ensure_ascii=False), - )] + return [ + TextContent( + type="text", + text=_json_dumps({"error": "table_names is required and non-empty"}), + ) + ] rec = recommend_template( table_names=table_names, @@ -288,10 +295,12 @@ async def handle_ai_recommend_template(arguments: Dict[str, Any]) -> List[TextCo "reasons": rec.reasons, "matched_tables": rec.matched_tables, } - return [TextContent( - type="text", - text=json.dumps(response, ensure_ascii=False, indent=2), - )] + return [ + TextContent( + type="text", + text=_json_dumps(response, indent=2), + ) + ] async def handle_ai_summarize_schema(arguments: Dict[str, Any]) -> List[TextContent]: @@ -301,10 +310,12 @@ async def handle_ai_summarize_schema(arguments: Dict[str, Any]) -> List[TextCont table_column_counts = arguments.get("table_column_counts", {}) if not table_names: - return [TextContent( - type="text", - text=json.dumps({"error": "table_names is required and non-empty"}, ensure_ascii=False), - )] + return [ + TextContent( + type="text", + text=_json_dumps({"error": "table_names is required and non-empty"}), + ) + ] summary = summarize_schema( table_names=table_names, @@ -321,10 +332,12 @@ async def handle_ai_summarize_schema(arguments: Dict[str, Any]) -> List[TextCont "core_entities": summary.core_entities, "relationships": summary.relationships, } - return [TextContent( - type="text", - text=json.dumps(response, ensure_ascii=False, indent=2), - )] + return [ + TextContent( + type="text", + text=_json_dumps(response, indent=2), + ) + ] async def handle_ai_metrics(arguments: Dict[str, Any]) -> List[TextContent]: @@ -346,7 +359,9 @@ async def handle_ai_metrics(arguments: Dict[str, Any]) -> List[TextContent]: GLOBAL_LLM_STATS.total_cache_creation = 0 GLOBAL_LLM_STATS.total_errors = 0 response["reset"] = True - return [TextContent( - type="text", - text=json.dumps(response, ensure_ascii=False, indent=2), - )] + return [ + TextContent( + type="text", + text=_json_dumps(response, indent=2), + ) + ] diff --git a/src/dbjavagenix/database/atomic_codegen_tools.py b/src/dbjavagenix/database/atomic_codegen_tools.py index 327a32d..17615a0 100644 --- a/src/dbjavagenix/database/atomic_codegen_tools.py +++ b/src/dbjavagenix/database/atomic_codegen_tools.py @@ -31,23 +31,25 @@ from ..core.exceptions import DatabaseConnectionError, MCPServiceError from ..database.connection_manager import connection_manager +from ..database.introspection import DatabaseIntrospector +from ..generator.template_context import apply_generation_options from ..utils.json_serialization import dumps as _json_dumps logger = logging.getLogger(__name__) -async def _run_db_call(callable_obj, *args): +async def _run_db_call(callable_obj, *args, **kwargs): """Run blocking database work outside the MCP event loop.""" - return await asyncio.to_thread(callable_obj, *args) + return await asyncio.to_thread(callable_obj, *args, **kwargs) async def _run_async_db_call(callable_obj, *args, **kwargs): - """Run the async analyzer in a worker because its introspection is synchronous.""" + """Run an async analyzer in a worker when it performs blocking I/O.""" def run(): return asyncio.run(callable_obj(*args, **kwargs)) - return await asyncio.to_thread(run) + return await _run_db_call(run) # ============================================================ @@ -104,6 +106,16 @@ def get_atomic_codegen_tools() -> List[Tool]: "include_swagger": {"type": "boolean", "default": True}, "include_lombok": {"type": "boolean", "default": True}, "include_mapstruct": {"type": "boolean", "default": True}, + "generate_dto": { + "type": "boolean", + "description": "Generate a DTO artifact when the template category does not provide one", + "default": False, + }, + "generate_vo": { + "type": "boolean", + "description": "Generate a VO artifact", + "default": False, + }, "project_path": { "type": "string", "description": "Optional target Spring Boot project path", @@ -253,6 +265,11 @@ async def handle_codegen_build_context(arguments: Dict[str, Any]) -> List[TextCo "useMapStruct": include_mapstruct, } ) + apply_generation_options( + context, + generate_dto=arguments.get("generate_dto"), + generate_vo=arguments.get("generate_vo"), + ) # 重写包路径(尊重 package_suffix) _rebuild_package_paths(context, package_name) @@ -512,48 +529,16 @@ def _compute_file_path( package_name = context.get("package", "com.example") package_path = package_name.replace(".", "/") if file_path.endswith(".java"): - normalized_path = "/".join(part for part in file_path.split("/") if part) - return f"{package_path}/{normalized_path}" + return f"{package_path}/{file_path}" return f"resources/{file_path}" -def _collect_all_table_names(connection_id: str, config) -> List[str]: +def _collect_all_table_names(connection_id: str, _config) -> List[str]: """收集库内所有表名(用于前缀分析)。失败时返回空列表。""" try: - - def collect(cursor) -> List[str]: - if config.type.name == "MYSQL": - cursor.execute("SHOW TABLES") - return [row[0] for row in cursor.fetchall()] - if config.type.name == "SQLITE": - cursor.execute( - "SELECT name FROM sqlite_master WHERE type='table' AND name NOT LIKE 'sqlite_%'" - ) - return [row[0] for row in cursor.fetchall()] - if config.type.name == "POSTGRESQL": - cursor.execute( - "SELECT table_name FROM information_schema.tables " - "WHERE table_catalog = current_database() " - "AND table_type = 'BASE TABLE' " - "AND table_schema NOT IN ('pg_catalog', 'information_schema') " - "ORDER BY table_schema, table_name" - ) - return [row[0] for row in cursor.fetchall()] - return [] - - if hasattr(connection_manager, "get_cursor"): - with connection_manager.get_cursor(connection_id) as cursor: - return collect(cursor) - - # Preserve the small test-double contract used by older integrations. - connection = connection_manager.get_connection(connection_id) - cursor = connection.cursor() - try: - return collect(cursor) - finally: - cursor.close() + return DatabaseIntrospector(connection_manager).list_tables(connection_id) except Exception as e: # noqa: BLE001 - logger.warning(f"_collect_all_table_names failed: {e}") + logger.warning("_collect_all_table_names failed: %s", e) return [] diff --git a/src/dbjavagenix/database/codegen_tools.py b/src/dbjavagenix/database/codegen_tools.py index 55bd311..7f2e172 100644 --- a/src/dbjavagenix/database/codegen_tools.py +++ b/src/dbjavagenix/database/codegen_tools.py @@ -1,437 +1,450 @@ -""" -代码生成集成工具 -将数据库分析结果与模板系统集成，提供 MCP 工具接口 -""" - -from typing import Dict, List, Any, Optional -from ..database.connection_manager import ConnectionManager -from ..generator.template_context import TemplateContextBuilder -from ..core.models import TableInfo, ColumnInfo, DatabaseType -from .introspection import DatabaseIntrospector - - -class CodegenAnalyzer: - """代码生成分析器 - 将数据库表结构转换为代码生成所需的格式""" - - def __init__(self, connection_manager: ConnectionManager): - self.connection_manager = connection_manager - self.introspector = DatabaseIntrospector(connection_manager) - - async def analyze_table_for_codegen( - self, - connection_id: str, - table_name: str, - all_table_names: Optional[List[str]] = None, - template_category: str = "Default", - project_root: Optional[str] = None, - schema: Optional[str] = None, - ) -> Dict[str, Any]: - """分析单个表的结构，返回代码生成所需的完整信息""" - - # 获取表基本信息 - config = self.introspector.get_config(connection_id) - table_info = self.introspector.get_table(connection_id, table_name, schema) - - # 获取列信息 - columns = self.introspector.get_columns(connection_id, table_name, schema) - - # 获取主键信息 - primary_keys = self.introspector.get_primary_keys(connection_id, table_name, schema) - primary_key_set = set(primary_keys) - for column in columns: - column["primary_key"] = ( - column.get("primary_key", False) or column["name"] in primary_key_set - ) - - # 获取外键信息 - foreign_keys = self.introspector.get_foreign_keys(connection_id, table_name, schema) - - # 获取索引信息 - indexes = self.introspector.get_indexes(connection_id, table_name, schema) - - # 获取数据库名称 - database_name = table_info.get("schema") or config.database or "unknown" - - # 构建 TableInfo 对象 - table_obj = self._build_table_info( - table_info, - columns, - primary_keys, - foreign_keys, - indexes, - database_name, - config.type, - ) - - # 构建代码生成上下文 - context_builder = TemplateContextBuilder( - author="ZXP", package_name="com.example", database_type=config.type - ) - context = context_builder.build_context( - table_obj, - template_category=template_category, - all_table_names=all_table_names, - project_root=project_root, - ) - - return { - "table_name": table_name, - "table_info": { - "name": table_obj.name, - "comment": table_obj.comment, - "schema": table_obj.schema, - "columns": [self._column_to_dict(col) for col in table_obj.columns], - }, - "template_context": context, - "java_types": self._extract_java_types(table_obj.columns, config.type), - "imports_needed": self._calculate_imports_needed(table_obj.columns, config.type), - "relationships": { - "primary_keys": primary_keys, - "foreign_keys": foreign_keys, - "indexes": indexes, - }, - } - - async def analyze_database_for_codegen( - self, connection_id: str, table_filter: Optional[List[str]] = None - ) -> Dict[str, Any]: - """分析整个数据库，返回所有表的代码生成信息""" - - # PostgreSQL table names are scoped by schema. Keep that identity through - # batch analysis so equally named tables cannot become ambiguous or overwrite - # one another in the returned mapping. - all_table_references = self.introspector.list_table_references(connection_id) - - def table_key(reference: Dict[str, str | None]) -> str: - return ( - f"{reference['schema']}.{reference['name']}" - if reference["schema"] - else reference["name"] - ) - - tables_to_analyze = [ - reference - for reference in all_table_references - if not table_filter - or reference["name"] in table_filter - or table_key(reference) in table_filter - ] - analysis_results = {} - for reference in tables_to_analyze: - name = reference["name"] - schema = reference["schema"] - result_key = table_key(reference) - try: - analysis_results[result_key] = await self.analyze_table_for_codegen( - connection_id, name, schema=schema - ) - except Exception as exc: - analysis_results[result_key] = {"error": str(exc)} - - return { - "database_info": { - "total_tables": len(all_table_references), - "analyzed_tables": len(tables_to_analyze), - "success_count": sum("error" not in item for item in analysis_results.values()), - "error_count": sum("error" in item for item in analysis_results.values()), - }, - "tables": analysis_results, - } - - def _build_table_info( - self, - table_info: Dict[str, Any], - columns: List[Dict[str, Any]], - primary_keys: List[str], - foreign_keys: List[Dict[str, Any]], - indexes: List[Dict[str, Any]], - database_name: str = "unknown", - database_type: DatabaseType = DatabaseType.MYSQL, - ) -> TableInfo: - """构建 TableInfo 对象""" - - column_objects = [] - for col in columns: - column_obj = ColumnInfo( - name=col["name"], - data_type=col["type"], - java_type=self._map_java_type(col["type"], database_type), - nullable=col["nullable"], - primary_key=col["primary_key"], - default_value=col["default_value"], - comment=col["comment"], - auto_increment=col["auto_increment"], - max_length=col["max_length"], - precision=col.get("precision"), - scale=col.get("scale"), - ) - column_objects.append(column_obj) - - table_obj = TableInfo( - name=table_info["name"], - schema=database_name, - comment=table_info["comment"], - columns=column_objects, - primary_keys=list(primary_keys), - foreign_keys={ - item["column_name"]: (f"{item['referenced_table']}.{item['referenced_column']}") - for item in foreign_keys - if item.get("column_name") - }, - indexes=[item["key_name"] for item in indexes if item.get("key_name")], - ) - - return table_obj - - def _column_to_dict(self, column: ColumnInfo) -> Dict[str, Any]: - """将 ColumnInfo 对象转换为字典""" - return { - "name": column.name, - "type": column.data_type, - "nullable": column.nullable, - "primary_key": column.primary_key, - "default_value": column.default_value, - "comment": column.comment, - "auto_increment": column.auto_increment, - "max_length": column.max_length, - "precision": column.precision, - "scale": column.scale, - "java_type": column.java_type, - } - - def _extract_java_types( - self, columns: List[ColumnInfo], database_type: DatabaseType = DatabaseType.MYSQL - ) -> List[str]: - """提取所需的 Java 类型列表""" - context_builder = TemplateContextBuilder(database_type=database_type) - java_types = set() - - for column in columns: - java_type = context_builder._map_java_type(column.data_type) - java_types.add(java_type) - - return sorted(list(java_types)) - - def _calculate_imports_needed( - self, columns: List[ColumnInfo], database_type: DatabaseType = DatabaseType.MYSQL - ) -> List[str]: - """计算需要导入的类列表""" - context_builder = TemplateContextBuilder(database_type=database_type) - imports = set() - - for column in columns: - java_type = context_builder._map_java_type(column.data_type) - - # 添加需要导入的类型 - if java_type == "BigDecimal": - imports.add("java.math.BigDecimal") - elif java_type == "LocalDate": - imports.add("java.time.LocalDate") - elif java_type == "LocalTime": - imports.add("java.time.LocalTime") - elif java_type == "LocalDateTime": - imports.add("java.time.LocalDateTime") - - return sorted(list(imports)) - - def _map_java_type( - self, database_type_name: str, database_type: DatabaseType = DatabaseType.MYSQL - ) -> str: - """将 MySQL 数据类型映射为 Java 类型""" - from ..generator.template_context import TemplateContextBuilder - - context_builder = TemplateContextBuilder(database_type=database_type) - return context_builder._map_java_type(database_type_name) - - -class CodegenGenerator: - """代码生成器 - 根据分析结果生成 Java 代码""" - - def __init__(self): - pass - - async def generate_code( - self, - analysis_result: Dict[str, Any], - template_category: str = "MybatisPlus-Mixed", - generation_config: Optional[Dict[str, Any]] = None, - ) -> Dict[str, Any]: - """根据分析结果生成代码""" - - from ..generator.template_context import TemplateConfigManager - - supported_categories = TemplateConfigManager.get_supported_categories() - if template_category not in supported_categories: - supported = ", ".join(supported_categories) - raise ValueError(f"不支持的模板分类: {template_category!r}；支持分类: {supported}") - - # 生成代码到内存中（不写入文件） - generated_code = {} - - # 获取模板文件列表 - template_config = TemplateConfigManager() - base_templates = template_config.get_template_files(template_category) - template_files = list(base_templates) - # 动态附加DTO/VO/MapStruct模板 & MyBatis-Plus配置 - tc = analysis_result.get("template_context", {}) - use_mapstruct = bool(tc.get("useMapStruct")) - include_dto_vo = bool(tc.get("includeDtoVo") or tc.get("include_dto_vo")) - extras: list[str] = [] - if include_dto_vo: - extras.extend(["dto.mustache", "vo.mustache"]) - if use_mapstruct: - extras.append("mapstruct_mapper.mustache") - # 为 MyBatis-Plus 路线附加配置类（分页拦截器） - if template_category in ("MybatisPlus", "MybatisPlus-Mixed"): - extras.append("mybatis_plus_config.mustache") - if extras: - template_files.extend(extras) - - # 使用分析结果中的模板上下文，但更新配置相关字段 - context = analysis_result["template_context"].copy() - - # 重新设置包名和作者信息 - if generation_config: - package_name = generation_config.get( - "package_name", context.get("package", "com.example") - ) - author = generation_config.get("author", context.get("author", "ZXP")) - - # 获取前缀后缀 - package_suffix = context.get("packageSuffix", "") - - # 重新构建组件包名 - if package_suffix: - controller_package = f"{package_name}.controller.{package_suffix}" - service_package = f"{package_name}.service.{package_suffix}" - entity_package = f"{package_name}.entity.{package_suffix}" - dao_package = f"{package_name}.dao.{package_suffix}" - dto_package = f"{package_name}.dto.{package_suffix}" - vo_package = f"{package_name}.vo.{package_suffix}" - # 修复serviceImpl包路径问题 - service_impl_package = f"{package_name}.service.impl.{package_suffix}" - else: - controller_package = f"{package_name}.controller" - service_package = f"{package_name}.service" - entity_package = f"{package_name}.entity" - dao_package = f"{package_name}.dao" - dto_package = f"{package_name}.dto" - vo_package = f"{package_name}.vo" - # 修复serviceImpl包路径问题 - service_impl_package = f"{package_name}.service.impl" - - # 更新上下文中的包相关信息 - context.update( - { - "package": package_name, - "packageName": package_name, - "hasPackageName": bool(package_name), - "basePackage": package_name, - "controllerPackage": controller_package, - "servicePackage": service_package, - "entityPackage": entity_package, - "daoPackage": dao_package, - "dtoPackage": dto_package, - "voPackage": vo_package, - # 添加serviceImpl包路径 - "serviceImplPackage": service_impl_package, - "author": author, - } - ) - - # 更新上下文中的配置相关字段 - context.update( - { - "templateCategory": template_category, - "isDefault": template_category == "Default", - "isMybatisPlus": template_category == "MybatisPlus", - "isMybatisPlusMixed": template_category == "MybatisPlus-Mixed", - } - ) - - # 为每个模板生成代码 - for template_file in template_files: - try: - # 附加模板从 common 目录加载 - effective_category = ( - template_category if template_file in base_templates else "common" - ) - code = await self._render_template(template_file, context, effective_category) - generated_code[template_file] = { - "filename": self._get_output_filename(template_file, context), - "code": code, - "template_file": template_file, - } - except Exception as e: - generated_code[template_file] = {"error": str(e), "template_file": template_file} - - return { - "table_name": analysis_result["table_name"], - "template_category": template_category, - "generated_code": generated_code, - "generation_statistics": { - "total_files": len(template_files), - "success_files": len([f for f in generated_code.values() if "error" not in f]), - "error_files": len([f for f in generated_code.values() if "error" in f]), - }, - } - - async def _render_template( - self, template_file: str, context: Dict[str, Any], category: str - ) -> str: - """渲染模板文件""" - from pathlib import Path - from ..generator.mustache_engine import MustacheTemplateEngine - - # 确定模板路径 - template_base_path = Path(__file__).parent.parent / "templates" / "java" - - if category == "common": - template_path = template_base_path / "common" / template_file - else: - template_path = template_base_path / category / template_file - - if not template_path.exists(): - raise FileNotFoundError(f"模板文件不存在: {template_path}") - - # 渲染模板 - engine = MustacheTemplateEngine() - return engine.render_file(str(template_path), context) - - def _get_output_filename(self, template_file: str, context: Dict[str, Any]) -> str: - """获取输出文件名""" - from ..generator.template_context import TemplateConfigManager - - template_config = TemplateConfigManager() - path_mapping = template_config.get_output_path_mapping() - - if template_file in path_mapping: - relative_path = path_mapping[template_file] - # 替换路径中的占位符 - file_path = relative_path.format(**context) - - # 添加包路径结构 - package_name = context.get("package", "com.example") - if package_name: - # 将包名转换为路径 - package_path = package_name.replace(".", "/") - # 对于Java源文件，添加包路径 - if file_path.endswith(".java"): - # 提取文件名和相对目录 - path_parts = file_path.split("/") - if len(path_parts) > 1: - # 移除表名路径，让所有表共享同一个包结构 - relative_dir = "/".join(part for part in path_parts[:-1] if part) - filename = path_parts[-1] - # 构建带包路径的完整路径，不包含表名子包 - return f"{package_path}/{relative_dir}/{filename}" - else: - return f"{package_path}/{file_path}" - else: - # 对于XML等资源文件，不添加包路径，但添加resources前缀 - return f"resources/{file_path}" - - return file_path - else: - # 默认文件名 - return template_file.replace(".mustache", ".java") +""" +代码生成集成工具 +将数据库分析结果与模板系统集成，提供 MCP 工具接口 +""" + +from typing import Dict, List, Any, Optional +from ..database.connection_manager import ConnectionManager +from ..generator.template_context import TemplateContextBuilder +from ..core.models import TableInfo, ColumnInfo, DatabaseType +from .introspection import DatabaseIntrospector + + +class CodegenAnalyzer: + """代码生成分析器 - 将数据库表结构转换为代码生成所需的格式""" + + def __init__(self, connection_manager: ConnectionManager): + self.connection_manager = connection_manager + self.introspector = DatabaseIntrospector(connection_manager) + + async def analyze_table_for_codegen( + self, + connection_id: str, + table_name: str, + all_table_names: Optional[List[str]] = None, + template_category: str = "Default", + project_root: Optional[str] = None, + schema: Optional[str] = None, + ) -> Dict[str, Any]: + """分析单个表的结构，返回代码生成所需的完整信息""" + + # 获取表基本信息 + config = self.introspector.get_config(connection_id) + table_info = self.introspector.get_table(connection_id, table_name, schema) + + # 获取列信息 + columns = self.introspector.get_columns(connection_id, table_name, schema) + + # 获取主键信息 + primary_keys = self.introspector.get_primary_keys(connection_id, table_name, schema) + primary_key_set = set(primary_keys) + for column in columns: + column["primary_key"] = ( + column.get("primary_key", False) or column["name"] in primary_key_set + ) + + # 获取外键信息 + foreign_keys = self.introspector.get_foreign_keys(connection_id, table_name, schema) + + # 获取索引信息 + indexes = self.introspector.get_indexes(connection_id, table_name, schema) + + # 获取数据库名称 + database_name = table_info.get("schema") or config.database or "unknown" + + # 构建 TableInfo 对象 + table_obj = self._build_table_info( + table_info, + columns, + primary_keys, + foreign_keys, + indexes, + database_name, + config.type, + ) + + # 构建代码生成上下文 + context_builder = TemplateContextBuilder( + author="ZXP", package_name="com.example", database_type=config.type + ) + context = context_builder.build_context( + table_obj, + template_category=template_category, + all_table_names=all_table_names, + project_root=project_root, + ) + + return { + "table_name": table_name, + "table_info": { + "name": table_obj.name, + "comment": table_obj.comment, + "schema": table_obj.schema, + "columns": [self._column_to_dict(col) for col in table_obj.columns], + }, + "template_context": context, + "java_types": self._extract_java_types(table_obj.columns, config.type), + "imports_needed": self._calculate_imports_needed(table_obj.columns, config.type), + "relationships": { + "primary_keys": primary_keys, + "foreign_keys": foreign_keys, + "indexes": indexes, + }, + } + + async def analyze_database_for_codegen( + self, connection_id: str, table_filter: Optional[List[str]] = None + ) -> Dict[str, Any]: + """分析整个数据库，返回所有表的代码生成信息""" + + # PostgreSQL table names are scoped by schema. Keep that identity through + # batch analysis so equally named tables cannot become ambiguous or overwrite + # one another in the returned mapping. + all_table_references = self.introspector.list_table_references(connection_id) + # Prefix analysis operates on bare table names. Deduplicate names so + # PostgreSQL tables with the same name in different schemas do not + # inflate a prefix group's table count. + all_table_names = sorted( + {str(reference["name"]) for reference in all_table_references if reference.get("name")} + ) + + def table_key(reference: Dict[str, str | None]) -> str: + return ( + f"{reference['schema']}.{reference['name']}" + if reference["schema"] + else reference["name"] + ) + + tables_to_analyze = [ + reference + for reference in all_table_references + if not table_filter + or reference["name"] in table_filter + or table_key(reference) in table_filter + ] + analysis_results = {} + for reference in tables_to_analyze: + name = reference["name"] + schema = reference["schema"] + result_key = table_key(reference) + try: + analysis_results[result_key] = await self.analyze_table_for_codegen( + connection_id, + name, + all_table_names=all_table_names, + schema=schema, + ) + except Exception as exc: + analysis_results[result_key] = {"error": str(exc)} + + return { + "database_info": { + "total_tables": len(all_table_references), + "analyzed_tables": len(tables_to_analyze), + "success_count": sum("error" not in item for item in analysis_results.values()), + "error_count": sum("error" in item for item in analysis_results.values()), + }, + "tables": analysis_results, + } + + def _build_table_info( + self, + table_info: Dict[str, Any], + columns: List[Dict[str, Any]], + primary_keys: List[str], + foreign_keys: List[Dict[str, Any]], + indexes: List[Dict[str, Any]], + database_name: str = "unknown", + database_type: DatabaseType = DatabaseType.MYSQL, + ) -> TableInfo: + """构建 TableInfo 对象""" + + column_objects = [] + for col in columns: + column_obj = ColumnInfo( + name=col["name"], + data_type=col["type"], + java_type=self._map_java_type(col["type"], database_type), + nullable=col["nullable"], + primary_key=col["primary_key"], + default_value=col["default_value"], + comment=col["comment"], + auto_increment=col["auto_increment"], + max_length=col["max_length"], + precision=col.get("precision"), + scale=col.get("scale"), + ) + column_objects.append(column_obj) + + table_obj = TableInfo( + name=table_info["name"], + schema=database_name, + comment=table_info["comment"], + columns=column_objects, + primary_keys=list(primary_keys), + foreign_keys={ + item["column_name"]: (f"{item['referenced_table']}.{item['referenced_column']}") + for item in foreign_keys + if item.get("column_name") + }, + indexes=[item["key_name"] for item in indexes if item.get("key_name")], + ) + + return table_obj + + def _column_to_dict(self, column: ColumnInfo) -> Dict[str, Any]: + """将 ColumnInfo 对象转换为字典""" + return { + "name": column.name, + "type": column.data_type, + "nullable": column.nullable, + "primary_key": column.primary_key, + "default_value": column.default_value, + "comment": column.comment, + "auto_increment": column.auto_increment, + "max_length": column.max_length, + "precision": column.precision, + "scale": column.scale, + "java_type": column.java_type, + } + + def _extract_java_types( + self, columns: List[ColumnInfo], database_type: DatabaseType = DatabaseType.MYSQL + ) -> List[str]: + """提取所需的 Java 类型列表""" + context_builder = TemplateContextBuilder(database_type=database_type) + java_types = set() + + for column in columns: + java_type = context_builder._map_java_type(column.data_type) + java_types.add(java_type) + + return sorted(list(java_types)) + + def _calculate_imports_needed( + self, columns: List[ColumnInfo], database_type: DatabaseType = DatabaseType.MYSQL + ) -> List[str]: + """计算需要导入的类列表""" + context_builder = TemplateContextBuilder(database_type=database_type) + # Use the same dialect-aware import table as generated template context. + return context_builder._build_imports(columns, template_category="Default") + + def _map_java_type( + self, database_type_name: str, database_type: DatabaseType = DatabaseType.MYSQL + ) -> str: + """将 MySQL 数据类型映射为 Java 类型""" + from ..generator.template_context import TemplateContextBuilder + + context_builder = TemplateContextBuilder(database_type=database_type) + return context_builder._map_java_type(database_type_name) + + +class CodegenGenerator: + """代码生成器 - 根据分析结果生成 Java 代码""" + + def __init__(self): + pass + + async def generate_code( + self, + analysis_result: Dict[str, Any], + template_category: str = "MybatisPlus-Mixed", + generation_config: Optional[Dict[str, Any]] = None, + ) -> Dict[str, Any]: + """根据分析结果生成代码""" + + from ..generator.template_context import ( + TemplateConfigManager, + apply_generation_options, + ) + + supported_categories = TemplateConfigManager.get_supported_categories() + if template_category not in supported_categories: + supported = ", ".join(supported_categories) + raise ValueError(f"不支持的模板分类: {template_category!r}；支持分类: {supported}") + + # 生成代码到内存中（不写入文件） + generated_code = {} + + # 获取模板文件列表 + template_config = TemplateConfigManager() + base_templates = template_config.get_template_files(template_category) + template_files = list(base_templates) + # 使用分析结果中的模板上下文，但更新配置相关字段 + context = analysis_result["template_context"].copy() + generation_options = generation_config or {} + + def option(*names: str) -> Any: + for name in names: + if name in generation_options: + return generation_options[name] + return None + + apply_generation_options( + context, + generate_dto=option("generate_dto", "generateDto"), + generate_vo=option("generate_vo", "generateVo"), + include_dto_vo=option("include_dto_vo", "includeDtoVo"), + ) + + # 动态附加 DTO/VO/MapStruct 模板 & MyBatis-Plus 配置 + use_mapstruct = bool(context.get("useMapStruct")) + extras: list[str] = [] + if context.get("generateDto") and "dto.mustache" not in base_templates: + extras.append("dto.mustache") + if context.get("generateVo") and "vo.mustache" not in base_templates: + extras.append("vo.mustache") + if use_mapstruct: + extras.append("mapstruct_mapper.mustache") + # 为 MyBatis-Plus 路线附加配置类（分页拦截器） + if template_category in ("MybatisPlus", "MybatisPlus-Mixed"): + extras.append("mybatis_plus_config.mustache") + if extras: + template_files.extend(extras) + + # 重新设置包名和作者信息 + if generation_config: + package_name = generation_config.get( + "package_name", context.get("package", "com.example") + ) + author = generation_config.get("author", context.get("author", "ZXP")) + + # 获取前缀后缀 + package_suffix = context.get("packageSuffix", "") + + # 重新构建组件包名 + if package_suffix: + controller_package = f"{package_name}.controller.{package_suffix}" + service_package = f"{package_name}.service.{package_suffix}" + entity_package = f"{package_name}.entity.{package_suffix}" + dao_package = f"{package_name}.dao.{package_suffix}" + dto_package = f"{package_name}.dto.{package_suffix}" + vo_package = f"{package_name}.vo.{package_suffix}" + # 修复serviceImpl包路径问题 + service_impl_package = f"{package_name}.service.impl.{package_suffix}" + else: + controller_package = f"{package_name}.controller" + service_package = f"{package_name}.service" + entity_package = f"{package_name}.entity" + dao_package = f"{package_name}.dao" + dto_package = f"{package_name}.dto" + vo_package = f"{package_name}.vo" + # 修复serviceImpl包路径问题 + service_impl_package = f"{package_name}.service.impl" + + # 更新上下文中的包相关信息 + context.update( + { + "package": package_name, + "packageName": package_name, + "hasPackageName": bool(package_name), + "basePackage": package_name, + "controllerPackage": controller_package, + "servicePackage": service_package, + "entityPackage": entity_package, + "daoPackage": dao_package, + "dtoPackage": dto_package, + "voPackage": vo_package, + # 添加serviceImpl包路径 + "serviceImplPackage": service_impl_package, + "author": author, + } + ) + + # 更新上下文中的配置相关字段 + context.update( + { + "templateCategory": template_category, + "isDefault": template_category == "Default", + "isMybatisPlus": template_category == "MybatisPlus", + "isMybatisPlusMixed": template_category == "MybatisPlus-Mixed", + } + ) + + # 为每个模板生成代码 + for template_file in template_files: + try: + # 附加模板从 common 目录加载 + effective_category = ( + template_category if template_file in base_templates else "common" + ) + code = await self._render_template(template_file, context, effective_category) + generated_code[template_file] = { + "filename": self._get_output_filename(template_file, context), + "code": code, + "template_file": template_file, + } + except Exception as e: + generated_code[template_file] = {"error": str(e), "template_file": template_file} + + return { + "table_name": analysis_result["table_name"], + "template_category": template_category, + "generated_code": generated_code, + "generation_statistics": { + "total_files": len(template_files), + "success_files": len([f for f in generated_code.values() if "error" not in f]), + "error_files": len([f for f in generated_code.values() if "error" in f]), + }, + } + + async def _render_template( + self, template_file: str, context: Dict[str, Any], category: str + ) -> str: + """渲染模板文件""" + from pathlib import Path + from ..generator.mustache_engine import MustacheTemplateEngine + + # 确定模板路径 + template_base_path = Path(__file__).parent.parent / "templates" / "java" + + if category == "common": + template_path = template_base_path / "common" / template_file + else: + template_path = template_base_path / category / template_file + + if not template_path.exists(): + raise FileNotFoundError(f"模板文件不存在: {template_path}") + + # 渲染模板 + engine = MustacheTemplateEngine() + return engine.render_file(str(template_path), context) + + def _get_output_filename(self, template_file: str, context: Dict[str, Any]) -> str: + """获取输出文件名""" + from ..generator.template_context import TemplateConfigManager + + template_config = TemplateConfigManager() + path_mapping = template_config.get_output_path_mapping() + + if template_file in path_mapping: + relative_path = path_mapping[template_file] + # 替换路径中的占位符 + file_path = relative_path.format(**context) + file_path = "/".join(part for part in file_path.split("/") if part) + + # 添加包路径结构 + package_name = context.get("package", "com.example") + if package_name: + # 将包名转换为路径 + package_path = package_name.replace(".", "/") + # 对于Java源文件，添加包路径 + if file_path.endswith(".java"): + # 提取文件名和相对目录 + path_parts = file_path.split("/") + if len(path_parts) > 1: + # 移除表名路径，让所有表共享同一个包结构 + relative_dir = "/".join(path_parts[:-1]) + filename = path_parts[-1] + # 构建带包路径的完整路径，不包含表名子包 + return f"{package_path}/{relative_dir}/{filename}" + else: + return f"{package_path}/{file_path}" + else: + # 对于XML等资源文件，不添加包路径，但添加resources前缀 + return f"resources/{file_path}" + + return file_path + else: + # 默认文件名 + return template_file.replace(".mustache", ".java") diff --git a/src/dbjavagenix/database/connection_manager.py b/src/dbjavagenix/database/connection_manager.py index 73ccf6f..b1319a0 100644 --- a/src/dbjavagenix/database/connection_manager.py +++ b/src/dbjavagenix/database/connection_manager.py @@ -1,10 +1,6 @@ """ Database connection manager for DBJavaGenix MCP tools """ -from copy import deepcopy -from collections import OrderedDict -import re -from threading import RLock import uuid import threading from typing import Dict, List, Any, Optional @@ -20,32 +16,6 @@ logger = logging.getLogger(__name__) -_SCHEMA_CHANGE_PATTERN = re.compile( - r"^\s*(?:CREATE|ALTER|DROP|RENAME|TRUNCATE|COMMENT)\b", re.IGNORECASE -) -_METADATA_CACHE_MAX_ENTRIES = 256 - - -def _is_schema_change_query(query: object) -> bool: - """Detect schema-changing SQL after optional leading comments.""" - if not isinstance(query, str): - return False - remaining = query.lstrip() - while remaining: - if remaining.startswith("--") or remaining.startswith("#"): - newline = remaining.find("\n") - if newline < 0: - return False - remaining = remaining[newline + 1 :].lstrip() - elif remaining.startswith("/*"): - end = remaining.find("*/", 2) - if end < 0: - return False - remaining = remaining[end + 2 :].lstrip() - else: - break - return bool(_SCHEMA_CHANGE_PATTERN.match(remaining)) - class ConnectionManager: """Manages database connections for MCP tools""" @@ -55,54 +25,6 @@ def __init__(self): self.connection_configs: Dict[str, DatabaseConfig] = {} self._registry_lock = threading.RLock() self._connection_locks: Dict[str, Any] = {} - self._metadata_cache: OrderedDict[ - tuple[str, str, Optional[str]], Dict[str, Any] - ] = OrderedDict() - self._metadata_cache_lock = RLock() - - def get_cached_metadata( - self, connection_id: str, table_name: str, schema: Optional[str] = None - ) -> Optional[Dict[str, Any]]: - """Return a deep copy of one connection-scoped metadata entry, if present.""" - key = (connection_id, table_name, schema) - with self._metadata_cache_lock: - cached = self._metadata_cache.get(key) - if cached is None: - return None - self._metadata_cache.move_to_end(key) - return deepcopy(cached) - - def cache_metadata( - self, - connection_id: str, - table_name: str, - schema: Optional[str], - metadata: Dict[str, Any], - ) -> None: - """Store a complete metadata document without sharing mutable references.""" - key = (connection_id, table_name, schema) - with self._metadata_cache_lock: - self._metadata_cache.pop(key, None) - self._metadata_cache[key] = deepcopy(metadata) - while len(self._metadata_cache) > _METADATA_CACHE_MAX_ENTRIES: - self._metadata_cache.popitem(last=False) - - def invalidate_metadata_cache(self, connection_id: Optional[str] = None) -> None: - """Invalidate all metadata or only entries belonging to one connection.""" - with self._metadata_cache_lock: - if connection_id is None: - self._metadata_cache.clear() - return - stale_keys = [ - key for key in self._metadata_cache if key[0] == connection_id - ] - for key in stale_keys: - self._metadata_cache.pop(key, None) - - def metadata_cache_size(self) -> int: - """Return the number of cached metadata documents for diagnostics/tests.""" - with self._metadata_cache_lock: - return len(self._metadata_cache) def create_connection(self, config: DatabaseConfig) -> str: """ @@ -155,14 +77,12 @@ def create_connection(self, config: DatabaseConfig) -> str: ) connection.autocommit = True elif config.type == DatabaseType.SQLITE: - # MCP database work runs in worker threads. SQLite's default - # thread affinity would reject a connection created elsewhere. + # MCP handlers may execute this connection in a worker thread. connection = sqlite3.connect(config.database, check_same_thread=False) connection.row_factory = sqlite3.Row # Enable dict-like access else: raise DatabaseConnectionError(f"Unsupported database type: {config.type}") - # Store the connection and its lock atomically with the registry. with self._registry_lock: self.connections[connection_id] = connection self._connection_locks[connection_id] = threading.RLock() @@ -192,8 +112,13 @@ def get_connection(self, connection_id: str) -> Any: Raises: DatabaseConnectionError: If connection not found """ - with self._connection_lock(connection_id): - connection = self._get_connection_unlocked(connection_id) + lock = self._connection_lock_for(connection_id) + with lock: + with self._registry_lock: + connection = self.connections.get(connection_id) + if connection is None: + raise DatabaseConnectionError(f"Connection {connection_id} not found") + try: if getattr(connection, "closed", 0): raise DatabaseConnectionError("connection is closed") @@ -201,45 +126,33 @@ def get_connection(self, connection_id: str) -> Any: connection.ping(reconnect=True) except Exception as exc: logger.warning("Connection %s is dead, removing: %s", connection_id, exc) - self._close_connection_unlocked(connection_id, connection) + self._remove_connection(connection_id, connection) raise DatabaseConnectionError( f"Connection {connection_id} is no longer valid" ) from exc return connection - @contextmanager - def _connection_lock(self, connection_id: str): - """Serialize operations for one connection without blocking the registry.""" + def _connection_lock_for(self, connection_id: str) -> Any: + """Return a per-connection lock, including for legacy test doubles.""" with self._registry_lock: - lock = self._connection_locks.get(connection_id) - if lock is None or connection_id not in self.connections: + if connection_id not in self.connections: raise DatabaseConnectionError(f"Connection {connection_id} not found") - with lock: - yield + # A few integrations inject a connection directly into the public + # mapping. Lazily creating the lock preserves that compatibility. + return self._connection_locks.setdefault(connection_id, threading.RLock()) - def _get_connection_unlocked(self, connection_id: str) -> Any: - with self._registry_lock: - connection = self.connections.get(connection_id) - if connection is None: - raise DatabaseConnectionError(f"Connection {connection_id} not found") - return connection - - def _close_connection_unlocked(self, connection_id: str, connection: Any) -> bool: + def _remove_connection(self, connection_id: str, connection: Any) -> None: """Close and remove a connection while its per-connection lock is held.""" try: connection.close() - closed = True except Exception as exc: logger.error("Error closing connection %s: %s", connection_id, exc) - closed = True finally: with self._registry_lock: self.connections.pop(connection_id, None) self.connection_configs.pop(connection_id, None) self._connection_locks.pop(connection_id, None) - self.invalidate_metadata_cache(connection_id) logger.info("Closed connection %s", connection_id) - return closed def close_connection(self, connection_id: str) -> bool: """ @@ -253,17 +166,16 @@ def close_connection(self, connection_id: str) -> bool: """ with self._registry_lock: if connection_id not in self.connections: - self.invalidate_metadata_cache(connection_id) return False lock = self._connection_locks.setdefault(connection_id, threading.RLock()) + with lock: - # A preceding close may have removed the connection while this - # caller was waiting for the per-connection lock. with self._registry_lock: - current = self.connections.get(connection_id) - if current is None: + connection = self.connections.get(connection_id) + if connection is None: return False - return self._close_connection_unlocked(connection_id, current) + self._remove_connection(connection_id, connection) + return True def get_connection_info(self, connection_id: str) -> Optional[DatabaseConfig]: """ @@ -312,8 +224,12 @@ def get_cursor(self, connection_id: str): Yields: Database cursor """ - with self._connection_lock(connection_id): - connection = self._get_connection_unlocked(connection_id) + lock = self._connection_lock_for(connection_id) + with lock: + with self._registry_lock: + connection = self.connections.get(connection_id) + if connection is None: + raise DatabaseConnectionError(f"Connection {connection_id} not found") try: if getattr(connection, "closed", 0): raise DatabaseConnectionError("connection is closed") @@ -321,21 +237,18 @@ def get_cursor(self, connection_id: str): connection.ping(reconnect=True) except Exception as exc: logger.warning("Connection %s is dead, removing: %s", connection_id, exc) - self._close_connection_unlocked(connection_id, connection) + self._remove_connection(connection_id, connection) raise DatabaseConnectionError( f"Connection {connection_id} is no longer valid" ) from exc try: cursor = connection.cursor() except Exception as exc: - logger.warning("Connection %s could not create a cursor: %s", connection_id, exc) - self._close_connection_unlocked(connection_id, connection) + self._remove_connection(connection_id, connection) raise DatabaseConnectionError( f"Connection {connection_id} is no longer valid" ) from exc try: - # Exceptions from the caller's SQL body must propagate without - # evicting a healthy connection from the registry. yield cursor finally: cursor.close() @@ -355,8 +268,11 @@ def execute_query(self, connection_id: str, query: str, params: Optional[tuple] Raises: DatabaseQueryError: If query execution fails """ - schema_change = _is_schema_change_query(query) + connection = None try: + # SQLite defaults to an implicit transaction. Keep its writes + # durable at this boundary without changing the caller's SQL API. + connection = self.get_connection(connection_id) with self.get_cursor(connection_id) as cursor: cursor.execute(query, params or ()) @@ -373,23 +289,30 @@ def execute_query(self, connection_id: str, query: str, params: Optional[tuple] else: # MySQL result.append(dict(zip(columns, row))) - if schema_change: - self.invalidate_metadata_cache(connection_id) - return result else: - if schema_change: - self.invalidate_metadata_cache(connection_id) - return [] # No results (e.g., INSERT/UPDATE/DELETE) + result = [] # No results (e.g., INSERT/UPDATE/DELETE) + + if isinstance(connection, sqlite3.Connection): + connection.commit() + return result except Exception as e: - if schema_change: - self.invalidate_metadata_cache(connection_id) + if isinstance(connection, sqlite3.Connection): + try: + connection.rollback() + except Exception as rollback_error: + logger.warning("SQLite rollback failed after query error: %s", rollback_error) logger.error(f"Query execution failed: {e}") raise DatabaseQueryError(f"Failed to execute query: {str(e)}") def __del__(self): """Clean up connections on destruction""" - for connection_id in list(self.connections.keys()): + try: + with self._registry_lock: + connection_ids = list(self.connections.keys()) + except Exception: + connection_ids = [] + for connection_id in connection_ids: try: self.close_connection(connection_id) except Exception: diff --git a/src/dbjavagenix/database/dialect.py b/src/dbjavagenix/database/dialect.py index fe3afe2..ef7426f 100644 --- a/src/dbjavagenix/database/dialect.py +++ b/src/dbjavagenix/database/dialect.py @@ -28,6 +28,18 @@ _PAREN_PATTERN = re.compile(r"\([^)]*\)") +_JAVA_TYPE_IMPORTS = { + "LocalDateTime": "java.time.LocalDateTime", + "LocalDate": "java.time.LocalDate", + "LocalTime": "java.time.LocalTime", + "OffsetDateTime": "java.time.OffsetDateTime", + "OffsetTime": "java.time.OffsetTime", + "Instant": "java.time.Instant", + "BigDecimal": "java.math.BigDecimal", + "BigInteger": "java.math.BigInteger", + "UUID": "java.util.UUID", +} + def _strip_paren(db_type: str) -> str: """剥离 `VARCHAR(64)` 这种带长度的括号部分,统一小写无空格。""" @@ -84,6 +96,12 @@ def jdbc_type_for(self, db_type: str) -> str: base = _strip_paren(db_type) return self.type_to_jdbc.get(base, "VARCHAR") + def java_imports_for(self, db_type: str) -> list[str]: + """Return imports required by the mapped Java type.""" + java_type = self.java_type_for(db_type) + import_path = _JAVA_TYPE_IMPORTS.get(java_type) + return [import_path] if import_path else [] + def is_string_type(self, db_type: str) -> bool: return _strip_paren(db_type) in self.string_types diff --git a/src/dbjavagenix/database/discovery_tools.py b/src/dbjavagenix/database/discovery_tools.py index 2447ea8..4a2a327 100644 --- a/src/dbjavagenix/database/discovery_tools.py +++ b/src/dbjavagenix/database/discovery_tools.py @@ -5,12 +5,12 @@ 返回的工具名,LLM 仍可直接 call_tool — server 端不限制 (progressive 仅影响 list_tools)。 """ -import json from typing import Any, Dict, List from mcp.types import TextContent, Tool from ..utils.tool_registry import is_progressive_mode_enabled, search_tools_by_query +from ..utils.json_serialization import dumps as _json_dumps def get_discovery_tools() -> List[Tool]: @@ -63,4 +63,4 @@ async def handle_search_tools(arguments: Dict[str, Any]) -> List[TextContent]: "无匹配工具。提示: 试试更宽泛的关键词,如 'connect' / 'table' / 'codegen' / 'spring'。" ) - return [TextContent(type="text", text=json.dumps(payload, ensure_ascii=False, indent=2))] + return [TextContent(type="text", text=_json_dumps(payload, indent=2))] diff --git a/src/dbjavagenix/database/mcp_tools.py b/src/dbjavagenix/database/mcp_tools.py index edfa186..a4a229b 100644 --- a/src/dbjavagenix/database/mcp_tools.py +++ b/src/dbjavagenix/database/mcp_tools.py @@ -1,10 +1,7 @@ """ MCP tools for database connection and basic query operations """ -from base64 import b64encode import asyncio -from datetime import date, datetime, time, timedelta -from decimal import Decimal import json import logging import os @@ -27,6 +24,7 @@ ) from ..database.connection_manager import connection_manager from ..database.introspection import DatabaseIntrospector +from ..database.dialect import get_dialect, list_supported_dialects from ..database.sql_identifiers import quote_mysql_identifier from ..database.capabilities import supported_database_type_values from ..config.config_manager import ConfigManager @@ -57,7 +55,54 @@ ("FOR", "KEY", "SHARE"), ("LOCK", "IN", "SHARE", "MODE"), ) -_DOLLAR_QUOTE_PATTERN = re.compile(r"\$(?:[A-Za-z_][A-Za-z0-9_]*)?\$") + + +async def _run_db_call(callable_obj, *args, **kwargs): + """Run one blocking database operation outside the MCP event loop.""" + return await asyncio.to_thread(callable_obj, *args, **kwargs) + + +async def _run_db_query(connection_id: str, query: str, params: Optional[tuple] = None): + """Execute SQL in a worker while preserving the two-argument call contract.""" + if params is None: + return await _run_db_call(connection_manager.execute_query, connection_id, query) + return await _run_db_call(connection_manager.execute_query, connection_id, query, params) + + +async def _run_async_db_call(callable_obj, *args, **kwargs): + """Run an async analyzer in a worker when its internals perform blocking I/O.""" + def run(): + return asyncio.run(callable_obj(*args, **kwargs)) + + return await _run_db_call(run) + + +def _connect_and_probe(config: DatabaseConfig) -> tuple[str, str]: + """Create a connection and perform the initial blocking connectivity probe.""" + connection_id = connection_manager.create_connection(config) + connection_manager.get_connection(connection_id) + server_info = "" + try: + if config.type == DatabaseType.MYSQL: + with connection_manager.get_cursor(connection_id) as cursor: + cursor.execute("SELECT VERSION() as version") + result = cursor.fetchone() + if result: + version = result[0] if isinstance(result, tuple) else result["version"] + server_info = f"MySQL {version}" + elif config.type == DatabaseType.POSTGRESQL: + with connection_manager.get_cursor(connection_id) as cursor: + cursor.execute("SELECT version() AS version") + result = cursor.fetchone() + if result: + version = result[0] if isinstance(result, tuple) else result["version"] + server_info = f"PostgreSQL {version}" + elif config.type == DatabaseType.SQLITE: + server_info = "SQLite" + except Exception as exc: + logger.warning("Could not get server info: %s", exc) + server_info = f"{config.type.value} (version unknown)" + return connection_id, server_info def _resolve_codegen_output_path(base_dir: Path, relative_path: object) -> Path: @@ -93,6 +138,12 @@ def _display_codegen_path(path: Path, project_root: Path) -> str: return str(path.absolute()) +def _write_codegen_file(path: Path, code: str) -> None: + """Create the parent directory and write one generated file.""" + path.parent.mkdir(parents=True, exist_ok=True) + path.write_text(code, encoding="utf-8") + + def _tokenize_read_only_sql(query: str) -> List[tuple[str, str]]: """Tokenize enough SQL to enforce the single, read-only statement contract.""" tokens: List[tuple[str, str]] = [] @@ -108,17 +159,6 @@ def _tokenize_read_only_sql(query: str) -> List[tuple[str, str]]: if query.startswith("/*", index) or query.startswith("--", index) or char == "#": raise MCPServiceError("SQL comments are not allowed in read-only queries") - dollar_quote = _DOLLAR_QUOTE_PATTERN.match(query, index) - if dollar_quote: - delimiter = dollar_quote.group(0) - end = query.find(delimiter, dollar_quote.end()) - if end < 0: - raise MCPServiceError("Unterminated dollar-quoted value in SQL query") - end += len(delimiter) - tokens.append(("quoted", query[index:end])) - index = end - continue - if char in "'\"`": quote = char start = index @@ -196,13 +236,6 @@ def _validate_read_only_query(query: Any) -> str: if _contains_locking_read_clause(tokens): raise MCPServiceError("Only non-locking read-only SELECT queries are allowed") - if re.search( - r"\bFETCH\s+(?:FIRST|NEXT)\s+\d+\s+ROWS?\s+WITH\s+TIES\b", - _mask_sql_literals(normalized), - re.IGNORECASE, - ): - raise MCPServiceError("FETCH WITH TIES is not supported because it can exceed the row limit") - depth = 0 top_level_select = first_word == "SELECT" for kind, value in tokens: @@ -231,8 +264,19 @@ def _validate_read_only_query(query: Any) -> str: def _has_top_level_limit_clause(query: str) -> bool: - """Return whether a query contains a cap understood by the rewriter.""" - return _top_level_limit_span(query) is not None + tokens = _tokenize_read_only_sql(query) + depth = 0 + for index, (kind, value) in enumerate(tokens): + if kind == "symbol": + if value == "(": + depth += 1 + elif value == ")": + depth -= 1 + elif kind == "word" and value == "LIMIT" and depth == 0: + next_token = tokens[index + 1] if index + 1 < len(tokens) else None + if next_token and (next_token[0] == "symbol" or next_token[1] == "ALL"): + return True + return False def _top_level_limit_span(query: str) -> tuple[int, int, int | None] | None: @@ -242,56 +286,9 @@ def _top_level_limit_span(query: str) -> tuple[int, int, int | None] | None: cannot affect the server-side result cap. The optional ``LIMIT offset,count`` form returns the row-count span rather than the offset span. """ - masked_query = _mask_sql_literals(query) - - def is_top_level(position: int) -> bool: - depth = 0 - for char in masked_query[:position]: - if char == "(": - depth += 1 - elif char == ")": - depth -= 1 - return depth == 0 - - offset_form = re.compile(r"\bLIMIT\s+\d+\s*,\s*(\d+)\b", re.IGNORECASE) - for match in offset_form.finditer(masked_query): - if is_top_level(match.start()): - return match.start(1), match.end(1), int(match.group(1)) - - simple_form = re.compile(r"\bLIMIT\s+(ALL|\d+)\b", re.IGNORECASE) - for match in simple_form.finditer(masked_query): - if is_top_level(match.start()): - value = match.group(1).upper() - return match.start(1), match.end(1), None if value == "ALL" else int(value) - - fetch_form = re.compile( - r"\bFETCH\s+(?:FIRST|NEXT)\s+(\d+)\s+ROWS?\s+ONLY\b", re.IGNORECASE - ) - for match in fetch_form.finditer(masked_query): - if is_top_level(match.start()): - return match.start(1), match.end(1), int(match.group(1)) - return None - - -def _mask_sql_literals(query: str) -> str: - """Replace quoted SQL literals with spaces while preserving offsets.""" masked = list(query) index = 0 while index < len(masked): - dollar_quote = _DOLLAR_QUOTE_PATTERN.match(query, index) - if dollar_quote: - delimiter = dollar_quote.group(0) - end = query.find(delimiter, dollar_quote.end()) - if end < 0: - for position in range(index, len(masked)): - masked[position] = " " - break - end += len(delimiter) - for position in range(index, end): - masked[position] = " " - index = end - continue - if masked[index] not in "'\"`": index += 1 continue @@ -311,7 +308,29 @@ def _mask_sql_literals(query: str) -> str: index += 2 continue index += 1 - return "".join(masked) + + masked_query = "".join(masked) + + def is_top_level(position: int) -> bool: + depth = 0 + for char in masked_query[:position]: + if char == "(": + depth += 1 + elif char == ")": + depth -= 1 + return depth == 0 + + offset_form = re.compile(r"\bLIMIT\s+\d+\s*,\s*(\d+)\b", re.IGNORECASE) + for match in offset_form.finditer(masked_query): + if is_top_level(match.start()): + return match.start(1), match.end(1), int(match.group(1)) + + simple_form = re.compile(r"\bLIMIT\s+(ALL|\d+)\b", re.IGNORECASE) + for match in simple_form.finditer(masked_query): + if is_top_level(match.start()): + value = match.group(1).upper() + return match.start(1), match.end(1), None if value == "ALL" else int(value) + return None def _apply_query_limit(query: str, limit: int) -> str: @@ -332,66 +351,6 @@ def _quote_mysql_identifier(identifier: Any) -> str: return quote_mysql_identifier(identifier) except ValueError as exc: raise MCPServiceError(str(exc)) from exc - - -async def _run_db_call(callable_obj, *args): - """Run one blocking database operation outside the MCP event loop.""" - return await asyncio.to_thread(callable_obj, *args) - - -async def _run_async_db_call(callable_obj, *args, **kwargs): - """Run an async analyzer whose internals contain blocking DB calls in a worker.""" - def run() -> Any: - return asyncio.run(callable_obj(*args, **kwargs)) - - return await asyncio.to_thread(run) - - -def _connect_and_probe(config: DatabaseConfig) -> tuple[str, str]: - """Create a connection and perform the initial blocking connectivity probe.""" - connection_id = connection_manager.create_connection(config) - connection_manager.get_connection(connection_id) - server_info = "" - try: - if config.type == DatabaseType.MYSQL: - with connection_manager.get_cursor(connection_id) as cursor: - cursor.execute("SELECT VERSION() as version") - result = cursor.fetchone() - if result: - server_info = f"MySQL {result[0] if isinstance(result, tuple) else result['version']}" - elif config.type == DatabaseType.POSTGRESQL: - with connection_manager.get_cursor(connection_id) as cursor: - cursor.execute("SELECT version() AS version") - result = cursor.fetchone() - if result: - server_info = ( - f"PostgreSQL {result[0] if isinstance(result, tuple) else result['version']}" - ) - elif config.type == DatabaseType.SQLITE: - server_info = "SQLite" - except Exception as exc: - logger.warning("Could not get server info: %s", exc) - server_info = f"{config.type.value} (version unknown)" - return connection_id, server_info - - -def _collect_all_table_names(connection_id: str, config: DatabaseConfig) -> List[str]: - """Collect table names for codegen prefix analysis inside a DB worker.""" - try: - with connection_manager.get_cursor(connection_id) as cursor: - if config.type == DatabaseType.MYSQL: - cursor.execute("SHOW TABLES") - elif config.type == DatabaseType.SQLITE: - cursor.execute( - "SELECT name FROM sqlite_master " - "WHERE type='table' AND name NOT LIKE 'sqlite_%'" - ) - else: - return [] - return [row[0] for row in cursor.fetchall()] - except Exception as exc: - logger.warning("Failed to get all table names for prefix analysis: %s", exc) - return [] def get_connection_tools() -> List[Tool]: @@ -402,8 +361,8 @@ def get_connection_tools() -> List[Tool]: List of MCP Tool objects """ return [ - Tool( - name="db_connect_test", + Tool( + name="db_connect_test", description="Test database connection and create connection session", inputSchema={ "type": "object", @@ -440,12 +399,28 @@ def get_connection_tools() -> List[Tool]: "default": "utf8mb4" } }, - "required": ["host", "port", "username", "password", "database_type"] - } - ), - - Tool( - name="db_query_databases", + "required": ["host", "port", "username", "password", "database_type"] + } + ), + + Tool( + name="db_disconnect", + description="Close an existing database connection session", + inputSchema={ + "type": "object", + "properties": { + "connection_id": { + "type": "string", + "description": "Connection identifier returned by db_connect_test", + "minLength": 1, + } + }, + "required": ["connection_id"], + }, + ), + + Tool( + name="db_query_databases", description="List all databases on the server", inputSchema={ "type": "object", @@ -685,7 +660,7 @@ def get_table_analysis_tools() -> List[Tool]: ] -async def handle_db_connect_test(arguments: Dict[str, Any]) -> List[TextContent]: +async def handle_db_connect_test(arguments: Dict[str, Any]) -> List[TextContent]: """ Handle database connection test @@ -707,7 +682,6 @@ async def handle_db_connect_test(arguments: Dict[str, Any]) -> List[TextContent] charset=arguments.get("charset", "utf8mb4") ) - # Connection setup and the initial probe are blocking driver calls. connection_id, server_info = await _run_db_call(_connect_and_probe, config) response = { @@ -755,9 +729,69 @@ async def handle_db_connect_test(arguments: Dict[str, Any]) -> List[TextContent] type="text", text=f"Unexpected error: {safe_error}\n\nRaw Response: {_json_dumps(error_response, ensure_ascii=False)}" )] - - -async def handle_db_query_databases(arguments: Dict[str, Any]) -> List[TextContent]: + + +async def handle_db_disconnect(arguments: Dict[str, Any]) -> List[TextContent]: + """Close a connection session and return a stable lifecycle response.""" + raw_connection_id = arguments.get("connection_id") if isinstance(arguments, dict) else None + connection_id = raw_connection_id.strip() if isinstance(raw_connection_id, str) else "" + + if not connection_id: + response = { + "success": False, + "error": "connection_not_found", + "connection_id": None, + "message": "Connection not found", + } + return [TextContent( + type="text", + text=f"Failed to disconnect connection: {response['message']}\n\n" + f"Raw Response: {_json_dumps(response, ensure_ascii=False)}", + )] + + try: + closed = await _run_db_call(connection_manager.close_connection, connection_id) + except Exception as exc: + safe_error = redact_sensitive_text(exc) + logger.error("Unexpected error in db_disconnect: %s", safe_error) + response = { + "success": False, + "error": "disconnect_failed", + "connection_id": connection_id, + "message": safe_error, + } + return [TextContent( + type="text", + text=f"Failed to disconnect connection: {safe_error}\n\n" + f"Raw Response: {_json_dumps(response, ensure_ascii=False)}", + )] + + if not closed: + response = { + "success": False, + "error": "connection_not_found", + "connection_id": connection_id, + "message": "Connection not found", + } + return [TextContent( + type="text", + text=f"Failed to disconnect connection: {response['message']}\n\n" + f"Raw Response: {_json_dumps(response, ensure_ascii=False)}", + )] + + response = { + "success": True, + "connection_id": connection_id, + "message": "Connection closed successfully", + } + return [TextContent( + type="text", + text=f"Database connection closed.\n\nRaw Response: " + f"{_json_dumps(response, ensure_ascii=False)}", + )] + + +async def handle_db_query_databases(arguments: Dict[str, Any]) -> List[TextContent]: """ Handle listing databases @@ -800,7 +834,7 @@ async def handle_db_query_databases(arguments: Dict[str, Any]) -> List[TextConte else: raise MCPServiceError(f"Listing databases not implemented for {config.type}") - results = await _run_db_call(connection_manager.execute_query, connection_id, query) + results = await _run_db_query(connection_id, query) # Extract database names databases = [] @@ -891,9 +925,9 @@ async def handle_db_query_tables(arguments: Dict[str, Any]) -> List[TextContent] raise MCPServiceError(f"Listing tables not implemented for {config.type}") if params is None: - results = await _run_db_call(connection_manager.execute_query, connection_id, query) + results = await _run_db_query(connection_id, query) else: - results = await _run_db_call(connection_manager.execute_query, connection_id, query, params) + results = await _run_db_query(connection_id, query, params) # Extract table names tables = [] @@ -979,9 +1013,7 @@ async def handle_db_query_table_exists(arguments: Dict[str, Any]) -> List[TextCo FROM information_schema.TABLES WHERE TABLE_SCHEMA = %s AND TABLE_NAME = %s """ - results = await _run_db_call( - connection_manager.execute_query, connection_id, query, (database, table) - ) + results = await _run_db_query(connection_id, query, (database, table)) elif config.type == DatabaseType.POSTGRESQL: schema_filter = "AND table_schema = %s" if schema else "" @@ -994,7 +1026,7 @@ async def handle_db_query_table_exists(arguments: Dict[str, Any]) -> List[TextCo {schema_filter} """.format(schema_filter=schema_filter) params = (database, table, schema) if schema else (database, table) - results = await _run_db_call(connection_manager.execute_query, connection_id, query, params) + results = await _run_db_query(connection_id, query, params) elif config.type == DatabaseType.SQLITE: query = """ @@ -1002,7 +1034,7 @@ async def handle_db_query_table_exists(arguments: Dict[str, Any]) -> List[TextCo FROM sqlite_master WHERE type='table' AND name = ? """ - results = await _run_db_call(connection_manager.execute_query, connection_id, query, (table,)) + results = await _run_db_query(connection_id, query, (table,)) else: raise MCPServiceError(f"Table existence check not implemented for {config.type}") @@ -1070,7 +1102,7 @@ async def handle_db_query_execute(arguments: Dict[str, Any]) -> List[TextContent query = _apply_query_limit(query, limit) - results = await _run_db_call(connection_manager.execute_query, connection_id, query) + results = await _run_db_query(connection_id, query) response = { "success": True, @@ -1097,7 +1129,9 @@ async def handle_db_query_execute(arguments: Dict[str, Any]) -> List[TextContent else: result_text = "Query executed successfully. No rows returned." - result_text += "\n\nRaw Response: " + _json_dumps(response) + result_text += "\n\nRaw Response: " + _json_dumps( + response, ensure_ascii=False + ) return [TextContent( type="text", @@ -1138,17 +1172,32 @@ def _get_java_type_mapping(db_type: DatabaseType, column_type: str, precision: O precision: Numeric precision scale: Numeric scale - Returns: - Dict with java_type and imports - """ - try: + Returns: + Dict with java_type and imports + + Supported runtime dialects use the same adapter as code generation. The + legacy YAML mapping remains a fallback for dialects without a registered + runtime adapter. + """ + try: + db_key = ( + db_type.value.lower() + if isinstance(db_type, DatabaseType) + else str(db_type).lower() + ) + if db_key in list_supported_dialects(): + dialect = get_dialect(db_key) + return { + "java_type": dialect.java_type_for(column_type), + "imports": dialect.java_imports_for(column_type), + } + config_manager = ConfigManager() if hasattr(config_manager, "get_type_mapping"): type_mapping = config_manager.get_type_mapping() else: type_mapping = _load_default_type_mapping() - db_key = db_type.value.lower() column_type_upper = column_type.upper().strip() base_type = re.sub(r"\([^)]*\)", "", column_type_upper) base_type = re.sub(r"\s+", " ", base_type).strip() @@ -1286,7 +1335,7 @@ async def handle_db_table_describe(arguments: Dict[str, Any]) -> List[TextConten for imp in sorted(java_imports): result_text += f"import {imp};\n" - result_text += f"\nRaw Response: {_json_dumps(response)}" + result_text += f"\nRaw Response: {_json_dumps(response, ensure_ascii=False)}" return [TextContent( type="text", @@ -1356,9 +1405,7 @@ async def handle_db_table_columns(arguments: Dict[str, Any]) -> List[TextContent WHERE TABLE_SCHEMA = %s AND TABLE_NAME = %s ORDER BY ORDINAL_POSITION """ - results = await _run_db_call( - connection_manager.execute_query, connection_id, query, (database, table) - ) + results = await _run_db_query(connection_id, query, (database, table)) elif config.type == DatabaseType.POSTGRESQL: columns = await _run_db_call( @@ -1430,7 +1477,7 @@ async def handle_db_table_columns(arguments: Dict[str, Any]) -> List[TextContent result_text += f" Comment: {row['COLUMN_COMMENT']}\n" result_text += "\n" - result_text += f"Raw Response: {_json_dumps(response)}" + result_text += f"Raw Response: {_json_dumps(response, ensure_ascii=False)}" return [TextContent( type="text", @@ -1494,9 +1541,7 @@ async def handle_db_table_primary_keys(arguments: Dict[str, Any]) -> List[TextCo AND CONSTRAINT_NAME = 'PRIMARY' ORDER BY ORDINAL_POSITION """ - results = await _run_db_call( - connection_manager.execute_query, connection_id, query, (database, table) - ) + results = await _run_db_query(connection_id, query, (database, table)) elif config.type == DatabaseType.POSTGRESQL: primary_keys = await _run_db_call( @@ -1540,7 +1585,7 @@ async def handle_db_table_primary_keys(arguments: Dict[str, Any]) -> List[TextCo else: result_text = f"No primary keys found for table {database}.{table}\n" - result_text += f"\nRaw Response: {_json_dumps(response)}" + result_text += f"\nRaw Response: {_json_dumps(response, ensure_ascii=False)}" return [TextContent( type="text", @@ -1612,9 +1657,7 @@ async def handle_db_table_foreign_keys(arguments: Dict[str, Any]) -> List[TextCo AND kcu.REFERENCED_TABLE_NAME IS NOT NULL ORDER BY kcu.ORDINAL_POSITION """ - results = await _run_db_call( - connection_manager.execute_query, connection_id, query, (database, table) - ) + results = await _run_db_query(connection_id, query, (database, table)) elif config.type == DatabaseType.POSTGRESQL: foreign_keys = await _run_db_call( @@ -1690,7 +1733,7 @@ async def handle_db_table_foreign_keys(arguments: Dict[str, Any]) -> List[TextCo else: result_text = f"No foreign keys found for table {database}.{table}\n" - result_text += f"Raw Response: {_json_dumps(response)}" + result_text += f"Raw Response: {_json_dumps(response, ensure_ascii=False)}" return [TextContent( type="text", @@ -1758,9 +1801,7 @@ async def handle_db_table_indexes(arguments: Dict[str, Any]) -> List[TextContent AND TABLE_NAME = %s ORDER BY INDEX_NAME, SEQ_IN_INDEX """ - results = await _run_db_call( - connection_manager.execute_query, connection_id, query, (database, table) - ) + results = await _run_db_query(connection_id, query, (database, table)) elif config.type == DatabaseType.POSTGRESQL: indexes = await _run_db_call( @@ -1854,7 +1895,7 @@ async def handle_db_table_indexes(arguments: Dict[str, Any]) -> List[TextContent else: result_text = f"No indexes found for table {database}.{table}\n" - result_text += f"Raw Response: {_json_dumps(response)}" + result_text += f"Raw Response: {_json_dumps(response, ensure_ascii=False)}" return [TextContent( type="text", @@ -1934,6 +1975,16 @@ def get_codegen_tools() -> List[Tool]: "description": "Java package name for generated code", "default": "com.example.generated" }, + "generate_dto": { + "type": "boolean", + "description": "Generate a DTO artifact when supported", + "default": False + }, + "generate_vo": { + "type": "boolean", + "description": "Generate a VO artifact when supported", + "default": False + }, "project_path": { "type": "string", "description": "Target Spring Boot project path (with src/main/java)", @@ -1995,6 +2046,16 @@ def get_codegen_tools() -> List[Tool]: "type": "string", "description": "Optional explicit output directory; defaults to the project source structure" }, + "generate_dto": { + "type": "boolean", + "description": "Generate a DTO artifact when supported", + "default": False + }, + "generate_vo": { + "type": "boolean", + "description": "Generate a VO artifact when supported", + "default": False + }, "include_swagger": { "type": "boolean", "description": "Include Swagger annotations in generated code", @@ -2036,7 +2097,9 @@ async def handle_db_codegen_analyze(arguments: Dict[str, Any]) -> List[TextConte schema = arguments.get("schema") or None template_category = arguments.get("template_category", "MybatisPlus-Mixed") author = arguments.get("author", "ZXP") - package_name = arguments.get("package_name", "com.example.generated") + package_name = arguments.get("package_name", "com.example.generated") + generate_dto = arguments.get("generate_dto") + generate_vo = arguments.get("generate_vo") # Validate connection exists config = connection_manager.get_connection_info(connection_id) @@ -2074,6 +2137,13 @@ async def handle_db_codegen_analyze(arguments: Dict[str, Any]) -> List[TextConte "isMybatisPlusMixed": template_category == "MybatisPlus-Mixed", "isSb35Java21": template_category == "sb35-java21", }) + from ..generator.template_context import apply_generation_options + + apply_generation_options( + analysis_result["template_context"], + generate_dto=generate_dto, + generate_vo=generate_vo, + ) # Format response text result_text = f"Code Generation Analysis: {table_name}\n" @@ -2122,7 +2192,7 @@ async def handle_db_codegen_analyze(arguments: Dict[str, Any]) -> List[TextConte # Keep a structured payload for non-MCP callers such as the CLI. result_text += "\n\nRaw Response: " + _json_dumps( - {"success": True, **analysis_result} + {"success": True, **analysis_result}, ensure_ascii=False ) return [TextContent( @@ -2178,6 +2248,8 @@ async def handle_db_codegen_generate(arguments: Dict[str, Any]) -> List[TextCont include_swagger = arguments.get("include_swagger", True) include_lombok = arguments.get("include_lombok", True) include_mapstruct = arguments.get("include_mapstruct", True) + generate_dto = arguments.get("generate_dto") + generate_vo = arguments.get("generate_vo") project_path = arguments.get("project_path") output_dir_arg = arguments.get("output_dir") @@ -2253,15 +2325,12 @@ async def handle_db_codegen_generate(arguments: Dict[str, Any]) -> List[TextCont if needs_attention: dependency_warnings.append("⚠️ Multiple dependency issues detected - Generated code may not compile correctly") - # ===== STEP 1: 获取数据库所有表名以支持前缀分析 ===== - logger.info("🔍 Getting all table names for package structure optimization...") - - # 获取数据库中的所有表名用于前缀分析 - config = connection_manager.get_connection_info(connection_id) - all_table_names = await _run_db_call(_collect_all_table_names, connection_id, config) - if not all_table_names: - all_table_names = [table_name] - logger.info("Found %s tables for prefix analysis: %s", len(all_table_names), all_table_names) + # ===== STEP 1: 获取数据库所有表名以支持前缀分析 ===== + logger.info("🔍 Getting all table names for package structure optimization...") + all_table_names = await _run_db_call( + _collect_codegen_table_names, connection_id, table_name + ) + logger.info(f"Found {len(all_table_names)} tables for prefix analysis: {all_table_names}") # ===== STEP 2: 分析表结构（包含前缀优化） ===== # Initialize analyzer and generator @@ -2326,12 +2395,22 @@ def with_suffix(kind: str) -> str: ctx["hasJakarta"] = False except Exception: pass - - # ===== STEP 3: 生成代码 ===== + + from ..generator.template_context import apply_generation_options + + apply_generation_options( + analysis_result["template_context"], + generate_dto=generate_dto, + generate_vo=generate_vo, + ) + + # ===== STEP 3: 生成代码 ===== generation_config = { "author": author, "package_name": package_name, - "output_dir": output_dir_arg or "generated_output" + "output_dir": output_dir_arg or "generated_output", + "generate_dto": generate_dto, + "generate_vo": generate_vo, } generation_result = await generator.generate_code( @@ -2403,6 +2482,7 @@ def with_suffix(kind: str) -> str: ) except ValueError as path_error: file_info["write_error"] = f"Unsafe output path: {path_error}" + file_info["write_error_kind"] = "unsafe_path" logger.warning( "Rejected generated file path %r for %s: %s", raw_relative_path, @@ -2411,22 +2491,18 @@ def with_suffix(kind: str) -> str: ) continue - if output_dir == resources_dir: - resource_files.append(str(full_output_path)) - else: - written_files.append(str(full_output_path)) - - # 确保父目录存在 - full_output_path.parent.mkdir(parents=True, exist_ok=True) - - # 写入文件 - try: - with open(full_output_path, 'w', encoding='utf-8') as f: - f.write(file_info["code"]) - logger.info(f"Successfully wrote file: {full_output_path}") - except Exception as write_error: - logger.error(f"Failed to write file {full_output_path}: {write_error}") - file_info["write_error"] = str(write_error) + # Register a path only after both directory creation and the write succeed. + try: + _write_codegen_file(full_output_path, file_info["code"]) + if output_dir == resources_dir: + resource_files.append(str(full_output_path)) + else: + written_files.append(str(full_output_path)) + logger.info(f"Successfully wrote file: {full_output_path}") + except Exception as write_error: + logger.error(f"Failed to write file {full_output_path}: {write_error}") + file_info["write_error"] = str(write_error) + file_info["write_error_kind"] = "write_failed" # ===== STEP 5: 格式化增强响应（包含包结构优化信息） ===== result_text = f"🚀 Code Generation Complete: {table_name}\n" @@ -2502,10 +2578,15 @@ def with_suffix(kind: str) -> str: resource_file_count = 0 for template_file, file_info in generated_files.items(): - if "error" in file_info: - result_text += f" ❌ {template_file}: {file_info['error']}\n" - elif "write_error" in file_info: - result_text += f" ⚠️ {file_info['filename']}: Generated but write failed - {file_info['write_error']}\n" + if "error" in file_info: + result_text += f" ❌ {template_file}: {file_info['error']}\n" + elif "write_error" in file_info: + error_kind = file_info.get("write_error_kind", "write_failed") + label = "Path rejected" if error_kind == "unsafe_path" else "Write failed" + result_text += ( + f" ⚠️ {file_info['filename']}: Generated but {label.lower()} - " + f"{file_info['write_error']}\n" + ) else: filename = file_info["filename"] code_lines = len(file_info["code"].split('\n')) @@ -2521,16 +2602,41 @@ def with_suffix(kind: str) -> str: result_text += f" ☕ {written_path} ({code_lines} lines)\n" java_file_count += 1 - # 文件统计 - total_written = len(written_files) + len(resource_files) - result_text += "\n📈 File Writing Summary:\n" - result_text += f" Java Files: {java_file_count} written to {java_source_dir.absolute()}\n" - result_text += f" Resource Files: {resource_file_count} written to {resources_dir.absolute()}\n" - result_text += f" Total Files: {total_written}\n" - - if total_written > 0: - result_text += "\n🎉 SUCCESS: All files written to SpringBoot project structure!\n" - result_text += f"📁 Working Directory: {Path.cwd().absolute()}\n" + # 文件统计 + total_written = len(written_files) + len(resource_files) + write_candidate_count = sum("error" not in file_info for file_info in generated_files.values()) + path_rejected_count = sum( + file_info.get("write_error_kind") == "unsafe_path" + for file_info in generated_files.values() + ) + write_failure_count = sum( + file_info.get("write_error_kind") == "write_failed" + for file_info in generated_files.values() + ) + write_attempt_count = write_candidate_count - path_rejected_count + result_text += "\n📈 File Writing Summary:\n" + result_text += f" Write Candidates: {write_candidate_count}\n" + result_text += f" Write Attempts: {write_attempt_count}\n" + result_text += f" Write Succeeded: {total_written}\n" + result_text += f" Paths Rejected: {path_rejected_count}\n" + result_text += f" Write Failed: {write_failure_count}\n" + result_text += f" Java Files: {java_file_count} written to {java_source_dir.absolute()}\n" + result_text += f" Resource Files: {resource_file_count} written to {resources_dir.absolute()}\n" + result_text += f" Total Files: {total_written}\n" + + generation_failure_count = stats["error_files"] + if ( + generation_failure_count == 0 + and path_rejected_count == 0 + and write_failure_count == 0 + and total_written == write_candidate_count + ): + result_text += "\n🎉 SUCCESS: All files written to SpringBoot project structure!\n" + result_text += f"📁 Working Directory: {Path.cwd().absolute()}\n" + elif total_written > 0: + result_text += "\n⚠️ PARTIAL: Some generated files were not written. Review the file errors above.\n" + else: + result_text += "\n❌ FAILED: No generated files were written. Review the file errors above.\n" # 简化的代码预览（仅显示文件名，不显示完整代码） result_text += "\n📝 Generated Code Preview:\n" @@ -2649,6 +2755,16 @@ def _detect_project_structure(project_path: Optional[str] = None) -> Dict[str, P """Resolve project structure from an explicit path or the current directory.""" start_dir = Path(project_path).expanduser() if project_path else None return detect_springboot_project_structure(start_dir) + + +def _collect_codegen_table_names(connection_id: str, fallback_table: str) -> List[str]: + """Collect table names through the shared introspection contract.""" + try: + table_names = DatabaseIntrospector(connection_manager).list_tables(connection_id) + return table_names or [fallback_table] + except Exception as exc: # noqa: BLE001 + logger.warning("Failed to get all table names for prefix analysis: %s", exc) + return [fallback_table] def get_springboot_project_tools() -> List[Tool]: diff --git a/src/dbjavagenix/database/observability_tools.py b/src/dbjavagenix/database/observability_tools.py index bc8ca51..a0549c7 100644 --- a/src/dbjavagenix/database/observability_tools.py +++ b/src/dbjavagenix/database/observability_tools.py @@ -2,7 +2,6 @@ P5: 可观测性工具 - server_metrics + server_health。 """ -import json import os import platform import sys @@ -12,6 +11,7 @@ from mcp.types import TextContent, Tool from ..utils.metrics import GLOBAL_TOOL_METRICS +from ..utils.json_serialization import dumps as _json_dumps def get_observability_tools() -> List[Tool]: @@ -62,10 +62,12 @@ async def handle_server_metrics(arguments: Dict[str, Any]) -> List[TextContent]: if arguments.get("reset"): GLOBAL_TOOL_METRICS.reset() payload["reset"] = True - return [TextContent( - type="text", - text=json.dumps(payload, ensure_ascii=False, indent=2), - )] + return [ + TextContent( + type="text", + text=_json_dumps(payload, indent=2), + ) + ] async def handle_server_health(arguments: Dict[str, Any]) -> List[TextContent]: @@ -92,18 +94,21 @@ async def handle_server_health(arguments: Dict[str, Any]) -> List[TextContent]: try: import mcp + mcp_version = getattr(mcp, "__version__", "unknown") except Exception: # noqa: BLE001 mcp_version = "unknown" try: from ..database.connection_manager import connection_manager + active_connections = len(connection_manager.list_connections()) except Exception: # noqa: BLE001 active_connections = -1 # not initialized try: import anthropic + anthropic_version = getattr(anthropic, "__version__", "installed") except ImportError: anthropic_version = "not_installed" @@ -127,20 +132,25 @@ async def handle_server_health(arguments: Dict[str, Any]) -> List[TextContent]: "active_connections": active_connections, }, "ai": { - "llm_available": bool(os.environ.get("ANTHROPIC_API_KEY")) and anthropic_version != "not_installed", - "progressive_mode": os.environ.get("DBJAVAGENIX_PROGRESSIVE", "").lower() in ("1", "true", "yes", "on"), + "llm_available": bool(os.environ.get("ANTHROPIC_API_KEY")) + and anthropic_version != "not_installed", + "progressive_mode": os.environ.get("DBJAVAGENIX_PROGRESSIVE", "").lower() + in ("1", "true", "yes", "on"), }, } - return [TextContent( - type="text", - text=json.dumps(health, ensure_ascii=False, indent=2), - )] + return [ + TextContent( + type="text", + text=_json_dumps(health, indent=2), + ) + ] def _read_version() -> str: """从 dbjavagenix.__init__ 读取版本""" try: import dbjavagenix + return getattr(dbjavagenix, "__version__", "0.2.0") except Exception: # noqa: BLE001 return "0.2.0" diff --git a/src/dbjavagenix/database/schema_algorithms_tools.py b/src/dbjavagenix/database/schema_algorithms_tools.py index eb48b93..8b04c5d 100644 --- a/src/dbjavagenix/database/schema_algorithms_tools.py +++ b/src/dbjavagenix/database/schema_algorithms_tools.py @@ -25,8 +25,10 @@ from ..algorithms import ( cluster_tables, find_cycles, + normalize_graph_input, topological_sort, ) +from ..utils.json_serialization import dumps as _json_dumps SCHEMA_TOPO_TOOL = Tool( @@ -113,13 +115,7 @@ def _parse_input(arguments: dict[str, Any]) -> tuple[list[str], list[tuple[str, str]]]: - tables = list(arguments.get("tables", [])) - raw_fks = arguments.get("fks", []) - fks: list[tuple[str, str]] = [] - for fk in raw_fks: - if isinstance(fk, (list, tuple)) and len(fk) == 2: - fks.append((str(fk[0]), str(fk[1]))) - return tables, fks + return normalize_graph_input(arguments.get("tables"), arguments.get("fks")) async def handle_schema_topo_order(arguments: dict[str, Any]) -> list[TextContent]: @@ -163,9 +159,7 @@ async def handle_schema_check_cycles( def _render(payload: dict[str, Any]) -> str: - import json - - return json.dumps(payload, ensure_ascii=False, indent=2) + return _json_dumps(payload, indent=2) SCHEMA_ALGORITHM_TOOLS = [ diff --git a/src/dbjavagenix/database/standards_tools.py b/src/dbjavagenix/database/standards_tools.py index db372d6..d064d0c 100644 --- a/src/dbjavagenix/database/standards_tools.py +++ b/src/dbjavagenix/database/standards_tools.py @@ -10,7 +10,6 @@ from __future__ import annotations -import json from typing import Any from mcp.types import TextContent, Tool @@ -22,6 +21,7 @@ generate_spotbugs_exclude_xml, generate_suppressions_xml, ) +from ..utils.json_serialization import dumps as _json_dumps STANDARDS_GENERATE_TOOL = Tool( @@ -103,9 +103,7 @@ async def handle_generate_quality_configs( { "path": _FILE_PATHS["checkstyle"], "language": "xml", - "content": generate_checkstyle_xml( - line_limit=line_limit, indent=indent - ), + "content": generate_checkstyle_xml(line_limit=line_limit, indent=indent), } ) if "checkstyle_suppressions" in include: @@ -153,7 +151,7 @@ async def handle_generate_quality_configs( return [ TextContent( type="text", - text=json.dumps(payload, ensure_ascii=False, indent=2), + text=_json_dumps(payload, indent=2), ) ] diff --git a/src/dbjavagenix/generator/java_generator.py b/src/dbjavagenix/generator/java_generator.py index f475972..84cff50 100644 --- a/src/dbjavagenix/generator/java_generator.py +++ b/src/dbjavagenix/generator/java_generator.py @@ -1,172 +1,175 @@ -""" -Java 代码生成器 -整合 EasyCode 模板移植功能，支持三种模板分类 -""" - -import os -from pathlib import Path -from typing import Dict, List, Optional -import logging - -from .mustache_engine import MustacheTemplateEngine -from .template_context import TemplateContextBuilder, TemplateConfigManager -from ..core.models import TableInfo, GenerationConfig - - -logger = logging.getLogger(__name__) - - -class JavaCodeGenerator: - """Java 代码生成器""" - - def __init__(self, config: GenerationConfig): - self.config = config - self.template_engine = MustacheTemplateEngine() # 无参数初始化 - self.context_builder = TemplateContextBuilder( - author=config.author, - package_name=config.package_name - ) - self.template_config = TemplateConfigManager() - - # 模板路径 - self.template_base_path = Path(__file__).parent.parent / "templates" / "java" - - def generate_from_table(self, table_info: TableInfo, output_dir: str, - template_category: str = "Default", - include_dto_vo: bool = True) -> Dict[str, str]: - """ - 根据表信息生成 Java 代码 - - Args: - table_info: 表信息 - output_dir: 输出目录 - template_category: 模板分类 (Default, MybatisPlus, MybatisPlus-Mixed) - include_dto_vo: 是否包含 DTO/VO - - Returns: - 生成的文件路径字典 - """ - logger.info(f"开始生成表 {table_info.name} 的 Java 代码，模板分类: {template_category}") - - # 构建模板上下文 - context = self.context_builder.build_context(table_info, template_category) - - # 生成文件 - generated_files = {} - - # 1. 生成主要模板文件 - template_files = self.template_config.get_template_files(template_category) - for template_file in template_files: - try: - file_path = self._generate_file( - template_file, context, output_dir, template_category - ) - generated_files[template_file] = file_path - logger.debug(f"生成文件: {file_path}") - except Exception as e: - logger.error(f"生成文件 {template_file} 失败: {e}") - raise - - # 2. 生成 DTO/VO/Mapper（如果需要） - if include_dto_vo: - additional_templates = self.template_config.get_additional_templates() - for template_file in additional_templates: - try: - file_path = self._generate_file( - template_file, context, output_dir, "common" - ) - generated_files[template_file] = file_path - logger.debug(f"生成附加文件: {file_path}") - except Exception as e: - logger.error(f"生成附加文件 {template_file} 失败: {e}") - # 附加文件生成失败不影响主流程 - continue - - logger.info(f"成功生成 {len(generated_files)} 个文件") - return generated_files - - # 注：数据库直连生成流程已由 CodegenAnalyzer + CodegenGenerator 覆盖； - # 这里不再提供 generate_from_database 实现。 - - def _generate_file(self, template_file: str, context: Dict, - output_dir: str, category: str) -> str: - """ - 生成单个文件 - - Args: - template_file: 模板文件名 - context: 模板上下文 - output_dir: 输出目录 - category: 模板分类 - - Returns: - 生成的文件路径 - """ - # 确定模板路径 - if category == "common": - template_path = self.template_base_path / template_file - else: - template_path = self.template_base_path / category / template_file - - if not template_path.exists(): - raise FileNotFoundError(f"模板文件不存在: {template_path}") - - # 生成代码 - code = self.template_engine.render_file(str(template_path), context) - - # 确定输出路径 - output_path = self._get_output_path(template_file, context, output_dir) - - # 确保输出目录存在 - os.makedirs(os.path.dirname(output_path), exist_ok=True) - - # 写入文件 - with open(output_path, 'w', encoding='utf-8') as f: - f.write(code) - - return output_path - - def _get_output_path(self, template_file: str, context: Dict, output_dir: str) -> str: - """ - 获取输出文件路径 - - Args: - template_file: 模板文件名 - context: 模板上下文 - output_dir: 输出目录 - - Returns: - 输出文件路径 - """ - path_mapping = self.template_config.get_output_path_mapping() - - if template_file not in path_mapping: - # 默认路径 - file_name = template_file.replace('.mustache', '.java') - relative_path = f"generated/{file_name}" - else: - relative_path = path_mapping[template_file] - - # 替换路径中的占位符 - relative_path = relative_path.format(**context) - - return os.path.join(output_dir, relative_path) - +""" +Java 代码生成器 +整合 EasyCode 模板移植功能，支持三种模板分类 +""" + +import os +from pathlib import Path +from typing import Dict, List, Optional +import logging + +from .mustache_engine import MustacheTemplateEngine +from .template_context import TemplateContextBuilder, TemplateConfigManager +from ..core.models import TableInfo, GenerationConfig + + +logger = logging.getLogger(__name__) + + +class JavaCodeGenerator: + """Java 代码生成器""" + + def __init__(self, config: GenerationConfig): + self.config = config + self.template_engine = MustacheTemplateEngine() # 无参数初始化 + self.context_builder = TemplateContextBuilder( + author=config.author, package_name=config.package_name + ) + self.template_config = TemplateConfigManager() + + # 模板路径 + self.template_base_path = Path(__file__).parent.parent / "templates" / "java" + + def generate_from_table( + self, + table_info: TableInfo, + output_dir: str, + template_category: str = "Default", + include_dto_vo: bool = True, + ) -> Dict[str, str]: + """ + 根据表信息生成 Java 代码 + + Args: + table_info: 表信息 + output_dir: 输出目录 + template_category: 模板分类 (Default, MybatisPlus, MybatisPlus-Mixed) + include_dto_vo: 是否包含 DTO/VO + + Returns: + 生成的文件路径字典 + """ + logger.info(f"开始生成表 {table_info.name} 的 Java 代码，模板分类: {template_category}") + + # 构建模板上下文 + context = self.context_builder.build_context(table_info, template_category) + + # 生成文件 + generated_files = {} + + # 1. 生成主要模板文件 + template_files = self.template_config.get_template_files(template_category) + for template_file in template_files: + try: + file_path = self._generate_file( + template_file, context, output_dir, template_category + ) + generated_files[template_file] = file_path + logger.debug(f"生成文件: {file_path}") + except Exception as e: + logger.error(f"生成文件 {template_file} 失败: {e}") + raise + + # 2. 生成 DTO/VO/Mapper（如果需要） + if include_dto_vo: + additional_templates = self.template_config.get_additional_templates() + for template_file in additional_templates: + try: + file_path = self._generate_file(template_file, context, output_dir, "common") + generated_files[template_file] = file_path + logger.debug(f"生成附加文件: {file_path}") + except Exception as e: + logger.error(f"生成附加文件 {template_file} 失败: {e}") + # 附加文件生成失败不影响主流程 + continue + + logger.info(f"成功生成 {len(generated_files)} 个文件") + return generated_files + + # 注：数据库直连生成流程已由 CodegenAnalyzer + CodegenGenerator 覆盖； + # 这里不再提供 generate_from_database 实现。 + + def _generate_file( + self, template_file: str, context: Dict, output_dir: str, category: str + ) -> str: + """ + 生成单个文件 + + Args: + template_file: 模板文件名 + context: 模板上下文 + output_dir: 输出目录 + category: 模板分类 + + Returns: + 生成的文件路径 + """ + # 确定模板路径 + if category == "common": + template_path = self.template_base_path / template_file + else: + template_path = self.template_base_path / category / template_file + + if not template_path.exists(): + raise FileNotFoundError(f"模板文件不存在: {template_path}") + + # 生成代码 + code = self.template_engine.render_file(str(template_path), context) + + # 确定输出路径 + output_path = self._get_output_path(template_file, context, output_dir) + + # 确保输出目录存在 + os.makedirs(os.path.dirname(output_path), exist_ok=True) + + # 写入文件 + with open(output_path, "w", encoding="utf-8") as f: + f.write(code) + + return output_path + + def _get_output_path(self, template_file: str, context: Dict, output_dir: str) -> str: + """ + 获取输出文件路径 + + Args: + template_file: 模板文件名 + context: 模板上下文 + output_dir: 输出目录 + + Returns: + 输出文件路径 + """ + path_mapping = self.template_config.get_output_path_mapping() + + if template_file not in path_mapping: + # 默认路径 + file_name = template_file.replace(".mustache", ".java") + relative_path = f"generated/{file_name}" + else: + relative_path = path_mapping[template_file] + + # 替换路径中的占位符 + relative_path = relative_path.format(**context) + # packageSuffix 为空时模板会产生双分隔符,在写盘边界统一归一化。 + relative_path = os.path.normpath(relative_path) + + return os.path.join(output_dir, relative_path) + def get_supported_categories(self) -> List[str]: """获取支持的模板分类""" return self.template_config.get_supported_categories() - - def validate_template_category(self, category: str) -> bool: - """验证模板分类是否支持""" - return category in self.get_supported_categories() - - def list_template_files(self, category: str) -> List[str]: - """列出指定分类的模板文件""" - if not self.validate_template_category(category): - raise ValueError(f"不支持的模板分类: {category}") - - return self.template_config.get_template_files(category) - - - # 批量/项目结构生成和 quick_generate 相关便捷接口已移除， - # 建议结合 CodegenAnalyzer + CodegenGenerator 与项目路径进行生成与落盘。 + + def validate_template_category(self, category: str) -> bool: + """验证模板分类是否支持""" + return category in self.get_supported_categories() + + def list_template_files(self, category: str) -> List[str]: + """列出指定分类的模板文件""" + if not self.validate_template_category(category): + raise ValueError(f"不支持的模板分类: {category}") + + return self.template_config.get_template_files(category) + + # 批量/项目结构生成和 quick_generate 相关便捷接口已移除， + # 建议结合 CodegenAnalyzer + CodegenGenerator 与项目路径进行生成与落盘。 diff --git a/src/dbjavagenix/generator/mustache_engine.py b/src/dbjavagenix/generator/mustache_engine.py index a1b9977..659a593 100644 --- a/src/dbjavagenix/generator/mustache_engine.py +++ b/src/dbjavagenix/generator/mustache_engine.py @@ -160,7 +160,7 @@ def build_entity_context(table: TableInfo, config: GenerationConfig) -> Dict[str for col in table.columns ], "primaryKeys": table.primary_keys, - "hasAutoIncrement": any(col.primary_key for col in table.columns), + "hasAutoIncrement": any(col.auto_increment for col in table.columns), "imports": TemplateContext._get_entity_imports(table, config), } diff --git a/src/dbjavagenix/generator/template_context.py b/src/dbjavagenix/generator/template_context.py index 76220f9..7e5df9d 100644 --- a/src/dbjavagenix/generator/template_context.py +++ b/src/dbjavagenix/generator/template_context.py @@ -10,6 +10,48 @@ from ..database.dialect import DialectAdapter, get_dialect +def apply_generation_options( + context: Dict[str, Any], + *, + generate_dto: Optional[bool] = None, + generate_vo: Optional[bool] = None, + include_dto_vo: Optional[bool] = None, +) -> Dict[str, Any]: + """Normalize DTO/VO switches shared by legacy and atomic generation paths. + + ``includeDtoVo`` and ``include_dto_vo`` are retained as compatibility + aliases for callers that historically enabled both artifacts together. + Explicit per-artifact switches take precedence over those aliases. + """ + if include_dto_vo is None: + include_dto_vo = context.get("includeDtoVo", context.get("include_dto_vo")) + + if include_dto_vo is not None: + if generate_dto is None: + generate_dto = include_dto_vo + if generate_vo is None: + generate_vo = include_dto_vo + + if generate_dto is None: + generate_dto = context.get("generateDto", False) + if generate_vo is None: + generate_vo = context.get("generateVo", False) + + dto_enabled = bool(generate_dto) + vo_enabled = bool(generate_vo) + context.update( + { + "generateDto": dto_enabled, + "generateVo": vo_enabled, + "hasDto": dto_enabled, + "hasVo": vo_enabled, + "includeDtoVo": dto_enabled or vo_enabled, + "include_dto_vo": dto_enabled or vo_enabled, + } + ) + return context + + class TemplateContextBuilder: """模板上下文构建器""" @@ -55,8 +97,10 @@ def build_context( # 基础上下文 - 修复变量名映射 class_name = self._to_pascal_case(table_info.name) entity_name_lower = self._to_camel_case(table_info.name) - primary_key_info = self._build_primary_key_context(table_info.columns) - columns_context = self._build_columns_context(table_info.columns) + primary_key_info = self._build_primary_key_context( + table_info.columns, table_info.primary_keys + ) + columns_context = self._build_columns_context(table_info.columns, table_info.primary_keys) # 前缀分析 - 新增功能 package_suffix = "" @@ -126,15 +170,19 @@ def build_context( # 列相关 "columns": columns_context, "primaryKey": primary_key_info, - "nonPrimaryColumns": self._build_non_primary_columns_context(table_info.columns), - "otherColumns": self._build_non_primary_columns_context(table_info.columns), # 别名 + "nonPrimaryColumns": self._build_non_primary_columns_context( + table_info.columns, table_info.primary_keys + ), + "otherColumns": self._build_non_primary_columns_context( + table_info.columns, table_info.primary_keys + ), # 别名 # 主键相关 - 添加缺失的主键字段 "primaryKeyName": primary_key_info["name"] if primary_key_info else "id", "primaryKeyType": primary_key_info["javaType"] if primary_key_info else "Long", "primaryKeyColumn": primary_key_info["dbName"] if primary_key_info else "id", # 数据库列名 - "capitalizedPrimaryKeyName": primary_key_info["javaName"].capitalize() + "capitalizedPrimaryKeyName": self._to_pascal_case(primary_key_info["javaName"]) if primary_key_info else "Id", # 导入相关 @@ -222,25 +270,29 @@ def _detect_technology_stack(self, project_root: Optional[str], template_categor tech_stack.is_modern_stack = True return tech_stack - def _build_columns_context(self, columns: List[ColumnInfo]) -> List[Dict[str, Any]]: + def _build_columns_context( + self, columns: List[ColumnInfo], primary_keys: Optional[List[str]] = None + ) -> List[Dict[str, Any]]: """构建列上下文""" column_contexts = [] + primary_key_names = self._effective_primary_key_names(columns, primary_keys) for i, column in enumerate(columns): java_name = self._to_camel_case(column.name) java_type = self._map_java_type(column.data_type) + is_primary_key = column.name in primary_key_names column_context = { # 基础字段信息 "name": column.name, # 数据库字段名 "javaName": java_name, # Java字段名 - "capitalizedJavaName": java_name.capitalize(), # 首字母大写的Java字段名 + "capitalizedJavaName": self._to_pascal_case(java_name), # 首字母大写的Java字段名 "dbName": column.name, "javaType": java_type, "jdbcType": self._map_jdbc_type(column.data_type), "comment": column.comment or column.name, # 字段属性 - "isPrimaryKey": column.primary_key, - "primaryKey": column.primary_key, # 兼容两种写法 + "isPrimaryKey": is_primary_key, + "primaryKey": is_primary_key, # 兼容两种写法 "isNullable": column.nullable, "nullable": column.nullable, "isAutoIncrement": column.auto_increment, @@ -248,7 +300,7 @@ def _build_columns_context(self, columns: List[ColumnInfo]) -> List[Dict[str, An "defaultValue": column.default_value, "maxLength": column.max_length, # 验证相关 - "required": not column.nullable and not column.primary_key, + "required": not column.nullable and not is_primary_key, "isString": self._is_string_type(column.data_type), "stringType": self._is_string_type(column.data_type), # 别名 "isStringType": self._is_string_type(column.data_type), # 另一个别名 @@ -262,9 +314,18 @@ def _build_columns_context(self, columns: List[ColumnInfo]) -> List[Dict[str, An return column_contexts - def _build_primary_key_context(self, columns: List[ColumnInfo]) -> Optional[Dict[str, Any]]: + def _build_primary_key_context( + self, columns: List[ColumnInfo], primary_keys: Optional[List[str]] = None + ) -> Optional[Dict[str, Any]]: """构建主键上下文""" - pk_column = next((col for col in columns if col.primary_key), None) + primary_key_names = self._effective_primary_key_names(columns, primary_keys) + ordered_names = [str(name) for name in (primary_keys or []) if name] + pk_column = next( + (column for name in ordered_names for column in columns if column.name == name), + None, + ) + if pk_column is None: + pk_column = next((col for col in columns if col.name in primary_key_names), None) if pk_column: java_name = self._to_camel_case(pk_column.name) @@ -275,13 +336,18 @@ def _build_primary_key_context(self, columns: List[ColumnInfo]) -> Optional[Dict "javaType": self._map_java_type(pk_column.data_type), "jdbcType": self._map_jdbc_type(pk_column.data_type), "comment": pk_column.comment or pk_column.name, + "isAutoIncrement": pk_column.auto_increment, + "autoIncrement": pk_column.auto_increment, } return None - def _build_non_primary_columns_context(self, columns: List[ColumnInfo]) -> List[Dict[str, Any]]: + def _build_non_primary_columns_context( + self, columns: List[ColumnInfo], primary_keys: Optional[List[str]] = None + ) -> List[Dict[str, Any]]: """构建非主键列上下文""" - non_pk_columns = [col for col in columns if not col.primary_key] + primary_key_names = self._effective_primary_key_names(columns, primary_keys) + non_pk_columns = [col for col in columns if col.name not in primary_key_names] column_contexts = [] for i, column in enumerate(non_pk_columns): @@ -291,14 +357,14 @@ def _build_non_primary_columns_context(self, columns: List[ColumnInfo]) -> List[ # 基础字段信息 "name": column.name, # 数据库字段名 "javaName": java_name, # Java字段名 - "capitalizedJavaName": java_name.capitalize(), # 首字母大写的Java字段名 + "capitalizedJavaName": self._to_pascal_case(java_name), # 首字母大写的Java字段名 "dbName": column.name, "javaType": java_type, "jdbcType": self._map_jdbc_type(column.data_type), "comment": column.comment or column.name, # 字段属性 - "isPrimaryKey": column.primary_key, - "primaryKey": column.primary_key, # 兼容两种写法 + "isPrimaryKey": False, + "primaryKey": False, # 兼容两种写法 "isNullable": column.nullable, "nullable": column.nullable, "isAutoIncrement": column.auto_increment, @@ -306,7 +372,7 @@ def _build_non_primary_columns_context(self, columns: List[ColumnInfo]) -> List[ "defaultValue": column.default_value, "maxLength": column.max_length, # 验证相关 - "required": not column.nullable and not column.primary_key, + "required": not column.nullable, "isString": self._is_string_type(column.data_type), "stringType": self._is_string_type(column.data_type), # 别名 "isStringType": self._is_string_type(column.data_type), # 另一个别名 @@ -320,6 +386,16 @@ def _build_non_primary_columns_context(self, columns: List[ColumnInfo]) -> List[ return column_contexts + @staticmethod + def _effective_primary_key_names( + columns: List[ColumnInfo], primary_keys: Optional[List[str]] = None + ) -> set[str]: + """Resolve the authoritative primary-key names for template contexts.""" + explicit_names = {str(name) for name in (primary_keys or []) if name} + if explicit_names and any(column.name in explicit_names for column in columns): + return explicit_names + return {column.name for column in columns if column.primary_key} + def _build_custom_mappings(self, columns: List[ColumnInfo]) -> Dict[str, bool]: """构建自定义映射规则""" column_names = [self._to_camel_case(col.name) for col in columns] @@ -332,25 +408,11 @@ def _build_custom_mappings(self, columns: List[ColumnInfo]) -> Dict[str, bool]: def _build_imports(self, columns: List[ColumnInfo], template_category: str) -> List[str]: """构建导入列表""" - imports = [] - java_types = {self._map_java_type(col.data_type) for col in columns} - - # 时间类型导入 - import_by_type = { - "LocalDateTime": "java.time.LocalDateTime", - "LocalDate": "java.time.LocalDate", - "LocalTime": "java.time.LocalTime", - "OffsetDateTime": "java.time.OffsetDateTime", - "OffsetTime": "java.time.OffsetTime", - "Instant": "java.time.Instant", - "BigDecimal": "java.math.BigDecimal", - "BigInteger": "java.math.BigInteger", - "UUID": "java.util.UUID", + imports = { + import_path + for column in columns + for import_path in self.dialect.java_imports_for(column.data_type) } - imports.extend( - import_by_type[java_type] for java_type in java_types if java_type in import_by_type - ) - return sorted(imports) def _has_date_field(self, columns: List[ColumnInfo]) -> bool: diff --git a/src/dbjavagenix/mcp_apps/code_diff.py b/src/dbjavagenix/mcp_apps/code_diff.py index 762df6b..84b935d 100644 --- a/src/dbjavagenix/mcp_apps/code_diff.py +++ b/src/dbjavagenix/mcp_apps/code_diff.py @@ -57,13 +57,15 @@ def build_code_diff_data( after = entry.get("code", "") before = _try_read_existing(root, file_path) if root else None - diff_files.append({ - "file_path": file_path, - "template_file": entry.get("template_file"), - "before": before, - "after": after, - "lines": entry.get("lines"), - }) + diff_files.append( + { + "file_path": file_path, + "template_file": entry.get("template_file"), + "before": before, + "after": after, + "lines": entry.get("lines"), + } + ) return { "language": language, @@ -76,9 +78,11 @@ def _try_read_existing(root: Path, relative_path: str) -> Optional[str]: if not relative_path: return None try: - target = root / relative_path + resolved_root = root.resolve() + target = (resolved_root / relative_path).resolve() + target.relative_to(resolved_root) if not target.exists() or not target.is_file(): return None return target.read_text(encoding="utf-8") - except (OSError, UnicodeDecodeError): + except (OSError, RuntimeError, ValueError, UnicodeDecodeError): return None diff --git a/src/dbjavagenix/mcp_apps/package_tree.py b/src/dbjavagenix/mcp_apps/package_tree.py index 6f75100..aae6e88 100644 --- a/src/dbjavagenix/mcp_apps/package_tree.py +++ b/src/dbjavagenix/mcp_apps/package_tree.py @@ -54,7 +54,9 @@ def build_package_tree_data( root = Path(project_root) if project_root else None for parts, original in zip(split_paths, file_paths): # 移除 root 前缀 - relative = parts[len(root_segments):] if parts[:len(root_segments)] == root_segments else parts + relative = ( + parts[len(root_segments) :] if parts[: len(root_segments)] == root_segments else parts + ) _insert_into_tree( tree, relative, @@ -90,9 +92,7 @@ def _common_prefix(paths: List[List[str]]) -> List[str]: return prefix -def _insert_into_tree( - node: Dict[str, Any], parts: List[str], file_status: str -) -> None: +def _insert_into_tree(node: Dict[str, Any], parts: List[str], file_status: str) -> None: """把 parts 路径插入树节点""" if not parts: return @@ -114,17 +114,21 @@ def _serialize_tree(node: Dict[str, Any]) -> List[Dict[str, Any]]: for name in sorted(children.keys()): child = children[name] if child.get("_file"): - out.append({ - "name": name, - "type": "file", - "status": child.get("_status", "new"), - }) + out.append( + { + "name": name, + "type": "file", + "status": child.get("_status", "new"), + } + ) else: - out.append({ - "name": name, - "type": "package", - "children": _serialize_tree(child), - }) + out.append( + { + "name": name, + "type": "package", + "children": _serialize_tree(child), + } + ) return out @@ -132,7 +136,12 @@ def _determine_status(project_root: Optional[Path], relative_path: str) -> str: """判断目标位置文件状态""" if not project_root: return "new" - target = project_root / relative_path - if target.exists() and target.is_file(): - return "modified" + try: + resolved_root = project_root.resolve() + target = (resolved_root / relative_path).resolve() + target.relative_to(resolved_root) + if target.exists() and target.is_file(): + return "modified" + except (OSError, RuntimeError, ValueError): + pass return "new" diff --git a/src/dbjavagenix/server/mcp_server.py b/src/dbjavagenix/server/mcp_server.py index 700ce6d..5a963b5 100644 --- a/src/dbjavagenix/server/mcp_server.py +++ b/src/dbjavagenix/server/mcp_server.py @@ -18,6 +18,7 @@ get_codegen_tools, get_springboot_project_tools, handle_db_connect_test, + handle_db_disconnect, handle_db_query_databases, handle_db_query_tables, handle_db_query_table_exists, @@ -72,6 +73,7 @@ from ..utils.metrics import GLOBAL_TOOL_METRICS from ..utils.security import redact_sensitive_data, redact_sensitive_text from ..utils.logging_config import configure_logging +from ..utils.json_serialization import dumps as _json_dumps from ..utils.tool_registry import filter_tools_for_listing # Configure logging (P5.2: plain / json via DBJAVAGENIX_LOG_FORMAT) @@ -140,7 +142,7 @@ def _tool_error_response(tool_name: str, error_code: str, error: object) -> list "tool": tool_name, "message": redact_sensitive_text(error), } - return [TextContent(type="text", text=json.dumps(payload, ensure_ascii=False))] + return [TextContent(type="text", text=_json_dumps(payload))] def _result_reports_error(result: Sequence[Any]) -> bool: diff --git a/src/dbjavagenix/templates/java/Default/mapper.xml.mustache b/src/dbjavagenix/templates/java/Default/mapper.xml.mustache index 78b9dcd..adfa4e7 100644 --- a/src/dbjavagenix/templates/java/Default/mapper.xml.mustache +++ b/src/dbjavagenix/templates/java/Default/mapper.xml.mustache @@ -55,12 +55,12 @@ </select> <!--新增所有列--> - <insert id="insert" keyProperty="{{primaryKeyName}}" useGeneratedKeys="true"> + <insert id="insert"{{#primaryKey.isAutoIncrement}} keyProperty="{{primaryKey.javaName}}" useGeneratedKeys="true"{{/primaryKey.isAutoIncrement}}> insert into {{tableName}}({{#otherColumns}}{{name}}{{#hasNext}}, {{/hasNext}}{{/otherColumns}}) values ({{#otherColumns}}#{ {{javaName}} }{{#hasNext}}, {{/hasNext}}{{/otherColumns}}) </insert> - <insert id="insertBatch" keyProperty="{{primaryKeyName}}" useGeneratedKeys="true"> + <insert id="insertBatch"{{#primaryKey.isAutoIncrement}} keyProperty="{{primaryKey.javaName}}" useGeneratedKeys="true"{{/primaryKey.isAutoIncrement}}> insert into {{tableName}}({{#otherColumns}}{{name}}{{#hasNext}}, {{/hasNext}}{{/otherColumns}}) values <foreach collection="entities" item="entity" separator=","> @@ -68,7 +68,7 @@ </foreach> </insert> - <insert id="insertOrUpdateBatch" keyProperty="{{primaryKeyName}}" useGeneratedKeys="true"> + <insert id="insertOrUpdateBatch"{{#primaryKey.isAutoIncrement}} keyProperty="{{primaryKey.javaName}}" useGeneratedKeys="true"{{/primaryKey.isAutoIncrement}}> insert into {{tableName}}({{#otherColumns}}{{name}}{{#hasNext}}, {{/hasNext}}{{/otherColumns}}) values <foreach collection="entities" item="entity" separator=","> diff --git a/src/dbjavagenix/templates/java/common/dto.mustache b/src/dbjavagenix/templates/java/common/dto.mustache new file mode 100644 index 0000000..6cb9d8a --- /dev/null +++ b/src/dbjavagenix/templates/java/common/dto.mustache @@ -0,0 +1,67 @@ +package {{dtoPackage}}; + +{{#imports}} +import {{.}}; +{{/imports}} +import java.io.Serializable; +{{#useLombok}} +import lombok.Data; +{{/useLombok}} +{{#useSwagger}} +{{#hasSpringDoc}} +import io.swagger.v3.oas.annotations.media.Schema; +{{/hasSpringDoc}} +{{#hasSwagger2}} +import io.swagger.annotations.ApiModel; +import io.swagger.annotations.ApiModelProperty; +{{/hasSwagger2}} +{{/useSwagger}} + +/** + * {{comment}}({{tableName}}) data transfer object. + * + * @author {{author}} + * @date {{date}} + */ +{{#useLombok}} +@Data +{{/useLombok}} +{{#useSwagger}} +{{#hasSpringDoc}} +@Schema(description = "{{comment}}DTO") +{{/hasSpringDoc}} +{{#hasSwagger2}} +@ApiModel(description = "{{comment}}DTO") +{{/hasSwagger2}} +{{/useSwagger}} +public class {{className}}DTO implements Serializable { + private static final long serialVersionUID = {{serialVersionUID}}L; + +{{#columns}} + /** + * {{comment}} + */ + {{#useSwagger}} + {{#hasSpringDoc}} + @Schema(description = "{{comment}}") + {{/hasSpringDoc}} + {{#hasSwagger2}} + @ApiModelProperty(value = "{{comment}}") + {{/hasSwagger2}} + {{/useSwagger}} + private {{javaType}} {{javaName}}; + +{{/columns}} +{{^useLombok}} +{{#columns}} + public {{javaType}} get{{capitalizedJavaName}}() { + return {{javaName}}; + } + + public void set{{capitalizedJavaName}}({{javaType}} {{javaName}}) { + this.{{javaName}} = {{javaName}}; + } + +{{/columns}} +{{/useLombok}} +} diff --git a/src/dbjavagenix/templates/java/common/vo.mustache b/src/dbjavagenix/templates/java/common/vo.mustache new file mode 100644 index 0000000..c4d1a09 --- /dev/null +++ b/src/dbjavagenix/templates/java/common/vo.mustache @@ -0,0 +1,67 @@ +package {{voPackage}}; + +{{#imports}} +import {{.}}; +{{/imports}} +import java.io.Serializable; +{{#useLombok}} +import lombok.Data; +{{/useLombok}} +{{#useSwagger}} +{{#hasSpringDoc}} +import io.swagger.v3.oas.annotations.media.Schema; +{{/hasSpringDoc}} +{{#hasSwagger2}} +import io.swagger.annotations.ApiModel; +import io.swagger.annotations.ApiModelProperty; +{{/hasSwagger2}} +{{/useSwagger}} + +/** + * {{comment}}({{tableName}}) view object. + * + * @author {{author}} + * @date {{date}} + */ +{{#useLombok}} +@Data +{{/useLombok}} +{{#useSwagger}} +{{#hasSpringDoc}} +@Schema(description = "{{comment}}VO") +{{/hasSpringDoc}} +{{#hasSwagger2}} +@ApiModel(description = "{{comment}}VO") +{{/hasSwagger2}} +{{/useSwagger}} +public class {{className}}VO implements Serializable { + private static final long serialVersionUID = {{serialVersionUID}}L; + +{{#columns}} + /** + * {{comment}} + */ + {{#useSwagger}} + {{#hasSpringDoc}} + @Schema(description = "{{comment}}") + {{/hasSpringDoc}} + {{#hasSwagger2}} + @ApiModelProperty(value = "{{comment}}") + {{/hasSwagger2}} + {{/useSwagger}} + private {{javaType}} {{javaName}}; + +{{/columns}} +{{^useLombok}} +{{#columns}} + public {{javaType}} get{{capitalizedJavaName}}() { + return {{javaName}}; + } + + public void set{{capitalizedJavaName}}({{javaType}} {{javaName}}) { + this.{{javaName}} = {{javaName}}; + } + +{{/columns}} +{{/useLombok}} +} diff --git a/src/dbjavagenix/utils/dependency_requirements.py b/src/dbjavagenix/utils/dependency_requirements.py index 53320af..ff094ea 100644 --- a/src/dbjavagenix/utils/dependency_requirements.py +++ b/src/dbjavagenix/utils/dependency_requirements.py @@ -223,6 +223,11 @@ def analyze_requirements(self, Returns: 按类别分组的依赖需求字典 """ + + # Version adaptation mutates dependency descriptors for compatibility + # with the existing public helper. Rebuild the catalog for every + # analysis so a previous project cannot affect this result. + self._initialize_dependency_catalog() # 根据Spring Boot版本调整依赖版本 if spring_boot_version: diff --git a/src/dbjavagenix/utils/tool_registry.py b/src/dbjavagenix/utils/tool_registry.py index de8ee8d..08039e9 100644 --- a/src/dbjavagenix/utils/tool_registry.py +++ b/src/dbjavagenix/utils/tool_registry.py @@ -20,6 +20,8 @@ from mcp.types import Tool +from ..database.capabilities import supported_database_display_names + @dataclass class ToolMetadata: @@ -43,7 +45,13 @@ class ToolMetadata: tags={"connect", "connection", "database", "test", "establish", "init"}, category="connection", always_visible=True, - description_brief="建立数据库连接 (MySQL/PostgreSQL/SQLite/Oracle/SqlServer)", + description_brief=("建立数据库连接 (" + "/".join(supported_database_display_names()) + ")"), + ), + "db_disconnect": ToolMetadata( + name="db_disconnect", + tags={"disconnect", "close", "connection", "release", "cleanup", "session"}, + category="connection", + description_brief="关闭并释放数据库连接会话", ), "db_query_databases": ToolMetadata( name="db_query_databases", diff --git a/tests/unit/test_async_db_boundary.py b/tests/unit/test_async_db_boundary.py index 5d17c1f..08ad458 100644 --- a/tests/unit/test_async_db_boundary.py +++ b/tests/unit/test_async_db_boundary.py @@ -7,8 +7,8 @@ import pytest from dbjavagenix.core.models import DatabaseConfig, DatabaseType -from dbjavagenix.database.connection_manager import ConnectionManager from dbjavagenix.database import mcp_tools +from dbjavagenix.database.connection_manager import ConnectionManager @pytest.fixture @@ -47,14 +47,12 @@ class _BlockingConnection: def __init__(self, state): self.state = state - self.closed_event = threading.Event() def cursor(self): return _BlockingCursor(self.state) def close(self): self.closed = 1 - self.closed_event.set() def _state(): @@ -119,20 +117,20 @@ async def test_sqlite_connection_can_be_used_by_worker_thread(sqlite_config): @pytest.mark.asyncio async def test_same_connection_queries_are_serialized(sqlite_config): - first_state = _state() - manager, connection_id = _manager_with_fake_connection(sqlite_config, first_state) + state = _state() + manager, connection_id = _manager_with_fake_connection(sqlite_config, state) first = asyncio.create_task(asyncio.to_thread(manager.execute_query, connection_id, "SELECT 1")) - assert await asyncio.to_thread(first_state["entered"].wait, 1) + assert await asyncio.to_thread(state["entered"].wait, 1) second = asyncio.create_task( asyncio.to_thread(manager.execute_query, connection_id, "SELECT 2") ) await asyncio.sleep(0.02) - assert first_state["max_active"] == 1 + assert state["max_active"] == 1 - first_state["release"].set() + state["release"].set() await asyncio.gather(first, second) - assert first_state["closed_cursors"] == 2 + assert state["closed_cursors"] == 2 manager.close_connection(connection_id) @@ -180,4 +178,3 @@ async def test_close_waits_for_query_and_cleans_connection(sqlite_config): assert connection_id not in manager.connections assert connection_id not in manager.connection_configs assert connection_id not in manager._connection_locks - assert state["closed_cursors"] == 1 diff --git a/tests/unit/test_atomic_codegen_tools.py b/tests/unit/test_atomic_codegen_tools.py index 0c83cb7..ac1dd5b 100644 --- a/tests/unit/test_atomic_codegen_tools.py +++ b/tests/unit/test_atomic_codegen_tools.py @@ -16,6 +16,7 @@ import pytest +from dbjavagenix.database import atomic_codegen_tools from dbjavagenix.database.atomic_codegen_tools import ( _compute_file_path, _collect_all_table_names, @@ -163,19 +164,38 @@ def test_render_tools_require_context(self): def test_collect_all_table_names_supports_postgresql(monkeypatch): - connection = _TableNameConnection() - monkeypatch.setattr( - "dbjavagenix.database.atomic_codegen_tools.connection_manager", - _TableNameManager(connection), - ) + class Introspector: + def __init__(self, manager): + assert manager is atomic_codegen_tools.connection_manager + + def list_tables(self, connection_id): + assert connection_id == "pg-1" + return ["users", "user_roles"] + + monkeypatch.setattr(atomic_codegen_tools, "DatabaseIntrospector", Introspector) names = _collect_all_table_names( "pg-1", type("Config", (), {"type": DatabaseType.POSTGRESQL})() ) assert names == ["users", "user_roles"] - assert "information_schema.tables" in connection.cursor_instance.query - assert "pg_catalog" in connection.cursor_instance.query + + +def test_collect_all_table_names_keeps_empty_fallback_on_introspection_error(monkeypatch): + class Introspector: + def __init__(self, manager): + pass + + def list_tables(self, connection_id): + if connection_id == "empty": + return [] + raise RuntimeError("metadata unavailable") + + monkeypatch.setattr(atomic_codegen_tools, "DatabaseIntrospector", Introspector) + config = type("Config", (), {"type": DatabaseType.POSTGRESQL})() + + assert _collect_all_table_names("empty", config) == [] + assert _collect_all_table_names("broken", config) == [] # ============================================================ diff --git a/tests/unit/test_cli_helpers.py b/tests/unit/test_cli_helpers.py new file mode 100644 index 0000000..e75b29a --- /dev/null +++ b/tests/unit/test_cli_helpers.py @@ -0,0 +1,82 @@ +"""Regression tests for the synchronous MCP-to-CLI response adapter.""" + +import pytest +from mcp.types import TextContent + +from dbjavagenix import cli_helpers + + +def _text_response(text: str) -> list[TextContent]: + return [TextContent(type="text", text=text)] + + +def test_connection_text_fallback_is_case_insensitive(monkeypatch): + async def fake_connect(_arguments): + return _text_response( + "Database connection successful!\n- Connection ID: conn-1\n- Server: SQLite" + ) + + monkeypatch.setattr(cli_helpers, "async_handle_db_connect_test", fake_connect) + + assert cli_helpers.handle_db_connect_test({}) == { + "success": True, + "connection_id": "conn-1", + "server_info": "SQLite", + } + + +@pytest.mark.parametrize( + ("wrapper_name", "handler_name"), + [ + ("handle_db_connect_test", "async_handle_db_connect_test"), + ("handle_db_query_databases", "async_handle_db_query_databases"), + ("handle_db_query_tables", "async_handle_db_query_tables"), + ("handle_db_codegen_analyze", "async_handle_db_codegen_analyze"), + ("handle_db_codegen_generate", "async_handle_db_codegen_generate"), + ("handle_springboot_read_config", "async_handle_springboot_read_config"), + ], +) +def test_all_wrappers_accept_direct_json(monkeypatch, wrapper_name, handler_name): + async def fake_handler(_arguments): + return _text_response('{"success": true, "value": 1}') + + monkeypatch.setattr(cli_helpers, handler_name, fake_handler) + + wrapper = getattr(cli_helpers, wrapper_name) + assert wrapper({}) == {"success": True, "value": 1} + + +def test_adapter_accepts_raw_response_before_human_text(): + async def fake_handler(_arguments): + return _text_response('Report\n\nRaw Response: {"success": true, "value": 2}') + + assert cli_helpers._run_mcp_sync(fake_handler, {}) == { + "success": True, + "value": 2, + } + + +@pytest.mark.parametrize( + "result", + [[], _text_response("not-json")], +) +def test_adapter_returns_stable_parse_errors(result): + async def fake_handler(_arguments): + return result + + payload = cli_helpers._run_mcp_sync(fake_handler, {}) + + assert payload["success"] is False + assert payload["error"] in {"No response", "Failed to parse response"} + + +def test_adapter_redacts_exception_text(): + async def fake_handler(_arguments): + raise RuntimeError("failed for jdbc:mysql://reader:db-secret@db/app") + + payload = cli_helpers._run_mcp_sync(fake_handler, {}) + + assert payload == { + "success": False, + "error": "failed for jdbc:mysql://reader:***@db/app", + } diff --git a/tests/unit/test_codegen_analyzer.py b/tests/unit/test_codegen_analyzer.py index 4541eeb..98db493 100644 --- a/tests/unit/test_codegen_analyzer.py +++ b/tests/unit/test_codegen_analyzer.py @@ -5,7 +5,7 @@ import pytest -from dbjavagenix.core.models import DatabaseType +from dbjavagenix.core.models import ColumnInfo, DatabaseType from dbjavagenix.database.codegen_tools import CodegenAnalyzer, CodegenGenerator from dbjavagenix.database.atomic_codegen_tools import get_atomic_codegen_tools from dbjavagenix.database.mcp_tools import get_codegen_tools @@ -67,6 +67,7 @@ class _BatchAnalyzer(CodegenAnalyzer): def __init__(self): super().__init__(connection_manager=object()) self.introspector = _FakeIntrospector() + self.calls = [] async def analyze_table_for_codegen( self, @@ -77,6 +78,7 @@ async def analyze_table_for_codegen( project_root=None, schema=None, ): + self.calls.append((connection_id, table_name, all_table_names, schema)) if table_name == "broken": raise RuntimeError("metadata unavailable") return {"table_name": table_name, "template_context": {}} @@ -84,7 +86,8 @@ async def analyze_table_for_codegen( @pytest.mark.asyncio async def test_batch_analysis_keeps_success_and_error_accounting(): - result = await _BatchAnalyzer().analyze_database_for_codegen("conn-1") + analyzer = _BatchAnalyzer() + result = await analyzer.analyze_database_for_codegen("conn-1") assert result["database_info"] == { "total_tables": 2, @@ -94,6 +97,10 @@ async def test_batch_analysis_keeps_success_and_error_accounting(): } assert result["tables"]["users"]["table_name"] == "users" assert result["tables"]["broken"]["error"] == "metadata unavailable" + assert analyzer.calls == [ + ("conn-1", "users", ["broken", "users"], None), + ("conn-1", "broken", ["broken", "users"], None), + ] class _PostgresBatchAnalyzer(CodegenAnalyzer): @@ -120,7 +127,7 @@ async def analyze_table_for_codegen( project_root=None, schema=None, ): - self.calls.append((connection_id, table_name, schema)) + self.calls.append((connection_id, table_name, all_table_names, schema)) return {"table_name": table_name, "table_info": {"schema": schema}} @@ -131,8 +138,8 @@ async def test_batch_analysis_keeps_postgresql_schema_identity(): result = await analyzer.analyze_database_for_codegen("pg-1") assert analyzer.calls == [ - ("pg-1", "users", "tenant_a"), - ("pg-1", "users", "tenant_b"), + ("pg-1", "users", ["users"], "tenant_a"), + ("pg-1", "users", ["users"], "tenant_b"), ] assert result["database_info"] == { "total_tables": 2, @@ -198,6 +205,27 @@ def get_columns(_connection_id, _table_name, _schema=None): assert column["scale"] is None +def test_imports_needed_match_dialect_aware_template_imports(): + analyzer = CodegenAnalyzer(object()) + columns = [ + ColumnInfo( + name="occurred_at", + data_type="TIMESTAMPTZ", + java_type="OffsetDateTime", + ), + ColumnInfo( + name="amount", + data_type="DECIMAL(12,2)", + java_type="BigDecimal", + ), + ] + + assert analyzer._calculate_imports_needed(columns, DatabaseType.POSTGRESQL) == [ + "java.math.BigDecimal", + "java.time.OffsetDateTime", + ] + + def test_codegen_entrypoints_expose_optional_schema(): atomic = next( tool for tool in get_atomic_codegen_tools() if tool.name == "codegen_build_context" diff --git a/tests/unit/test_codegen_options.py b/tests/unit/test_codegen_options.py new file mode 100644 index 0000000..c93f8df --- /dev/null +++ b/tests/unit/test_codegen_options.py @@ -0,0 +1,179 @@ +"""Regression tests for the shared DTO/VO generation option contract.""" + +import pytest + +from dbjavagenix.database.atomic_codegen_tools import get_atomic_codegen_tools +from dbjavagenix.database.codegen_tools import CodegenGenerator +from dbjavagenix.database.mcp_tools import get_codegen_tools +from dbjavagenix.core.models import ColumnInfo, TableInfo +from dbjavagenix.generator.template_context import TemplateContextBuilder, apply_generation_options + + +def _analysis(category: str = "Default") -> dict: + return { + "table_name": "users", + "template_context": { + "className": "User", + "package": "com.example", + "packageName": "com.example", + "packageSuffix": "", + "templateCategory": category, + "useMapStruct": False, + }, + } + + +def test_apply_generation_options_preserves_legacy_aliases(): + context = {"generateDto": False, "generateVo": False} + + apply_generation_options(context, include_dto_vo=True) + + assert context["generateDto"] is True + assert context["generateVo"] is True + assert context["includeDtoVo"] is True + assert context["include_dto_vo"] is True + + +def test_apply_generation_options_explicit_switches_override_alias(): + context = {} + + apply_generation_options(context, generate_dto=False, generate_vo=True, include_dto_vo=True) + + assert context["generateDto"] is False + assert context["generateVo"] is True + assert context["hasDto"] is False + assert context["hasVo"] is True + + +@pytest.mark.asyncio +async def test_codegen_generator_supports_dto_only_and_rebuilds_package(monkeypatch): + generator = CodegenGenerator() + rendered = [] + + async def render(template_file, _context, _category): + rendered.append(template_file) + return template_file + + monkeypatch.setattr(generator, "_render_template", render) + + result = await generator.generate_code( + _analysis(), + template_category="Default", + generation_config={"package_name": "org.demo", "generate_dto": True}, + ) + + assert "dto.mustache" in result["generated_code"] + assert "vo.mustache" not in result["generated_code"] + assert result["generated_code"]["dto.mustache"]["filename"] == "org/demo/dto/UserDTO.java" + assert rendered.count("dto.mustache") == 1 + + +@pytest.mark.asyncio +async def test_codegen_generator_supports_vo_only(monkeypatch): + generator = CodegenGenerator() + + async def render(template_file, _context, _category): + return template_file + + monkeypatch.setattr(generator, "_render_template", render) + + result = await generator.generate_code( + _analysis(), + template_category="Default", + generation_config={"generate_vo": True}, + ) + + assert "dto.mustache" not in result["generated_code"] + assert "vo.mustache" in result["generated_code"] + + +@pytest.mark.asyncio +async def test_codegen_generator_honors_context_legacy_alias(monkeypatch): + generator = CodegenGenerator() + analysis = _analysis() + analysis["template_context"]["includeDtoVo"] = True + + async def render(template_file, _context, _category): + return template_file + + monkeypatch.setattr(generator, "_render_template", render) + + result = await generator.generate_code(analysis, template_category="Default") + + assert "dto.mustache" in result["generated_code"] + assert "vo.mustache" in result["generated_code"] + + +@pytest.mark.asyncio +async def test_codegen_generator_does_not_duplicate_sb35_dto(monkeypatch): + generator = CodegenGenerator() + rendered = [] + + async def render(template_file, _context, _category): + rendered.append(template_file) + return template_file + + monkeypatch.setattr(generator, "_render_template", render) + + result = await generator.generate_code( + _analysis("sb35-java21"), + template_category="sb35-java21", + generation_config={"generate_dto": True}, + ) + + assert rendered.count("dto.mustache") == 1 + assert result["generation_statistics"]["total_files"] == 6 + + +@pytest.mark.asyncio +async def test_codegen_generator_renders_common_dto_and_vo(): + table = TableInfo( + name="users", + schema="public", + comment="Users", + columns=[ + ColumnInfo( + name="id", + data_type="BIGINT", + java_type="Long", + primary_key=True, + nullable=False, + ) + ], + primary_keys=["id"], + ) + context = TemplateContextBuilder(package_name="com.example").build_context(table, "Default") + + result = await CodegenGenerator().generate_code( + {"table_name": "users", "template_context": context}, + template_category="Default", + generation_config={ + "package_name": "org.demo", + "generate_dto": True, + "generate_vo": True, + }, + ) + + assert result["generation_statistics"] == { + "total_files": 8, + "success_files": 8, + "error_files": 0, + } + assert "package org.demo.dto;" in result["generated_code"]["dto.mustache"]["code"] + assert "class UsersDTO" in result["generated_code"]["dto.mustache"]["code"] + assert "package org.demo.vo;" in result["generated_code"]["vo.mustache"]["code"] + assert "class UsersVO" in result["generated_code"]["vo.mustache"]["code"] + + +def test_codegen_schemas_expose_individual_dto_vo_switches(): + legacy = {tool.name: tool for tool in get_codegen_tools()} + atomic = {tool.name: tool for tool in get_atomic_codegen_tools()} + + for name in ("db_codegen_analyze", "db_codegen_generate"): + properties = legacy[name].inputSchema["properties"] + assert properties["generate_dto"]["default"] is False + assert properties["generate_vo"]["default"] is False + + properties = atomic["codegen_build_context"].inputSchema["properties"] + assert properties["generate_dto"]["default"] is False + assert properties["generate_vo"]["default"] is False diff --git a/tests/unit/test_codegen_output_dir.py b/tests/unit/test_codegen_output_dir.py index 2ab69ac..e1f58d4 100644 --- a/tests/unit/test_codegen_output_dir.py +++ b/tests/unit/test_codegen_output_dir.py @@ -41,16 +41,15 @@ def cursor(self): return _Cursor() -@pytest.mark.asyncio -async def test_codegen_generate_writes_to_explicit_output_dir(monkeypatch, tmp_path): - project = tmp_path / "project" - output = tmp_path / "custom-output" +def _stub_codegen_handler(monkeypatch, project: Path, generated_code: dict) -> None: + """Install deterministic handler dependencies for write-result tests.""" structure = _structure(project) config = SimpleNamespace(type=DatabaseType.SQLITE, database="app") monkeypatch.setattr(mcp_tools.connection_manager, "get_connection_info", lambda _id: config) monkeypatch.setattr(mcp_tools.connection_manager, "get_connection", lambda _id: _Connection()) monkeypatch.setattr(mcp_tools, "_detect_project_structure", lambda _path=None: structure) + monkeypatch.setattr(mcp_tools, "_collect_codegen_table_names", lambda *_args: ["users"]) async def validate(_args): return [TextContent(type="text", text="Project Structure: ✅ OK")] @@ -73,18 +72,14 @@ async def analyze_table_for_codegen(self, *_args, **_kwargs): class FakeGenerator: async def generate_code(self, *_args, **_kwargs): + success_files = sum("error" not in info for info in generated_code.values()) return { - "generated_code": { - "entity.mustache": { - "filename": "com/example/User.java", - "code": "class User {}", - }, - "mapper.xml.mustache": { - "filename": "resources/mapper/User.xml", - "code": "<mapper />", - }, + "generated_code": generated_code, + "generation_statistics": { + "total_files": len(generated_code), + "success_files": success_files, + "error_files": len(generated_code) - success_files, }, - "generation_statistics": {"total_files": 2, "success_files": 2, "error_files": 0}, } import dbjavagenix.database.codegen_tools as codegen_tools @@ -92,6 +87,62 @@ async def generate_code(self, *_args, **_kwargs): monkeypatch.setattr(codegen_tools, "CodegenAnalyzer", FakeAnalyzer) monkeypatch.setattr(codegen_tools, "CodegenGenerator", FakeGenerator) + +def test_collect_codegen_table_names_uses_shared_introspector(monkeypatch): + calls = [] + + class Introspector: + def __init__(self, manager): + assert manager is mcp_tools.connection_manager + + def list_tables(self, connection_id): + calls.append(connection_id) + return ["sys_user", "sys_role"] + + monkeypatch.setattr(mcp_tools, "DatabaseIntrospector", Introspector) + + assert mcp_tools._collect_codegen_table_names("pg-1", "sys_user") == [ + "sys_user", + "sys_role", + ] + assert calls == ["pg-1"] + + +def test_collect_codegen_table_names_falls_back_on_empty_or_error(monkeypatch): + class Introspector: + def __init__(self, manager): + pass + + def list_tables(self, connection_id): + if connection_id == "empty": + return [] + raise RuntimeError("metadata unavailable") + + monkeypatch.setattr(mcp_tools, "DatabaseIntrospector", Introspector) + + assert mcp_tools._collect_codegen_table_names("empty", "orders") == ["orders"] + assert mcp_tools._collect_codegen_table_names("broken", "orders") == ["orders"] + + +@pytest.mark.asyncio +async def test_codegen_generate_writes_to_explicit_output_dir(monkeypatch, tmp_path): + project = tmp_path / "project" + output = tmp_path / "custom-output" + _stub_codegen_handler( + monkeypatch, + project, + { + "entity.mustache": { + "filename": "com/example/User.java", + "code": "class User {}", + }, + "mapper.xml.mustache": { + "filename": "resources/mapper/User.xml", + "code": "<mapper />", + }, + }, + ) + await mcp_tools.handle_db_codegen_generate( { "connection_id": "conn-1", @@ -110,6 +161,92 @@ async def generate_code(self, *_args, **_kwargs): assert not (project / "src" / "main" / "java" / "com" / "example" / "User.java").exists() +@pytest.mark.asyncio +async def test_codegen_generate_reports_actual_write_failures(monkeypatch, tmp_path): + project = tmp_path / "project" + output = tmp_path / "custom-output" + _stub_codegen_handler( + monkeypatch, + project, + { + "entity.mustache": { + "filename": "com/example/User.java", + "code": "class User {}", + }, + "mapper.xml.mustache": { + "filename": "resources/mapper/User.xml", + "code": "<mapper />", + }, + }, + ) + + real_writer = mcp_tools._write_codegen_file + + def flaky_writer(path: Path, code: str) -> None: + if path.name == "User.xml": + raise OSError("disk full") + real_writer(path, code) + + monkeypatch.setattr(mcp_tools, "_write_codegen_file", flaky_writer) + + response = await mcp_tools.handle_db_codegen_generate( + { + "connection_id": "conn-1", + "table_name": "users", + "template_category": "Default", + "package_name": "com.example", + "project_path": str(project), + "output_dir": str(output), + } + ) + + text = response[0].text + assert (output / "com" / "example" / "User.java").read_text(encoding="utf-8") == "class User {}" + assert not (output / "resources" / "mapper" / "User.xml").exists() + assert "Write Attempts: 2" in text + assert "Write Succeeded: 1" in text + assert "Write Failed: 1" in text + assert "PARTIAL" in text + assert "All files written" not in text + + +@pytest.mark.asyncio +async def test_codegen_generate_reports_rejected_paths_separately(monkeypatch, tmp_path): + project = tmp_path / "project" + output = tmp_path / "custom-output" + _stub_codegen_handler( + monkeypatch, + project, + { + "entity.mustache": { + "filename": "../outside/User.java", + "code": "class User {}", + }, + }, + ) + + response = await mcp_tools.handle_db_codegen_generate( + { + "connection_id": "conn-1", + "table_name": "users", + "template_category": "Default", + "package_name": "com.example", + "project_path": str(project), + "output_dir": str(output), + } + ) + + text = response[0].text + assert "Write Candidates: 1" in text + assert "Write Attempts: 0" in text + assert "Write Succeeded: 0" in text + assert "Paths Rejected: 1" in text + assert "Write Failed: 0" in text + assert "path rejected" in text + assert "FAILED: No generated files were written" in text + assert not (tmp_path / "outside" / "User.java").exists() + + def test_codegen_schema_exposes_output_dir(): tool = next( tool for tool in mcp_tools.get_codegen_tools() if tool.name == "db_codegen_generate" diff --git a/tests/unit/test_commit_metadata.py b/tests/unit/test_commit_metadata.py index 3f0cb2c..c0feed8 100644 --- a/tests/unit/test_commit_metadata.py +++ b/tests/unit/test_commit_metadata.py @@ -8,6 +8,7 @@ validate_title = _POLICY["validate_title"] validate_pr_body = _POLICY["validate_pr_body"] validate_issue_body = _POLICY["validate_issue_body"] +validate_main = _POLICY["main"] VALID_PR_BODY = """## 关联 Issue @@ -39,6 +40,48 @@ 模板变为必填；回滚本提交。 """ +VALID_BUG_ISSUE_BODY = """## 版本与环境 + +Python 3.12，固定 SQLite fixture。 + +## 问题与预期行为 + +实际结果与预期结果不一致。 + +## 最小复现 + +运行最小测试命令即可复现。 + +## 验收标准 + +- [ ] 回归测试通过。 + +## 非目标、风险与安全 + +不涉及凭据或生产数据；恢复单一提交即可回滚。 +""" + +VALID_FEATURE_ISSUE_BODY = """## 问题与用户价值 + +统一合同，降低维护成本。 + +## 建议方案与替代方案 + +复用共享 helper；不采用重复实现。 + +## 验收标准 + +- [ ] 新旧入口行为一致。 + +## 架构、兼容性与测试计划 + +保持公共 API，运行单元测试和静态检查。 + +## 非目标与风险 + +不改变数据库 schema；回滚单一提交。 +""" + def test_valid_gitmoji_conventional_title(): assert validate_title(":sparkles: feat(generator): add nullable columns") == [] @@ -81,24 +124,6 @@ def test_rejects_likely_encoding_corruption(): assert any("UTF-8 encoding" in error for error in errors) -def test_rejects_replacement_character_pseudo_newline_and_placeholder_title(): - replacement_errors = validate_title(":books: docs(governance): 修复\ufffd") - pseudo_newline_errors = validate_title(":books: docs(governance): 修复\\n占位") - placeholder_errors = validate_title(":books: docs(governance): <请填写摘要>") - - assert any("replacement character" in error for error in replacement_errors) - assert any("literal" in error for error in pseudo_newline_errors) - assert any("placeholder" in error for error in placeholder_errors) - - -def test_rejects_generic_title_and_sentence_punctuation(): - assert validate_title("init") - assert any( - "sentence punctuation" in error - for error in validate_title(":books: docs(readme): 更新说明。") - ) - - def test_accepts_complete_pr_body(): assert validate_pr_body(VALID_PR_BODY) == [] @@ -110,79 +135,61 @@ def test_rejects_incomplete_pr_body_and_encoding_corruption(): assert any("missing required section" in error for error in errors) -def test_rejects_empty_pr_sections_and_unreplaced_placeholders(): - body = VALID_PR_BODY.replace("固化契约。", "").replace( - "模板变为必填；回滚本提交。", "<填写风险>" - ) - - errors = validate_pr_body(body) - +def test_rejects_empty_and_out_of_order_pr_sections(): + empty = VALID_PR_BODY.replace("更新模板和 CI。", "") + errors = validate_pr_body(empty) assert any("section is empty" in error for error in errors) - assert any("placeholder" in error for error in errors) - - -VALID_ISSUE_BODY = """## 问题与用户价值 - -贡献者需要可追溯的元数据规范。 - -## 建议方案与替代方案 - -扩展纯 Python 校验器；不依赖网络。 - -## 验收标准 - -- [ ] 覆盖正文结构和编码信号。 - -## 架构、兼容性与测试计划 - -运行单元测试，保持运行时 API 不变。 - -## 非目标与风险 - -不重写历史提交；回滚治理提交。 -""" - - -def test_accepts_complete_issue_body(): - assert validate_issue_body(VALID_ISSUE_BODY) == [] - - -def test_accepts_issue_form_headings(): - body = """### 环境信息 - -Python 3.12；SQLite fixture。 - -### 问题与预期行为 - -描述实际和预期行为。 - -### 复现步骤 -运行脱敏的最小命令。 + out_of_order = ( + VALID_PR_BODY.replace("## 任务（Task）", "## TEMP") + .replace("## 行动（Action）", "## 任务（Task）") + .replace("## TEMP", "## 行动（Action）") + ) + assert any("out of order" in error for error in validate_pr_body(out_of_order)) -### 验收标准 -- [ ] 回归测试通过。 +def test_rejects_replacement_control_and_literal_escape_characters(): + replacement_errors = validate_pr_body(VALID_PR_BODY.replace("固化契约。", "固化\ufffd契约。")) + assert any("replacement characters" in error for error in replacement_errors) -### 非目标、风险与安全信息 + control_errors = validate_pr_body(VALID_PR_BODY.replace("固化契约。", "固化\x01契约。")) + assert any("control characters" in error for error in control_errors) -不包含凭据；可回滚。 -""" + escape_errors = validate_pr_body(VALID_PR_BODY.replace("固化契约。", "固化\\r\\n契约。")) + assert any("literal escape sequences" in error for error in escape_errors) - assert validate_issue_body(body) == [] +def test_accepts_windows_paths_in_body_text(): + body = VALID_PR_BODY.replace("固化契约。", r"记录 C:\tmp\report，固化契约。") + assert validate_pr_body(body) == [] -def test_rejects_issue_body_with_missing_sections_and_pseudo_newline(): - errors = validate_issue_body("## 问题与用户价值\\n\n说明") - assert any("complete bug or feature section set" in error for error in errors) - assert any("literal" in error for error in errors) +def test_validates_bug_and_feature_issue_bodies(): + assert validate_issue_body(VALID_BUG_ISSUE_BODY, ":bug: fix(database): 修复事务") == [] + assert ( + validate_issue_body(VALID_FEATURE_ISSUE_BODY, ":sparkles: feat(database): 增加能力") == [] + ) -def test_rejects_issue_body_with_empty_section_and_control_character(): - body = VALID_ISSUE_BODY.replace("扩展纯 Python 校验器；不依赖网络。", "") + "\x0b" +def test_rejects_issue_body_with_wrong_template_and_encoding(): + errors = validate_issue_body( + VALID_BUG_ISSUE_BODY.replace("## 最小复现", "## 复现步骤").replace( + "固定 SQLite", "固定?? SQLite" + ), + ":bug: fix(database): 修复事务", + ) + assert any("consecutive '?'" in error for error in errors) + assert any("missing required section" in error for error in errors) - errors = validate_issue_body(body) - assert any("section is empty" in error for error in errors) - assert any("control character" in error for error in errors) +def test_issue_event_cli_validates_title_and_body(tmp_path, capsys): + event_path = tmp_path / "issues.json" + event_path.write_text( + '{"issue": {"title": ":sparkles: feat(database): 增加能力", ' + '"body": ' + repr(VALID_FEATURE_ISSUE_BODY).replace("'", '"') + "}}", + encoding="utf-8", + ) + assert validate_main(["--issue-event", str(event_path)]) == 0 + output = capsys.readouterr() + assert "OK: issue title" in output.out + assert "OK: issue body" in output.out diff --git a/tests/unit/test_connection_manager.py b/tests/unit/test_connection_manager.py index d233bf4..d1e1261 100644 --- a/tests/unit/test_connection_manager.py +++ b/tests/unit/test_connection_manager.py @@ -156,12 +156,76 @@ def test_bad_sql_raises(self, manager_with_conn): # A statement error must not evict an otherwise healthy connection. assert cid in mgr.connections + def test_failed_sql_rolls_back_pending_sqlite_transaction(self, manager_with_conn): + mgr, cid = manager_with_conn + mgr.execute_query(cid, "CREATE TABLE pending (id INTEGER PRIMARY KEY, value TEXT)") + + # Simulate a caller-owned transaction started on the same connection. + connection = mgr.get_connection(cid) + connection.execute("INSERT INTO pending (id, value) VALUES (1, 'uncommitted')") + + with pytest.raises(DatabaseQueryError): + mgr.execute_query(cid, "INVALID SQL STATEMENT") + + assert mgr.execute_query(cid, "SELECT * FROM pending") == [] + def test_empty_result_table(self, manager_with_conn): mgr, cid = manager_with_conn mgr.execute_query(cid, "CREATE TABLE e (x INT)") rows = mgr.execute_query(cid, "SELECT * FROM e") assert rows == [] + def test_sqlite_file_writes_survive_connection_reopen(self, tmp_path): + database = tmp_path / "persisted.db" + config = DatabaseConfig( + type=DatabaseType.SQLITE, + host="", + port=0, + database=str(database), + username="", + password="", + ) + + first = ConnectionManager() + first_id = first.create_connection(config) + first.execute_query(first_id, "CREATE TABLE items (id INTEGER PRIMARY KEY, name TEXT)") + first.execute_query(first_id, "INSERT INTO items (id, name) VALUES (?, ?)", (1, "saved")) + first.close_connection(first_id) + + second = ConnectionManager() + second_id = second.create_connection(config) + try: + assert second.execute_query(second_id, "SELECT name FROM items") == [{"name": "saved"}] + finally: + second.close_connection(second_id) + + def test_sqlite_returning_write_is_committed(self, tmp_path): + database = tmp_path / "returning.db" + config = DatabaseConfig( + type=DatabaseType.SQLITE, + host="", + port=0, + database=str(database), + username="", + password="", + ) + manager = ConnectionManager() + connection_id = manager.create_connection(config) + try: + manager.execute_query(connection_id, "CREATE TABLE items (id INTEGER PRIMARY KEY)") + assert manager.execute_query( + connection_id, "INSERT INTO items DEFAULT VALUES RETURNING id" + ) == [{"id": 1}] + finally: + manager.close_connection(connection_id) + + reopened = ConnectionManager() + reopened_id = reopened.create_connection(config) + try: + assert reopened.execute_query(reopened_id, "SELECT id FROM items") == [{"id": 1}] + finally: + reopened.close_connection(reopened_id) + class TestGetCursor: def test_context_manager_yields_cursor(self, manager_with_conn): diff --git a/tests/unit/test_dependency_requirements.py b/tests/unit/test_dependency_requirements.py new file mode 100644 index 0000000..f33b94c --- /dev/null +++ b/tests/unit/test_dependency_requirements.py @@ -0,0 +1,50 @@ +"""Regression tests for dependency requirement version isolation.""" + +from dbjavagenix.utils.dependency_requirements import DependencyRequirements + + +def _required_coordinate(requirements, group_id: str, artifact_id: str) -> tuple[str, str, str]: + for dependency in requirements["required"]: + if dependency.group_id == group_id and dependency.artifact_id == artifact_id: + return dependency.group_id, dependency.artifact_id, dependency.version + raise AssertionError(f"missing required dependency {group_id}:{artifact_id}") + + +def test_spring_boot_version_adaptation_does_not_leak_between_analyses(): + catalog = DependencyRequirements() + + boot2 = catalog.analyze_requirements("MybatisPlus", "mysql", spring_boot_version="2.7.18") + modern_defaults = catalog.analyze_requirements("MybatisPlus", "mysql") + + assert _required_coordinate(boot2, "mysql", "mysql-connector-java") == ( + "mysql", + "mysql-connector-java", + "8.0.33", + ) + assert _required_coordinate(modern_defaults, "com.mysql", "mysql-connector-j") == ( + "com.mysql", + "mysql-connector-j", + "8.4.0", + ) + assert _required_coordinate( + modern_defaults, + "com.baomidou", + "mybatis-plus-spring-boot3-starter", + ) == ("com.baomidou", "mybatis-plus-spring-boot3-starter", "3.5.7") + + +def test_version_adaptation_is_order_independent(): + first = DependencyRequirements() + first_modern = first.analyze_requirements("MybatisPlus", "mysql", spring_boot_version="3.5.5") + first_boot2 = first.analyze_requirements("MybatisPlus", "mysql", spring_boot_version="2.7.18") + + second = DependencyRequirements() + second_boot2 = second.analyze_requirements("MybatisPlus", "mysql", spring_boot_version="2.7.18") + second_modern = second.analyze_requirements("MybatisPlus", "mysql", spring_boot_version="3.5.5") + + assert _required_coordinate( + first_modern, "com.mysql", "mysql-connector-j" + ) == _required_coordinate(second_modern, "com.mysql", "mysql-connector-j") + assert _required_coordinate( + first_boot2, "mysql", "mysql-connector-java" + ) == _required_coordinate(second_boot2, "mysql", "mysql-connector-java") diff --git a/tests/unit/test_dialect.py b/tests/unit/test_dialect.py index 64a1528..9d351b2 100644 --- a/tests/unit/test_dialect.py +++ b/tests/unit/test_dialect.py @@ -104,6 +104,10 @@ def test_is_decimal_type(self): assert self.d.is_decimal_type("DECIMAL") assert not self.d.is_decimal_type("INT") + def test_java_imports_follow_mapped_type(self): + assert self.d.java_imports_for("DATETIME") == ["java.time.LocalDateTime"] + assert self.d.java_imports_for("INT") == [] + class TestPostgreSQLDialect: def setup_method(self): @@ -173,6 +177,10 @@ def test_jdbc_for_special_types(self): def test_unknown_falls_back_to_string(self): assert self.d.java_type_for("WEIRD_PG_TYPE") == "String" + def test_java_imports_follow_mapped_type(self): + assert self.d.java_imports_for("TIMESTAMPTZ") == ["java.time.OffsetDateTime"] + assert self.d.java_imports_for("UUID") == [] + class TestSQLiteDialect: def setup_method(self): diff --git a/tests/unit/test_java_generator.py b/tests/unit/test_java_generator.py index 3a36c5b..cd6e7c4 100644 --- a/tests/unit/test_java_generator.py +++ b/tests/unit/test_java_generator.py @@ -115,13 +115,51 @@ def test_unknown_template_uses_fallback(self, gen, simple_table, gen_config): assert "generated" in path assert path.endswith("brand_new.java") + def test_empty_package_suffix_does_not_create_empty_path_segment( + self, gen, simple_table, gen_config + ): + from dbjavagenix.generator.template_context import TemplateContextBuilder + + context = TemplateContextBuilder( + author=gen_config.author, package_name=gen_config.package_name + ).build_context(simple_table, "Default") + path = gen._get_output_path("entity.mustache", context, gen_config.output_dir) + + relative = Path(path).relative_to(Path(gen_config.output_dir)).as_posix() + assert relative == "entity/Account.java" + + def test_package_suffix_remains_a_distinct_path_segment(self, gen, gen_config): + from dbjavagenix.generator.template_context import TemplateContextBuilder + + table = TableInfo( + name="sys_account", + schema="public", + columns=[ + ColumnInfo( + name="id", + data_type="BIGINT", + java_type="Long", + primary_key=True, + ) + ], + ) + context = TemplateContextBuilder( + author=gen_config.author, package_name=gen_config.package_name + ).build_context( + table, + "Default", + all_table_names=["sys_account", "sys_role"], + ) + path = gen._get_output_path("entity.mustache", context, gen_config.output_dir) + + relative = Path(path).relative_to(Path(gen_config.output_dir)).as_posix() + assert relative == "entity/system/SysAccount.java" + class TestRealGeneration: """跑一次真实生成, 验证文件产生且包含关键 token""" - def test_default_generates_entity_dao_service_controller( - self, gen, simple_table, gen_config - ): + def test_default_generates_entity_dao_service_controller(self, gen, simple_table, gen_config): output_dir = gen_config.output_dir Path(output_dir).mkdir(parents=True, exist_ok=True) result = gen.generate_from_table( @@ -136,6 +174,7 @@ def test_default_generates_entity_dao_service_controller( # 文件实际存在 entity_path = Path(result["entity.mustache"]) assert entity_path.exists() + assert entity_path.relative_to(Path(output_dir)).as_posix() == "entity/Account.java" content = entity_path.read_text(encoding="utf-8") assert "class Account" in content assert "private Long id" in content @@ -153,9 +192,7 @@ def test_sb35_java21_generates_record_dto(self, gen, simple_table, gen_config): dto_content = Path(result["dto.mustache"]).read_text(encoding="utf-8") assert "public record AccountDTO" in dto_content - def test_sb35_java21_entity_is_class_not_record( - self, gen, simple_table, gen_config - ): + def test_sb35_java21_entity_is_class_not_record(self, gen, simple_table, gen_config): """回归测试: P1.2 修复后 sb35-java21 entity 应为 class (JPA 兼容), 不是 record""" output_dir = gen_config.output_dir Path(output_dir).mkdir(parents=True, exist_ok=True) diff --git a/tests/unit/test_mcp_apps_code_diff.py b/tests/unit/test_mcp_apps_code_diff.py index 3e3b48f..b1818b7 100644 --- a/tests/unit/test_mcp_apps_code_diff.py +++ b/tests/unit/test_mcp_apps_code_diff.py @@ -72,12 +72,14 @@ def test_project_root_file_exists(self, tmp_path): target.parent.mkdir(parents=True) target.write_text("public class User { /* old */ }", encoding="utf-8") - files = [{ - "template_file": "entity.mustache", - "file_path": target_rel, - "code": "public class User { /* new */ }", - "lines": 1, - }] + files = [ + { + "template_file": "entity.mustache", + "file_path": target_rel, + "code": "public class User { /* new */ }", + "lines": 1, + } + ] data = build_code_diff_data(files, project_root=str(tmp_path)) assert len(data["files"]) == 1 f = data["files"][0] @@ -106,3 +108,13 @@ def test_returns_none_for_binary(self, tmp_path): f.write_bytes(b"\xff\xfe\xfd\xfc") # 非 utf-8 时返回 None assert _try_read_existing(tmp_path, "binary.bin") is None + + def test_returns_none_for_parent_path_escape(self, tmp_path): + outside = tmp_path.parent / "outside.txt" + outside.write_text("must not read", encoding="utf-8") + assert _try_read_existing(tmp_path, "../outside.txt") is None + + def test_returns_none_for_absolute_path_escape(self, tmp_path): + outside = tmp_path.parent / "outside-absolute.txt" + outside.write_text("must not read", encoding="utf-8") + assert _try_read_existing(tmp_path, str(outside)) is None diff --git a/tests/unit/test_mcp_apps_package_tree.py b/tests/unit/test_mcp_apps_package_tree.py index 1424fc9..17b1251 100644 --- a/tests/unit/test_mcp_apps_package_tree.py +++ b/tests/unit/test_mcp_apps_package_tree.py @@ -21,9 +21,11 @@ def test_empty(self): assert data == {"root": "", "children": []} def test_single_file(self): - data = build_package_tree_data([ - "com/example/entity/User.java", - ]) + data = build_package_tree_data( + [ + "com/example/entity/User.java", + ] + ) assert data["root"] == "com.example.entity" # 仅有一层 children: 文件本身 assert len(data["children"]) == 1 @@ -34,11 +36,13 @@ def test_single_file(self): } def test_common_prefix_detected(self): - data = build_package_tree_data([ - "com/example/entity/User.java", - "com/example/dao/UserDao.java", - "com/example/service/UserService.java", - ]) + data = build_package_tree_data( + [ + "com/example/entity/User.java", + "com/example/dao/UserDao.java", + "com/example/service/UserService.java", + ] + ) assert data["root"] == "com.example" # 三个一级 children: entity / dao / service (packages) names = [c["name"] for c in data["children"]] @@ -48,10 +52,12 @@ def test_common_prefix_detected(self): assert "service" in names def test_nested_packages(self): - data = build_package_tree_data([ - "com/example/entity/sub/Inner.java", - "com/example/entity/Outer.java", - ]) + data = build_package_tree_data( + [ + "com/example/entity/sub/Inner.java", + "com/example/entity/Outer.java", + ] + ) assert data["root"] == "com.example.entity" # children: Outer.java (file) + sub (package) children = data["children"] @@ -60,11 +66,13 @@ def test_nested_packages(self): assert types["sub"] == "package" def test_alpha_sort(self): - data = build_package_tree_data([ - "z.java", - "a.java", - "m.java", - ]) + data = build_package_tree_data( + [ + "z.java", + "a.java", + "m.java", + ] + ) names = [c["name"] for c in data["children"]] assert names == ["a.java", "m.java", "z.java"] @@ -98,10 +106,12 @@ def test_no_common(self): assert _common_prefix([["a", "X.java"], ["b", "Y.java"]]) == [] def test_partial(self): - assert _common_prefix([ - ["a", "b", "c", "X.java"], - ["a", "b", "d", "Y.java"], - ]) == ["a", "b"] + assert _common_prefix( + [ + ["a", "b", "c", "X.java"], + ["a", "b", "d", "Y.java"], + ] + ) == ["a", "b"] def test_empty_list(self): assert _common_prefix([]) == [] @@ -118,3 +128,13 @@ def test_file_exists_returns_modified(self, tmp_path): def test_file_missing_returns_new(self, tmp_path): assert _determine_status(tmp_path, "missing.txt") == "new" + + def test_parent_path_escape_returns_new(self, tmp_path): + outside = tmp_path.parent / "outside.txt" + outside.write_text("must not inspect", encoding="utf-8") + assert _determine_status(tmp_path, "../outside.txt") == "new" + + def test_absolute_path_escape_returns_new(self, tmp_path): + outside = tmp_path.parent / "outside-absolute.txt" + outside.write_text("must not inspect", encoding="utf-8") + assert _determine_status(tmp_path, str(outside)) == "new" diff --git a/tests/unit/test_mcp_disconnect.py b/tests/unit/test_mcp_disconnect.py new file mode 100644 index 0000000..f7c18c5 --- /dev/null +++ b/tests/unit/test_mcp_disconnect.py @@ -0,0 +1,117 @@ +"""Regression tests for the public MCP connection lifecycle tool.""" + +import json + +import pytest + +from dbjavagenix.core.models import DatabaseConfig, DatabaseType +from dbjavagenix.database import mcp_tools +from dbjavagenix.database.connection_manager import ConnectionManager +from dbjavagenix.utils.tool_registry import search_tools_by_query + + +def _raw_payload(response): + return json.loads(response[0].text.split("Raw Response:", 1)[1].strip()) + + +def _sqlite_config(database=":memory:"): + return DatabaseConfig( + type=DatabaseType.SQLITE, + host="", + port=0, + database=database, + username="", + password="secret", + ) + + +def test_disconnect_tool_schema_requires_only_non_empty_connection_id(): + tool = next(tool for tool in mcp_tools.get_connection_tools() if tool.name == "db_disconnect") + + assert tool.inputSchema["required"] == ["connection_id"] + assert tool.inputSchema["properties"] == { + "connection_id": { + "type": "string", + "description": "Connection identifier returned by db_connect_test", + "minLength": 1, + } + } + + +@pytest.mark.asyncio +async def test_disconnect_success_removes_connection_and_masked_config(monkeypatch): + manager = ConnectionManager() + connection_id = manager.create_connection(_sqlite_config()) + monkeypatch.setattr(mcp_tools, "connection_manager", manager) + + response = await mcp_tools.handle_db_disconnect({"connection_id": connection_id}) + + assert _raw_payload(response) == { + "success": True, + "connection_id": connection_id, + "message": "Connection closed successfully", + } + assert connection_id not in manager.connections + assert connection_id not in manager.connection_configs + + +@pytest.mark.asyncio +@pytest.mark.parametrize("arguments", [{}, {"connection_id": ""}, {"connection_id": " "}]) +async def test_disconnect_empty_id_returns_connection_not_found(monkeypatch, arguments): + def fail_if_called(_connection_id): + raise AssertionError("empty IDs must be rejected before touching the manager") + + monkeypatch.setattr(mcp_tools.connection_manager, "close_connection", fail_if_called) + + response = await mcp_tools.handle_db_disconnect(arguments) + + assert _raw_payload(response) == { + "success": False, + "error": "connection_not_found", + "connection_id": None, + "message": "Connection not found", + } + + +@pytest.mark.asyncio +async def test_disconnect_unknown_and_repeated_id_are_stable_failures(monkeypatch): + manager = ConnectionManager() + connection_id = manager.create_connection(_sqlite_config()) + monkeypatch.setattr(mcp_tools, "connection_manager", manager) + + first = _raw_payload(await mcp_tools.handle_db_disconnect({"connection_id": connection_id})) + repeated = _raw_payload(await mcp_tools.handle_db_disconnect({"connection_id": connection_id})) + unknown = _raw_payload(await mcp_tools.handle_db_disconnect({"connection_id": "missing"})) + + assert first["success"] is True + assert repeated == { + "success": False, + "error": "connection_not_found", + "connection_id": connection_id, + "message": "Connection not found", + } + assert unknown["error"] == "connection_not_found" + assert unknown["success"] is False + + +@pytest.mark.asyncio +async def test_disconnect_exception_is_redacted(monkeypatch): + def fail(_connection_id): + raise RuntimeError("jdbc:mysql://reader:db-secret@db/app") + + monkeypatch.setattr(mcp_tools.connection_manager, "close_connection", fail) + + response = await mcp_tools.handle_db_disconnect({"connection_id": "conn-1"}) + payload = _raw_payload(response) + + assert payload["success"] is False + assert payload["error"] == "disconnect_failed" + assert payload["message"] == "jdbc:mysql://reader:***@db/app" + assert "db-secret" not in response[0].text + + +def test_disconnect_is_discoverable_by_progressive_registry(): + results = search_tools_by_query("disconnect") + + assert results + assert results[0]["name"] == "db_disconnect" diff --git a/tests/unit/test_mcp_json_boundary.py b/tests/unit/test_mcp_json_boundary.py new file mode 100644 index 0000000..43080f0 --- /dev/null +++ b/tests/unit/test_mcp_json_boundary.py @@ -0,0 +1,59 @@ +"""Regression tests for shared JSON serialization at MCP response boundaries.""" + +import asyncio +import json +from decimal import Decimal +from types import SimpleNamespace + +from dbjavagenix.database import ai_tools, discovery_tools, observability_tools +from dbjavagenix.database.ai_tools import handle_ai_recommend_template +from dbjavagenix.database.discovery_tools import handle_search_tools +from dbjavagenix.database.observability_tools import handle_server_metrics + + +def test_ai_response_serializes_nested_driver_values(monkeypatch): + monkeypatch.setattr( + ai_tools, + "recommend_template", + lambda **_: SimpleNamespace( + template="MybatisPlus", + options={"sample": Decimal("1.25")}, + pattern="Custom", + confidence="medium", + score=Decimal("0.875"), + reasons=[], + matched_tables=[], + ), + ) + + result = asyncio.run(handle_ai_recommend_template({"table_names": ["orders"]})) + payload = json.loads(result[0].text) + + assert payload["score"] == "0.875" + assert payload["options"]["sample"] == "1.25" + + +def test_discovery_response_serializes_driver_values(monkeypatch): + monkeypatch.setattr( + discovery_tools, + "search_tools_by_query", + lambda *_args, **_kwargs: [{"name": "custom", "score": Decimal("2.5")}], + ) + + result = asyncio.run(handle_search_tools({"query": "custom"})) + payload = json.loads(result[0].text) + + assert payload["results"][0]["score"] == "2.5" + + +def test_observability_response_serializes_driver_values(monkeypatch): + monkeypatch.setattr( + observability_tools.GLOBAL_TOOL_METRICS, + "snapshot", + lambda: {"latency": Decimal("3.75")}, + ) + + result = asyncio.run(handle_server_metrics({})) + payload = json.loads(result[0].text) + + assert payload["metrics"]["latency"] == "3.75" diff --git a/tests/unit/test_mustache_engine.py b/tests/unit/test_mustache_engine.py index fee989f..5062709 100644 --- a/tests/unit/test_mustache_engine.py +++ b/tests/unit/test_mustache_engine.py @@ -185,6 +185,31 @@ def test_comment_fallback(self, sample_config): ctx = TemplateContext.build_entity_context(t, sample_config) assert "entity" in ctx["comment"].lower() + @pytest.mark.parametrize( + "primary_key, auto_increment, expected", + [(True, False, False), (False, True, True), (False, False, False)], + ) + def test_has_auto_increment_uses_column_flag( + self, sample_config, primary_key, auto_increment, expected + ): + table = TableInfo( + name="account", + schema="public", + columns=[ + ColumnInfo( + name="id", + data_type="BIGINT", + java_type="Long", + primary_key=primary_key, + auto_increment=auto_increment, + ) + ], + ) + + context = TemplateContext.build_entity_context(table, sample_config) + + assert context["hasAutoIncrement"] is expected + class TestTemplateContextDto: def test_excludes_primary_key(self, sample_table, sample_config): diff --git a/tests/unit/test_postgresql_mcp_tools.py b/tests/unit/test_postgresql_mcp_tools.py index 8717d65..d41ad38 100644 --- a/tests/unit/test_postgresql_mcp_tools.py +++ b/tests/unit/test_postgresql_mcp_tools.py @@ -7,6 +7,7 @@ from dbjavagenix.core.models import DatabaseType from dbjavagenix.database import mcp_tools +from dbjavagenix.database.dialect import get_dialect @pytest.fixture @@ -221,12 +222,23 @@ def execute_query(connection_id, query, params=None): def test_postgresql_java_mapping_normalizes_catalog_type_names(): + assert mcp_tools._get_java_type_mapping(DatabaseType.POSTGRESQL, "time without time zone") == { + "java_type": "LocalTime", + "imports": ["java.time.LocalTime"], + } + assert mcp_tools._get_java_type_mapping( + DatabaseType.POSTGRESQL, "timestamp without time zone" + ) == {"java_type": "LocalDateTime", "imports": ["java.time.LocalDateTime"]} + assert mcp_tools._get_java_type_mapping(DatabaseType.POSTGRESQL, "timestamptz") == { + "java_type": "OffsetDateTime", + "imports": ["java.time.OffsetDateTime"], + } assert mcp_tools._get_java_type_mapping( DatabaseType.POSTGRESQL, "timestamp with time zone" ) == {"java_type": "OffsetDateTime", "imports": ["java.time.OffsetDateTime"]} assert mcp_tools._get_java_type_mapping(DatabaseType.POSTGRESQL, "uuid") == { - "java_type": "UUID", - "imports": ["java.util.UUID"], + "java_type": "String", + "imports": [], } assert mcp_tools._get_java_type_mapping(DatabaseType.POSTGRESQL, "jsonb") == { "java_type": "String", @@ -241,6 +253,40 @@ def test_postgresql_java_mapping_normalizes_catalog_type_names(): ) == {"java_type": "OffsetDateTime", "imports": ["java.time.OffsetDateTime"]} +@pytest.mark.parametrize( + ("database_type", "column_type"), + [ + (DatabaseType.MYSQL, "TINYINT(1)"), + (DatabaseType.MYSQL, "SMALLINT"), + (DatabaseType.POSTGRESQL, "INT2"), + (DatabaseType.POSTGRESQL, "UUID"), + (DatabaseType.SQLITE, "INTEGER"), + (DatabaseType.SQLITE, "DATETIME"), + ], +) +def test_supported_mcp_mapping_matches_codegen_dialect(database_type, column_type): + expected = get_dialect(database_type.value) + + assert mcp_tools._get_java_type_mapping(database_type, column_type) == { + "java_type": expected.java_type_for(column_type), + "imports": expected.java_imports_for(column_type), + } + + +def test_supported_mcp_mapping_unknown_type_matches_codegen_fallback(): + assert mcp_tools._get_java_type_mapping(DatabaseType.POSTGRESQL, "CUSTOM_DOMAIN") == { + "java_type": "String", + "imports": [], + } + + +def test_unregistered_dialect_keeps_legacy_yaml_mapping_fallback(): + assert mcp_tools._get_java_type_mapping(DatabaseType.SQLSERVER, "INT") == { + "java_type": "Integer", + "imports": [], + } + + @pytest.mark.asyncio async def test_table_describe_uses_normalized_introspection_with_schema(monkeypatch): class Introspector: diff --git a/tests/unit/test_schema_algorithms_tools.py b/tests/unit/test_schema_algorithms_tools.py index ffc1e0d..9e90a7c 100644 --- a/tests/unit/test_schema_algorithms_tools.py +++ b/tests/unit/test_schema_algorithms_tools.py @@ -119,3 +119,26 @@ def test_malformed_fk_tuple_skipped(self): result = _run(handle_schema_topo_order({"tables": ["a"], "fks": [["a"], ["a", "b", "c"]]})) payload = json.loads(result[0].text) assert payload["order"] == ["a"] + + def test_string_containers_are_not_split_into_nodes(self): + result = _run(handle_schema_topo_order({"tables": "users", "fks": "invalid"})) + payload = json.loads(result[0].text) + assert payload == { + "order": [], + "unresolved": [], + "levels": {}, + "has_cycle": False, + } + + def test_duplicate_tables_are_normalized_before_rendering(self): + result = _run( + handle_schema_cluster_tables( + { + "tables": ["users", "users", "orders"], + "fks": [["orders", "users"], ["orders", "users"]], + } + ) + ) + payload = json.loads(result[0].text) + assert payload["num_clusters"] == 1 + assert payload["clusters"][0]["members"] == ["orders", "users"] diff --git a/tests/unit/test_schema_cluster.py b/tests/unit/test_schema_cluster.py index 414f3f2..773d5c6 100644 --- a/tests/unit/test_schema_cluster.py +++ b/tests/unit/test_schema_cluster.py @@ -1,4 +1,5 @@ """Unit tests for schema_cluster (Union-Find clustering).""" + import pytest from dbjavagenix.algorithms.schema_cluster import ClusterResult, cluster_tables @@ -79,3 +80,14 @@ def test_cluster_member_order_deterministic(self): [("zeta", "alpha"), ("mu", "alpha")], ) assert r.clusters[0] == ["alpha", "mu", "zeta"] + + def test_duplicate_tables_do_not_duplicate_cluster_members(self): + r = cluster_tables( + ["users", "users", "orders", "orders"], + [("orders", "users"), ("orders", "users")], + ) + assert r.clusters == [["orders", "users"]] + + def test_malformed_graph_values_are_ignored(self): + r = cluster_tables(["users", "", None], [("users", ""), (None, "users")]) + assert r.clusters == [["users"]] diff --git a/tests/unit/test_schema_cycle_check.py b/tests/unit/test_schema_cycle_check.py index 198bae6..a653d63 100644 --- a/tests/unit/test_schema_cycle_check.py +++ b/tests/unit/test_schema_cycle_check.py @@ -1,4 +1,5 @@ """Unit tests for schema_cycle_check (DFS cycle detection).""" + import pytest from dbjavagenix.algorithms.schema_cycle_check import CycleResult, find_cycles @@ -28,9 +29,7 @@ def test_simple_2cycle(self): def test_3cycle(self): # a -> b -> c -> a - r = find_cycles( - ["a", "b", "c"], [("a", "b"), ("b", "c"), ("c", "a")] - ) + r = find_cycles(["a", "b", "c"], [("a", "b"), ("b", "c"), ("c", "a")]) assert not r.safe assert len(r.cycles) == 1 assert set(r.cycles[0]) == {"a", "b", "c"} @@ -42,9 +41,7 @@ def test_self_reference_not_a_cycle(self): def test_chain_then_cycle(self): # x (alone) ; a -> b -> a - r = find_cycles( - ["x", "a", "b"], [("a", "b"), ("b", "a")] - ) + r = find_cycles(["x", "a", "b"], [("a", "b"), ("b", "a")]) assert not r.safe assert len(r.cycles) == 1 assert set(r.cycles[0]) == {"a", "b"} @@ -78,3 +75,14 @@ def test_safe_property(self): assert r1.safe is True r2 = find_cycles(["a", "b"], [("a", "b"), ("b", "a")]) assert r2.safe is False + + def test_duplicate_tables_and_edges_are_normalized(self): + r = find_cycles( + ["a", "a", "b"], + [("a", "b"), ("b", "a"), ("a", "b"), ("", "a")], + ) + assert r.cycles == [["a", "b"]] + + def test_malformed_graph_values_are_ignored(self): + r = find_cycles(["a", "b"], "not-an-edge-list") + assert r.safe diff --git a/tests/unit/test_schema_topo.py b/tests/unit/test_schema_topo.py index 6eb730b..28e20f0 100644 --- a/tests/unit/test_schema_topo.py +++ b/tests/unit/test_schema_topo.py @@ -1,4 +1,5 @@ """Unit tests for schema_topo (Kahn's topological sort).""" + import pytest from dbjavagenix.algorithms.schema_topo import TopoResult, topological_sort @@ -35,9 +36,7 @@ def test_rbac_pattern(self): def test_cycle_detection(self): # a -> b -> c -> a (cycle) - r = topological_sort( - ["a", "b", "c"], [("b", "a"), ("c", "b"), ("a", "c")] - ) + r = topological_sort(["a", "b", "c"], [("b", "a"), ("c", "b"), ("a", "c")]) assert r.has_cycle assert set(r.unresolved) == {"a", "b", "c"} assert r.order == [] @@ -65,6 +64,22 @@ def test_deterministic_order(self): r = topological_sort(["z", "y", "x"], []) assert r.order == ["x", "y", "z"] + def test_duplicate_tables_and_edges_are_normalized(self): + r = topological_sort( + ["users", "users", "orders"], + [("orders", "users"), ("orders", "users")], + ) + assert r.order == ["users", "orders"] + assert r.unresolved == [] + assert r.levels == {"users": 0, "orders": 1} + + def test_malformed_graph_values_are_ignored(self): + r = topological_sort( + ["users", "", None, "orders"], + [("orders", "users"), ("orders", None), ["orders"], "invalid"], + ) + assert r.order == ["users", "orders"] + def test_levels_computed(self): # diamond: a -> b, a -> c, b -> d, c -> d r = topological_sort( diff --git a/tests/unit/test_server_dispatch.py b/tests/unit/test_server_dispatch.py index 56aa93d..fd03f6b 100644 --- a/tests/unit/test_server_dispatch.py +++ b/tests/unit/test_server_dispatch.py @@ -54,6 +54,22 @@ async def fake_health(arguments): assert calls == [{"verbose": True}] +@pytest.mark.asyncio +async def test_disconnect_is_dispatchable_from_canonical_name(monkeypatch): + calls = [] + + async def fake_disconnect(arguments): + calls.append(arguments) + return [TextContent(type="text", text="closed")] + + monkeypatch.setattr(mcp_server, "handle_db_disconnect", fake_disconnect) + + result = await mcp_server.handle_call_tool("db_disconnect", {"connection_id": "conn-1"}) + + assert result[0].text == "closed" + assert calls == [{"connection_id": "conn-1"}] + + @pytest.mark.asyncio async def test_unknown_tool_returns_structured_error(monkeypatch): monkeypatch.setattr(mcp_server, "_tool_handlers", lambda: {}) diff --git a/tests/unit/test_template_context_builder.py b/tests/unit/test_template_context_builder.py index 4892c9b..8ef482d 100644 --- a/tests/unit/test_template_context_builder.py +++ b/tests/unit/test_template_context_builder.py @@ -3,6 +3,8 @@ 聚焦命名转换、类型映射、上下文字典结构,不依赖项目根目录探测。 """ +from pathlib import Path + import pytest from dbjavagenix.core.models import ColumnInfo, DatabaseType, TableInfo @@ -15,6 +17,7 @@ TemplateConfigManager, TemplateContextBuilder, ) +from dbjavagenix.generator.mustache_engine import MustacheTemplateEngine from dbjavagenix.utils.pom_analyzer import PomAnalyzer, TechnologyStack @@ -262,6 +265,34 @@ def test_columns_structure(self, builder, rbac_user_table): assert "isPrimaryKey" in first assert "isLast" in first + def test_capitalized_names_preserve_camel_case(self, builder): + table = TableInfo( + name="api_client", + schema="public", + columns=[ + ColumnInfo( + name="api_url", + data_type="VARCHAR(255)", + java_type="String", + primary_key=True, + ), + ColumnInfo( + name="display_name", + data_type="VARCHAR(255)", + java_type="String", + ), + ], + primary_keys=["api_url"], + ) + + context = builder.build_context(table, "Default") + + assert [column["capitalizedJavaName"] for column in context["columns"]] == [ + "ApiUrl", + "DisplayName", + ] + assert context["capitalizedPrimaryKeyName"] == "ApiUrl" + def test_postgresql_context_includes_dialect_imports(self): table = TableInfo( name="audit_event", @@ -295,6 +326,161 @@ def test_primary_key_info(self, builder, rbac_user_table): assert len(pk_cols) == 1 assert pk_cols[0]["javaName"] == "userId" + def test_primary_key_list_normalizes_column_context_and_preserves_order(self, builder): + table = TableInfo( + name="order_item", + schema="public", + columns=[ + ColumnInfo(name="line_no", data_type="INT", java_type="Integer"), + ColumnInfo(name="order_id", data_type="BIGINT", java_type="Long"), + ColumnInfo(name="sku", data_type="VARCHAR(32)", java_type="String"), + ], + primary_keys=["order_id", "line_no"], + ) + + context = builder.build_context(table, "Default") + + assert context["primaryKey"]["dbName"] == "order_id" + assert [column["name"] for column in context["columns"] if column["isPrimaryKey"]] == [ + "line_no", + "order_id", + ] + assert [column["name"] for column in context["nonPrimaryColumns"]] == ["sku"] + + def test_unmatched_primary_key_list_falls_back_to_column_flags(self, builder): + table = TableInfo( + name="account", + schema="public", + columns=[ + ColumnInfo( + name="id", + data_type="BIGINT", + java_type="Long", + primary_key=True, + ) + ], + primary_keys=["missing_id"], + ) + + context = builder.build_context(table, "Default") + + assert context["primaryKey"]["dbName"] == "id" + assert context["columns"][0]["isPrimaryKey"] is True + + @pytest.mark.parametrize("auto_increment", [True, False]) + def test_primary_key_context_preserves_auto_increment(self, builder, auto_increment): + table = TableInfo( + name="account", + schema="public", + columns=[ + ColumnInfo( + name="id", + data_type="BIGINT", + java_type="Long", + primary_key=True, + auto_increment=auto_increment, + ) + ], + primary_keys=["id"], + ) + + primary_key = builder.build_context(table, "Default")["primaryKey"] + + assert primary_key["isAutoIncrement"] is auto_increment + assert primary_key["autoIncrement"] is auto_increment + + @pytest.mark.parametrize("auto_increment", [True, False]) + def test_default_mapper_renders_auto_increment_key_option(self, builder, auto_increment): + table = TableInfo( + name="account", + schema="public", + columns=[ + ColumnInfo( + name="id", + data_type="BIGINT", + java_type="Long", + primary_key=True, + auto_increment=auto_increment, + ) + ], + primary_keys=["id"], + ) + context = builder.build_context(table, "Default") + template = ( + Path(__file__).parents[2] + / "src" + / "dbjavagenix" + / "templates" + / "java" + / "Default" + / "mapper.mustache" + ) + + rendered = MustacheTemplateEngine().render_file(str(template), context) + + assert ('useGeneratedKeys="true"' in rendered) is auto_increment + + @pytest.mark.parametrize("auto_increment", [True, False]) + def test_default_xml_mapper_gates_generated_key_options(self, builder, auto_increment): + table = TableInfo( + name="account", + schema="public", + columns=[ + ColumnInfo( + name="account_code", + data_type="VARCHAR(32)", + java_type="String", + primary_key=True, + auto_increment=auto_increment, + ), + ColumnInfo(name="display_name", data_type="VARCHAR(64)", java_type="String"), + ], + primary_keys=["account_code"], + ) + context = builder.build_context(table, "Default") + template = ( + Path(__file__).parents[2] + / "src" + / "dbjavagenix" + / "templates" + / "java" + / "Default" + / "mapper.xml.mustache" + ) + + rendered = MustacheTemplateEngine().render_file(str(template), context) + insert_lines = [line.strip() for line in rendered.splitlines() if "<insert id=" in line] + + assert len(insert_lines) == 3 + for line in insert_lines: + assert ('keyProperty="accountCode"' in line) is auto_increment + assert ('useGeneratedKeys="true"' in line) is auto_increment + + def test_default_xml_mapper_without_primary_key_has_no_generated_key_options(self, builder): + table = TableInfo( + name="audit_log", + schema="public", + columns=[ColumnInfo(name="message", data_type="TEXT", java_type="String")], + ) + context = builder.build_context(table, "Default") + template = ( + Path(__file__).parents[2] + / "src" + / "dbjavagenix" + / "templates" + / "java" + / "Default" + / "mapper.xml.mustache" + ) + + rendered = MustacheTemplateEngine().render_file(str(template), context) + insert_lines = [line for line in rendered.splitlines() if "<insert id=" in line] + + assert len(insert_lines) == 3 + assert all( + "keyProperty" not in line and "useGeneratedKeys" not in line for line in insert_lines + ) + def test_with_prefix_analysis_creates_suffix(self, builder, rbac_user_table): # 提供同前缀的表名集合,前缀分析器应识别出 sys 前缀 all_tables = ["sys_user", "sys_role", "sys_permission"] diff --git a/tests/unit/test_tool_registry.py b/tests/unit/test_tool_registry.py index 4e1191e..347e1ff 100644 --- a/tests/unit/test_tool_registry.py +++ b/tests/unit/test_tool_registry.py @@ -17,6 +17,7 @@ get_discovery_tools, handle_search_tools, ) +from dbjavagenix.database.capabilities import supported_database_display_names from dbjavagenix.database.atomic_codegen_tools import get_atomic_codegen_tools from dbjavagenix.server.mcp_server import _all_tools from dbjavagenix.utils.tool_registry import ( @@ -50,6 +51,14 @@ def test_search_tools_in_registry(self): assert meta.always_visible is True assert "search" in meta.tags + def test_connection_description_matches_runtime_capabilities(self): + meta = get_metadata("db_connect_test") + assert meta is not None + expected = "建立数据库连接 (" + "/".join(supported_database_display_names()) + ")" + assert meta.description_brief == expected + assert "Oracle" not in meta.description_brief + assert "SQL Server" not in meta.description_brief + def test_always_visible_names(self): names = get_always_visible_names() # 至少包含 SKILL 工作流前几步需要的核心工具