docs: add SeaTunnel Source and Sink quick start - #464
Conversation
Bumped default release version to 1.7.0 and updated default GPG user in the release workflow. Added installation step for subversion on Ubuntu. Appended WARP.md to .gitignore.
Deleted validate-release-in-local.sh and enhanced validate-release.sh to support both SVN and local directory validation. Added color-coded output, improved argument handling, and included Java version checks for better usability and error reporting.
Major rewrite of validate-release.sh for Apache HugeGraph, adding modular structure, improved logging, error/warning collection, colorized output, and comprehensive validation steps for source and binary packages. New features include dependency checks, GPG key management, license compliance, file size and binary checks, version consistency, and automated server/toolchain testing. Usage instructions and help output are expanded for clarity.
…s V3 Updated CI workflow to support additional OS and architectures (arm64, macOS 14). Improved documentation and script usage instructions. The license header check now covers more file types and excludes generated/vendor files. Maven build commands in docs and scripts now use '-DskipTests' and '-Dcheckstyle.skip=true' for consistency. Added a detailed README for the release validation script.
Adds contextual error and warning reporting with step and package information, enhances license category and header checks, improves version consistency logic, and refines summary output with execution time and clearer formatting. These changes make validation results more actionable and easier to interpret, especially for multi-package and multi-step validations.
The JSON license was added to the CATEGORY_X regex in the binary package validation step to ensure packages with this license are properly flagged during release validation.
[pull] master from apache:master
* chore: update release workflow and .gitignore entries Bumped default release version to 1.7.0 and updated default GPG user in the release workflow. Added installation step for subversion on Ubuntu. Appended WARP.md to .gitignore. * refactor: unify release validation script and add local path support Deleted validate-release-in-local.sh and enhanced validate-release.sh to support both SVN and local directory validation. Added color-coded output, improved argument handling, and included Java version checks for better usability and error reporting. * refactor: revamp release validation script with enhanced checks V2 Major rewrite of validate-release.sh for Apache HugeGraph, adding modular structure, improved logging, error/warning collection, colorized output, and comprehensive validation steps for source and binary packages. New features include dependency checks, GPG key management, license compliance, file size and binary checks, version consistency, and automated server/toolchain testing. Usage instructions and help output are expanded for clarity. * refactor: enhance release validation for multi-arch and license checks V3 Updated CI workflow to support additional OS and architectures (arm64, macOS 14). Improved documentation and script usage instructions. The license header check now covers more file types and excludes generated/vendor files. Maven build commands in docs and scripts now use '-DskipTests' and '-Dcheckstyle.skip=true' for consistency. Added a detailed README for the release validation script. * refactor: improve validation script error context and reporting V4 Adds contextual error and warning reporting with step and package information, enhances license category and header checks, improves version consistency logic, and refines summary output with execution time and clearer formatting. These changes make validation results more actionable and easier to interpret, especially for multi-package and multi-step validations. * fix: add JSON to CATEGORY_X license validation The JSON license was added to the CATEGORY_X regex in the binary package validation step to ensure packages with this license are properly flagged during release validation. * introduce new version of validation * Update GPG username and regex in workflow --------- Co-authored-by: imbajin <jin@apache.org>
…#415) * fixed mvn version to 1.7.0 added graphspace part for docs of client changed client examples to NEWER version fixed parameters in loader docs
[pull] master from apache:master
[pull] master from apache:master
dosubot
Bot
added
size:L
docs: add Apache SeaTunnel connector integration guide (CN only) Add HugeGraph-SeaTunnel Connector documentation as an ecosystem entry point for users who want to sync data between HugeGraph and external systems via Apache SeaTunnel. Changes: - Add CN SeaTunnel connector doc in quickstart/toolchain/ - Add SeaTunnel to toolchain table and architecture diagram in CN introduction page - Follow existing Spark-connector documentation style and structure Co-Authored-By: Claude <noreply@anthropic.com> @
cf93179 to
4ae78ff
Compare
dosubot
Bot
added
size:M
imbajin
left a comment
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: The PR adds an unrelated release workflow that uses obsolete artifact rules, and its SeaTunnel guide contains unsupported claims and non-runnable configuration. Evidence: actionlint /tmp/validate-release-new-464.yml; exact-head patch; and comparison with the parent release workflow.
imbajin
left a comment
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: The new release workflow still fails valid artifacts or validates them incompletely, and the SeaTunnel documentation has unsupported compatibility and reference claims. Evidence: exact-head workflow/source review, actionlint, macOS grep reproduction, and current Apache documentation checks at 4ae78ff.
docs: add SeaTunnel integration architecture diagrams Add GPT-generated illustrations for the SeaTunnel Connector documentation: - Architecture diagram: HugeGraph + SeaTunnel data integration overview - Source/Sink diagram: bidirectional data flow between HugeGraph and external systems - Full-stack ecosystem diagram: complete HugeGraph toolchain visualization Placed in connector doc and introduction page for visual enhancement. Co-Authored-By: Claude <noreply@anthropic.com> @
docs: fix PR review issues for SeaTunnel connector doc Address reviewer feedback: - B1: Fix SeaTunnel installation link (/docs/start-v2/ -> /docs/getting-started/) - B2: Replace fake config params (url/graph/label) with real connector parameters (host/port/graph_name/mappings) from SeaTunnel docs - C: Add version compatibility matrix, note Source connector is dev-only - D: Add English docs synchronization (connector doc + introduction page) - E: Fix ASCII diagram border alignment (all lines now 70 chars wide) Co-Authored-By: Claude <noreply@anthropic.com> @
dosubot
Bot
added
size:L
imbajin
left a comment
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: The Chinese introduction page contains an unmatched fenced code block, so the rest of the page can render as code and its navigation/content is broken. Evidence: exact-head source has five triple-backtick fences with the extra opener at line 41; the latest deploy check is green but does not catch this Markdown structure.
fix: correct version matrix and connector doc link - Java: 8+ -> 11+ (HugeGraph Client 1.5.0+ requires Java 11) - SeaTunnel: 2.3.12+ -> 2.3.13+ (bundles HugeGraph Client 1.5.0) - HugeGraph Server: 1.0.0+ -> 1.5.0+ (match client version in connector) - Link: generic /docs/connector-v2/ -> specific HugeGraph Sink page Co-Authored-By: Claude <noreply@anthropic.com> @
- add a single-node import walkthrough with verification - document HugeGraph Source reads and graph clone flows - align examples and limits with SeaTunnel dev - remove unrelated workflow and obsolete assets
imbajin
changed the title
imbajin
left a comment
imbajin
left a comment
There was a problem hiding this comment.
Final review on a136574: +1. The final diff is scoped to the Chinese SeaTunnel guide and its diagram, aligns the Source/Sink examples with current Next/dev behavior, documents the cross-graph ID constraints, and passes the Hugo build. Independent review score: 9.3/10, with no Critical or Important findings.
- add Loader vs SeaTunnel selection guide for new users - add Kafka / Flink+Spark engines / graph migration scenarios with collapsible sections - add mermaid and ASCII diagrams per scenario - align config options with connector-hugegraph source - point SeaTunnel doc links to latest pages (Source doc links to dev branch file)
- move version matrix to top and deduplicate - add Docker-first deployment guide (K8s link, binary as reference) - replace FakeSource examples with LocalFile CSV - collapse engine section to doc links - add graph_space to all examples - add multi-graph batch migration example - drop license section, unnumbered references at end
imbajin
left a comment
imbajin
left a comment
There was a problem hiding this comment.
Blocking: yes. Summary: The quick-start mixes the released 2.3.13 distribution with dev-only HugeGraph connector APIs and contains several non-runnable Docker, Kafka, and migration examples. Evidence: the 2.3.13 HugeGraphSinkFactory requires schema_config; official SeaTunnel Docker/Source/Kafka documentation and exact-head source checks confirm the remaining incompatibilities.
| # username = "admin" | ||
| # password = "admin" | ||
| # schema_save_mode = CREATE_SCHEMA_WHEN_NOT_EXIST | ||
| mappings = [ |
There was a problem hiding this comment.
mappings API, but the 2.3.13 tag's HugeGraphSinkOptions exposes only schema_config, and HugeGraphSinkFactory requires it. mappings and schema_save_mode are from the later dev implementation, so the released setup fails configuration validation. Scope these examples to an exact dev/SNAPSHOT build, or rewrite them for the released schema_config API and update the version matrix.
| **Docker（推荐）**。拉取镜像，把作业配置所在目录挂载进容器提交，宿主机不用装 Java 环境： | ||
|
|
||
| ```bash | ||
| docker pull apache/seatunnel:2.3.13 |
There was a problem hiding this comment.
apache/seatunnel:2.3.13 image cannot run the shown job as-is: SeaTunnel's official Docker documentation says this image uses openjdk:8 and includes only the fake/console connectors, while this guide requires Java 11+ and uses LocalFile/HugeGraph. The later binary example installs plugins, but this Docker path does not. Add a tested JDK 11+ image plus matching connector installation, or provide a custom image containing these plugins.
|
|
||
| sink { | ||
| HugeGraph { | ||
| host = "127.0.0.1" |
There was a problem hiding this comment.
127.0.0.1:8080 points to that container rather than the host's HugeGraph Server or another HugeGraph container. The same endpoint pattern is used in the edge example. Use a shared Docker network/service name or document host.docker.internal (and the Linux host-gateway equivalent) so the quick-start actually connects.
| │ HugeGraph 顶点 │ | ||
| │ id = person:marko │ | ||
| │ label = person │ | ||
| │ props = { age: 29 } │ |
There was a problem hiding this comment.
name from props, although the mapping explicitly includes properties = ["name", "age"] and the dev VertexMapper always retains PRIMARY_KEY idFields as properties. Show name in the resulting properties (or change the mapping) so the example matches the connector's actual output.
| properties = { since: 2020 } | ||
| ``` | ||
|
|
||
| > 边的端点 ID 策略从服务端已有的 VertexLabel 读取，`sourceConfig.idFields` / `targetConfig.idFields` 必须能拼出端点 ID。默认 `check_vertex = false`，顶点和边允许乱序写入，跑完图最终一致；设成 `true` 则服务端直接拒绝端点不存在的边。 |
There was a problem hiding this comment.
check_vertex = false does not provide the stated eventual-consistency guarantee. The official Sink documentation says missing endpoints may remain orphan edges or trigger phantom-vertex creation. Replace "最终一致" with the actual behavior and advise loading vertices first or enabling check_vertex = true when orphan edges are unacceptable.
| source { | ||
| Kafka { | ||
| bootstrap.servers = "localhost:9092" | ||
| topics = "user-events" |
There was a problem hiding this comment.
topic (a comma-separated list is supported), not topics. With this block, the required topic option is missing and the example will not start. Rename the key to topic and validate it against the target SeaTunnel version.
|
|
||
| source { | ||
| HugeGraph { | ||
| host = "graph-a:8080" |
There was a problem hiding this comment.
host = "graph-a:8080" is combined with port = 8080, while the connector builds protocol://host:port; the resulting address is http://graph-a:8080:8080. The same defect appears for graph-b. Set host to graph-a/graph-b and keep the port separate.
| graph_name = "${graph}" | ||
| graph_space = "DEFAULT" | ||
| # mappings 按目标图的 label 逐一列出，源、目标同名 label 一一对应 | ||
| mappings = [...] |
There was a problem hiding this comment.
mappings = [...] is only a placeholder and provides no per-table source_table binding. A mapping without that binding is activated in every writer, so simply replacing ... with multiple mappings can cross-apply labels. Provide complete per-label mappings with source_table, or mark this as pseudocode and remove the directly runnable Docker command.
| ```bash | ||
| # 每个图跑两次：先顶点（VERTEX）、再边（EDGE），图名不变 | ||
| for g in graph1 graph2 graph3; do | ||
| docker run --rm -it -v /path/to/job:/config apache/seatunnel:2.3.13 \ |
There was a problem hiding this comment.
apache/seatunnel:2.3.13. In addition, the config fixes label_type = "VERTEX" and the loop invokes only one job per graph despite saying to run twice, so edges are never migrated. Use an exact dev build containing Source and parameterize/execute both VERTEX and EDGE jobs.
| ./bin/seatunnel.sh -e local -c /config/hugegraph-sync.conf | ||
| ``` | ||
|
|
||
| 更多用法见官方 [Docker 部署文档](https://seatunnel.apache.org/docs/getting-started/docker/docker/)。 |
There was a problem hiding this comment.
🧹 minor — This Docker deployment link currently returns 404 (/docs/getting-started/docker/docker/). Link to the valid official page at /docs/getting-started/docker/.
- use schema_config (2.3.13 API) in quick-start and Kafka examples, scope mappings/Source to dev preview - fix Docker path: note JDK8 image and missing plugins, add JDK11 Dockerfile sample, correct doc link - fix container networking guidance (host.docker.internal / docker network) - fix Kafka option topics -> topic - fix host:port duplication in migration examples - correct check_vertex description and diagram output props - mark batch migration as pseudocode with per-table binding note
- clarify Tools, Loader, and SeaTunnel selection\n- align JDBC, Kafka, and dev migration examples\n- add concise diagrams and fold long configs\n- replace stale architecture asset and version links
imbajin
left a comment
imbajin
left a comment
There was a problem hiding this comment.
Blocking: no. Summary: The graph2graph prerequisite wording conflicts with the exact dev connector contract and should be corrected before users follow the migration steps. Evidence: the linked f1a1a0a SeaTunnel HugeGraph Sink documentation and the exact-head guide at line 259.
|
|
||
|  | ||
|
|
||
| `mappings` 默认会创建缺失的 Schema。边映射的源和目标顶点标签仍需存在，因此要按下面的顺序先跑顶点任务，再跑边任务；如果把 `schema_save_mode` 改成 `ERROR_WHEN_SCHEMA_NOT_EXIST`，请提前创建目标图 Schema。 |
There was a problem hiding this comment.
Blocking: no. The statement that the edge mapping's source and target labels still need to exist conflicts with the exact f1a1a0a HugeGraph Sink contract linked above: mappings defaults to CREATE_SCHEMA_WHEN_NOT_EXIST and creates missing VertexLabel, EdgeLabel, and PropertyKey elements before writing. This makes the graph2graph prerequisite misleading. Please remove that precondition, or set schema_save_mode = "ERROR_WHEN_SCHEMA_NOT_EXIST" and explain why pre-creating the schema is required.
6 participants
目的
把 HugeGraph-SeaTunnel 中文快速开始整理成一条普通开发者可以直接阅读的路径，并明确发布版与 dev 预览版的配置边界。
Before → After
sql2graph、Kafkakafka2graph，以及固定 dev commit 的graph2graph。schema_config，HugeGraph Source 和mappings明确标为 dev 预览。Visual preview
Before
After
验证
hugo --minify通过。git diff --check通过。<details>配对、四张图片引用和固定 upstream 链接检查通过。本轮没有安装或编译 SeaTunnel，也没有启动数据库、Kafka 或其他服务。文档中的运行命令和配置经过 upstream 文档与源码核对，但没有宣称运行时 smoke test 结果。
主要依据