Skip to content

feat(drivers): add Teldrive V2 driver - #3000

Open
totza2010 wants to merge 1 commit into
OpenListTeam:mainfrom
totza2010:feat/teldrive-v2
Open

feat(drivers): add Teldrive V2 driver#3000
totza2010 wants to merge 1 commit into
OpenListTeam:mainfrom
totza2010:feat/teldrive-v2

Conversation

@totza2010

@totza2010 totza2010 commented Aug 30, 2026

Copy link
Copy Markdown
Contributor

Summary / 摘要

Adds a driver for TelDrive v2.

TelDrive v2 is a full rewrite of the server's HTTP API, not a version bump. It is
not backward compatible with the v1 API that the existing drivers/teldrive
speaks, so the two cannot be served by one client:

v1 (drivers/teldrive) v2 (this PR)
Auth access_token cookie X-Api-Key header
Addressing path strings UUIDs (parentId)
Listing offset + meta.totalPages opaque cursor (nextCursor)
Mutations plain requests Idempotency-Key header required
Upload per-chunk POST durable session: create → PUT parts → complete
Overwrite client-side check server-side conflictPolicy
Integrity none BLAKE3 tree hash, 16 MiB blocks

Because listing, addressing, auth and upload all differ, a shared client would be
two implementations behind one set of if v2 {} branches. This PR adds a
separate package instead and does not touch drivers/teldrive — existing v1
storages keep working untouched.

User-visible behaviour

  • New storage driver Teldrive V2 in the driver dropdown.
  • Uploads are resumable: an interrupted upload re-attaches to its existing server
    session and only sends the parts that are missing.
  • Optional per-part BLAKE3 checksum so the server rejects corruption in transit.
  • Optional direct 302 download links via TelDrive share tokens, cached and
    auto-renewed before expiry.

Implementation notes

  • pkg/utils/hash/tdhash.go registers TelDrive's BLAKE3 tree hash
    (blake3_tree) with the existing hash registry, following the gcid
    precedent. Verified byte-for-byte against a live TelDrive server and against
    rclone's backend/teldrive/tdhash.
  • Part uploads use a dedicated resty client. base.RestyClient has a 30s
    timeout and 3 built-in retries, both wrong for a part upload — the server only
    responds once the part has been relayed to Telegram, and an automatic retry
    would replay a body behind the driver's own retry policy.
  • Progress is reported per completed part. Byte-level progress is not achievable
    here: instrumentation showed the request body drains in ~0.3s of a ~26s
    request, so the remaining time is the server relaying to Telegram.
  • PreferProxy: true — without a share token the download URL is authenticated
    by a header, which a browser following a 302 cannot send.

Dependencies

go.mod changes by one line. github.com/zeebo/blake3 v0.2.4 was already in
the tree, pulled in by rclone/rclone v1.75.0, and is only moving out of the
// indirect block because pkg/utils/hash/tdhash.go now imports it directly —
this is what go mod tidy produces. go.sum is unchanged, which is the
proof that no new module enters the build.

  • This PR has breaking changes.
  • This PR changes public API, config, storage format, or migration behavior.
  • This PR requires corresponding changes in related repositories.

Related repository PRs / 关联仓库 PR:

Related Issues / 关联 Issue

Relates to #2034 (the v1 driver).

Testing / 测试

Tested against a real TelDrive v2 server (ghcr.io/tgdrive/teldrive:v2, commit
e3142b5) backed by a real Telegram account, driven through the OpenList web UI.

Exercised end to end: list, mkdir, rename, move, copy, remove (trash and hard
delete), download through the proxy, download via 302 share link, multi-part
upload, upload resume after an interrupted transfer, and conflict handling.

Hash agreement checked three ways — this driver, the TelDrive server, and
rclone 1.75.1's teldrive hash — on the same 36 MiB file, whole-file and
per-part. The vectors are pinned in pkg/utils/hash/tdhash_test.go.

  • go test ./...
  • Manual test / 手动测试: Windows 11, Docker Desktop, TelDrive v2 + Telegram
go build ./...
go vet ./drivers/teldrive_v2/
go test ./drivers/teldrive_v2/ ./pkg/utils/hash/

One note on go test ./...: it is currently red on main as well, before this
change. Go 1.27's vet runs a stricter printf check, and ten pre-existing
packages (drivers/123, drivers/189, drivers/189pc, drivers/chaoxing,
drivers/google_drive, drivers/google_photo, drivers/lanzou, and three
under internal/offline_download/) fail it with "non-constant format string".
I reproduced this on a clean upstream/main checkout — it is unrelated to this
PR, and none of the packages this PR adds or touches are affected. Happy to open
a separate PR for those if that would be useful.

Unit tests cover the pure logic that does not need a server: path normalisation,
chunk-size rounding, share-expiry parsing, retry classification, idempotency-key
stability, error formatting, and the BLAKE3 tree hash vectors.

Checklist / 检查清单

  • I have read CONTRIBUTING.
  • I confirm this contribution follows the repository license, contribution policy, and code of conduct.
  • I have formatted the changed code with gofmt, go fmt, or prettier where applicable.
  • I have requested review from relevant maintainers or code owners where applicable.

AI Disclosure / AI 使用声明

  • This PR includes AI-assisted content.

Tools used / 使用工具:

  • Claude

Usage scope / 使用范围:

  • Code generation / 代码生成

  • Refactoring / 重构

  • Documentation / 文档

  • Tests / 测试

  • Translation / 翻译

  • Review assistance / 审查辅助

  • I have reviewed and validated all AI-assisted content included in this PR.

  • I have ensured that all AI-assisted commits include Co-Authored-By attribution.

  • I can reproduce all AI-assisted content included in this PR without any AI tools.

I used Claude Code as an assistant while writing this driver: mapping the v2
OpenAPI surface onto OpenList's driver interfaces, drafting and refactoring the
implementation, and writing the tests. Every design decision - separate package
over a version flag, part-level progress, the share-token cache, which optional
interfaces to implement - was mine, and I validated the result against a live
TelDrive v2 server and a real Telegram account rather than against the spec
alone. The commit carries a Co-Authored-By trailer accordingly.

@pikachuren pikachuren left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🙏 感谢 @kkbdvl 提交!
🤖 AI 自动审核声明:本评审报告由 AI 自动生成,当前使用 Claude Opus 5 模型进行分析。
⚠️ AI 分析结果仅供参考,可能存在误判或遗漏。如您发现任何问题或有不同意见,欢迎随时提出讨论和纠正。
⚠️ 重要提醒:即使 AI 评审认为代码质量良好且建议合并,最终是否合并仍需由项目维护者进行人工判定。项目维护者会综合考虑代码质量、项目规划、技术方向、团队资源等多方面因素做出决策。

🎯 结论

✅ 建议 Approve(由维护者人工确认)— 代码注释解释了「为什么」而非「做了什么」,是本轮质量最高的新驱动之一

📖 概要

feat(drivers): add Teldrive V2 driver · 新增 Teldrive V2 驱动。
核心改动:新增 drivers/teldrive_v2 包(含单元测试),实现 tree-hash 算法,鉴权从 v1 的 cookie 改为 X-Api-Key 头。

🧭 整体方案

技术路线是针对 Teldrive v2 API 的完整适配。这个 PR 给我印象最深的是注释质量——几处关键设计都解释清楚了背后的约束,而不是复述代码:

  • shareToken 上方的注释说明了为什么需要缓存:Teldrive 的 GET /files/{id}/shares 只返回 share 的 id、过期时间和下载次数,唯独不返回 token,token 只在创建时返回一次。因此复用 share 就必须自己记住 token。这是外部 API 的非显然约束,注释点明后,shareCache 的存在就完全合理了。
  • driver_test.go 中注释说明「teldrive 把多个文档化状态都collapse 到 409」,解释了为何要对整个 4xx 做统一处理。
  • treeHashBlockMiB 注明了是 teldrive 内部 internal/treehash 的固定块大小。

这类注释正是代码里最该写的东西。此外驱动附带了 driver_test.gotdhash_test.go 两个测试文件,在新驱动 PR 中很少见。

📊 变更统计

10 个文件(+1439 / -1 行) | 功能 ⭐⭐⭐⭐⭐ | 最小改动 ⭐⭐⭐⭐ | 前向兼容 ⭐⭐⭐⭐⭐ | 方案设计 ⭐⭐⭐⭐⭐

🚨 关键问题

P0(阻塞合并):无

P1(建议修复):无

P2(可选)

  • 💡 shareCache 使用 sync.Map 缓存 file UUID → share token,但看起来没有容量上限或淘汰机制(仅按 expiresAt 判断有效性)。长期运行且访问大量不同文件的实例,这个 map 会持续增长。是否考虑加一个定期清理过期项的协程,或改用带上限的 LRU 呢?
  • 💡 创建 share 是一个有副作用的写操作(在 Teldrive 侧生成公开分享链接)。请问驱动被移除或存储被删除时,这些已创建的 share 会被清理吗?如果不会,可能在用户的 Teldrive 上遗留大量公开链接,建议在文档中说明~
  • 💡 分享 token 会拼进 URL 并返回给客户端,意味着持有该 URL 者无需鉴权即可下载。这是 share 机制的固有语义、也是实现直链所必需,但建议在 driver 的 help 文案中提示用户注意~
  • 💡 go.mod 有 1 行改动,看起来是依赖版本微调。建议在 PR 描述中说明改动原因~
  • 💡 pkg/utils/hash/tdhash.go 把 tree-hash 放在通用 hash 包下。若该算法仅 Teldrive 使用,是否考虑放在驱动包内以减少通用包的表面积?当然如果预期会被复用,放这里也合理~

🔐 安全审查

  • 审查方式:扫描命令执行、TLS 绕过、硬编码凭证、可疑外连
  • 安全评估:✅ 未发现恶意代码
  • 详细结论:无 exec.Command、无 InsecureSkipVerify、无硬编码密钥;ApiKey 由用户配置并通过 X-Api-Key 请求头传递(优于放在 URL 中);请求目标为用户自建的 Teldrive 实例地址,Init 中对 urlapi_key 均做了必填校验。

📂 逐文件分析

drivers/teldrive_v2/driver.go / util.go / upload.go

改动意图:实现驱动主体、工具函数与上传。
代码逻辑shareToken 优先查缓存,未命中或已过期才创建新 share 并记录 token;Link 用 token 拼接直链。
问题分析:逻辑正确,对「token 不可事后取回」这一约束的处理是恰当的;缓存增长与 share 清理可优化(P2)。

drivers/teldrive_v2/driver_test.go / pkg/utils/hash/tdhash_test.go

问题分析:新驱动自带测试,覆盖 hash 算法与错误状态处理,加分项。

drivers/teldrive_v2/meta.go

问题分析ApiKey 的 help 写明了获取路径(Settings → API keys),对用户友好。

✅ 待处理清单

  • [P2] 为 shareCache 增加过期清理或容量上限
  • [P2] 说明驱动移除时已创建 share 的清理策略
  • [P2] 在 help 中提示分享链接的免鉴权特性
  • [P2] 说明 go.mod 改动原因

🎯 结论:✅ 建议 Approve — 实现完整、鉴权方式得当、自带测试,注释质量尤其突出,无阻塞项。

TelDrive v2 rewrites its HTTP API: every path moves under /api/v1, the
listing is cursor-paginated instead of page-numbered, objects are addressed
by UUID rather than path, uploads go through durable server-side sessions,
and every mutating endpoint requires an Idempotency-Key. None of it is
reachable from the existing Teldrive driver, so this adds a separate one
and leaves the v1 driver untouched.

Notable differences from the v1 driver:

- Copy is a single request; the server copies a whole subtree
  transactionally, so no client-side recursion is needed.
- Uploads create a session, PUT each part, then complete. Parts are
  idempotent on (uploadId, partNo), and an interrupted upload resumes by
  reusing its session and skipping parts the server already stored.
- The client mtime is preserved, which the v1 driver could not do.
- Files carry a BLAKE3 tree hash. It is registered as blake3_tree in
  pkg/utils/hash and reported on every object; when hash_enabled is set,
  each part is also sent with a checksum for the server to verify.

Part uploads use a dedicated resty client: the shared one caps requests at
30s, which a part cannot meet because the server only responds once it has
relayed the part to Telegram, and it would buffer the whole part in memory
to compute Content-Length.

Requires a TelDrive v2 server at e3142b5 or newer, where the move endpoint
began accepting a conflictPolicy other than "fail".

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@totza2010

Copy link
Copy Markdown
Contributor Author

Thanks for the detailed review. I've pushed changes for two of the five P2 items and would like to explain my reasoning on the other three.

shareCache growth — fixed

You're right, and this was inconsistent of me: I had already dropped an earlier pathCache-into-toObj write for exactly this reason and left shareCache unbounded.

I looked at internal/cache.KeyedCache first, since it has both a TTL and an hourly GC cron. It turns out not to be usable from a driver instance: NewKeyedCache appends its GC method to the package-level gcFuncs slice, unsynchronised and never removed. Every call site in the repo today is a package-level singleton created once (internal/op/cache.go, internal/op/archive.go), so that's fine there — but calling it per storage in Init would grow gcFuncs on every re-init and race two storages initialising concurrently. WeakCacheMap doesn't fit either: nothing else holds a reference to a share token, so it would be collected immediately and every Link() would mint a new share.

So I've added an explicit sweep instead. It runs on the mint path — already an HTTP round trip — and no more than once per shareRenewMargin, so a busy storage doesn't pay the walk on every miss:

func (d *TeldriveV2) sweepShares(now time.Time) { ... }

Since every entry expires within share_expire, the map is now bounded by the number of distinct files linked within one expiry window rather than by everything ever linked. Covered by TestSweepShares, which also pins the rate limit.

✅ Share cleanup and unauthenticated links — documented

These two are really the same disclosure, so I've folded them into the use_share_link help text (and mirrored it into the frontend PR):

Each link carries a TelDrive share token, so anyone holding the URL can download that file without signing in, and OpenList can no longer count or revoke those accesses. Shares are left on your TelDrive until they expire, including after this storage is removed.

On revoking in Drop(): I did consider it — POST /files/{id}/shares returns the share id, so it's technically possible. I decided against it deliberately. Drop() isn't guaranteed to run (a killed process never calls it), so a cleanup there would read as a guarantee it can't make. Shares do expire on their own within share_expire, which is capped at 168h, so nothing accumulates indefinitely. Saying so plainly in the help text seemed more honest than code that usually works. Happy to add revocation as well if you'd prefer belt and braces.

go.mod — already covered in the description

This one is described under Dependencies in the PR body. In short: github.com/zeebo/blake3 v0.2.4 was already in the tree as an indirect dependency of rclone/rclone v1.75.0. It moves out of the // indirect block only because pkg/utils/hash/tdhash.go now imports it directly — that's what go mod tidy produces, same version. go.sum is unchanged, which is the proof that no new module enters the build.

tdhash.go placement — I'd prefer to keep it in pkg/utils/hash

Two reasons. There's precedent: pkg/utils/hash/gcid.go is PikPak-specific and lives there. More importantly the hash has to be registered in the global registry to be usable at all — utils.RegisterHash is what lets a model.Obj carry the value and lets the rest of OpenList display and compare it. Moving the type into the driver package would still require registering it centrally, so it would split the algorithm from its registration without shrinking the surface.


Also rebased onto current main (picking up the three security fixes and the dependency bumps) — go.mod merged cleanly and go.sum is still untouched.

One thing I can't do myself: Test Build and Beta Release (Docker) are sitting at action_required since this is a fork PR. They need a maintainer to approve the workflow run.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants