Skip to content

fix(converter): cap doc_name length to avoid Windows path-length errors #252

Description

@sebastianbraun25

Problem

openkb add can fail with a Windows-specific OSError [WinError 206] ("The filename or extension is too long") when ingesting a source file whose name is long. The sanitized wiki name (doc_name, derived from the original filename's stem) is used, unmodified and unbounded in length, as a directory/file name component in several paths — notably twice within the per-add staging tree (.openkb/staging/add-{doc_name}-{uuid8}/wiki/sources/images/{doc_name}/). Combined with an already-nested vault path, the total path length exceeds Windows' ~260-char limit.

Reproduction

  • OS: Windows, fork integration / upstream main as of 2026-09.
  • Command: openkb add <file> inside a moderately nested KB directory (e.g. ...\vaults\<kb-name>\).
  • File: a source document whose filename stem is ~70+ characters (e.g. a Salesforce bulk-query export like bulkQuery_result_7501o00000T3gpIAAR_7511o00000cwd9eAAA_7521o00000DfL77.csv).
  • Observed:
    [ERROR] Conversion failed: [WinError 206] Der Dateiname oder die Erweiterung ist zu lang: 'C:\...\.openkb\staging\add-bulkQuery_result_7501o00000T3gpIAAR_7511o00000cwd9eAAA_7521o00000DfL77-3e00e804\wiki\sources\images\bulkQuery_result_7501o00000T3gpIAAR_7511o00000cwd9eAAA_7521o00000DfL77'
    
  • Expected: ingest succeeds regardless of original filename length; long filenames are handled the same way _sanitize_filename in openkb/url_ingest.py already caps URL-derived filenames (_MAX_FILENAME_STEM = 80).

Kontext

  • Root cause: _sanitize_stem() in openkb/converter.py only replaces disallowed characters, it never bounds length. doc_name (the return value, also used for collision-suffixing via resolve_doc_name/resolve_doc_name_from_key) is then used unmodified in cli.py::_staging_dir_for (staging directory name) and converter.py::convert_document (wiki/sources/images/{doc_name}/), so an overlong original filename appears twice in the same path.
  • openkb/url_ingest.py already caps stems at 80 chars in _sanitize_filename (_MAX_FILENAME_STEM) for URL-derived downloads, but that cap isn't shared with the local-file ingest path (_sanitize_stem).

Vorschlag

Cap _sanitize_stem()'s output length (e.g. 40 chars, conservative because doc_name can appear twice in the staging path) and append a short deterministic hash suffix when truncated, so uniqueness/re-ingest stability is preserved (unlike url_ingest.py's counter-based _unique_path, converter.py's existing collision mechanism is registry-based and needs a deterministic, non-incremental suffix).


This issue was drafted with the assistance of an AI assistant.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions