A self-hosted document ingestion and structured parsing service.
StructaDoc turns uploaded documents into stable, structured data. It combines a user-facing Vue workspace, a controlled administration area, a versioned HTTP API, durable background processing, and local or S3-compatible storage in one self-hostable application.
Users can upload and manage PDF, Word, Excel, and PowerPoint files; start parsing; read the result as rendered Markdown, as Blocks in reading order, or as a page-layout map of where each Block sits; download assets and artifacts; share documents; and export Markdown, HTML, ZIP, or PDF. Administrators additionally manage parsing Providers and API clients.
MinerU Cloud and self-hosted MinerU are adapters behind a Provider-neutral boundary. Consumers never need to understand Provider task protocols, output layouts, or version-specific raw JSON.
Pull the published image and start it with the example script:
docker pull ghcr.io/philfanzhou/structadoc:latest
git clone https://github.com/philfanzhou/StructaDoc.git
cd StructaDoc
sudo ./scripts/start-container.shThen open http://localhost:8080. The first visitor is sent to /setup to create the administrator. The official MinerU endpoint is already configured as the default Provider, with no token: parsing becomes available once that administrator enters an API token for it under /admin.
scripts/start-container.sh is an example for Linux and WSL, and running it again is the normal way to apply a new image: it stops and removes an existing structadoc container before starting the current one. Documents, results, and the administrator live in ./data on the host, which is what sudo is for — that directory has to belong to the image's non-root user. Change the image tag, the published port, the memory limit, or the data path by editing the variables at the top of the script.
StructaDoc is in early development, but the main end-to-end platform is implemented:
- Vue 3 user workspace and administration area;
- generic OpenID Connect authentication using
(issuer, subject)as the stable external identity; - optional local break-glass administrator and separate scoped API-client credentials;
- document ownership, explicit sharing, and resource-level authorization;
- Local and S3-compatible storage;
- SQLite, PostgreSQL, MySQL, and MariaDB persistence;
- immutable Provider configuration versions and durable Parse Runs;
- atomic claims, leases, heartbeats, retries, cancellation, and crash recovery;
- MinerU Cloud signed upload and MinerU Local 3.x multipart / opt-in 4.x task adapters;
- constrained LibreOffice Office-to-PDF fallback;
- resumable large-PDF segmentation and deterministic result merging;
- bounded Provider ZIP intake and deterministic canonical normalization;
- stable result DTOs, authorized downloads, rendered Markdown, Block and layout views, and exports;
- persistent cleanup jobs that complete object deletion before removing relational data.
The production Docker image, PostgreSQL/MySQL/MariaDB contract suites, and Chromium workspace smoke test are exercised by GitHub Actions. A Worker sends documents to a Provider once an administrator has configured and enabled one under /admin; a deployment with no Provider makes no outbound requests, because a Parse Run cannot be created without one.
See the documentation index for architecture decisions, specifications, implementation notes, and deployment guidance.
StructaDoc owns the reliable conversion from files to normalized document data:
- ingest and retain original documents;
- create and execute durable asynchronous Parse Runs;
- integrate external or local parsing Providers;
- normalize Provider output into Documents, Pages, Blocks, Assets, and Artifacts;
- retain parsing history and raw results for traceability;
- expose versioned APIs and a user-facing workspace.
Alongside that lifecycle, ADR-0013 records one explicit exception: a non-archiving image layout detection mode, exposed as POST /api/v1/layout/detect, in which an authenticated API client holding layout:detect submits a single JPEG or PNG and receives Provider-neutral normalized layout regions in the same response, while StructaDoc creates no Document or Parse Run and stores no request image or result. The mode detects page layout only — it returns no recognized text, table HTML, formula LaTeX, or assets. It needs an administrator-configured layout runtime to answer anything; StructaDoc maintains an optional one — a separate Python worker in runtime/layout-worker/ running the MinerU SDK layout path — deployable beside the main image or replaceable by any service speaking the same protocol (Layout Runtime).
StructaDoc intentionally does not include:
- full-text search or OpenSearch;
- vector search, embeddings, or RAG pipelines;
- LLM-generated metadata or domain entities such as questions, vocabulary, invoice records, or contract records;
- online editing for Office documents;
- direct consumer access to its database or object storage;
- raw Provider JSON as a stable public contract.
Consumers may build search, vectorization, knowledge bases, or domain extraction on top of StructaDoc's versioned output.
Install the .NET 10 SDK and Node.js 24, then run:
dotnet restore StructaDoc.slnx
dotnet tool restore
cd web
npm ci
npm run build
cd ..
dotnet build StructaDoc.slnx --no-restore
dotnet test StructaDoc.slnx --no-build --no-restore
dotnet run --project src/StructaDoc.HostThe Host serves the compiled web application and API on one address:
/— the document workspace for signed-in users;/setup— first-run administrator creation, reachable only while no administrator exists;/admin— the administration area, restricted to administrators;/api/v1/...— the service API.
Useful unauthenticated endpoints are:
GET /api/v1/openapi.jsonand/api/v1/docs— the consumer API description and a page that browses it;GET /api/v1/system/info— service identity and version;GET /health/live— process liveness, always200while the process runs;GET /health/ready(aliasGET /health) — control-plane and business-database readiness, as a JSON body; only the status code is the deployment contract.
SQLite creates its database file automatically. PostgreSQL, MySQL, and MariaDB require a
pre-created target database: startup and --migrate-business-database fail safely when the target
is missing, unreachable, or rejects credentials. Create the target through your deployment's
database administration tools, then rerun the command or restart the service. See
Database Support for recovery
and upgrade requirements.
For CI coverage and local reproduction, see Continuous Integration.
The root Dockerfile builds the Vue application and .NET Host, then creates one non-root runtime image containing ASP.NET Core, LibreOffice no-GUI components, and common fonts. The image is the whole deployment unit: build it, then run one container with /data mounted.
docker build --tag structadoc:local .
docker run --detach --name structadoc \
--publish 8080:8080 \
--volume /srv/structadoc/data:/data \
--env Authentication__BootstrapAdministratorUsername='structadoc-admin' \
--env Authentication__BootstrapAdministratorPassword='use-a-secret-manager-or-a-long-random-value' \
structadoc:localPowerShell:
docker build --tag structadoc:local .
docker run --detach --name structadoc `
--publish 8080:8080 `
--volume D:\StructaDoc\data:/data `
--env Authentication__BootstrapAdministratorUsername='structadoc-admin' `
--env Authentication__BootstrapAdministratorPassword='use-a-secret-manager-or-a-long-random-value' `
structadoc:localTo run the published image instead of a locally built one, use scripts/start-container.sh from Quick Start. The default address is http://localhost:8080, with the workspace at / and the administration area at /admin. The bootstrap variables are optional: omit them and the first visitor is sent to /setup to create the administrator. The example values show the required shape and are not default credentials. Inject production secrets through the deployment platform and remove bootstrap credentials after the first administrator exists.
For restricted networks, the repository includes explicit official, china, and connectivity-based auto build modes:
bash ./scripts/build-container.sh autopowershell -NoProfile -ExecutionPolicy Bypass -File ./scripts/build-container.ps1 -MirrorMode AutoReview Single-Container Deployment for image contents, mirror trust, non-root permissions, persistent data, backup requirements, and runtime limits.
flowchart LR
User["Interactive user"] --> Host
Admin["Administrator"] --> Host
Client["API client"] --> Host
subgraph Image["StructaDoc application image"]
Host["ASP.NET Core Host\nVue workspace + API"]
Worker["Durable Parse Worker"]
Converter["LibreOffice conversion"]
Normalizer["Result normalization"]
Cleanup["Durable cleanup Worker"]
Host --> Worker
Worker -. "when the Provider needs PDF" .-> Converter
Worker --> Normalizer
Host --> Cleanup
end
Host --> Database["SQLite / PostgreSQL / MySQL / MariaDB\nmetadata, structured content, durable jobs"]
Host --> Storage["Local or S3-compatible storage\noriginals and generated resources"]
Worker --> Cloud["MinerU Cloud"]
Worker --> Local["Self-hosted MinerU"]
Normalizer --> Database
Normalizer --> Storage
Cleanup --> Storage
The Worker uses the configured relational database as the authoritative queue. It does not rely on an in-process queue, so work can resume from stored leases and checkpoints after a restart. SQLite supports one StructaDoc instance; server databases support multiple competing Worker instances. The administration control plane stays per-instance in every form — administrator accounts, setup, and browser-stored settings are local to each container and not shared — so a multi-instance deployment pins its shared configuration rather than administering it from one browser session. See Database Support.
Interactive users authenticate through configurable OIDC Authorization Code flow with PKCE. StructaDoc depends on standard discovery, token validation, and claim mapping, not on a Provider-specific SDK. SignaCore, Keycloak, Authentik, Entra ID, and other standards-compliant Providers can be used without changing business code. An administrator can configure the Provider under /admin, with a test that fetches its discovery document, so a deployment never needs a command line to let its users in.
The stable identity key is (issuer, subject). Email and display name are presentation attributes, not authorization keys. OIDC tokens remain in the Host; browser JavaScript receives only an encrypted HttpOnly application session.
The local administrator remains available for bootstrap and break-glass recovery. Machine clients use separate API keys with explicit scopes:
documents:readdocuments:writeparses:readparses:writelayout:detect(non-archiving layout detection only; API clients only — browser sessions do not reach this endpoint)
See User Workspace and OIDC and Authentication.
Provider configuration is administrator-controlled and immutable by version. Each Parse Run snapshots its Provider type, configuration version, model/backend, and non-sensitive options. Changing the default Provider never changes existing work.
When a Provider supports the source format, StructaDoc submits the original. Otherwise, if the source is a supported Office format and the Provider accepts PDF, the constrained LibreOffice adapter creates a separate normalized-pdf Artifact. The original is never overwritten.
Every successful Provider result becomes a canonical Parse Bundle containing:
| Content | Purpose |
|---|---|
| Pages | Provider-neutral page identity and dimensions |
| Blocks | Ordered text, headings, tables, formulas, images, and other content |
| Assets | Extracted images and binary resources |
| Artifacts | Markdown, normalized PDF, Provider archive, layout, and model output |
| Provider metadata | Sanitized Provider, model, and option facts |
Raw ZIP and JSON may be retained as authorized Artifacts, but raw fields never become the public Block contract. See the Canonical Document Model.
The versioned API includes:
POST /api/v1/documents
GET /api/v1/documents
GET /api/v1/documents/{documentId}
GET /api/v1/documents/{documentId}/content
DELETE /api/v1/documents/{documentId}
POST /api/v1/documents/{documentId}/parse-runs
GET /api/v1/documents/{documentId}/parse-runs
GET /api/v1/parse-runs/{parseRunId}
POST /api/v1/parse-runs/{parseRunId}/cancel
GET /api/v1/parse-runs/{parseRunId}/pages
GET /api/v1/parse-runs/{parseRunId}/blocks
GET /api/v1/parse-runs/{parseRunId}/assets
GET /api/v1/parse-runs/{parseRunId}/artifacts
GET /api/v1/parse-runs/{parseRunId}/markdown
GET /api/v1/parse-runs/{parseRunId}/markdown/preview
GET /api/v1/parse-runs/{parseRunId}/exports/{format}
Content is retrieved through authorized endpoints; internal storageRef values never appear in public DTOs. markdown returns the stored Markdown Artifact and markdown/preview returns it rendered as a self-contained page for display. Parse Run creation supports Idempotency-Key. Block listing uses stable sequence pagination. Deletion returns an accepted lifecycle transition and is completed by a durable cleanup job.
Cancellation is best-effort and idempotent: it stops local processing and durably completes as cancelled, which is also how a Document is released for deletion when its Parse Run will never finish on its own. StructaDoc does not yet propagate upstream cancellation, including to MinerU versions that expose a task cancellation route, so submitted work may keep consuming remote resources.
The API-client surface is published as an OpenAPI 3.1 document at GET /api/v1/openapi.json, with stable operation IDs and explicit multipart, idempotency, authentication, pagination, caching, and range contracts suitable for SDK generation. Browser-only administration and session routes are separated into GET /api/v1-browser/openapi.json; /api/v1/docs browses both. See API Description.
Administrator endpoints manage local sessions, API clients, and Provider configurations under /api/v1/admin. Cookie-authenticated writes require an antiforgery token; API-key requests do not use browser cookies. A scope authorizes the endpoint, and ownership or an explicit grant authorizes the resource, so an API client reaches what it uploaded and what was shared with it rather than the whole workspace. See ADR-0008.
Configuration uses standard ASP.NET Core keys; environment variables replace : with __. Most keys are also settable under /admin, where an environment variable, if present, wins and the setting is shown as managed by the deployment — including the Worker lease/heartbeat/retry/polling rhythm, the Provider result intake and normalization limits, the LibreOffice conversion limits, and the administrator session lifetime and login limits, not just the connections. See Service Settings for the full settable list and the short deployment-minimum set that stays with the container (ports, control-plane and key ring paths, reverse-proxy trust, the migration gate, and bootstrap credentials).
| Area | Important keys |
|---|---|
| Database | Database__Provider, Database__ConnectionString, Database__ServerVersion, Database__ApplyMigrationsOnStartup |
| Worker | Worker__Enabled (deployment only), Worker__MaxConcurrency, Worker__MaxExecutionDuration, lease, heartbeat, retry, and polling limits |
| Storage | Storage__Provider, Storage__RootPath, S3 endpoint, bucket, prefix, region, and credential settings |
| Documents | Documents__UploadApiEnabled, Documents__MaxUploadBytes |
| OIDC | Oidc__Enabled, Oidc__Authority, Oidc__ClientId, Oidc__ClientSecret, scopes and role mapping; all but scopes and the callback paths are also settable under /admin |
| Local administration | session lifetime and login limits under Authentication__*, also settable under /admin; bootstrap credentials and the Data Protection key path, persistence form, and root key under Authentication__* are deployment only |
| Conversion | concurrency, timeout, and byte/disk limits under LibreOffice__*; the executable and temporary paths are deployment only |
| Provider results | archive, entry, expansion, compression-ratio, and normalization limits; the temporary path is deployment only |
Connection strings, bootstrap passwords, OIDC secrets, Provider tokens, and storage credentials must come from deployment secrets, never committed configuration.
| Component | Technology |
|---|---|
| Host, API, and Workers | .NET 10 and ASP.NET Core 10 |
| Web workspace | Vue 3, TypeScript, and Vite |
| Persistence | EF Core with SQLite, PostgreSQL, MySQL, and MariaDB migrations |
| File storage | Local filesystem or S3-compatible object storage |
| Office conversion | Constrained LibreOffice headless subprocess |
| Parsing | MinerU Cloud and MinerU Local Provider adapters |
| Deployment | One StructaDoc image; SQLite volume or external server database; optional layout inference worker beside it |
The runtime image does not contain Node.js, the .NET SDK, or Python. Build tools exist only in multi-stage build stages.
StructaDoc is independent of Ruoyu.Study, SignaCore, Consul, and their internal models. It defines its own generic OIDC, storage, parsing, and public API boundaries.
- Detect file type from content and structure; do not trust client MIME types.
- Bound file size, pages, processing time, memory, temporary disk, archive expansion, and conversion concurrency.
- Keep Provider, OIDC, storage, and API credentials out of logs and responses.
- Apply SSRF controls to Provider URLs and signed transfer URLs.
- Make external data transfer through online Providers visible to administrators and users.
- Authorize every document, result, Asset, Artifact, export, share, and deletion operation at resource level.
- Back up the database, object storage, and Data Protection key ring as one recoverable set.
StructaDoc is not an official MinerU project and does not copy or maintain MinerU source code. MinerU is used only as a configurable external parsing Provider.
When using MinerU, follow its current open-source license and service terms. If MinerU is used to provide an online service to third parties, follow its attribution requirements in the product UI or public documentation.
Discuss changes to public APIs, the canonical model, parsing Providers, authentication, storage, job execution, or deployment architecture before implementation. Bug fixes, tests, documentation improvements, and small behavior-preserving refactors are welcome as pull requests.
StructaDoc is licensed under the Apache License 2.0.