Skip to content
alchemist-editorPublic

About

OpenTimelineIO Rust

Resources

Stars

2 stars

Watchers

0 watching

Forks

Latest commit

 

History

476 Commits

Folders and files

Repository files navigation

otio-rust

A Rust implementation of OpenTimelineIO — the interchange format for editorial timeline information — of the time math library beneath it, of the file-format adapters around it, and of the language bindings on top.

The goal is a pure-Rust core with no C++ and no Python anywhere under it, and everything else generated from that core rather than written by hand.

  opentime ── otio-core ─┬─ otio-adapter ─┬─ otio-ale          ALE
                         │                ├─ otio-cmx3600      EDL
                         │                ├─ otio-fcp7    ┐    FCP 7 XML
                         │                ├─ otio-fcpx    ┘    FCP X XML   (over otio-xml)
                         │                └─ otio-aaf ─ aaf    AAF
                         │
                         ├─ otio-python ─── the `opentimelineio` Python package
                         ├─ otio-capi ───── libotio ─┬─ Go · Swift · Zig · C++ · C# · Objective-C
                         └─ otio-wasm ───── .wasm ───┴─ TypeScript
                                                     generated by otio-sdk-gen

Status

Everything described below is on main and covered by CI.

.otio, ALE, EDL and both FCP XML flavours can be read and written from Rust, Python, C, Go, Swift, Zig, C++, C#, Objective-C and TypeScript. AAF can be read and written from Rust and Python. What is not done is the section worth reading before you plan around any of this.

Core

Crate What it is
opentime Rational time, time ranges, SMPTE timecode. Upstream's own test suite passes against it.
otio-core The timeline object model and .otio serialization, round-tripping upstream's sample documents. Objects live in a generational arena and are named by handle, per ADR 0001.
otio-adapter The trait every file-format adapter implements, plus the error type and the metadata shapes they share. Each format names its own typed read and write options instead of upstream's keyword-argument bag. Also upstream's media linkers and hooks, as named functions in a registry, run around a read or write in upstream's order.
otio-xml A small XML tree, parser and pretty printer for the XML adapters. Not general purpose; it exists because the workspace takes no third-party dependencies.
otio-bundle .otioz and .otiod bundles: a timeline packaged with its media, as upstream's bundle.h. Carries its own zip and DEFLATE for the same no-dependencies reason.

Adapters

Crate Format State
otio-ale Avid Log Exchange Read and write, round-tripping upstream's sample files byte for byte
otio-cmx3600 CMX 3600 EDL Read and write, with upstream's own test suite as the measure
otio-fcp7 Final Cut Pro 7 interchange XML Read and write, round-tripping upstream's sample files
otio-fcpx Final Cut Pro X XML Read and write, round-tripping upstream's sample files
aaf The AAF container and object model, a port of pyaaf2 Read and write. Reading is checked against manifests pyaaf2 produced from the same files; writing produces pyaaf2's files byte for byte for the same operations, importing DNxHD and WAV essence and copying objects in from another file among them, and so does changing an existing file, as pyaaf2's 'r+' mode does
otio-aaf AAF mapped to OpenTimelineIO Read and write. Reading runs the transcription and all three of upstream's passes, and bakes keyframes and logs as it does, matching otio-aaf-adapter byte for byte on every sample file in its test suite. Writing matches the files that adapter writes, byte for byte, on every sample it can write, embedding media included

Everything these write is meant to open unchanged in existing OpenTimelineIO tools, so arithmetic, rounding, timecode behaviour and output formatting follow upstream exactly.

This is a port, not a reimplementation with its own ideas. Where upstream has a quirk — or an outright bug, and there are a couple — it is reproduced here rather than tidied, because every file already in the world carries what upstream produced, and agreeing with upstream is the only way to agree with every other reader. Knowing departures are documented on the item itself, and are always in the direction of accepting more input, never of producing different output for input upstream accepts.

Bindings and SDKs

Three binding layers sit directly on the core:

Crate What it is
otio-python PyO3 bindings publishing a package named opentimelineio, aiming to be a drop-in replacement for upstream's, adapters and plugin system included: manifests, media linkers, hooks and schemadefs. Measured by vendoring upstream's whole Python test suite and four of its adapter suites, and running them unmodified.
otio-capi The C ABI: libotio plus a hand-written otio.h, proven by a C program linked against it in CI. Objects are handles, strings are copied out, failures are status codes. This is the layer every non-Python SDK sits on.
otio-wasm The same C ABI built for wasm32-unknown-unknown, with no wasm-bindgen step, plus the TypeScript package that wraps it for browsers and Node.

The language SDKs on top of those are generated, not hand-written, and neither is the description they are generated from:

Crate What it is
otio-sdk-model Reads the Rust source of otio-capi — not the header, not a hand-written schema — and emits a description of the interface, committed as sdk/api.json so a change to the C ABI shows up as a readable diff
otio-sdk-gen Turns that description into one SDK per language, one backend module each
SDK Where it lives Over
Go sdk/go libotio via cgo
Swift sdk/swift libotio
Zig sdk/zig libotio
C++ sdk/cpp libotio
C# sdk/csharp libotio, loaded at run time
Objective-C sdk/objc libotio, on Apple's runtime and GNUstep's
TypeScript crates/otio-wasm/ts WebAssembly, in browsers and Node

The TypeScript one lives beside the wasm crate rather than under sdk/ because it ships with the .wasm module that crate builds. It is published to npm as @alchemist-edit/otio (npm install @alchemist-edit/otio). The others, and the C library itself for each supported target, are published as GitHub releases. How both kinds of release go out is in docs/releasing.md.

Each backend owns the shape of its own language — a generated API that reads like transliterated C is a failure even if every function is present — and the shape of the API itself follows upstream OpenTimelineIO's own bindings for that language. Deliberate departures are recorded in ADR 0003. Every SDK is built and tested by its own toolchain in CI, and the generated files are checked in, so drift from the C ABI fails a build rather than reaching a user.

Media linkers and hook scripts work from every SDK, not only Python: a function of your own is registered under a name, a read names the linker it wants, and scripts attached to post_adapter_read and the other hooks run where upstream runs them (guide, ADR 0009).

More detail in sdk/README.md and crates/otio-wasm/README.md.

Extensions

extensions/ holds work built on this core that is not part of OpenTimelineIO, kept apart from it as ADR 0010 sets out: its own crates, its own C header, its own module in every SDK, and off until a program enables it. The first is timeline sync (ADR 0011, epic #115), which so far is the extension and its switch; identity, patches and sessions follow.

Documentation

site/ is a documentation site: what OpenTimelineIO is, what this port does, and every code sample shown in each of the languages above, with a switcher along the top of the sample. Its reference section is generated from the same sdk/api.json the SDKs are, so it cannot describe a call the library does not have. See site/README.md.

What is not done

Stated plainly, because most of it is somewhere a reader would otherwise assume completeness.

AAF writing has only been checked against upstream, not against Media Composer. otio-aaf writes the same bytes as upstream's adapter given the same clock and identifiers, on every sample upstream can write, so a file that upstream's would get into Media Composer this one gets in too. No file written here has been imported into Media Composer as part of testing, so that is inferred from byte parity, not observed. That includes embedding media in the file (upstream's embed_essence), which copies essence out of another AAF or imports a raw DNxHD stream as upstream does, and, like upstream, cannot embed a WAV file. From Python, upstream's four AAF hooks run where upstream runs them, so an otio_aaf_pre_write_transcribe hook can transcode other media into something embeddable; they are handed None where upstream hands them the open pyaaf2 file (ADR 0007).

AAF's reading log is Rust and Python only. Every SDK reads and writes AAF with upstream's options, but transcribe_log prints as it reads, which would need a callback across the C ABI, so the C ABI leaves it out.

Bundles are not in the TypeScript package. Every other SDK reads and writes .otioz and .otiod through a path, with upstream's media policy. A bundle is a directory, or an archive of media copied in from disk, and the WebAssembly module has no file system, so the package leaves both formats out rather than offer them only to refuse.

otio-core's error messages are not upstream's wording yet. opentime's are, because an upstream test compares one exactly; the rest reach Python as ValueError text that does not match.

Open work is tracked in issues, with the long-running pieces as epics.

Design decisions

Decisions that shape the whole project are recorded as ADRs in docs/adr:

Each crate's own README.md covers the decisions local to it; they are worth reading before changing one.

Development

The workspace takes no third-party Rust dependencies outside the Python bindings, so the Rust side needs only a toolchain. What CI runs:

cargo build --workspace --exclude otio-python --all-targets
cargo test  --workspace --exclude otio-python

cargo fmt --all --check
cargo clippy --workspace --all-targets -- -D warnings
RUSTDOCFLAGS=-D warnings cargo doc --workspace --no-deps

Build before you test, as two commands. The C ABI's test suite links a real C program against libotio, so it needs the library on disk first.

How CI arranges the same work — what gates what, and why a documentation change runs nothing at all — is in docs/ci.md.

otio-python is excluded above because it is a Python extension module — linking it on macOS needs flags that come from maturin rather than cargo. It is built and tested on its own:

cd crates/otio-python
pip install .                          # runs maturin
python tests/run_upstream_tests.py     # upstream's test files, unmodified

The minimum supported Rust version is 1.85, and a recent toolchain will happily accept syntax newer than that — let-chains, stable since 1.88, are the easy trap. CI checks it; checking locally saves a round trip:

cargo +1.85.0 build --workspace

The SDKs

Each needs its own toolchain and a built C library:

cargo build -p otio-capi --release

cp target/release/libotio.a sdk/go/lib/    && (cd sdk/go    && go test ./...)
cp target/release/libotio.a sdk/swift/lib/ && (cd sdk/swift && swift test -Xlinker -L"$PWD/lib")
cp target/release/libotio.a sdk/zig/lib/   && (cd sdk/zig   && zig build test)
cp target/release/libotio.a sdk/cpp/lib/   && \
  cmake -S sdk/cpp -B sdk/cpp/build -DCMAKE_BUILD_TYPE=Release && \
  cmake --build sdk/cpp/build && ctest --test-dir sdk/cpp/build --output-on-failure
cp target/release/libotio.a sdk/objc/lib/  && (cd sdk/objc  && make check)

C# loads the shared library at run time instead of linking the static one:

cp target/release/libotio.so sdk/csharp/lib/ && \
  (cd sdk/csharp && dotnet run --project tests)   # libotio.dylib on macOS

TypeScript builds the wasm module instead of linking the static library, and runs the same test suite in Node and in Chromium:

cd crates/otio-wasm/ts
npm install
npm run build          # cargo build --target wasm32-unknown-unknown, then tsc
npm test               # in Node
npm run test:browser   # the same suite, in Chromium

The documentation site

A Next.js app with a toolchain of its own, and no relationship to the cargo workspace:

cd site
npm install
npm run dev        # http://localhost:3000
npm run verify     # what CI runs: samples, types, lint, tests, build

The generated SDKs are checked in. Regenerate them, all or one at a time:

cargo run -p otio-sdk-gen              # every target
cargo run -p otio-sdk-gen -- ts        # go | swift | ts | zig | cpp | csharp | objc
cargo run -p otio-sdk-gen -- --check   # fails if what is committed is stale

cargo test -p otio-sdk-gen runs that same check, and is how the Rust jobs catch a C ABI change nobody regenerated for.

The SDK jobs and the C ABI job do not run on Windows: linking a Rust static library with MSVC needs an environment the runners do not have set up.

License

Apache-2.0, matching upstream OpenTimelineIO.

About

OpenTimelineIO Rust

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages