Keyboard shortcuts

Press ← or → to navigate between chapters

Press S or / to search in the book

Press ? to show this help

Press Esc to hide this help

OpenTimelineIO adapters, per-schema versioning with upgrade/downgrade functions and UnknownSchema preservation

Document status: reviewed. Canonical source.

Summary

OpenTimelineIO (OTIO) is an interchange format and API for editorial cut information maintained under the Academy Software Foundation. Its canonical serialization is JSON in which every object carries an OTIO_SCHEMA label of the form Name.Version. Each schema is versioned independently; the C++ TypeRegistry holds per-schema upgrade functions keyed by target version and downgrade functions keyed by source version. On read, an object whose version is older than the registered one is upgraded by applying upgrade functions in order (gaps permitted); an object whose version is newer than the registered one is rejected with SCHEMA_VERSION_UNSUPPORTED. On write, a caller may pass target_schema_versions (or select a family/label such as OTIO_CORE:0.14.0 via OTIO_DEFAULT_TARGET_VERSION_FAMILY_LABEL) and the writer downgrades dictionaries step by step (no gaps allowed).

Objects whose schema name is not registered are instantiated as UnknownSchema, which stores the original schema name, original version and the raw dictionary, and writes them back verbatim; the test suite asserts equivalence after a serialize/deserialize round trip. Adapters are plugins declared in plugin_manifest.json (Adapter.1 entries with name, filepath, suffixes) and discovered via the OTIO_PLUGIN_MANIFEST_PATH environment variable or setuptools entry points; they implement any of read_from_file, read_from_string, write_to_file, write_to_string. Adapter test conventions rely on sample_data fixtures, JSON baselines and equivalence assertions (assertIsOTIOEquivalentTo, assertJsonEqual) with explicit disk-to-memory-to-disk round-trip tests.

Evidence

  • Schema label format "SimpleClass.2"; schema_name()/schema_version() accessors; @otio.core.upgrade_function_for(SimpleClass, 2) and C++ TypeRegistry::instance().register_upgrade_function(name, 2, fn); @otio.core.downgrade_function_from(SimpleClass, 2) — https://opentimelineio.readthedocs.io/en/latest/tutorials/versioning-schemas.html (retrieved 2026-08-29).
  • “upgrade functions will be called in order, but they need not cover every version number” and “Downgrade functions must be called in order with no gaps” — docs/tutorials/versioning-schemas.md lines 321–323 (main).
  • schema_version_targets argument on serialization: schemas above the target “will be converted to AnyDictionary and run through the necessary downgrade functions before being serialized” — docs/tutorials/versioning-schemas.md line 124.
  • Families and labels: OTIO_CORE family with labels such as 0.14.0, 0.15.0; custom families in .plugin_manifest.json under version_manifests; otio.versioning.fetch_map("OTIO_CORE", "0.15.0"); env var OTIO_DEFAULT_TARGET_VERSION_FAMILY_LABEL=OTIO_CORE:0.14.0; otioconvert -A target_schema_versions=... — same tutorial.
  • CORE_VERSION_MAP.cpp is generated by make version-map, is “part of the unit tests suite”, and maps label → {schema → version}, e.g. 0.14.0 has Clip 1, Marker 2; 0.15.0 has Clip 2 — src/opentimelineio/CORE_VERSION_MAP.cpp lines 1–60.
  • TypeRegistry::_instance_from_schema: unregistered name → new UnknownSchema(schema_name, schema_version); newer version → ErrorStatus::SCHEMA_VERSION_UNSUPPORTED; older version → iterate upgrade_functions where schema_version <= e.first <= registered — src/opentimelineio/typeRegistry.cpp lines 360–420.
  • Registered upgrades/downgrades in core: register_upgrade_function(Marker::Schema::name, 2/3, ...), register_upgrade_function(Clip::Schema::name, 2, ...), register_downgrade_function(Marker, 3), register_downgrade_function(Clip, 2) — typeRegistry.cpp lines 96–220.
  • UnknownSchema class: UnknownSchema(std::string const& original_schema_name, int original_schema_version), original_schema_name(), original_schema_version(), data(), is_unknown_schema() override — src/opentimelineio/unknownSchema.h.
  • UnknownSchema::read_from swaps the reader dictionary into _data and erases OTIO_SCHEMA; write_to writes every stored key; _schema_name_for_reference returns the original name so the label is re-emitted — src/opentimelineio/unknownSchema.cpp lines 8–45.
  • Writer downgrade path: _downgrade_version_manifest, _downgrade_dictionary, error “No downgrader function available for …” when a step is missing — src/opentimelineio/serialization.cpp lines 112–114, 421–506, 1040–1052.
  • tests/test_unknown_schema.py: fixture embeds "OTIO_SCHEMA": "MyOwnDangSchema.3" inside media_reference.metadata; test_serialize_deserialize asserts assertIsOTIOEquivalentTo(self.orig, test_otio); test_unknown_to_dict checks data returns a copy and nested known schemas (RationalTime) are decoded — lines 9–100.
  • tests/test_version_manifest.py: test_fetch_map, test_env_variable_downgrade, test_two_version_manifests — lines 65–131.
  • Adapter contract: implement read_from_file, read_from_string, write_to_file, write_to_string; manifest {"OTIO_SCHEMA": "PluginManifest.1", "adapters": [{"OTIO_SCHEMA": "Adapter.1", "name", "filepath", "suffixes"}]}; discovery via OTIO_PLUGIN_MANIFEST_PATH or opentimelineio.plugins entry points — https://opentimelineio.readthedocs.io/en/latest/tutorials/write-an-adapter.html.
  • Core adapter tests: tests/test_adapter_plugin.py loads tests/baselines/adapter_example.json, checks has_feature("read_from_file"), argument pass-through (suffix=3), media-linker hooks, manifest lookup by suffix/name, env-var path deduplication and manifest ordering — lines 22–302; baselines directory holds empty_*.json per schema and adapter_plugin_manifest.plugin_manifest.json (listing retrieved 2026-08-29).
  • otio_test_utils.OTIOAssertions provides assertJsonEqual (with trailing-decimal-zero normalization) and assertIsOTIOEquivalentTo — src/py-opentimelineio/opentimelineio/test_utils.py lines 15–31.
  • External adapter convention (CMX3600): tests/sample_data/*.edl fixtures; test_edl_round_trip_mem2disk2mem (write_to_string → read_from_string → assertJsonEqual), test_edl_round_trip_disk2mem2disk (read → write to tmp → read → assertIsOTIOEquivalentTo), test_regex_flexibility — https://github.com/OpenTimelineIO/otio-cmx3600-adapter tests/test_cmx_3600_adapter.py lines 16–286.
  • Adapters were moved out of core into separate packages bundled by OpenTimelineIO-Plugins (otio-aaf-adapter, otio-cmx3600-adapter, otio-fcp-adapter, otio-svg-adapter, …) — https://github.com/OpenTimelineIO/OpenTimelineIO-Plugins pyproject.toml lines 17–26.
  • License Apache-2.0 — LICENSE.txt; tags v0.18.1, v0.18.0, v0.17.0 (retrieved 2026-08-29).

Mechanism

Serialization is a typed dictionary encoding. Every SerializableObject writes its fields plus an OTIO_SCHEMA label; nested objects recurse, so schema labels appear at every level including inside free-form metadata maps. Deserialization dispatches on the label: the registry finds the _TypeRecord for the name, constructs the object, and if the on-disk version is lower it mutates the raw AnyDictionary through each registered upgrade function whose target version lies in the interval before calling read_from. Upgrades are therefore dictionary-to-dictionary transforms independent of the C++ class layout, and may skip versions. Reading a newer version than the library knows is a hard error rather than a best-effort parse.

Downgrading is symmetric but stricter. The writer receives a schema_version_map (from an explicit argument, an environment-selected family label, or a plugin manifest’s version_manifests). For any object whose registered version exceeds the target, the writer serializes it to a dictionary and applies downgrade functions one version at a time until it reaches the target, failing loudly if a step is missing. The CORE_VERSION_MAP snapshot ties human-readable labels (library releases) to full schema-version sets and is regenerated and unit-tested whenever schemas change, giving a reproducible target for “write for OTIO 0.14.0”.

Unknown schemas are preserved by construction. When the name lookup fails the registry constructs UnknownSchema with the original name and version, and read_from takes ownership of the entire dictionary minus the label. Nested known objects inside that dictionary are still decoded, so the preserved blob is a typed tree rather than raw bytes. On write the original label is re-emitted with the stored fields, which yields an idempotent round trip; the object is inert (no upgrade functions run on it) and reports is_unknown_schema() so tools can flag it.

Adapters are thin translators between an external format and the in-memory object model; the object model, not the adapter, owns versioning and preservation. Tests fix the contract from both ends: JSON baselines for the OTIO side and sample files for the foreign side, with equivalence checks after mem→disk→mem and disk→mem→disk cycles.

NUIF relevance

Borrow

  • Version each NUIF schema/dialect construct independently and encode name.version on every serialized node, so migration is local rather than a monolithic document version bump.
  • Implement migrations as ordered dictionary transforms registered per target version, with gap-tolerant upgrades and gap-free downgrades, and reject newer-than-known versions explicitly.
  • Publish a generated, unit-tested label → schema-version-set manifest (the CORE_VERSION_MAP pattern) so nuif migrate --target <label> is reproducible.
  • Model foreign or unregistered constructs as a first-class UnknownSchema-like value that keeps name, version and payload, decodes nested known values, re-emits verbatim and is queryable via an is_unknown predicate.
  • Adopt the adapter test convention of paired fixtures (foreign sample data plus canonical JSON baselines) with both round-trip directions asserted through a canonical equivalence function.

Adapt

  • OTIO preserves unknowns structurally (typed dictionary); NUIF additionally needs byte-exact preservation for extension payloads whose encoding is not NUIF’s own, per spec/07 and ADR 0004.
  • OTIO’s downgrade discards data silently inside downgrade functions; NUIF downgrades must emit fidelity records (approximated, unsupported) per dropped field.
  • Family/label selection via an environment variable is convenient for scripts but should be an explicit CLI/API argument in NUIF’s headless contract, not ambient state.
  • OTIO unknown objects do not participate in upgrades; NUIF should allow dialects to register migrations for previously unknown namespaces when they become known.

Reject

  • JSON with embedded schema labels at every node is verbose and lacks a canonical form; NUIF keeps nuif-text-0/nuif-cbor-0 canonicalization rules rather than free-form JSON.
  • Adapter discovery via setuptools entry points and manifest environment paths is Python-specific; NUIF adapters are Rust crates or WASM components with declared capabilities.
  • Equivalence assertions that normalize trailing decimal zeros ad hoc should be replaced by NUIF numeric canonicalization rules in spec/08.

Open questions

  • Whether per-node schema labels are needed in the binary profile or can be replaced by a document-level dialect version table plus per-entity type references.
  • How to compose migrations across dialects when one dialect’s downgrade changes a construct another dialect references.
  • Whether NUIF should support “read newer than known” via opaque preservation of the unknown fields instead of OTIO’s hard failure.
  • How to keep the unknown-object equivalence check stable under canonicalization when preserved payloads contain floating-point values.