08 — Serialization and package format

Document status: draft. Canonical source.

The NUIF logical model is encoding-independent.

Initial profiles:

• nuif-text-0 — deterministic human-readable canonical form for fixtures/review.
• nuif-cbor-0 — deterministic CBOR following draft-ietf-cbor-serialization §4.1 (preferred serialization) and §5.1 (bytewise-lexicographic map key order), with the narrowing rules of RFC 0005 stated by value.

cargo xtask codec-benchmark is the non-normative decision harness for these profiles. It records size, latency and allocation only after exact semantic, canonical and unknown-data edit fixpoints pass. Native partial loading is a separate capability: profile-0 decoders currently load the complete document. Schema-generated candidates MUST NOT be compared using a partial logical model; the active next-candidate investigation is Cap’n Proto because its encoding specification defines a canonical form. No schema-generated NUIF profile is currently accepted.

The experimental nuif-package-0 profile assigns .nuif to a deterministic ZIP container. Bare encodings use .nuif.json and .nuif.cbor. Historical alpha files that used .nuif for bare bytes MAY be recognized read-only through content detection, but new .nuif output MUST be a package once this profile is accepted.

nuif-package-0 is proposed by RFC 0010. Its reference codec, cross-writer byte fixture, package/resource identity relations, explicit resolver and hostile archive/one-over suite are executable through cargo xtask gate-i-package. The separate nuif-png-rgba8-0 segment is executable through cargo xtask gate-i-image; nuif-opentype-static-single-0 is executable through cargo xtask gate-i-font. This is not full Gate I: broader PNG and OpenType interpretation plus cross-platform/external package and media evidence remain incomplete.

cargo xtask gate-i-font-metadata is research evidence for RFC 0013 axis decoding and coordinate normalization; cargo xtask gate-i-font-shaping adds an isolated shaping and internal location-coherence trial. Those gates alone do not admit packages; cargo xtask gate-i-font-package and cargo xtask gate-i-font-runtime separately require capability-gated typed admission and direct coordinate delivery. The direct API, CLI, generated Node/browser WASM, stdio MCP and linked POSIX C consumer now return the same complete interior-package snapshot report. A three-host CI collector compares all twelve resource reports and requires the variable-font runtime payload to remain exact; a retained passing hosted aggregate is still required before cross-platform promotion.

Numeric and string rules (RFC 0005)

• Numeric kinds are integer (signed 64-bit) and real (binary64). Authored reals MUST be finite. Negative zero is not distinct from zero.
• In nuif-cbor-0, integers use major type 0 or 1 and reals use the shortest IEEE 754 floating-point width that round-trips, including integral reals. Integer and real are distinct logical values. Both real zeros use positive floating-point zero; integer zero remains distinct. Integer heads MUST be shortest; lengths MUST be definite; map keys MUST be strictly increasing in bytewise order of their complete deterministic encoding; no tags and no simple values other than false, true and null appear; extension and unknown-kind payloads are byte strings hashed verbatim (RFC 0008).
• Decoders used for hashing and conformance MUST reject non-canonical input rather than re-canonicalize it.
• In nuif-text-0, reals print as the shortest round-trip decimal in the fixed layout of RFC 0005 rule 15; NaN and infinities are parse errors; keys are in UTF-8 byte order; layout is not significant. Text key order is intentionally independent from CBOR encoded-key order (RFC 0008).
• Identifiers (namespaces, keys, kind names, extension names) match [a-z0-9][a-z0-9_.:-]*. String values are stored verbatim as valid UTF-8 and are never normalized by canonicalization.

Hash

The canonical hash of a document is SHA-256 over its nuif-cbor-0 bytes. The text profile has no separate hash: hash(text) = hash(cbor(parse(text))). Published content identifiers carry the profile identifier. Canonical hashes MUST exclude transport-only compression differences.

Package, resource and semantic hashes are distinct:

• document_hash is SHA-256 of canonical document.cbor and covers semantic asset bindings;
• resource_digest is SHA-256 of exact resource bytes;
• package_hash is SHA-256 of the complete deterministic package bytes.

Package-only caches or reports may change package_hash without changing document_hash. Replacing a resource bound to an asset changes the resource digest and semantic document hash while preserving stable AssetId.

Proposed package profile 0

The package member set is:

mimetype
manifest.cbor
document.cbor
blobs/sha256/<digest-hex>

The first member is stored mimetype with exact ASCII value application/nuif+zip. This media type remains provisional until registration. manifest.cbor and document.cbor are canonical nuif-cbor-0.

The manifest declares package profile/version, the canonical document descriptor, required capabilities, stable assets and every immutable resource descriptor. A descriptor includes media type, SHA-256 digest, size, role and an embedded or explicit linked locator. The manifest is not self-addressed.

Required capabilities are a set of at most 256 identifiers, each at most 128 ASCII bytes and matching [a-z0-9][a-z0-9_.:-]*. Structural package decode MUST validate and preserve this set without pretending the host implements it. Before claiming full package support, a host MUST compare the complete set with capabilities it explicitly declares. Missing requirements MUST be returned as an exact deterministic set. Inspection, preservation and migration tools MAY operate after structural decode without claiming full semantic or behavioral support and MUST NOT execute a resource merely because its capability is declared.

Profile 0 uses stored ZIP members only; mimetype is first and other names are bytewise sorted. Names are exact ASCII registered paths. Writers use fixed timestamps/header attributes, no comments/extra fields/data descriptors, encryption, directories, ZIP64 or split archives. The manual reference writer and zip 8.6.0 independently reproduce the exact header fixture.

Readers MUST reject duplicate decoded names, non-ASCII/backslash/absolute/dot paths, directories, symlinks, encryption, unsupported compression, inconsistent headers, unknown members, undeclared blobs, missing required blobs and size/digest mismatches. Readers MUST NOT extract package members to a filesystem.

Portable packages embed every resource required by their declared profile. Linked resources are explicit and never fetched implicitly; a caller-supplied resolver verifies expected size and digest before use. Credentials MUST NOT be stored in resource locators.

Schema versions

Every serialized record kind carries a schema version. Migrations are registered pure functions per kind; reading a record whose version is newer than the implementation knows is an error with a diagnostic, never silent loss.

Parsers MUST enforce resource limits and reject cycles where the relevant graph is specified acyclic. The experimental package limits are 80 MiB per archive, 32 MiB per resource, 64 MiB total embedded resources, 8,192 descriptors and 256 required capabilities of at most 128 bytes each. nuif-png-rgba8-0 additionally limits encoded input to 32 MiB, each dimension to 8,192, pixels to 16,777,216 and chunks to 4,096. nuif-opentype-static-single-0 limits encoded input to 32 MiB, tables to 256, family names to 256, coverage ranges to 65,536 and feature settings to 64. Broader image interpretation and general font-format/policy limits remain experiment-required and MUST be accepted through later media profiles before implementations claim those capabilities.