Documentation index
This page is generated from docs/catalog.json and document frontmatter. The site describes a pre-standard research project.
- Source digest:
sha256:d554926f107ca96300cd2ac76757a15e9c9cbdc7ced95047e6d1f380ebc3712c - Published Markdown files: 266
- Navigation sections: 10
Status inventory
| Status | Documents |
|---|---|
accepted | 16 |
draft | 16 |
executable | 1 |
exploratory | 1 |
informational | 71 |
pre-draft | 1 |
proposed | 7 |
provisional | 1 |
reviewed | 122 |
verified | 30 |
NUIF authored-interface interchange research manuscript
Architecture, executable profiles, resource packaging, reconstruction research and open risks
Refpath contributors
Version working-2026-08-31 · 2026-08-31
Manuscript status: working technical manuscript; not peer reviewed. This generated document does not change the status of any included specification module.
Abstract
This manuscript evaluates a layered authored-interface interchange model through prior-art synthesis, bounded adapter profiles and executable conformance experiments. It also separates proposed resource packaging, source-backed capture and screenshot reconstruction from the narrower implemented alpha evidence. It records the architectural hypothesis, implemented evidence, unresolved risks and governance conditions without claiming specification stability, reconstruction accuracy or universal format coverage.
Reproducibility
The body is compiled from 13 canonical whitepaper modules at source digest sha256:d554926f107ca96300cd2ac76757a15e9c9cbdc7ced95047e6d1f380ebc3712c. Editorial changes belong in those source modules.
NUIF foundation
NUIF investigates a portable, vendor-neutral draft specification for authored user-interface documents. The candidate model is intended to preserve meaning across editors and implementation targets rather than treating a rendered bitmap, a vendor scene graph, or source-language AST as the universal truth.
Thesis
A useful portable interface specification must coordinate several representations instead of collapsing them into one:
- semantic/document containment;
- component and instance identity;
- authored layout and responsive constraints;
- resolved geometry at explicit evaluation contexts;
- geometry, paint, typography, and assets;
- design-token references and themes;
- interaction/state and data-binding graphs;
- source/tool provenance and correspondence;
- extension payloads that can survive unknown intermediaries;
- deterministic operations, diff, patch, and reconciliation.
Portable resources add a second identity boundary: editable semantic assets retain stable IDs, while exact image/font bytes use immutable content digests. Package paths and source URLs are locators/provenance, not identity.
NUIF therefore treats portability as a synchronization problem as much as a serialization problem.
Architectural hypothesis
The working model is a small canonical core plus coordinated graphs and extension dialects. The containment tree answers ownership and order. Typed relationship graphs express constraints, components, tokens, interactions, provenance, dependencies, and other relationships that do not belong in a tree.
The reference implementation will preserve both authored and resolved state. Resolved state is always scoped to an evaluation context and is never allowed to silently replace authored intent.
Fidelity model
Every adapter and transformation must classify material mappings:
lossless— semantics are preserved exactly;representable— equivalent target semantics exist, even if encoded differently;approximated— a declared approximation is produced;preserved_unrenderable— data survives as an extension but the target cannot render/edit it;unsupported— data cannot currently be represented or preserved safely.
Silent loss is a conformance failure.
Explicit non-goals
NUIF does not promise to infer the unique original source program from pixels, reproduce arbitrary JavaScript execution, make every platform text renderer bit-identical, or force every target to support every capability. The draft specification should make such boundaries inspectable and machine-readable.
Screenshot reconstruction is therefore an optional inference client, not a new canonical truth. It may propose a validated editable hypothesis and calibrated alternatives, but screenshot-only evidence cannot be classified as lossless authored source.
Reference implementation role
The Rust implementation and editor are executable research instruments and conformance references. They do not define semantics by accident; normative behavior belongs in spec/ and must be testable independently.
NUIF architecture thesis
NUIF is a specification-first authored-interface model. Its center is neither a vendor editor nor a source framework.
Core thesis
A portable interface document must retain intent, structure, relationships and evaluated results simultaneously. A single flattened scene tree cannot preserve enough information for loss-minimizing round trips across editors and runtime frameworks.
The recommended architecture is a layered hybrid:
Document containment tree
│ stable IDs
├── component / instance graph
├── token / theme graph
├── layout constraint graph
├── interaction / state graph
├── provenance / correspondence graph
└── asset dependency graph
Authored model ──evaluate/lower──► resolved model ──► render scene
▲ │
└──────── reconcile / lift ◄────┘
Borrowed foundations
- MLIR: dialects, explicit lowering, partial legality and multiple abstraction levels.
- OpenUSD: non-destructive composition, references, layers and variants.
- glTF: small core, extension registry, used/required capabilities.
- DTCG: token interchange.
- SVG/Unicode/OpenType: geometry and text foundations.
- Retentive/symmetric lenses: synchronization with preserved source regions.
New work required
NUIF must define the missing combination: authored UI semantics + resolved state + cross-tool provenance + structural loss accounting + source patch synchronization.
Canonical layers
- Document layer — identity, containment, semantics, accessibility.
- Component layer — definitions, instances, slots, parameters, variants and overrides.
- Layout layer — authored sizing/layout intent independent of resolved geometry.
- Visual layer — geometry, paint, text and effects.
- Behavior layer — interactions, states, animation and data bindings.
- Resolved layer — computed layout, shaped text, flattened paint/effect plans for a declared evaluation context.
- Provenance layer — source/destination correspondence and fidelity diagnostics.
- Resource layer — stable semantic assets bound to content-addressed bytes, package/resolver locators and derivation records.
No lower layer is permitted to silently erase a higher-level authored construct. Lowerings that cannot represent a construct must emit fidelity records.
Stable identity
Identity is semantic and independent of path, order and display name. Moving an entity does not change its ID. Content hashes identify immutable resources and canonical snapshots, not editable semantic entities.
Compiler and reconstruction ports
Deterministic source adapters and probabilistic screenshot reconstruction meet at the operation boundary:
retained source + resolved host observations ─┐
├─> typed operations -> core
pixels + OCR/CV/model hypotheses ─────────────┘ -> render/evaluate
Source-backed and screenshot-only inputs retain distinct evidence classes. A model/provider is replaceable and cannot redefine the operation grammar, validator, layout semantics, resource identity or fidelity ceilings.
Falsifiability
The architecture fails if the v0 experiment cannot preserve a non-trivial responsive component through editor→HTML→NUIF→editor while retaining component identity, token bindings, layout intent, an opaque foreign extension and a minimal source patch after an edit.
The resource/reconstruction extension fails if independent package writers cannot reproduce the proposed bytes, if browser capture cannot be pinned without secret leakage, if visual objectives reward flat screenshot copies, or if adaptation fails to beat the untuned tool-assisted baseline on a frozen holdout.
Layout and rendering research synthesis
Layout is not geometry
NUIF separates authored constraints from resolved boxes. Fixed x/y/width/height are valid authored values for freeform content, but they are not the universal layout representation.
The initial layout vocabulary contains families rather than one universal algorithm:
freeform— transforms/anchors and explicit geometry.stack— one-dimensional flow with intrinsic sizing, distribution, alignment and gaps.flex— web-compatible flexible layout semantics.grid— bounded explicit fixed/frtracks, spans and deterministic no-implicit-track placement in profile 0; broader CSS Grid features remain capability-reported adapter input.constraint— relational linear constraints for editor/native-layout cases.custom— extension/dialect-defined evaluator with declared fallback/resolved geometry.
Common sizing primitives are normalized across families: fixed, intrinsic-min, intrinsic-max, fit-content, fill/available, percentage, min/max clamps, aspect ratio and content measurement.
Taffy is the recommended first evaluator for CSS-compatible block/flex/grid behavior because it implements web algorithms in Rust. SwiftUI’s proposal-response model and Cassowary-style constraints demonstrate why the canonical schema must remain a superset rather than serializing Taffy’s Style directly.
Evaluation context
Resolved layout is keyed by an explicit context including viewport/container size, pixel ratio, locale, writing direction, font set, token/theme selection and feature/dialect capabilities. Multiple resolved snapshots may coexist as caches or conformance fixtures.
Rendering semantics
The draft specification defines the visual meaning of paths, fills, strokes, transforms, clipping, masks, gradients, compositing, images, text and supported effects. It does not specify GPU command buffers or a renderer implementation.
The reference renderer uses a backend trait. Vello/wgpu is the leading interactive experiment, but conformance requires deterministic raster comparisons and must permit CPU reference rendering where GPU differences would make tests unstable.
Text
Canonical text remains Unicode text + style runs + semantic annotations + font references. Shaping produces resolved glyph IDs, clusters, advances and offsets using pinned font data and a declared Unicode/shaping version. A glyph cache never replaces semantic text.
Portability reports must distinguish font substitution, missing glyphs, line-break differences and rasterization differences from document-model loss.
Protocol, portability and synchronization
NUIF treats portability as an ongoing synchronization problem.
Operations
The protocol operates on stable entities and semantic properties. Operations include create/delete/move, set/unset property, list/set relation edits, component/instance overrides, token bindings, extension edits and transactions. Editor gestures lower to these operations.
A drag inside a stack should usually become a reorder or layout-property edit; a drag in freeform space may become a transform edit. GUI coordinates are input data, not the protocol abstraction.
Patch model
A patch is a deterministic ordered set of operations with base snapshot identity, optional preconditions, transaction metadata and provenance. Patches can be replayed headlessly.
Three-way merge uses stable identity first and structural matching only when identity is absent. Conflicts are typed: property, delete/edit, ordering, relationship, extension and semantic-lowering conflicts.
Correspondence
Adapters maintain correspondence records between NUIF entities/properties and foreign constructs such as DOM nodes, CSS declarations, Svelte component props or design-tool node IDs. Correspondence is separable from the canonical design so source-specific metadata can be detached when unnecessary.
Fidelity classes
Every adapter/evaluator may report:
lossless— semantics preserved and reconstructable.representable— equivalent semantics represented through different constructs.approximated— visible/behavioral approximation with known semantic loss.preserved_unrenderable— data retained opaquely but not understood/rendered.unsupported— data could not be safely preserved.
Silent degradation is a conformance failure.
Serialization, collaboration and governance
Logical model before encoding
NUIF defines one logical model with multiple conforming encodings.
Text form
A canonical, reviewable representation is implemented for examples, fixtures,
diffs and Git workflows. nuif-text-0 fixes number formatting, UTF-8 key order,
layout and strict decode/canonicalize behavior; later text profiles may evolve
only through explicit versioning.
Binary form
Deterministic CBOR is the profile-0 binary form because the NUIF profile closes the choices left by RFC 8949 without coupling the logical model to generated code. The executable codec gate finds it near 41% of canonical-text size at 4,096 entities on an Apple M5 Pro run, while its typed decode path is slower than text. That result supports CBOR as a compact canonical form, not as a universal latency winner.
A candidate is timed only after complete-model round trip, canonical fixpoint and unknown-data preservation through a neighboring edit. Protobuf does not specify canonical binary output. FlatBuffers deliberately permits different byte layouts and old readers ignore new fields, so a rebuilding editor needs a separate retention strategy. Cap’n Proto specifies a schema-agnostic canonical form and is the preferred next experiment, but it still needs a complete NUIF mapping, bounded old-reader edit trial and two agreeing canonical writers. Compiled zero-copy runtime caches remain separate, explicitly noncanonical profiles rather than replacements for authoring interchange.
The experimental package form separates manifest/document records from
content-addressed resources. RFC 0010 selects a candidate deterministic ZIP
profile with fixed mimetype, canonical manifest/document records and
SHA-256-addressed blobs. Bare encodings use explicit .nuif.json and
.nuif.cbor names. Exact ZIP header fixtures, two independent local writers and
bounded image/font segments now exist. Cross-platform and externally authored
writer evidence remains required before package-profile acceptance.
Semantic document, resource and package hashes have different scopes. Stable asset identity is not content addressing. Unknown extension payloads remain explicit typed bytes/values and must not depend on accidental codec unknown- field behavior.
Collaboration
Canonical documents do not require CRDT tombstones, clocks or replica metadata. A collaboration profile maps NUIF operations to an append-only/change structure and can use Automerge, Yjs or another convergent transport. Checkpoints serialize back to canonical NUIF.
This keeps offline files simple and permits multiple collaboration engines.
The executable register profile uses causal multi-value registers. The separate existing-tree profile replays uniquely ordered moves, rejects cycles, models deletion as profile trash and orders siblings through stable RGA-style origins. Semantic move and deletion conflicts remain visible even when the profile can choose a deterministic checkpoint. Automerge is presently tested as an operation-set transport, not claimed as an implementation of the tree algorithm.
Governance
Early development occurs in refpath/nuif, but the architecture assumes eventual neutral stewardship. A plausible progression is:
- OSS research/reference implementation under Refpath.
- public RFC process + implementer registry.
- independent community/working group once two independent implementations exist.
- investigate W3C Community Group for UI/document semantics and/or Khronos-style governance if renderer/asset vendors become primary stakeholders.
Specification text, schemas and conformance tests need clear royalty-free contribution/IP terms before claiming standards-track stability.
Prior art and competitive map
No surveyed system currently combines the whole NUIF thesis. Several solve important subsets.
| System | Strongest reusable idea | Gap relative to NUIF |
|---|---|---|
| Penpot | open inspectable design document; SVG mapping | shape-centric; not a cross-runtime synchronization standard |
| OpenPencil | programmable editor, Figma codec, DOM/CSS, CLI/MCP | editor ecosystem, not neutral standards governance |
| Figma | mature component/layout authoring semantics | proprietary canonical model and evolving vendor format |
| W3C UI Specification Schema CG | implementation-agnostic UI field/schema goal | closed 2026; schema approach lacked executable renderer/protocol proof |
| Open UI | component anatomy/states/accessibility research | web-control scope, not authored visual document exchange |
| SVG | vector geometry/paint interoperability | lacks high-level components/responsive authored layout |
| Lottie/Rive | portable animation and state-machine runtimes | animation/runtime focus rather than general UI authoring |
| DTCG | neutral token semantics | intentionally only tokens |
| OpenUSD | non-destructive layers/references/variants | 3D scene domain, not UI semantics |
| glTF | compact core + extension governance | delivery/runtime asset rather than authoring model |
| MaterialX | renderer-independent typed graph | material domain |
| MLIR | dialects/multi-level lowering | compiler infrastructure rather than document semantics |
| CSS | rigorous layout families and authored→formatting pipeline | web-specific cascade/DOM/runtime semantics |
| IFC/STEP | long-lived semantic interchange and profiles | complexity warns against over-generalizing the core |
Directly borrow
Stable standards concepts: SVG geometry, DTCG token values, Unicode/OpenType text foundations, CSS-compatible algorithms for matching profiles, glTF-style capability declarations, OpenUSD-style composition principles, MLIR-style dialect/lowering discipline.
Adapt
Retentive lenses → property/source correspondence; CRDTs → collaboration profile; WebRender/Vello/Skia → renderer boundary and conformance strategy; Tree-sitter → source-preserving adapter infrastructure.
Invent/prove
The proposed integration combines authored and resolved UI state, stable cross-tool semantic identity, opaque extension retention, fidelity accounting, bidirectional semantic patches and a native open editor whose internal state is the draft model itself.
Implementation language and runtime choice
Decision: Rust reference core
Rust is the strongest default for the reference implementation because the project simultaneously requires untrusted binary parsing, graph/document transforms, geometry, text shaping, native/WASM embedding, GPU access, fuzzing and stable C-compatible boundaries.
Alternatives
- C++ has the deepest graphics ecosystem and mature Skia/Yoga integration, but expands memory-safety risk in parsers/plugins and makes a browser/WASM-safe reference core less attractive.
- Zig offers excellent systems control and C interoperability but has a smaller mature graphics/text/schema ecosystem and less API stability for a standards reference implementation.
- Go is strong for services/tooling but weaker for low-level rendering/WASM/native GUI integration and deterministic allocation-sensitive engines.
- TypeScript is ideal at web/editor adapter boundaries but unsuitable as the only renderer/codec/reference-core implementation.
Stack
- Rust: document model, operations, layout abstraction, codec, renderer scene, conformance, WASM bindings.
- Taffy: initial CSS-compatible evaluator behind NUIF types.
- Vello/wgpu: interactive renderer experiment behind a NUIF renderer trait.
- HarfBuzz-compatible shaping: text experiment with pinned font inputs.
- Masonry + AccessKit: reference editor shell (ADR 0006, accepted; toolchain 1.98.0, MSRV 1.96); Svelte 5 + TypeScript for the later browser demonstration over the WASM bindings.
- Tree-sitter/language-native parsers: source adapters where concrete syntax retention is required.
Adapters MAY be written in the ecosystem-native language; conformance is against behavior/protocol, not implementation language.
Risk register and impossibility boundaries
Fundamental boundaries
- Rendered output is underdetermined. Pixels/boxes cannot uniquely reveal whether layout came from flex, grid, constraints, absolute positioning or runtime code. Imported foreign content must mark inferred intent.
- Arbitrary program behavior is not serializable as UI structure. NUIF does not promise to recover arbitrary JavaScript/Swift/Dart application logic.
- Text is environment-sensitive. Font files, shaping versions, fallback and rasterization can differ. Exact conformance requires pinned inputs; portability must classify substitution separately.
- Platform-native controls differ. Semantic equivalence may be possible while exact visuals/behavior are platform-specific.
- Effects/shaders can exceed a portable core. Extensions may be preserved without being renderable.
- Standard complexity can kill adoption. IFC/STEP demonstrate the cost of excessive semantic scope.
- A single reference implementation can accidentally become the spec. Independent implementation is a standards gate.
- Resource bytes carry legal and security constraints. Exact font/image preservation does not imply permission to redistribute or safe decoding.
- Visual metrics are gameable. A flat screenshot can look exact while discarding editability, semantics, accessibility and responsive behavior.
- Model confidence can be misleading. Raw probability is not calibrated correctness and cannot upgrade inferred evidence into source truth.
- Capture can leak secrets. Browser/network/accessibility observations may expose credentials, personal data or proprietary content unless collection, export, retention and training are separately bounded.
Containment strategies
- explicit fidelity reports and inference confidence;
- authored + resolved snapshots instead of pretending either alone is canonical truth;
- opaque extension preservation;
- deterministic capability/evaluation contexts;
- small core + profiles;
- reference implementation backed by normative conformance fixtures;
- fuzzing/resource budgets for all untrusted inputs;
- early independent implementation and adapter experiments.
- stable asset IDs separated from resource digests, locators and provenance;
- verify resource size/digest before decoding and never fetch implicitly;
- typed operation output for models, atomic validation and finite correction loops;
- structural/text/resource/edit-task metrics alongside visual diagnostics;
- calibrated decision-level confidence, alternatives and abstention;
- private captures default to local processing, no retention and no training.
Current implementation risks
- The macOS graphics fork is a maintenance boundary. The editor pins
refpath/xilemcommit1b96eb8; its parenteabfe0amoves the active renderer from wgpu 28 and metal-rs to wgpu 29 and the objc2 Metal bindings. The fork changes public API call sites and does not patch the Objective-C blocks ABI. Each fork update requires the editor tests, reverse dependency trace, and macOS Metal window smoke test recorded innuif:research:macos-metal-block-future-incompatibility. A separaterefpath/metal-rsmove-to-block2branch exists only for review; NUIF does not depend on the deprecated binding or that experimental patch. - Release signing is credential-bound. The editor packaging gate builds, archives and smoke-tests an unsigned host package. Platform signing and notarisation require release credentials and remain separate from source conformance.
- Portable resources are only narrowly implemented. The deterministic package, RGBA8 PNG and static single-face TrueType subsets have executable gates, while CPU profile 0 remains unchanged. RFC 0010 cannot be accepted until broad media/font matrices, the configured Linux/Windows/macOS jobs produce passing hosted evidence, external reproduction, calibrated aggregate budgets and interoperability review pass.
- Capture and reconstruction accuracy remains unestablished. RFC 0011 and specification 14 now have bounded fixed-input contracts, a local pinned browser fixture and a typed synthetic evaluation report. They establish interfaces, refusal rules and metric consistency, not portable browser-capture or screenshot-reconstruction accuracy. No current release makes those accuracy claims.
Thesis falsifiers
The project should rethink its architecture if ordinary source round trips require broad regeneration, if unknown extension preservation cannot survive routine edits, if the layout vocabulary becomes a vendor-property dump, or if a second implementation cannot reproduce v0 behavior from specification + fixtures alone. The resource/reconstruction path should additionally narrow if package bytes cannot reproduce across writers, correction loops improve pixels by deleting semantics, confidence cannot support useful risk/coverage, or tuned models do not beat the untuned tool-assisted baseline.
Governance and standardization strategy
NUIF starts under Refpath because research and implementation need a concrete home, but the target is neutral stewardship.
Repository governance now
- Public RFCs for semantic changes.
- ADRs for reference-implementation choices.
- Research evidence is distinct from normative requirements.
- Extension registry changes require examples and conformance fixtures.
- No vendor adapter can redefine core semantics.
Standardization path
A W3C Community Group is a plausible early venue for document/component semantics and coordination with Open UI/DTCG, but NUIF should not enter formal standards work before an executable v0 and at least one external implementer exist. Khronos/ASWF-style governance offers useful precedent for graphics/rendering and extension registries. A neutral foundation can become appropriate once multiple vendors/projects depend on the format.
IP/licensing goals
- reference code: permissive dual MIT/Apache-2.0;
- specification/schema/conformance text: permissive terms compatible with standards adoption;
- contributions: explicit patent/IP policy before standards-track claims;
- trademarks/conformance branding: separate from implementation copyright.
The project must not call itself an industry standard merely because the repository is public. Conformance, multiple implementations and neutral governance are prerequisites.
Naming and project identity
NUIF is the current working name and repository slug, not yet a cleared standards trademark.
A public reconnaissance found prior acronym use in an old Nexus User Input Framework and in a 2024 computer-vision paper, plus unrelated uses. None currently appears to occupy the same open UI-authoring interchange category, but collision risk is non-zero.
The architecture therefore separates human branding from protocol identity. Stable schema namespaces, extension IDs and version identifiers must not depend on a product trademark remaining unchanged.
Before v0.1 branding is promoted outside the research project:
- perform repository/package/domain/trademark clearance in key jurisdictions;
- decide whether
NUIFis an acronym or simply a proper project name; - reserve crate/package/extension namespaces;
- define conformance-mark governance separately from the open specification;
- make any rename before third-party persisted documents become common.
Research coverage and continuous completeness
A research repository cannot truthfully claim to contain every paper that will ever be relevant. NUIF instead defines operational completeness: every planned architectural front must have an explicit status, evidence links, unresolved questions and an experiment/decision path.
research/coverage.yaml is the machine-readable coverage contract. It maps the founding research plan to research IDs, specification modules, RFCs/ADRs, code seams and experiments. The project can therefore identify gaps structurally rather than relying on prose search to infer that a topic was forgotten.
Current state
All founding fronts are represented. The resource, browser-capture,
reconstruction-evaluation, adaptation/distillation and AI artifact-governance
fronts are now explicit rather than hidden inside “serialization” or
“inference.” Decisions that can safely be made from mature prior art are marked
covered. Questions whose answer would be premature without an implementation
are marked experiment-required. Areas whose evidence base will continuously
evolve—prior art, adapters, reconstruction and data/model governance—remain
ongoing by design.
This distinction is important: marking an open research problem as finished would be less rigorous than preserving it as a first-class graph node.
Additional boundaries from the final sweep
- WAI-ARIA and accessibility API mappings support a semantic-role/state layer
distinct from platform-specific accessibility trees. The bounded
nuif-web-accessibility-0lowering now proves computed role/name/state agreement for one eleven-node fixture across pinned Chromium, Firefox and WebKit while retaining native-platform and behavior non-claims. - KHR_interactivity provides contemporary precedent for portable,
capability-aware behavior graphs rather than arbitrary scripts embedded in
visual nodes. The smaller
nuif-behavior-state-machine-0sidecar now has exact Rust/Node traces for ordered guards, state, effects and explicit required/optional capability handling without claiming a final semantic schema. Its first wire experiment is one inert canonical-CBOR, content-addressed package resource: a Rust gate validates document binding and hostile cases while an independent Python ZIP reader checks exact container bytes. The attachment remains outside the canonicalDocumentand never grants execution authority. A separate one-way web lowering maps the bounded effects through native activation,hiddenand an ARIA status region; five events agree across pinned Chromium, Firefox and WebKit under one exact CSP-hash-authorized runtime without extending that evidence to native UI or screen-reader speech. - ReverseORC and related layout-inference work show that multiple viewport observations materially improve recovery of responsive intent.
- Screenshot-to-code research continues to show that visual reconstruction is not equivalent to recovering authored layout or behavior.
- Merkle/content addressing is appropriate for immutable assets and snapshots but not for editable semantic identity.
- EPUB OCF and OCI descriptors support a narrow manifest-driven package with size/digest verification; NUIF now has an in-repository independent-writer fixture. A three-OS CI matrix now exercises the package, image and font gates, while successful hosted evidence and external reproduction remain open.
- OpenType and Fontations evidence now supports one executable static TrueType
package baseline with a pinned HarfBuzz metadata oracle. The retired
ttf-parserdecision remains documented; broad font formats, portability outcomes and shaping/raster integration remain experiment-required. Warmed parser and packaged-validation allocation ceilings cover every accepted fixture. - Browser source capture and screenshot reconstruction are different evidence
lanes and cannot share a blanket
losslessclaim. - Current screenshot-to-code work supports OCR/region/hierarchical and render- correction experiments, not a claim that authored UI recovery is solved.
- LoRA, quantized adaptation and distillation are conditional experiment techniques; evaluation, rights-cleared traces and artifact governance precede training.
The continuous-research process should periodically re-run topic searches, append or supersede research records, and update research/coverage.yaml only when new evidence or experiments change the status of a front.
Cross-industry patterns: evidence, adoption and rejection
This document synthesizes 52 research records added on 2026-08-29 from visual-effects interchange, game-engine asset systems, programming-language research on bidirectional transformation and layout verification, distributed-systems testing, and 2D-rendering conformance practice. Each pattern below is classified as borrowed (adopted as is), adapted (adopted with a stated change) or rejected (ruled out with the reason). Record identifiers (nuif:research:*) carry the locators; this document does not repeat them.
Method
The 52 records synthesized here were reviewed from primary sources (specifications, source code at a named commit and papers with DOI). reviewed does not mean every material claim has completed locator-level verification; research/AUDIT.md defines the stricter verified state. Claims resting on one record inherit its evidence status as well as its confidence. Source conflicts remain explicit until an RFC or experiment resolves them.
Patterns
Document model and composition
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
| Opinion strength ordering over composition arcs (LIVERPS) | openusd-composition-and-crate | Adapt: NUIF needs a total, documented resolution order for library, theme, variant and instance-override opinions; six arc kinds are more than a UI document needs | spec/03-components-and-composition.md, question composition-strength |
| Flatten as an explicit, named lowering that discards composition | openusd-composition-and-crate, alembic | Borrow: flattening is a lowering with a fidelity record, never the save format | spec/00-conformance.md, docs/whitepaper/01-architecture.md |
| Authored network versus cooked output with pull-based, memoized evaluation and push-based dirtying | houdini-pdg-and-hda, hydra-render-delegate | Borrow for the evaluator: resolved snapshots are pull-evaluated per context and invalidated by hierarchical locator sets rather than global dirty bits | crates/nuif-layout, ADR 0002 |
| Prefab override as a sparse modification set against a source definition | unity-prefabs-and-yaml-merge | Borrow: instance overrides are sparse property sets keyed by stable identity and property path | spec/03-components-and-composition.md |
| Resolved-only interchange (baked samples) | alembic | Reject as a canonical form; accept as an explicit cache profile | spec/08-serialization.md |
Identity and ordering
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
| File-local numeric identity with global identity through a second key | unity-prefabs-and-yaml-merge, godot-tscn-scene-format | Reject: file-local identities orphan cross-file references on replacement; NUIF identities are global from creation | spec/02-identity-and-properties.md |
| Path-based addressing in patches | json-patch-rfc6902-and-merge-patch | Reject for entities; retain for property paths inside an identity-addressed operation | spec/06-operations-and-patches.md |
| Parent link and fractional position as one atomic property | figma-multiplayer-and-rendering-engineering | Adapt: parent and anchor move together; collaboration profiles may use list identifiers internally, while canonical operations use Start/After(id) anchors | RFC 0006, crates/nuif-protocol |
| Tree move with undo/redo of concurrent operations and cycle rejection, mechanized proof | crdt-tree-move-operation | Adapt for the collaboration profile; the canonical document keeps a totally ordered log and needs no replica metadata | spec/10-collaboration-profile.md |
| Random resource identifiers with path fallback and warnings | godot-tscn-scene-format | Borrow the fallback discipline for asset references; a resolved-by-path reference must be diagnosed | spec/09-provenance-and-fidelity.md |
Unknown data preservation and schema evolution
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
UnknownSchema keeps name, version and raw payload and re-emits it verbatim; round trip asserted by a test | opentimelineio | Borrow verbatim: this is the executable form of nuif:claim:opaque-preservation | rfcs/0002-extension-preservation.md, experiment unknown-extension-roundtrip |
| Unknown node class preserved as a placeholder that records its original class and properties, written back on save | godot-tscn-scene-format | Borrow: entities of unknown kind are preserved, not dropped, and their original kind is restored on export | spec/07-extensions-and-dialects.md |
| Unknown data ignored and not re-saved | blender-dna-rna-and-headless | Reject: this is the failure mode NUIF exists to prevent | risk register |
| Per-schema version numbers with gap-tolerant upgrade functions and a generated version manifest | opentimelineio, unreal-asset-versioning-and-automation | Adapt: each core record kind carries a version; migrations are pure functions registered per kind; reading a newer version than known is an error, not silent loss | spec/08-serialization.md, migrate command |
| Self-describing struct layout embedded in the file | blender-dna-rna-and-headless | Reject: NUIF encodings are schema-versioned, not struct-layout-described; the deterministic CBOR profile already carries structure | ADR 0004 |
| Extension prefix registry with a status ladder that requires validator support before release | gltf-validator-and-sample-assets | Borrow: EXT namespaces are promoted only with a conformance fixture and validator rule | question extension-governance |
Operations, undo and merge
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
| Every editor action is an operator with typed parameters, invocable from scripts | blender-dna-rna-and-headless | Borrow: every editor gesture lowers to a protocol operation that a script can invoke | RFC 0004 |
| Memento-based transaction snapshots | unreal-asset-versioning-and-automation, blender-dna-rna-and-headless | Reject for the canonical log; inverse operations are recorded instead, because snapshots do not commute and cannot be merged | spec/06-operations-and-patches.md |
| Undo restores expected user state under concurrent edits; undo rewrites redo history | figma-multiplayer-and-rendering-engineering, command-pattern-undo-and-event-sourcing | Adapt: the invariant “undo, copy, redo leaves the document unchanged” becomes a metamorphic relation in the operations suite | conformance/HARNESS.md |
| Structural three-way merge keyed by class and identity with declared set-valued fields and float epsilons | unity-prefabs-and-yaml-merge | Borrow: a merge-rules declaration per property kind (ordered list, identity set, scalar with tolerance) | spec/06-operations-and-patches.md |
| Tree matching becomes the identity map when stable identifiers exist; the residual problem is move and order conflicts | ast-diff-gumtree-and-structural-merge | Borrow: no heuristic matching in NUIF-native merges; GumTree-style matching is reserved for adapters without identity | docs/whitepaper/03-protocol-and-portability.md |
| Conflicts as first-class states rather than failures | patch-theory-darcs-pijul | Borrow: typed conflict objects are document state until resolved | spec/06-operations-and-patches.md |
| Operational transformation with server serialization | operational-transformation-vs-crdt | Reject as a canonical model; permissible as a collaboration profile | ADR 0005 |
Canonical encoding
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
| RFC 8949 §4.2 core deterministic encoding rules | canonicalization-rfc8785-and-cbor-deterministic | Borrow | nuif-cbor-0 |
| Float handling: CDE keeps numeric kinds while dCBOR reduces integral floats and zero | same, cbor-data-model-and-key-order-correction | Preserve NUIF’s declared integer/real kinds; canonicalize both real zeros to positive floating zero | RFC 0008 |
RFC 8785 number serialization via shortest round-trip and -0 to 0 | same | Adapt for nuif-text-0 | spec/08-serialization.md |
| Content-addressed deduplication of array values | openusd-composition-and-crate, alembic | Borrow for the package asset store; reject for editable entities | ADR 0004 |
Layout
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
Fixture tests generated from browser layout through WebDriver, compared at < 0.1 px, with structural (not numeric) handling of known divergences | taffy-and-yoga-browser-generated-tests, differential-testing | Borrow: the layout differential suite is generated, never edited by hand; divergences are classified per case | experiment layout-differential |
| Layout as SMT-solvable constraints with a visual assertion logic | cassius-web-layout-verification | Adapt: the assertion vocabulary (no overlap, containment, alignment, text fits) becomes a fixture-level oracle; the SMT encoding itself is out of scope because no formalization covers flex or grid | conformance/HARNESS.md |
| Relational constraint synthesis from multi-device examples | inferui-and-layout-synthesis | Adapt for import inference only; results are marked inferred with confidence | experiment layout-inference |
| Flexbox §9.9.1.2 placeholder and grid intrinsic-sizing divergences | css-flexbox-grid-algorithm-specs | Record: NUIF conformance cannot claim exact agreement where the CSS specification is implementation-defined; such cases are tolerance-tiered | spec/04-layout.md |
Rendering determinism
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
| Renderer as a pluggable backend behind a stable scene abstraction | hydra-render-delegate | Borrow (already ADR 0003) | crates/nuif-render |
| GPU shaders as ground truth compared by a perceptual mean | vello-testing-and-cpu-reference | Reject as the conformance oracle; borrow the threshold values for the interactive backend | conformance/HARNESS.md |
CPU f32 pipeline with tolerance 0 inside Vello’s own harness | vello-testing-and-cpu-reference, resvg-test-suite | Treat as candidate evidence, not proof of cross-platform identity; calibrate NUIF’s path with pinned assets and a CI matrix | render-tolerance experiment, ADR 0003 |
Per-test numeric and perceptual thresholds (idiff, oiiotool, WPT fuzzy, WebRender fuzzy(max,count)) | hydra-render-delegate, blender-dna-rna-and-headless, skia-gold-and-gm-tests, webrender-reftests | Adapt into three declared tiers: exact, bounded per-channel delta with pixel count, perceptual (ꟻLIP mean) | conformance/HARNESS.md |
| Reftests (two documents that must render identically) over pixel baselines | skia-gold-and-gm-tests | Borrow for equivalence-preserving rewrites | metamorphic relation class 1 |
| Scene capture to a text serialization for deterministic replay | webrender-reftests | Borrow: render scenes are serializable fixtures | crates/nuif-render |
| WGSL and WebGPU leave rounding, reassociation, sample locations and edge inclusion implementation-defined | gpu-rendering-nondeterminism | Record: GPU output is never normative | spec/05-geometry-paint-text.md |
Text
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
Shaping fixtures as glyph strings with font hash, options and expected glyph=cluster@dx,dy+adv output | text-rendering-reproducibility | Borrow the format for the text suite | experiment text-pinning |
| Hinting off, grayscale coverage, declared subpixel quantum, font SHA-256, Unicode and shaper versions pinned | same | Borrow | spec/05-geometry-paint-text.md |
Testing methodology
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
| Deterministic simulation: single-threaded scheduler, seeded PRNG, all nondeterminism behind injectable interfaces, reproduction by seed | deterministic-simulation-testing | Borrow: the trial loop is seed-driven and prints the seed on failure | conformance/HARNESS.md |
| Swarm testing (random feature subsets per run) | same | Borrow for operation generators | same |
| Metamorphic relations with tolerant equality; reduction by reversing recorded transformations | metamorphic-testing-graphics | Borrow: nine relation classes are defined in the record | same |
| ddmin over operation sequences, hierarchical reduction over the document, choice-sequence shrinking over generated values | delta-debugging-and-test-case-reduction | Borrow: three-level reducer | QA contract item 9 |
| Model-based testing with a small reference model and precondition-preserving shrinking | property-based-testing-state-machines | Borrow: proptest-state-machine over an ordered-forest model | same |
| Structure-aware fuzzing with explicit depth and allocation budgets | fuzzing-structured-inputs | Borrow: arbitrary does not bound value depth; NUIF bounds depth and node count explicitly | spec/11-security.md |
| Snapshot testing with redactions, sorted output and a single update variable | golden-master-and-snapshot-testing, libtest-mimic-and-data-driven-fixtures | Borrow: NUIF_UPDATE_EXPECT is the only regeneration switch | conformance/HARNESS.md |
| Machine-readable validation report with severity codes, pointers and per-code policy | gltf-validator-and-sample-assets | Borrow as the report schema for validate, import, export | spec/12-cli-api-and-automation.md |
| Sample-asset corpus with per-asset metadata, tags and CI validation | same | Borrow for conformance/fixtures | conformance/HARNESS.md |
Editor automation
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
Headless execution with a script (--background --python, hython, commandlets, -nullrhi) | blender-dna-rna-and-headless, houdini-pdg-and-hda, unreal-asset-versioning-and-automation | Borrow: the editor binary runs a session script without a window | apps/editor/UI-SPEC.md |
| Plugin API as a programmable surface, but no headless mode and read-mostly REST | figma-plugin-and-rest-api-as-automation-surface | Record as the gap NUIF closes; borrow pluginData-style opaque per-entity stores as adapter evidence | adapters/README.md |
| Accessibility tree as the semantic query and action surface for UI tests | accesskit-semantic-ui-testing, egui-and-egui-kittest, masonry-xilem-and-linebender-test-harness | Borrow: entity identifiers are carried in the accessibility tree; tests query by role and label and dispatch actions without pointer synthesis | ADR 0006 |
| Same-frame scene and accessibility outputs with virtual time and CPU rasterization | masonry-xilem-and-linebender-test-harness | Borrow | ADR 0006 |
| Pixel-based UI screenshot tests with per-OS thresholds | egui-and-egui-kittest | Reject as the primary editor oracle; permitted only for shell wiring | apps/editor/QA.md |
Ruled out
The following were examined and excluded from the architecture; the reason is recorded so the question is not reopened without new evidence.
- A self-describing binary struct layout (Blender DNA): solves version drift for one implementation but does not preserve data it cannot re-save and does not compose with a schema-versioned interchange model.
- Memento undo as the canonical history: does not commute, cannot be merged, and bloats logs; inverse operations are required by
spec/06. - Integer child indices in
MoveandInsertas the only order representation: non-commutative under concurrency; a list identifier is required for the collaboration profile and harmless for the canonical form. - GPU rendering as a normative oracle: implementation-defined by the WebGPU and WGSL specifications.
- Perceptual UI screenshot tests as the primary editor test: platform-dependent text rendering makes them a shell-wiring check only.
- Heuristic tree matching for NUIF-native merges: unnecessary with stable identity and a source of spurious moves.
- Whole-project regeneration as the synchronization model: contradicted by the lens and delta-lens laws that NUIF’s patch model must satisfy (
lenses-foster-boomerang,bidirectional-evaluation-direct-manipulation).
Consequences for the specification
The records imply the following changes. Items 1–3 were decided by follow-up research on 2026-08-29 and are recorded as accepted RFCs; items 4–6 remain proposals.
- Sibling order is a canonical array without keys;
InsertandMoveuse anchors (Start,After(id)); the collaboration profile maps anchors onto a Fugue-family list CRDT (RFC 0006). nuif-cbor-0follows deterministic preferred serialization while preserving integer/real data-model identity; real zero is positive floating zero, text and CBOR key orders are distinct, strict decoders reject non-canonical input, text hashes through CBOR and strings remain verbatim (RFCs 0005 and 0008).- Entities of unknown kind load as
Unknownwith typed core fields and an opaque payload; ignorant implementations preserve bytes, knowing ones may re-encode; validation severities follow the glTF pattern (RFC 0007). - Every serialized record kind carries a schema version; migrations are registered pure functions; newer-than-known versions load as
Unknownfor entities (RFC 0007) and are diagnosed for other records. - Validation, import and export reports follow one schema with stable codes, severities and pointers.
- Layout conformance declares tolerance tiers per case and classifies every divergence from a browser reference as schema loss, evaluator defect or implementation-defined behavior.
Open questions raised by this synthesis
Recorded in research/questions.yaml: cbor-float-zero (decided, RFC 0005), sibling-order-identifier (decided, RFC 0006), unknown-kind-preservation (decided, RFC 0007), editor-toolchain-msrv (ADR 0006), layout-assertion-vocabulary, render-tolerance-tiers.
Resources, capture and model-neutral reconstruction
NUIF’s next research front is not “add an AI converter.” It is a coordinated resource, capture and reconstruction architecture with explicit truth boundaries. Images and fonts must survive as verified resources. Source-backed browser imports must retain authored/resolved evidence. Screenshot-only imports must remain honest probabilistic hypotheses. Every path converges through one core operation, validation, rendering and fidelity contract.
Decision summary
The recommended direction is:
- define stable assets separately from immutable byte resources;
- make
.nuifa deterministic portable package after cross-writer proof; - grow the executable narrow PNG and static TrueType resource baselines only through named profiles and measured hostile-input budgets;
- add a pinned browser-capture adapter separate from static source sync;
- build screenshot reconstruction as a replaceable observation/proposal loop;
- freeze a structural and visual evaluation suite before training;
- consider adaptation or distillation only after the untuned loop exposes a repeatable learnable error distribution.
RFC 0010 and RFC 0011 remain proposed contracts. Their bounded package, narrow PNG/static-font and capture/reconstruction experiments are implementation evidence only for the named subsets; they are not published conformance or standards claims.
One core, two import lanes
Source-backed lane
HTML/CSS + browser execution + resource responses
-> retained source + resolved observations
-> deterministic adapter/lowering + explicit inference where needed
|
v
typed NUIF operations
|
v
core validation/apply
|
v
resource-aware NUIF
^
|
Screenshot-only lane |
pixels + context |
-> OCR/CV/grounding observations
-> hierarchy/layout/resource hypotheses
-> typed operations -> render/diff/correct
The lanes differ in evidence, not in their mutation authority. Both use the same typed operations. Neither provider can write core structs directly. A browser observation may support an exact resolved value under one pinned context, but it does not automatically reveal authored intent. A screenshot cannot establish source equivalence regardless of visual score.
Portable resource model
Four identities must remain distinct:
| Concern | Identity | Change behavior |
|---|---|---|
| editable semantic asset | AssetId | stable when its content is replaced |
| immutable encoded bytes | ResourceDigest | changes for any byte change |
| package/resolver location | locator | may change without changing bytes |
| source/derivation history | provenance record | may grow without renaming asset/bytes |
An asset points to a content descriptor containing media type, SHA-256 and byte length. A package path or external URL only locates candidate bytes; size and digest are checked before decoding. External resolution is opt-in. Opening a document never triggers network access.
Resource roles clarify hash and retention behavior:
source: original encoded bytes;authoring: exact bytes needed to evaluate/edit semantics;derived: crop, trace, selected frame, conversion or generated result with input digests and transformation identity;cache: decoded pixels, GPU textures or acceleration state that can be deleted without changing the semantic document.
This avoids a common failure: storing only a decoded bitmap and calling the source image preserved, or hashing an editable asset by its current bytes and therefore breaking every reference after replacement.
Candidate .nuif package
The proposed first package is a deliberately small deterministic ZIP profile:
mimetype
manifest.cbor
document.cbor
blobs/sha256/<digest>
mimetype is first and stored. Manifest and document are deterministic CBOR.
Every embedded blob is addressed by exact SHA-256 bytes. The first profile uses
stored members only, fixed metadata and sorted ASCII paths to make independent
writer byte equality attainable and to avoid compression-version variability.
There are three hashes:
- semantic document hash over canonical
document.cbor; - resource digest over each exact blob;
- package hash over the exact ZIP artifact.
Cache or report changes may change the package hash while leaving the semantic
document hash untouched. Bare canonical forms remain .nuif.json and
.nuif.cbor. Historical alpha .nuif raw files need read-only detection during
migration; new .nuif output becomes the package only after RFC acceptance.
The package reader rejects duplicate or unsafe paths, symlinks, directories, encryption, split archives, unsupported compression, inconsistent headers, undeclared/missing blobs and digest mismatch. It does not extract to a filesystem. Exact byte fixtures, member/resource limits and two independent writers are acceptance gates.
Package-to-session handoff uses shared immutable buffers. The release gate passes an 8 MiB resource through package, handle map and session with the same allocation pointer while keeping handoff allocator traffic and retained bookkeeping below 1 MiB. This prevents a host from paying one full resource copy merely to enter the core.
Images
The original encoded image is authoritative. A semantic image asset records its resource digest and intrinsic interpretation; each image paint records fit, crop, transform, sampling, opacity and color conversion. Derived decoded pixels and GPU textures are caches.
PNG is the correct first format because its current W3C specification covers
lossless encoded pixels, alpha and explicit colour metadata. The executable
nuif-png-rgba8-0 baseline chooses an intentionally smaller contract:
non-interlaced RGBA8, no ancillary metadata or one valid sRGB chunk, encoded
samples interpreted as sRGB, straight decoded alpha, identity encoded orientation,
declared fit/crop/sampling/opacity and bounded integer CPU composition. png
0.18.1 and zune-png 0.5.2 must emit identical RGBA bytes for the accepted
fixtures; encoded resources remain digest-identical through package edits.
This avoids pretending that decoder agreement on simple images settles PNG
Third Edition. The compatible nuif-png-basic-rgba8-1 profile now admits the
lossless-to-RGBA8 subset: 1/2/4/8-bit greyscale and indexed images, RGB8,
greyscale-alpha8, RGBA8 and valid palette/colour-key transparency. It preserves
encoded bytes and requires exact normalized RGBA agreement between both
decoders. It is separately named so profile zero never changes meaning.
Image-paint affine semantics are orthogonal to decoder choice. The executable
matrix [a c tx; b d ty; 0 0 1] maps crop-local source coordinates forward
into the fitted rectangle. The CPU reference inverse-maps destination pixel
centers, clips to the entity, and rejects singular or numerically unbounded
matrices. Flip, rotation and translation fixtures make composition order
observable; live host trials are still required for vendor interoperability.
Decoded pixels are interned once per digest/profile in the renderer-independent scene and commands carry compact deterministic handles. A 64 MiB total is preflighted before each new inflation. The release gate retains one 1 MiB surface for 1,024 image instances under 8 MiB allocated and 4 MiB retained, and rejects a 64 MiB plus 16 byte declared total before the second decode.
Any broader profile still has to pin:
- accepted chunks and metadata conflicts;
- color-space precedence and output space;
- Exif orientation;
- conversion and premultiplication points for every accepted colour signal;
- sampling and compositing;
- encoded, pixel, decoded-byte, chunk and metadata limits;
- independent decoder and malformed-input fixtures.
16-bit/interlaced PNG, CICP/ICC/gamma/chromaticity, Exif, animation, perspective/tiling and host-specific affine equivalence are not claimed. A Linux/Windows/macOS CI matrix runs the profile, but the cross-platform claim remains withheld until its hosted artifacts pass. JPEG, WebP, AVIF, video and SVG follow as separate profiles. Freezing a frame or tracing a screenshot crop is a derived approximation, not recovery of the original asset. Generative upscaling/inpainting requires an explicit user policy and cannot silently become canonical source evidence.
Fonts
Exact typography depends on exact font bytes, face/collection index, variation axes, features, coverage, shaping inputs and renderer parameters. Packaging also depends on redistribution policy.
The proposed policy states are portable, private_authoring, linked,
substituted and unavailable. OpenType fsType is preserved as machine-
readable evidence, including restricted/preview/editable/no-subsetting/bitmap
flags, but is not treated as a complete legal license decision.
The executable nuif-opentype-static-single-0 baseline accepts only one
canonically packed, checksummed TrueType-outline sfnt face at index zero.
Skrifa 0.46.2 supplies package-facing metadata after NUIF validates the
directory, ranges, packing and checksums and directly checks required sfnt and
OS/2 fields. A committed hb-info 14.4.0 capture independently checks Ahem
metrics, family, tables and Unicode coverage. Exact bytes, family names,
coverage, fsType, license expression and explicit embedding review must
agree. Package encode/decode and caller-resolved linked bytes run the same
validation. Four static TrueType fixtures are accepted, while six package
trials distinguish portable, private-authoring, linked, substituted and
unavailable outcomes. Each accepted inspection and packaged-font validation is
also measured after warmup against a 4 MiB allocator-traffic and 2 MiB retained
reference ceiling; these are implementation regressions, not format semantics.
Six additional trials retain requested identity separately from a stable font
asset, render with an available declared replacement as approximated, and
emit no text command with item-level unsupported fidelity when replacement
bytes or the font are unavailable.
This is intentionally not general OpenType support. TTC, CFF/CFF2, variable,
color, bitmap, SVG and WOFF/WOFF2 sources, historic ambiguous permission
combinations, subsetting, cluster-level fallback, arbitrary packaged-font
shaping and cross-platform raster behavior remain separate fixtures and
profiles. The configured three-OS parser/package matrix does not establish
cross-platform raster behavior. Parser acceptance and fsType do not grant
redistribution rights.
Browser capture can identify platform fonts used for a node and capture downloaded web-font response bodies. It generally cannot retrieve arbitrary local font bytes. A family/PostScript name is therefore never exact resource identity. Missing bytes produce a link, substitution or unavailable fidelity record instead of a false portable-font claim.
Source-backed browser capture
Static Tree-sitter source synchronization and live browser capture solve different problems. The former preserves source spans for a bounded authored subset. The latter runs a pinned browser to observe actual cascade, layout, fonts, resource responses, accessibility and pixels. They should correlate through provenance, not become one oversized adapter.
The first browser-capture profile records browser/protocol build, OS, viewport, DPR, page scale, locale, timezone, color/reduced-motion preferences, font environment, scroll/pseudo state, navigation identity, readiness/network policy and animation freeze. It collects:
- original HTML/CSS and stylesheet text where accessible;
- DOM snapshots including available frame/template/shadow content;
- boxes, inline text boxes, paint order and a bounded style set;
- downloaded image/font/style response bodies and hashes;
- platform-font usage and font readiness;
- accessibility tree;
- reference screenshots with exact parameters.
Canvas, WebGL, video and worklets are bounded observation surfaces; a frame can be preserved without pretending its generating program was reconstructed. Cookies, authorization headers, credentials, storage and secret form values are not exported. Scripts remain inert.
Multiple viewports and states are more valuable than one oversized capture: they constrain layout hypotheses and permit held-out responsive evaluation.
The automated nuif-cdp-live-0 segment now implements that boundary for one
loopback fixture and exact Chrome for Testing 152.0.7977.64. It starts fresh
temporary profiles, retains a structured runtime context, waits for the exact
navigation loader and declared freeze/readiness point, captures bounded
DOM/layout/background/font/accessibility/resource/PNG evidence, and replaces
opaque browser node IDs with deterministic preorder identities. Four runs at
360, 768, held-out 900 and repeated 360 px retain exactly the expected five
response bodies and repeat the narrow capture bytes. Five exercised
query/cookie/storage/authorization/header canaries are absent from serializable
capture, observations, proposals and package bytes. The two fitted viewports
beat the one-viewport freeform baseline on the held-out fixture.
The separate bounded nuif-layout-inference-0 artifact ranks row stack,
column stack, Grid, linear constraint and fixed freeform candidates using only
the 360/768 px observations. It then evaluates the untouched 900 px holdout,
where the selected constraint records 0.0626 normalized error versus 0.2918
for freeform. All alternatives and exact geometry observation identities are
retained; confidence is raw and uncalibrated, and the result remains
inferred. The trial tests a mechanism on one fixture, not general accuracy or
recovery of original authored intent.
This is a falsifiable local baseline, not the entire profile described above. Cross-browser/OS reproduction, opaque frames and response bodies, full matched-style/source correlation, canvas/video frame capture, authenticated sites and licensed real-page evaluation remain open. WebDriver BiDi is the standards-track transport to revisit as its implemented evidence surface grows; Playwright is the higher-level candidate when NUIF owns a real multi-engine matrix and can make one tool the browser-version authority.
Screenshot reconstruction
A screenshot supplies visible samples but not the unique scene graph or layout program. The recommended pipeline is:
screenshots + contexts
-> OCR and baselines
-> deterministic regions, colors, edges, repetitions and asset candidates
-> optional replaceable UI grounding
-> typed observation graph with confidence and evidence regions
-> replaceable reasoner proposes hierarchy/layout and NUIF operations
-> core validates and applies atomically
-> deterministic layout/render
-> text/structure/geometry/resource/visual differences
-> bounded corrective operations
The model emits typed operations, not an unconstrained full document or code to execute. Invalid and stale transactions fail without partial state. High- resolution full views, overlapping tiles and semantic crops share explicit coordinate transforms; duplicate or conflicting observations remain visible.
The result contains a valid document or no-result, accepted operation log, observations, derived resources, item fidelity, alternatives/abstentions, evaluation report and exact pipeline artifact identities.
Evaluation before training
The benchmark has separate synthetic-exact, licensed real screenshot and source-backed suites. Synthetic NUIF rendering provides exact entities, properties, operations and resources. Real images need human-reviewed visible targets and must preserve ambiguity. Source-backed cases evaluate retained bytes and observations unavailable to the screenshot-only route.
Required metrics include:
- valid operation/document rate;
- OCR region recall, character/word error and baselines;
- element precision/recall and hierarchy error;
- property/geometry accuracy;
- held-out viewport behavior;
- exact resource digest only when bytes exist;
- provenance/fidelity honesty;
- accessibility evidence where justified;
- raw pixels, FLIP, SSIM and pinned LPIPS diagnostics;
- calibrated confidence, abstention and risk/coverage;
- latency, peak RAM/VRAM, iterations and cost.
No pixel score is sufficient. A page-sized screenshot can be visually perfect and semantically useless. Structural/text/resource/edit-task metrics prevent that reward shortcut. Dataset splits group by origin, template, component, font, resource and generator; near duplicates cannot cross splits.
The executable nuif-reconstruction-corpus-manifest-0 turns that rule into a
bounded audit. It pins the data snapshot, dataset card, evaluator and every
input/target by digest; records public/restricted/withheld disclosure and
evaluation/calibration/adaptation/redistribution permission independently; and rejects
exact artifact or declared family reuse across adaptation, calibration,
validation and test. Private/authenticated records also require explicit
authorization and a withdrawal-policy artifact. This is declaration integrity,
not automated legal review or duplicate discovery; real records and their group
assignments still need independent human/tool review.
The ablation ladder is deterministic OCR/CV, one-shot reasoner, observation- assisted reasoner, hierarchical crops, multi-viewport ranking, correction loop, then any tuned or distilled student. Every addition uses the same frozen holdout and budget.
Adaptation and distillation
Training is justified only after the untuned loop and error taxonomy are reproducible, rights-cleared traces exist, and the remaining errors appear learnable. Training examples contain input hashes, observation versions, proposals, diagnostics, accepted operations, intermediate renders/differences and final package/fidelity reports. Positive sequence targets are validated accepted transitions, not raw model transcripts.
Compare prompt/schema/tool improvements and retrieval before fine-tuning. If adaptation remains justified, compare ordinary supervised tuning, low-rank adaptation and quantized low-rank adaptation under equal data and evaluation. Quantized adaptation is a memory technique, not an accuracy claim.
Sequence-level distillation may train a smaller student from the best evaluated teacher pipeline. The teacher is a measured system of tools plus a model, not a provider name. Distillation transfers errors too, so render validation and held-out evaluation remain mandatory.
Models, processors, adapters and datasets are separately versioned optional
artifacts with digests, model cards, dataset datasheets, license lineage and
training manifests. They never redefine nuif-core or travel as ordinary
document resources.
The executable provider boundary uses a deliberately small canonical wrapper, not a new AI bill-of-materials vocabulary. It binds NUIF capabilities, execution modes and observation/proposal profiles to exact implementation, model, processor, adapter, quantization, prompt and tool artifacts. Observation bundles carry the complete manifest registry, so a digest cannot dangle and a proposal cannot substitute an unpublished provider before mutation. Released or learned providers point to content-addressed SPDX 3.0.1 or CycloneDX 1.7 inventory; learned providers also point to a model card. This complements runtime packaging such as MLflow or ONNX external data without making either a required NUIF dependency.
Private/authenticated captures default to local processing, no retention and no training. Remote transfer, telemetry, retention and training are independent consent/policy decisions.
Maturity boundary
The current 0.1.0-alpha.3 label belongs to the developer editor application.
It provides no evidence that the broad image/font resource, portable browser
capture or screenshot reconstruction accuracy profiles are complete. A
deterministic package, narrow PNG/static-font segments, fixed provider-input
contracts and one pinned local live-browser segment are implemented, but their
deliberately narrow evidence does not promote the broader profiles.
Promotion requires the package/resource cross-writer fixtures, pinned capture reproduction, baseline/closed-loop/calibration harness, leak-resistant licensed evaluation data, independent result reproduction and at least one real edit workflow that benefits from the inferred semantics. Until then the work is research and proposed specification text, not a standard or production reconstruction promise.
Primary research records
nuif:research:resource-packaging-and-source-capture-synthesisnuif:research:model-agnostic-screenshot-reconstruction-and-trainingnuif:research:provider-artifact-manifests-and-ai-bomsnuif:research:epub-ocf-package-containernuif:research:oci-resource-descriptorsnuif:research:opentype-font-embedding-and-portabilitynuif:research:ttf-parsernuif:research:fontationsnuif:research:chromium-source-backed-ui-capturenuif:research:live-chromium-cdp-capturenuif:research:design2code-real-world-benchmarknuif:research:pix2struct-screenshot-parsing-pretrainingnuif:research:screenai-ui-annotationnuif:research:confidence-calibration-and-selective-predictionnuif:research:lora-low-rank-adaptationnuif:research:qlora-quantized-adaptationnuif:research:sequence-level-knowledge-distillation
NUIF — Neutral User Interface Format
Authored intent, resolved state, stable identity and explicit loss accounting in one portable document model for user interfaces.
Citation metadata is provided in CITATION.cff. The current
software citation identifies the latest published editor prerelease. It does
not cite the draft specification as an accredited standard.
Problem · Scope · Architecture · Repository · Method · Status · Contributing · License
Problem
Interface designs move between design editors, design systems, source frameworks and automation tools by regeneration. Each export flattens authored constraints, components, token bindings and provenance into pixels, a vendor scene graph or generated code; each import guesses them back. Three consequences follow: round trips lose information, the loss is silent, and entity identity does not survive the trip, so later edits cannot be synchronized as patches.
NUIF treats portability as a synchronization problem. One canonical document keeps the authored intent and the resolved evaluation state side by side under stable identifiers, records every lossy mapping as a typed fidelity class, preserves data it does not understand, and exposes every mutation as a replayable semantic operation. Editors, runtimes and adapters become peers of the document rather than owners of it.
Scope
| NUIF is | NUIF is not |
|---|---|
| A draft specification for a vendor-neutral authored-interface document model: identity, containment, relationship graphs, layout intent, resolved geometry, paint, text, components, tokens, behavior, provenance, extensions | A Figma clone or a Figma file format; vendor formats are adapters |
| A Rust reference engine (model, protocol, layout, render scene, codecs, query, headless API and CLI) that falsifies the specification | A renderer specification; GPU command streams are implementation detail behind a scene boundary |
| A reference test editor that replicates a conventional design-editor layout and authors NUIF state directly | A product editor; its feature set is fixed to what conformance testing, import and export require |
| An executable conformance kit: fixtures, operation replay, layout matrices, reference rasterization, fidelity diagnostics | A claim that pixels uniquely recover an original program, resource or behavior |
| A proposed model-neutral observation/reconstruction contract using typed operations, validation and explicit inference provenance | A required AI model, provider, training framework or promise of screenshot-lossless import |
| A machine-readable research corpus with claims, questions and experiments | A standard; that status requires conformance profiles, neutral governance and independent implementations |
Architecture
flowchart LR
subgraph Clients
CLI[CLI]
Editor[Reference test editor]
WASM[Browser and plug-in WASM]
QA[Automated conformance clients]
end
subgraph Engine["Rust reference engine"]
API[Headless API]
Proto[Operations, transactions, patches]
Doc[(Canonical document<br/>containment tree + relationship graphs<br/>authored intent)]
Eval[Layout evaluation<br/>per evaluation context]
Res[(Resolved snapshot<br/>boxes, shaped text, diagnostics)]
Scene[Render scene]
Codec[Codecs<br/>canonical text, deterministic CBOR]
Query[Query and diagnostics]
end
subgraph Peers["Adapters (peers, not parents)"]
HTML[HTML/CSS, Svelte, React]
SVG[SVG]
Design[Penpot, Figma]
Native[SwiftUI, Compose, Flutter]
end
Ref[Renderer backends<br/>Vello/wgpu, CPU reference]
Conf[Conformance suites<br/>fixtures, replay, reference images]
CLI --> API
Editor --> API
WASM --> API
QA --> API
API --> Proto --> Doc
Doc --> Eval --> Res --> Scene --> Ref
Doc <--> Codec
Doc --> Query
Codec <--> Peers
Peers -- fidelity records + correspondence --> Doc
Ref --> Conf
Res --> Conf
Proto -- replay log --> Conf
Principles that the diagram encodes:
- the canonical document is the only owner of authored state; every client, including the editor, mutates it through semantic operations;
- resolved state is derived per evaluation context and never overwrites authored intent;
- identity is semantic and independent of path, order and geometry;
- adapters emit fidelity records (
lossless,representable,approximated,preserved_unrenderable,unsupported) and correspondence records; silent loss is a conformance failure; - unknown extensions survive transit byte-for-byte;
- collaboration is a profile above canonical documents, not part of them;
- the headless API and CLI are the primary test surface; GUI automation is supplementary.
Repository
| Path | Content |
|---|---|
docs/whitepaper/ | Research synthesis, architecture thesis, risk register, cross-industry patterns |
research/ | Evidence records, claims, open questions, experiment registry, coverage contract, schema |
spec/ | Draft normative modules (model, identity, components, layout, paint/resources/text, operations, extensions, serialization/package, provenance, collaboration, security, automation, semantics, observation/reconstruction) |
rfcs/ · adrs/ | Proposals for the specification · decisions for the reference implementation |
crates/ | Rust workspace: model, protocol, layout, pinned text shaping, render, codecs, query, API, CLI, WebAssembly binding and shared seeded testing |
apps/editor/ | Reference test editor: architecture, source installation, headless contract, UI specification |
conformance/ | Suite plan, test-harness architecture, fixtures including the v0 falsification experiment |
adapters/ · schemas/ | Adapter contracts and interchange schemas |
tools/ | Research graph ingestion, validators, commit lint |
Method
Research is structured data. Each record in research/items/ has a stable identifier, a primary source with locators, a confidence value, typed relations to claims and other records, and links to the specification, decisions, code and experiments it informs. research/coverage.yaml maps every architectural front to its evidence and status (covered, experiment-required, ongoing), so gaps are detected structurally. Claims become normative only through the RFC process and an executable conformance fixture.
Testing is designed for automated trial loops: generate or load a document, apply operations, export and import through adapters, compare canonical forms, resolved layouts and reference images, minimize failures, and record a machine-readable report. The harness design is in conformance/HARNESS.md.
Status
| Surface | Maturity | Boundary |
|---|---|---|
| Reference editor | Research preview; current release 0.1.0-alpha.3 | Semantic Versioning applies to the editor application only |
| Draft specification | Pre-draft | No normative conformance profile is published |
| Executable adapter and conformance profiles | Experimental | Results apply only to each declared profile and evaluation matrix |
| WebAssembly binding | Experimental nuif-wasm-api-0 | Byte-oriented text/CBOR, validation, patch and history parity; no host authority or browser-layout claim |
| Direct Rust SDK | Experimental nuif-api façade | Package-aware load/validate/apply/export over one core; no stable C ABI or crates.io publication claim |
| Package/resources | Experimental implementation; Gate I incomplete | Deterministic package plus narrow independently parsed RGBA8 PNG and static TrueType resource paths are executable; a three-OS CI matrix is configured, while broad media/font matrices, external reproduction and successful hosted matrix evidence remain open |
| Capture/reconstruction | Experimental contracts and deterministic baselines | Browser/screenshot normalization plus one pinned local live-CDP segment, typed proposals, calibration primitives and a finite loop exist; no portable capture, broad accuracy or model claim |
| Behavior runtime | Experimental nuif-behavior-state-machine-0, content-addressed nuif-behavior-package-resource-0 transport and nuif-web-behavior-0 lowering | Package/CBOR binding, Rust/Node traces and three-engine browser effects pass; behavior remains outside the canonical semantic Document, native UI and assistive-technology evidence |
| Project | Open research project; not a standard | Standards status requires neutral governance and independent implementations |
Gates B through H are complete under the bounded, quantified criteria in research/AUDIT.md. The workspace executes structural validation, anchored atomic operations, replay/inversion, canonical text and deterministic CBOR, measured hostile-input limits, responsive profile-0 layout, bounded explicit fixed/fr Grid tracks and placement, exact CPU rasterization, pinned NUIF/Taffy/Chrome layout trials, seeded reports and headless and native-shell editor drivers. Gate C covers sparse row/column flow, explicit placement and spans without a schema-loss exemption. Gate D pins shaping, outlines, hard-line layout, encoded-sRGB paint and integer composition; scene and raw-RGBA hashes reproduce on macOS/aarch64, Linux/aarch64 and Linux/x86_64, while PNG encoding is non-normative and paths, images, instances and extension paint remain property-attributed fidelity records. Seven retentive source/package adapter profiles are integrated across HTML/CSS, SVG 2, DTCG 2025.10, Penpot v3 packages, static React JSX and static Svelte, with import, export, synchronization, hostile-input checks and CLI conformance; Svelte additionally passes a pinned official-compiler oracle. An eighth executable profile covers normalized Figma Plugin API snapshot and mutation-plan mapping plus a compiled no-network review shell, without claiming live host execution. The ninth is a one-way bounded HTML/ARIA accessibility projection whose computed role/name/state surface agrees across pinned Chromium, Firefox and WebKit test engines. The tenth maps the bounded behavior sidecar to native button activation, hidden visibility and an ARIA status region through one CSP-hash-restricted finite runtime, with event-by-event agreement in the same three engines. A machine-audited inventory separates these executable profiles from four researched or externally bounded targets. Complete fixture authoring, AccessKit-driven deterministic GUI trials, standard-library-only Python v0 reproduction, metadata-free register and existing-tree collaboration checkpoints, a pinned Automerge operation-transport oracle, hostile editor interaction trials, a scaling benchmark suite, native host packaging and cross-checked WebAssembly and MCP developer surfaces are automated. The native shell exposes the complete model-backed profile-zero editing surface while leaving future-profile sections of the draft UI specification explicit; concurrent entity creation and collaboration garbage collection, a foreign tree materializer, a general-purpose second implementation, signed native distribution and external interoperability review remain incomplete.
Gates I through L are not complete. The package segment of Gate I is now
executable: .nuif is a deterministic bounded package, stable assets and exact
resources have distinct identities, resolution is explicit, and the CLI/editor
preserve package resources. cargo xtask gate-i-package records cross-writer,
fixpoint, identity, resolver and hostile/one-over evidence.
cargo xtask gate-i-image independently decodes the narrow
nuif-png-rgba8-0 subset, preserves exact encoded resources and repeats
resource-aware CPU rendering. cargo xtask gate-i-font compares the narrow
nuif-opentype-static-single-0 subset across two Rust parser families, enforces
exact package metadata and explicit embedding review, and rejects malformed and
one-over resources. Gate I still lacks the broader PNG and OpenType matrices,
external writer and successful hosted cross-platform package/media evidence.
The CI workflow now runs all three narrow resource gates independently on
Linux, Windows and macOS and archives each platform report; that configuration
does not become reproduction evidence until the hosted jobs pass. RFC
0011/specification 14 have executable
provider-neutral observation with canonical artifact-manifest registries,
browser/screenshot baseline, typed-proposal,
flat-copy rejection, calibration and finite-loop primitives. The local Gate J
segment now drives pinned Chromium, retains exact resources/font/accessibility
evidence, excludes exercised secret canaries and beats a one-view baseline at a
held-out viewport. Gate J still requires portable cross-OS/browser and broader
source evidence; Gate K still requires reconstruction accuracy and an
independent evaluator. No LoRA/QLoRA artifact or distillation is implemented. The editor’s
0.1.0-alpha.3 version is not maturity evidence for these research fronts.
Specifications remain drafts and no standards conformance profile is published.
The separate bounded web-accessibility gate projects portable semantics to inert
HTML/ARIA and compares computed role, name and state across exact Playwright
Chromium, Firefox and WebKit engines; it does not claim native platform or
application-behavior equivalence.
The bounded behavior gate first binds one canonical program to a deterministic
package resource and independently inspects its ZIP bytes, then executes the
same stable-identity state machine through Rust and Node runtimes. The separate web-behavior gate
lowers that trace contract to native button clicks, hidden visibility and an
ARIA status region through one exact CSP-hash-authorized runtime and compares
every event in pinned Chromium, Firefox and WebKit. Timers, internal events,
numeric computation, networking, arbitrary scripts, native UI and
assistive-technology speech remain excluded.
Run the automated baseline:
cargo xtask all # installs/reuses the pinned browser and runs every gate
cargo xtask browser-install # optional browser prefetch
cargo xtask gate-wasm # pinned-browser initialization and Node/native byte parity
cargo xtask wasm-package # downloadable direct-browser developer archive
cargo xtask gate-mcp # current stateless stdio protocol and native byte parity
cargo xtask mcp-package # live-tested host developer archive
cargo xtask gate-web-behavior # three-engine native web behavior lowering
cargo xtask gate-behavior-package # inert package attachment and independent ZIP checks
cargo xtask cli-package # exercised standalone developer-tool archive
cargo xtask trial 24301 100
cargo xtask gate-b # 10,000 patches; raster sample every 100 patches
cargo xtask hostile-inputs # boundary/one-over time and allocator report
cargo xtask reduction-profile # hierarchical/choice reduction and fixture evidence
cargo xtask editor-hostile-inputs # semantic, parser and snapshot rejection report
cargo xtask fuzz-smoke # bounded AddressSanitizer campaigns over five core surfaces
cargo xtask adapter-audit # research/profile/gate coverage for all advertised targets
cargo xtask diagnostic-audit # public model/layout/trial diagnostic registry drift
cargo xtask performance # portable budgets plus one execution of every Criterion path
cargo xtask codec-benchmark # codec size, latency, allocation and admission evidence
cargo xtask gate-c # NUIF/Taffy/pinned-Chrome layout report
cargo xtask gate-d-text # HarfBuzz golden shaping + separate raster report
cargo xtask editor-trial # author the v0 fixture and emit editor evidence
cargo xtask editor-gui-trial # exercise AccessKit and reproduce shell pixels
cargo xtask editor-package # build, smoke-test and archive the host application
cargo xtask editor-launch # package and open the native application
cargo xtask editor-install --user --channel source # persistent local build
cargo xtask editor-doctor --user # verify receipt, binary and integration
cargo xtask editor-update --user --channel alpha --check # resolve only
cargo xtask editor-update --user --channel alpha # verified explicit update
cargo xtask editor-rollback --user # reactivate the retained previous build
cargo xtask editor-uninstall --user # remove only marked managed paths
cargo xtask gate-f # retentive HTML/CSS subset synchronization
cargo xtask gate-f-v0 # full-v0 model sync plus editor/CLI source bridge
cargo xtask gate-g # independent Python v0 parse/write/layout/render
cargo xtask gate-h # exhaustive collaboration register convergence
cargo xtask gate-i-package # deterministic package/resource evidence
cargo xtask gate-i-image # independent RGBA8 PNG/resource/render evidence
cargo xtask gate-i-font # independent static TrueType/resource/policy evidence
cargo xtask gate-figma # pure Plugin API snapshot/plan mapping (not a live host)
cargo xtask gate-behavior # package binding plus deterministic Rust/Node behavior traces
cargo xtask capture-baselines # bounded capture/reconstruction contract evidence
cargo xtask gate-j-live # pinned live Chromium/resource/secret/held-out evidence
cargo xtask gate-accessibility # three-engine computed accessibility evidence
cargo run --locked -p nuif-cli -- fixture v0-responsive-card /tmp/v0.nuif
cargo run --locked -p nuif-editor -- --headless \
--script conformance/fixtures/v0-responsive-card/editor-trial.jsonl \
--document /tmp/v0.nuif --output /tmp/edited.nuif
cargo run --locked -p nuif-editor # launch the native editor
cargo xtask all bootstraps the pinned Python research-validator environment, wasm-bindgen toolchain and Chrome for Testing under ignored target/, then runs research validation, Rust verification, WebAssembly/native API parity, the short full-raster trial, the 10,000-patch Gate B trial, release-mode hostile-input, codec-decision and performance trials, the Gate C differential layout trial, both Gate D text/render trials, complete headless/native editor trials, bounded retentive adapter bridges, the bounded Rust/Node behavior differential, the independent Gate G reproduction, exhaustive Gate H collaboration-register convergence, the Gate I package, narrow-image and narrow-font segments, and the bounded capture/reconstruction contract report. Each measured run leaves a JSON report or snapshot under target/; target/verification-manifest.json indexes the complete evidence set and records success or the first failed step. CI archives both the individual evidence and this manifest.
Contributing
Research, specification, conformance, implementation and adapter contributions are accepted; see CONTRIBUTING.md for the record schema, the writing register and the single-line commit rule. Governance is described in GOVERNANCE.md; security reports in SECURITY.md.
License
Reference code is dual-licensed under Apache-2.0 or MIT (LICENSE-APACHE,
LICENSE-MIT). The repository has not adopted specification-wide copyright or
patent terms. docs/whitepaper/08-governance-and-standardization.md records the
licensing requirements that precede standards-track publication.
Research and implementation roadmap
The evidence gates and quantified acceptance criteria are normative for project planning in research/AUDIT.md; these phases describe implementation order only.
Phase 0 — foundation (complete)
Exit: research graph schema, architectural RFCs, compilable core seams, CI and v0 falsification fixture exist.
Phase 1 — canonical model (complete)
Implement typed properties, relations, components, tokens, extensions, deterministic IDs, validation and operation replay. Exit: structural conformance suite and canonical hash stability.
Phase 2a — responsive layout falsifier (complete)
Implement freeform + stack/flex subset and a pinned NUIF/Taffy/Chrome context matrix. Exit: responsive-card layout agreement at three viewports, measured per-fixture bounds and classification of every generated divergence.
Phase 2b — bounded explicit Grid (complete)
Profile 0 now defines positive fixed/fr tracks, sparse row/column auto-flow,
zero-based explicit placement, positive spans, no implicit tracks and bounded
resource use. The independent NUIF evaluator implements those rules directly;
Taffy and CSS are lowering targets, not hidden runtime dependencies. Gate C
exercises simple, explicit and spanning Grid cases and passes with no classified,
blocking or unexplained divergence. Intrinsic, percentage, named, repeated,
subgrid, masonry and implicit CSS tracks remain outside this bounded profile.
Phase 3 — visual/text (complete for profile 0)
Pinned Ahem/HarfRust shaping matches HarfBuzz glyph goldens; unhinted Skrifa 0.46.2 outlines match normalized hb-vector goldens; hard-line layout, rectangles, ellipses, encoded-sRGB color and integer composition have normative scene/raw-RGBA baselines across macOS/aarch64, Linux/aarch64 and Linux/x86_64. PNG hashes remain deterministic artifact diagnostics but are not pixel-conformance boundaries. Path, image, instance and extension paint remain explicit unsupported/preserved fidelity rather than hidden fallbacks. Full UAX #14 soft wrapping and expanded vector paints belong to a future profile.
Phase 4a — bare serialization/protocol (complete for profile 0)
Canonical text + deterministic CBOR plus patch/diff/query CLI. Exit: byte-stable
cycles and measured hostile-input limits. The separate active codec decision
gate records four-scale size, latency, allocation, canonicalization and
decode-then-select evidence. Both implemented codecs pass opaque-data edit
preflight; schema candidates are not timed on partial models. Cap’n Proto is
the next candidate only after a complete mapping. This phase does not include the
portable .nuif package, images or general font resources.
Phase 4b — portable package and resources (active; container segment implemented)
RFC 0010 now has a package layer above nuif-codec, stable assets in the core,
explicit verified resource resolution and package-preserving CLI/editor I/O.
The manual writer and an independent ZIP writer produce identical bytes;
semantic/resource/package hashes obey distinct fixtures; hostile archives and
package/resource/count one-over cases are blocking through
cargo xtask gate-i-package. Existing raw .nuif inputs migrate read-only and
new bare forms use .nuif.json/.nuif.cbor. The executable
nuif-png-rgba8-0 baseline independently decodes a deliberately narrow PNG
subset. The separately named nuif-png-basic-rgba8-1 expansion now covers
non-interlaced lossless-to-RGBA8 greyscale, indexed, RGB, greyscale-alpha and
RGBA forms plus valid transparency; both retain encoded bytes and repeat
package-aware CPU image rendering through cargo xtask gate-i-image. Gate I
remains open for 16-bit/interlaced/colour-managed PNG and live host/GPU affine
equivalence. Package/session handoff and decoded image surfaces now have
measured sharing, total-byte and allocation ceilings. Static-font inspection
and packaged validation now have warmed allocation ceilings across every
accepted fixture. A Linux/Windows/macOS resource-gate matrix is configured;
successful hosted artifacts are still required before a cross-platform
reproduction claim. The
separate nuif-opentype-static-single-0 baseline validates one exact static
TrueType face through package encoding/resolution, compares Skrifa results with
a pinned HarfBuzz metadata capture and rejects malformed/policy/one-over cases through
cargo xtask gate-i-font. TTC, CFF/CFF2, variable/color/bitmap/WOFF2 fonts,
cluster-level fallback, arbitrary packaged-font shaping, successful hosted
cross-platform evidence and external implementations remain open. Stable
text-to-font asset bindings now distinguish requested, replacement and
unavailable identities with six blocking layout/render fidelity trials.
Phase 5 — editor (complete for the headless profile-0 instrument)
The entire v0 fixture is authored from an empty document through identity-addressed semantic actions. Direct generation, editor output and operation replay are byte-identical, and the editor writes canonical document, context, layout, scene, CPU raster and fidelity report artifacts. The Rust-native Masonry shell from ADR 0006 and the later Svelte/WASM demonstration are non-normative interface work and cannot redefine this headless result.
Phase 5b — native editor research preview (complete through alpha.3)
The native shell exposes the semantic driver through identity-backed canvas selection, a file menu with canonical and declared adapter import/export routes, document-aligned background grid and pixel rulers, layer and component browsing, insertion tools, evaluation widths, zoom, inspector transactions, bounded explicit Grid authoring and source-built developer installation. Captured pointer movement for freeform children previews locally and commits one semantic position operation on release; it snaps to whole pixels by default and supports Control-suspended snapping. Stack/Flex drags infer the effective responsive axis from resolved siblings and commit one same-parent Move; unchanged order creates no history, while Grid, Constraint, cross-parent and instance-child cases fail closed. Freeform selections expose eight handles; managed-layout children expose the three trailing handles. Resize previews resolved geometry and atomically commits the changed fixed axes plus an anchored freeform position when required; Shift preserves corner aspect ratio and invalid, root or semantically ineffective paths fail closed. Grid track, flow, atomic item position and span edits use the same validated operations as the headless and accessibility surfaces. Open packages pass their digest-verified embedded resources through the same bounded session used by CLI render/snapshot, so the narrow RGBA8 image segment renders without implicit fetching. cargo xtask editor-gui-trial, cargo xtask editor-hostile-inputs and cargo xtask editor-install-trial exercise the semantic, visual, adversarial and lifecycle boundaries. The broader apps/editor/UI-SPEC.md remains a draft; multi-selection, persisted aspect-ratio constraints, object smart guides, cross-parent/tree drag, Grid/Constraint reorder and managed leading-edge resize, token authoring and expanded paint are not claimed by this phase.
Packages declaring capabilities outside the editor’s explicit empty support set open structurally but read-only. The driver, accessibility surface and changed-package save boundary return the exact missing set, while an unmodified copy stays byte-identical. This conservative boundary is included in the editor hostile-interaction gate.
Phase 5c — browser and plug-in binding (complete for nuif-wasm-api-0)
The byte-oriented WebAssembly module wraps nuif-api, canonical text/CBOR,
deterministic packages, explicit package-capability negotiation and semantic
patches without copying the model into JavaScript. A Node/native differential
checks exact edited bare and package bytes, packaged-resource preservation and
typed missing-capability failures. Structural requirement-bearing packages are
read-only until complete-set authorization, including semantic patches and
mode conversion; the direct-browser target initializes its package API in
pinned headless Chrome. Its JavaScript, TypeScript and WASM are
packaged as a CI and tagged-release developer artifact. The module declares no
filesystem, network or host-document authority. The Figma review shell now
compiles against pinned official typings and crosses a mock snapshot into the
Rust core, but its assigned-ID live-host run remains separate. Browser-layout
execution, a WASI CLI, npm publication and live Figma, Affinity and Canva
adapters remain
separate profiles and version streams.
Phase 5d — external agent binding (complete for nuif-mcp-tools-0)
The MCP process is a stateless stdio adapter over the same API and semantic patch layer. Its four inline-text tools carry no host authority, support only the current 2026-07-28 lifecycle, and are differentially checked against the native CLI through a real child process. Five native release jobs package and attest the separately versioned binary; source installation remains available without an application store. Live compatibility with named third-party MCP hosts, large-document resource handles and any authenticated HTTP service are separate trials and are not claimed by this phase.
Phase 5e — direct SDK and foreign binding boundary (direct SDK complete)
nuif-api::NuifDocument is the package-aware, byte-oriented façade over the
canonical codecs, verified package/resources and typed session operations.
Text/CBOR load, validation, transaction application, hashes, undo/redo and
bare/package export have one implementation; the WASM binding delegates to it
and the system benchmark suite measures direct text, CBOR and package calls.
The façade separates inert structural package access from session
authorization: requirement-bearing packages reject evaluation, mutation,
history and mode conversion until exact complete-set negotiation succeeds.
The unified performance gate executes every Criterion path once and audits
per-profile adapter direction coverage; controlled benchmarks include package
capability negotiation and all ten integrated adapter profiles without treating
shared CI timing noise as a regression threshold.
No stable C ABI is claimed while the semantic API remains 0.0.x. ADR 0011
requires a separately reviewed unsafe nuif-ffi boundary, stable ownership and
error contracts, cbindgen header/symbol checks, sanitizer-backed C consumers,
pinned UniFFI Swift/Kotlin consumers and real XCFramework/AAR packages before
that surface becomes integrated. This is a promotion gate, not missing logic
that should be guessed into the core.
Phase 5f — standalone developer CLI package (complete locally)
nuif-cli-tools-0 is packaged for the same Linux x86-64, Linux AArch64,
Windows x86-64, macOS Apple Silicon and macOS Intel release matrix as the native
tools. Each archive contains the binary, licenses, developer instructions and a
smoke report produced by that release binary. The gate requires version and
capability identity, then generates, validates, canonicalizes and inspects a
real profile-0 document before the archive can be indexed. Sibling manifests
bind the source revision, platform, binary/archive digests, command inventory
and explicit filesystem/standard-stream authority; the release index records
the packages under tools and includes a separate CycloneDX SBOM. Local
macOS/AArch64 packaging passes. Successful hosted jobs and attestations remain
release-time evidence, not a claim made from workflow configuration.
The CLI declares an empty extension-capability support set. Capability-bearing packages remain available for structural inspection, bare extraction and exact copying, while evaluation, external-format conversion, semantic package rewrites and package-mode changes fail with the exact requirement set. Native package import/export preserves resources and manifest requirements.
Phase 6a — first adapters/sync falsifier (complete for bounded HTML/CSS profile 0)
nuif-html-css-0 maps a declared container/text/finite-token subset through real DOM/CSS syntax with byte-span correspondence. Text, token and four-edge padding edits change only their six spans; comments and unmapped markup survive exactly; unsupported semantics have target/property fidelity. HTML/CSS was intentionally tested before SVG because Gate F and the architecture stop condition concern minimal source patches. This narrow profile remains independently automated even after the full-v0 follow-on; arbitrary HTML/CSS and SVG remain broader adapter work.
Phase 6b — full-v0 HTML/CSS sync (complete)
nuif-html-css-v0 carries the complete responsive-card model through 181 retained correspondences. The full trial applies eight local token/padding/text/responsive edits while preserving all other source bytes and opaque payloads; the editor bridge applies name and width edits through semantic actions and the public CLI, then re-imports to byte-identical canonical NUIF. Browser path rendering, instance materialization and unknown visuals remain explicit target limitations.
Phase 6c — bounded SVG sync (complete)
nuif-svg-0 maps a fixed surface, freeform groups, rectangles, ellipses and literal pinned-font text to SVG 2 XML. The trial applies seven identity, geometry, paint, text and accessibility edits through 45 retained correspondences, preserves unmarked XML, and rejects scripts, external resources and unsupported SVG geometry before synchronization.
Phase 6d — bounded DTCG sync (complete)
nuif-dtcg-scalar-0 maps flat boolean, string and number tokens to the Design Tokens Format Module 2025.10. Namespaced metadata retains NUIF document and token identity and distinguishes integer from real values; the trial applies eight edits through 21 correspondences while preserving unknown extension bytes. Groups, aliases, composite types and token-local extensions require a token-model RFC and a separate profile.
Phase 6e — adapter inventory (complete for advertised targets)
adapters/index.json enumerates twelve advertised targets. The blocking adapter audit requires a primary research record, integration surface, next bounded profile and exclusion boundary for every target; executable entries additionally require a crate, profile document and routed conformance gate. Ten profiles are integrated: the seven retentive HTML/CSS, SVG, DTCG, Penpot, static React JSX and static Svelte profiles; nuif-figma-plugin-snapshot-0; the one-way nuif-web-accessibility-0 projection; and the one-way nuif-web-behavior-0 host lowering. The Figma profile proves normalized mapping, CLI parity and static compilation of its no-network shell, not plug-in execution in Figma. Affinity, Canva, SwiftUI, Jetpack Compose and Flutter remain explicitly researched or externally bounded rather than carrying unsupported implementation claims. Affinity is a user-mediated SVG bridge until a public API exists; Canva is a stable-API current-page app profile with Connect and marketplace claims kept separate. Svelte uses Tree-sitter only for retained spans and exact official svelte/compiler 5.57.0 as its foreign parse/compile oracle.
Phase 6f — bounded web accessibility projection (automated)
nuif-web-accessibility-0 lowers ten roles, role-specific Boolean states and
five stable-identity relationships to inert native HTML/ARIA. It rejects
unsupported roles and state combinations, ambiguous direct/relationship names,
unnamed labels and duplicate relationships before output. The foreign oracle
pins Playwright 1.62.1 and its Chromium, Firefox and WebKit engines, then
compares computed role, accessible name and every admitted Boolean state for
eleven entities. The first macOS/arm64 run has identical full snapshots across
Chromium 151.0.7922.34, Firefox 153.0 and WebKit 26.5. Native platform APIs,
keyboard/focus traces, application behavior and broader semantic value types
remain separate work.
Phase 6g — bounded behavior state-machine sidecar (automated)
nuif-behavior-state-machine-0 executes stable-entity activate events through
one flat deterministic state machine. Ordered guarded transitions, sequential
Boolean/string actions, visibility/announcement effects, required capability
refusal and explicit optional no-op degradation have complete traces. The Rust
reference and independently written Node interpreter agree for both capability
sets over the five-event fixture. RFC 0012 now carries the same program as one
canonical-CBOR, content-addressed source resource under
nuif-behavior-package-resource-0 without adding it to the semantic
Document. The package gate proves document/package hash separation, exact
round trip, hostile refusal and independent Python ZIP inspection; generic
package decode remains inert, the SDK reports exact missing package
requirements before a full-support claim, and runtime effect authorization
stays separate. Timers, internal events, numeric computation,
navigation, animation, networking, scripts, native effects and browser effects
beyond the following projection remain separate profiles and wire-design work.
Phase 6h — bounded web behavior projection (automated)
nuif-web-behavior-0 composes the behavior sidecar and accessibility
projection without accepting authored JavaScript. Enabled native button/switch
clicks select the same transitions as the reference runtime; visibility uses
hidden, and one advisory announcement per transition uses an unfocused polite
status region. Delimiter-safe program data is interpreted by one finite runtime
authorized by an exact CSP hash. Separate pointer and alternating Enter/Space
keyboard sequences pass all five events’ state, transition, retained visibility
and announcement comparisons in Playwright
Chromium 151.0.7922.34, Firefox 153.0 and WebKit 26.5 on macOS/arm64. Checkbox,
radio, disabled-control, focus, navigation, animation, screen-reader speech,
native UI and arbitrary script remain explicit exclusions.
Phase 7a — collaboration property registers (complete)
nuif-collab-registers-0 keeps causal metadata outside canonical documents and materializes concurrent register-like semantic operations through operation-set and replica-log algorithms. Every delivery of the three-replica trial converges, and distinct concurrent values remain explicit property conflicts.
Phase 7b — bounded existing-tree structural collaboration (complete)
nuif-collab-tree-0 implements move, reorder, trash deletion and later rescue for identities already present in one canonical base. Unique Lamport ordering plus cycle rejection preserves one-parent/acyclic structure; RGA-style stable origins preserve deterministic sibling order without putting clocks, tombstones or position IDs in canonical NUIF. Move/move, delete/move, deleted-parent, delete/descendant-move, cycle and anchor conflicts remain explicit. Two materializers converge over all 5,040 deliveries of a fixture that includes a causal moved-position anchor, plus a 4,096-change scale trial. Pinned Automerge 3.4.1 reproduces the exact immutable operation set under three merge orders, duplicate merge and save/load; it is a foreign transport oracle, not an independent implementation of the tree algorithm. Concurrent creation, causally stable garbage collection, combined property/structure transactions and external tree-materializer reproduction remain future profiles.
Phase 8a — mechanical independent reproduction (complete for v0 profile 0)
The standard-library-only Python implementation reads, writes, lays out and rasterizes the v0 profile without importing, invoking or linking the Rust packages. Its differential trial is exact at 360, 768 and 1,440 pixels and stays in the unified CI loop.
Phase 8b — external reproduction and standards review
Package the schema/conformance kit and obtain reproduction by an externally authored implementation. External provenance, interoperability review, neutral governance and a published conformance profile remain prerequisites for credible standards status; the in-repository mechanical reproduction and source adapter do not establish them.
Phase 9a — canonical research publication (complete)
cargo xtask docs-check compiles the repository Markdown into one machine-readable catalog. cargo xtask docs-build renders that catalog without a second editable documentation source. cargo xtask docs-paper composes the thirteen canonical whitepaper modules into a working technical manuscript and a verified PDF. Pull requests build retained artifacts, while default-branch workflow runs deploy the static site through GitHub Pages. CITATION.cff describes the tagged alpha.3 software release; no DOI or peer-review claim is present.
Phase 9b — implementer draft and incubation (blocked on external evidence)
Meet the implementer-draft gate in docs/STANDARDS-ROADMAP.md, including a general-purpose externally maintained implementation, requirement-to-test traceability, legal review of specification and patent terms and organizational supporters. Venue selection follows the resulting scope: W3C for Web and design-tool incubation, Khronos for graphics/content-tool conformance, or OASIS for a governed document protocol. Application alpha versions do not advance this phase.
Phase 10 — source-backed browser capture (active; local live segment automated)
Create a dedicated browser-capture adapter instead of expanding the retentive
Tree-sitter adapter into a runtime. Pin browser/protocol/OS/context and collect
bounded source, DOM/layout/style, downloaded-resource, font-use, accessibility
and screenshot observations. Exit: repeated normalized observations/resource
hashes reproduce; multi-viewport evidence predicts a held-out context; canvas,
video, cross-origin and local-font gaps remain explicit; secret canaries never
enter exported evidence. cargo xtask capture-baselines proves repeatable
normalization, exact resource retention, query-secret redaction, typed proposal
application and cycle rejection from fixed provider input. cargo xtask gate-j-live additionally drives exact Chrome for Testing 152.0.7977.64 with at
most three recorded fresh-profile attempts per viewport. It records a structured runtime context, retains exactly the
five declared response bodies, observes actual downloaded-font and
accessibility results, repeats 360 px bytes exactly, excludes exercised query,
cookie, storage, authorization and custom-header canaries, and beats the 360 px
freeform geometry at held-out 900 px using the 360/768 px observations. The
separate target/layout-inference-report.json ranks row stack, column stack,
Grid, linear constraint and freeform alternatives without consulting the
holdout, selects the constraint candidate, and records its 0.0626 normalized
held-out error against 0.2918 for fixed freeform. This one fixture is an
executable falsifier, not an accuracy distribution or proof of authored intent. Cross-OS/browser
reproduction, opaque and cross-origin fixtures, complete matched-style/source
correlation, canvas/video frame handling and real licensed pages remain open,
so the broader phase does not yet exit.
Phase 11 — screenshot reconstruction baseline (active contract baseline)
Implement the vendor/model-neutral observation and typed-operation boundary from RFC 0011/specification 14. Compare deterministic OCR/CV, one-shot proposal, observation-assisted proposal, hierarchical crops, multi-viewport ranking and a bounded render/difference correction loop. Exit: one harness reports validity, text, element, tree, geometry, resources, held-out layout, provenance, visual, confidence, latency and memory/cost; flat screenshot copies fail the editable profile; an independent evaluator reproduces the main result.
The executable baseline currently proves observation-codec fixpoints, explicit
observed/inferred evidence and omissions, typed atomic proposals, default
flat-copy rejection, deterministic loop termination and training-only ranking
of five bounded layout hypotheses against a live held-out viewport. The
correction loop now has explicit success, no_improvement, and
repeated_state outcomes, with a caller-provided objective threshold so a
perfect score is never assumed by the core. The typed
nuif-reconstruction-evaluation-0 report now covers every required per-example
family, preserves empty denominators as unscored, rejects screenshot-only
source-resource recall claims and keeps unavailable hardware measurements
explicit. Its deterministic synthetic fixture validates the evaluator
contract, not reconstruction quality. The missing OCR/model baselines, licensed
real held-out corpus, independently reviewed group/near-duplicate assignments,
predeclared FLIP thresholds/viewing
contexts, statistical uncertainty method and independent evaluator keep the
phase open. The pinned test-only LDR-FLIP implementation proves that exact and
perceptual diagnostics can coexist and refuses implicit alpha handling; its one
synthetic local error is not threshold calibration. A deterministic three-example fixture exercises typed
distribution aggregation, including micro/macro separation and explicit
missingness, without presenting it as empirical accuracy evidence.
The typed corpus manifest and audit now pin snapshot/card/evaluator/artifact
digests, separate disclosure and allowed-use policy, and reject exact or
family-level leakage across all four partitions. Its four synthetic policy
records prove the auditor, not the existence, legality, coverage or independence
of a real corpus. The provider-manifest gate now binds every browser,
screenshot, OCR, proposal and correction identity to canonical manifest bytes,
requires that observation bundles carry the complete bounded registry and
rejects dangling proposal identities before mutation. Released/learned
fixtures require external SPDX 3.0.1 or CycloneDX 1.7 inventory identity, and
learned fixtures require a model card. Synthetic digests and source-bundle
development providers prove the contract only; no released model or accuracy
result exists.
Phase 12 — calibration and conditional adaptation (calibration primitive active; adaptation blocked on Phase 11)
Calibrate decision-level confidence and establish review/abstain risk thresholds on disjoint data. Only if a stable learnable error distribution remains, create rights-cleared validated operation traces and compare prompt/tool changes, retrieval, supervised tuning, LoRA, QLoRA where compatible and sequence-level distillation. Exit: a candidate beats the untuned closed-loop baseline under the same frozen holdout/budget without validity, calibration, privacy, licensing or maintenance regression. Training is skipped if that gate is not met.
The current interpolation/selective-review fixture and
cargo xtask confidence-calibration smoke test exercise only typed evaluator
arithmetic, disjoint split enforcement, shifted holdouts and selective-review
policy. They are not evidence of calibrated risk coverage on real data or of a
production threshold.
Early falsifiers
Stop/rethink if: semantic model requires pervasive vendor-specific exceptions; opaque extensions cannot survive common operations; source synchronization routinely requires whole-file regeneration; independent implementation cannot reproduce normative layout/visual behavior from the spec; deterministic packages do not reproduce across writers; reconstruction optimizes pixels by discarding semantics; or tuning cannot beat the untuned tool-assisted baseline fairly.
Standards-development roadmap
NUIF remains a pre-standard research project with a draft specification and
reference implementation. The editor version 0.1.0-alpha.3 identifies an
application prerelease. It does not establish specification stability,
interoperability, resource-package conformance, reconstruction accuracy or
external review.
Current publication boundary
The repository can publish source, generated documentation, schemas, conformance fixtures, implementation reports and a citable technical manuscript without joining a standards body. This stage uses the project code licenses and current contribution process. It does not create specification-wide patent commitments.
The following work is executable without external approval:
- publish the default-branch documentation through GitHub Pages;
- publish tagged source and native editor prereleases through GitHub Releases;
- expose
CITATION.cffthrough GitHub’s repository citation interface; - compile a technical manuscript from the canonical whitepaper modules;
- validate schemas, frontmatter, links, adapter coverage and conformance reports in GitHub Actions;
- archive a later public release with Zenodo after its GitHub integration is enabled by an authorized organization account.
GitHub adds a repository citation control when CITATION.cff exists on the
default branch. Zenodo can ingest public GitHub releases after an account owner
connects and enables the repository. Each release receives a DOI. Locators:
GitHub Docs, “About CITATION files” and “Referencing and citing content”;
Zenodo, “Enable a repository” and “Archive a release from GitHub”, retrieved
2026-08-30:
https://docs.github.com/en/repositories/managing-your-repositorys-settings-and-features/customizing-your-repository/about-citation-files,
https://docs.github.com/en/repositories/archiving-a-github-repository/referencing-and-citing-content,
https://help.zenodo.org/docs/github/enable-repository/ and
https://help.zenodo.org/docs/github/archive-software/github-upload/.
Implementer-draft gate
An implementer draft requires all of the following evidence:
- one bounded model and serialization profile whose normative requirements are internally consistent;
- a versioned schema and conformance suite mapped to every normative requirement;
- two independently maintained implementations that pass the same profile;
- at least one foreign-system round trip that preserves declared opaque data and reports every loss;
- security, privacy, accessibility and internationalization review records;
- a compatibility and deprecation policy for profile revisions;
- an extension namespace and registration process;
- specification copyright, patent, contribution and trademark terms reviewed by qualified counsel.
The present repository has an independent Python profile-zero evaluator and ten executable adapter profiles, including a normalized Figma mapping and compiled no-network review shell that explicitly exclude live host behavior, plus bounded three-engine web-accessibility and finite web-behavior projections. The behavior program also has one deterministic content-addressed package transport with an independent ZIP reader; it remains outside the canonical semantic document and is not a second complete implementation. These results cover bounded subsets. They do not constitute two independent implementations of the complete draft.
RFCs 0010, 0011 and 0012 are proposed research inputs. Their package, resource, behavior-attachment, capture and reconstruction profiles are not prerequisites for a small core implementer draft unless the selected charter includes them. If included, each requires its own independent implementation/evaluator report; a model demo or editor alpha does not satisfy interoperability.
Incubation gate
Standards-body incubation begins only after the implementer-draft gate and evidence of external participation. Venue selection follows the demonstrated adoption surface.
| Venue | Entry condition | Suitable scope | Current disposition |
|---|---|---|---|
| W3C Community Group | proposer plus four supporters; no participation fee | Web authoring, browser semantics, design tokens and Web APIs | Preferred incubation path if Web and design-tool stakeholders participate |
| Khronos New Initiative | Board-reviewed proposal, industry sponsors and later member participation | graphics, 2D/3D content tools, GPU-adjacent interchange and a trademarked conformance program | Alternative if graphics-tool vendors become the primary implementers |
| OASIS Open Project | contributor agreements, project sponsors and governed specification advancement | document packages, APIs and protocols with a path to OASIS and international approval | Alternative if protocol and package governance dominate |
| Ecma Technical Committee | General Assembly formation and at least three supporting members for new work | mature multi-company software-platform specifications | Deferred until member organizations request a committee |
| Community Specification | contributor agreement, scope, notices and specification license | repository-based multi-party specification incubation | Legal framework candidate before or alongside an organization-hosted process |
W3C Community Group reports are not W3C Standards. Khronos detailed Working
Group design requires membership. Ecma and OASIS formal paths require member or
sponsor governance. These conditions are process boundaries rather than
technical quality rankings. Primary locators are recorded in
research/items/standards-development-venue-comparison.md.
Formal-advancement gate
Formal advancement requires a stable chartered scope, recorded consensus, public review, resolved intellectual-property terms, interoperable implementations and conformance results. The selected body defines the exact ballot, review and patent procedures. NUIF cannot claim accreditation before that process completes.
The corresponding repository evidence includes:
- immutable specification snapshots and versioned conformance profiles;
- implementation reports linked to exact source revisions;
- test-suite coverage for every normative requirement;
- issue dispositions for security, privacy, accessibility and internationalization reviews;
- a public list of implementers, exclusions and withdrawn participants;
- release and errata procedures that distinguish compatible corrections from new feature levels.
Vendor adoption
Figma, Affinity, Canva and other hosts can evaluate the draft without replacing their internal document models. API hosts map a declared NUIF profile through a supported plug-in or service boundary. Affinity currently uses a separately declared file-interchange trial because no stable public document API was found. The headless CLI and WASM binding remain the semantic test surfaces. Vendor- private state is either represented, preserved as declared opaque data or reported through structured fidelity diagnostics.
The current Figma mapping and static shell, the Affinity SVG bridge draft and
the Canva Apps SDK draft describe feasible boundaries but do not include their
required live vendor-runtime trials. A vendor adoption claim therefore requires
a signed test fixture, host-version matrix, import/export or transaction report
and maintainer outside the reference-core implementation. Canva marketplace
approval and native NUIF Connect support are separate upstream outcomes. The
integration boundary is specified in docs/HOST-INTEGRATION.md and ADRs 0008
and 0012.
Decision authority
Technical evidence can identify a preferred venue and automate every repository-side prerequisite. It cannot accept patent obligations, sign a contributor agreement for another entity, create independent implementations or record stakeholder consensus. Those acts remain attributable to their human or organizational participants.
Contributing
NUIF accepts research, specification, conformance, implementation and adapter contributions.
Research contributions
Add a stable research record under research/items/ using the repository schema. Prefer primary sources; include retrieval date, source version/commit where available, confidence, claims and explicit graph relationships. Do not silently replace conflicting evidence.
Specification contributions
Use an RFC for semantic/protocol changes. Every new normative behavior should include or identify a conformance fixture. Vendor-specific behavior belongs in an adapter or extension unless it demonstrates a broadly portable primitive.
Code contributions
Keep the Rust workspace formatted and warning-free. Core crates must remain independent of editor UI and vendor adapters. New parsing/rendering paths must document untrusted-input/resource-limit considerations.
Writing register
All persisted prose (documents, specification, research records, comments) uses the technical register defined in .claude/skills/research-register/SKILL.md: established terminology from the source field, no marketing language, no invented names for known concepts, a locator for every non-obvious claim. The glossary in .claude/skills/research-register/references/terminology.md lists preferred terms.
Commits
A commit message is exactly one line: <type>: <subject> with type in docs, research, spec, rfc, adr, feat, fix, test, refactor, perf, build, ci, chore; imperative mood; lower-case first letter; no trailing period; at most 72 characters. No body, no trailers and no attribution of tools, models or assistants. Enable the local hook with git config core.hooksPath .githooks; CI runs the same check (tools/git/commit-lint.sh).
Run tools/research/validate.sh after editing research/.
Governance
NUIF is currently an experimental Refpath-hosted open research project, not an accredited standard.
Maturity surfaces
- The reference editor is a research preview. Its
0.1.0-alpha.Napplication version does not version the draft specification. - The specification is pre-draft. No conformance profile is published as normative.
- Executable adapter profiles are experimental and apply only to their declared subset and evaluation matrix.
- A tag freezes one research instrument revision; it does not certify the architectural thesis or vendor interoperability.
Decision surfaces
- Research records collect evidence and may contradict one another.
- RFCs propose normative model/protocol changes.
- ADRs select reference-implementation techniques and do not automatically define the draft specification.
- Specifications become normative only when their status explicitly says so.
- Conformance fixtures are the executable interpretation of normative requirements.
Change process
Semantic changes require an RFC with motivation, alternatives, compatibility/loss analysis, security considerations, conformance changes and evidence links. Implementation-only changes may use an ADR when they establish a durable architectural choice.
Neutrality
Figma, Penpot, HTML/CSS, Svelte, React, Flutter, SwiftUI, Compose and future systems are adapters/implementers, not privileged sources of truth.
The long-term goal is neutral stewardship once the project has an executable stable profile and at least two independent implementations.
Pre-standard boundary
Standards-track publication requires an operative specification license, royalty-free patent commitments, contribution terms, named editor and maintainer roles, consensus and appeal procedures, a conflict-of-interest and antitrust policy, trademark rules and conformance-mark governance. The current MIT and Apache-2.0 terms license reference code; they do not establish those specification-development terms.
Security policy
NUIF treats documents, assets, extensions and adapter inputs as untrusted.
Binary and text decoders, archive decompression, fonts, images, vector paths,
shader and effect extensions, GPU allocations and collaboration inputs are
security-sensitive surfaces. The normative threat model is
spec/11-security.md.
Supported releases
NUIF has no stable release. The latest editor prerelease receives security corrections when a report affects its declared profile. Earlier prereleases and unpublished source revisions do not receive a separate support period. This policy does not change the experimental status of the draft specification or its conformance profiles.
Private reporting
Report a suspected vulnerability through GitHub private vulnerability reporting. Include the affected revision or release, input profile, reproduction steps, observed result and expected boundary. Attach a minimized input when disclosure of that input does not create additional risk.
Do not publish exploitable parser, renderer, installer or updater details in a public issue before a coordinated correction is available. Non-sensitive bugs and already-public dependency advisories may use the public issue tracker.
Maintainers triage reports in the private advisory, determine the affected profiles and releases, prepare regression coverage and coordinate disclosure with the reporter. A correction is not complete until its hostile-input or regression case is exercised by the repository harness. Credit is recorded when requested and when disclosure does not expose private information.
GitHub documents the private reporting and advisory workflow in Privately reporting a security vulnerability, retrieved 2026-08-30: https://docs.github.com/en/code-security/security-advisories/working-with-repository-security-advisories/privately-reporting-a-security-vulnerability.
Code of conduct
Contributors are expected to keep technical discussion professional, evidence-based and focused on the work. Harassment, threats, discrimination and deliberate disruption are not acceptable.
Disagreement over architecture, standards evidence or implementation choices is expected. Critique claims and code directly; do not turn technical disagreement into personal attacks.
Maintainers may moderate or remove participation that repeatedly violates these expectations.
Documentation publication
Repository Markdown is the sole editable documentation source. Files remain
beside the implementation, specification module, decision or research record
that they describe. docs/catalog.json defines site membership and navigation
without copying document bodies.
Local commands
The documentation compiler is part of xtask:
cargo xtask docs-check
cargo xtask docs-build
cargo xtask docs-serve
cargo xtask docs-paper
docs-check validates catalog paths, unique document identifiers, required
frontmatter, file budgets and relative links. Every repository Markdown source
must be published or named in the catalog’s explicit exclusion list, so a new
document cannot silently disappear from the site. The command writes the
machine-readable catalog and report under target/. docs-build stages
Markdown with generated navigation and invokes mdBook 0.5.4. The static site is written to
target/docs-site. docs-serve rebuilds the staging tree and starts the local
mdBook server. docs-paper builds the site and prints the generated technical
manuscript to target/docs-site/downloads/nuif-research-manuscript.pdf through
the repository’s pinned Chrome for Testing binary.
docs-paper keeps Chrome’s process sandbox enabled by default. A disposable,
externally isolated CI runner that cannot create Chrome’s Linux namespace may
set NUIF_CHROME_NO_SANDBOX=1; do not use that override for routine local
rendering.
The pinned renderer can be installed through:
cargo xtask docs-setup
cargo xtask browser-install
Generated navigation, search indexes and HTML are build artifacts. They are not committed.
The manuscript body is generated from the whitepaper modules listed in
docs/catalog.json. The composition metadata does not copy chapter text.
Publication does not imply peer review or specification maturity.
Metadata boundary
Specification modules, RFCs, ADRs and research records require YAML
frontmatter. docs/schema/document.schema.json defines the common identifier,
kind and status fields. Research records retain their additional schema in
research/schema/research-item.schema.json.
The compiler accepts a maximum of 2 MiB per Markdown file and 64 KiB per frontmatter block. It uses typed YAML deserialization with duplicate-key rejection and parser resource budgets. YAML metadata does not define the NUIF interchange syntax.
Hosted publication
The Pages workflow builds the same staged source on pull requests and on the
default branch. Pull requests retain a build artifact and do not deploy. A
default-branch build uploads target/docs-site as the GitHub Pages artifact and
deploys through the github-pages environment. GitHub documents custom Actions
workflows for generators other than Jekyll and for repositories that do not
want compiled output on a publication branch. Locator: GitHub Docs,
“Configuring a publishing source for your GitHub Pages site”, “Publishing with
a custom GitHub Actions workflow”, retrieved 2026-08-30:
https://docs.github.com/en/pages/getting-started-with-github-pages/configuring-a-publishing-source-for-your-github-pages-site.
The workflow pins every action to a full commit. It grants read-only repository
contents to the build job. Only the deployment job receives pages: write and
id-token: write; it does not receive source or release write permission.
GitHub Wiki is not a publication target. GitHub stores each wiki in a separate Git repository, so enabling direct edits would create a second source history. GitHub also documents restricted search-engine indexing for wiki content and recommends GitHub Pages for public documentation. Locator: GitHub Docs, “Adding or editing wiki pages” and “About wikis”, retrieved 2026-08-30: https://docs.github.com/en/communities/documenting-your-project-with-wikis/adding-or-editing-wiki-pages and https://docs.github.com/en/communities/documenting-your-project-with-wikis/about-wikis.
GitBook bidirectional synchronization is not enabled. Its Git Sync can modify
SUMMARY.md and synchronize edits from GitBook to the connected branch. That
mode would add another authoring surface. Locator: GitBook documentation,
“Git Sync” and “Configuration”, retrieved 2026-08-30:
https://gitbook.com/docs/getting-started/git-sync
and
https://gitbook.com/docs/getting-started/git-sync/content-configuration.
Immutable records
The Pages site represents the default branch and is not an immutable
specification release. Git tags, release notes and source archives remain the
versioned record. A later citable research release can add CITATION.cff, a
Zenodo concept DOI and a version DOI without changing the canonical Markdown
source. Publication prerequisites and venue constraints are recorded in
research/items/scholarly-publication-and-citation-workflow.md.
Host integration and vendor adoption
Figma and Canva use NUIF through small host adapters, not by embedding the native NUIF editor. Affinity is currently a file-interchange and foreign-runtime target because no stable public Affinity document-object or scripting API was located in the reviewed official material. In every case the product continues to own its canvas, document and undo lifecycle. NUIF maps only a declared profile and emits evidence describing what was exact, approximated, preserved or unsupported.
The architectural decision is ADR 0008; ADR 0012 sets the current vendor
priority and evidence boundaries. Primary evidence is recorded in
nuif:research:figma-plugin-and-rest-api-as-automation-surface and
nuif:research:affinity-interchange-and-adoption and
nuif:research:canva-apps-and-connect-adoption.
Deliverables for a host
A production host integration consists of five independently reviewable parts:
- A bounded profile that lists mapped host object kinds and properties, resource limits, unsupported semantics and identity rules.
- Pure import/export mapping functions covered by checked-in host snapshots and expected canonical NUIF documents.
- For API hosts, a thin plug-in shell for file selection, host permission prompts, undo and user-visible fidelity confirmation. For interchange-only hosts, a documented user-mediated trial and exact input/output artifact set.
- A
nuif-adapter::HostAdapterReportfor every direction. The report records the host/API version and revision, canonical hash, host-object correspondences, property fidelity and preservation result. - A host-specific package or interchange kit, version stream and release gate. These artifacts do not inherit the native editor’s version.
The portable contract is canonical NUIF plus the report. Rust is optional for a vendor implementation. A TypeScript or JavaScript plug-in may implement the same mapping directly from the specification and conformance fixtures.
Rust hosts use the package-aware nuif-api::NuifDocument façade documented in
docs/SDK-AND-BINDINGS.md. Browser plug-ins use the thin WASM wrapper over that
façade. A stable C/Swift/Kotlin ABI is deliberately not claimed during the
0.0.x semantic-API phase; ADR 0011 defines the native-binding promotion gate.
Browser binding boundary
nuif-wasm-api-0 packages parsing, validation, canonical text/CBOR,
deterministic .nuif load/export, explicit manifest-capability negotiation and
bounded semantic patch/history operations for browser and JavaScript consumers.
It retains verified embedded resources without exposing a second mutable
JavaScript model. The generated module has no filesystem, network, Figma,
Canva or Affinity authority.
For Figma, the module belongs in the UI iframe, where Figma documents normal
browser APIs including WebAssembly. The plug-in main thread still owns
SceneNode access and exchanges bounded messages with that iframe. Canva apps
also run in an iframe; Canva’s current content-security policy admits packaged
WebAssembly while blocking third-party scripts, nested frames and workers. A
Canva build must therefore bundle the NUIF module with the reviewed app and let
the Apps SDK own every design mutation. Affinity has no equivalent documented
embedding boundary, so its first profile invokes the existing SVG adapter
outside the product and uses user-mediated import/export. The WASM module can
validate, edit and re-encode complete NUIF packages locally, but it is not a
host adapter and cannot justify a vendor fidelity claim by itself.
Local package evaluation
A Rust host that opens a package passes only its verified embedded resources to the in-process session:
#![allow(unused)]
fn main() {
let package = NuifPackage::decode(bytes)?;
package.require_capabilities(&host_capabilities)?;
let session = Session::with_resources(
package.document.clone(),
package.embedded_resources(),
)?;
let snapshot = session.snapshot(&context)?;
}
Inspection or extraction tools may instead call capability_report and retain
unknown required resources without claiming full support. Structural decode
never executes a capability and does not authorize a semantic rewrite: a tool
must negotiate the complete set or explicitly detach the package before
migration. A rendering or behavior host must pass the exact declared supported
set before it presents the package as fully evaluated.
Session::with_resources rechecks every SHA-256 binding and enforces count,
single-resource and total-byte limits before it can render. It grants no linked
resource or network authority. Package/session handoff shares immutable byte
buffers, so cloning a package or opening a render session does not duplicate the
complete embedded-resource payload. The CLI and reference editor use this path,
so opening a package containing a nuif-png-rgba8-0 or
nuif-png-basic-rgba8-1 image resolves it locally;
a bare document or unresolved link continues to emit item-level fidelity.
Hosts that need authenticated or remote resources keep that policy outside the
session, resolve explicitly, verify against the descriptor, and then create a
new bounded session.
The release allocation trial hands an 8 MiB embedded buffer from package to
session with the same allocation pointer; map/session construction allocates
under 1 MiB and retains under 1 MiB. Scene lowering separately stores one
decoded surface for repeated image uses and enforces a 64 MiB unique decoded
surface total before inflation. These are regression ceilings measured by
cargo xtask gate-i-package and cargo xtask gate-i-image, not promises about
an arbitrary host allocator.
Figma path
The executable pure-mapping profile is
adapters/figma/SNAPSHOT-PROFILE.md; the live-host promotion contract remains
adapters/figma/PROFILE-DRAFT.md.
Figma Plugin main thread
├─ reads/writes current PageNode and SceneNode objects
├─ owns host mutation and undo grouping
└─ exchanges typed messages with
Figma UI iframe
├─ user selects or downloads .nuif/report files
└─ parses/serializes the bounded profile
nuif-figma implements the normalized JSON boundary between those two
threads. It maps one visible, opaque, fixed-size frame subtree to canonical
NUIF and produces a deterministic mutation-plan tree in the other direction.
adapters/figma/plugin compiles the thin main-thread and iframe shell against
the pinned official typings. The release-mode gate covers exact round trips,
identity repair, unsupported-property evidence, hostile bounds, static message
validation, a no-network bundle and a TypeScript fixture imported by Rust. It
does not execute figma.create*, font loading, page loading, undo or plug-in
messaging in Figma; those remain live-host promotion evidence.
The manifest template targets only figma, requires dynamic page loading and
declares no network domains. Figma assigns the required plug-in ID; a reviewer
passes that assigned value at packaging time, and CI never substitutes a fake
ID. The shell only reads the selected subtree on the current page. Import
validates a precomputed plan, stays disabled until confirmation, removes nodes
created by a failed attempt and commits success as one undo step. Export does
not mutate the file.
Host node IDs are recorded in correspondence evidence. A shared nuif
plug-in-data namespace may carry document/entity identifiers for
synchronization, subject to the 100 kB entry limit. Because Figma does not
document every copy/duplicate persistence case, the bridge must scan imported
identifiers, replace duplicates and report the repair. A changed plug-in ID
must not be used as the only identity store because private plug-in data becomes
inaccessible under another ID.
The REST API is suitable for authenticated read snapshots and server-side validation, not the primary write path. The writable path is the user-run Plugin API. Dev Mode plug-ins are read-only and are not the import target.
Affinity path
The first Affinity contract is adapters/affinity/PROFILE-DRAFT.md. The
all-new Affinity combines vector, photo and page-layout tools in one no-cost
desktop product, which makes live interchange trials accessible to contributors.
That product position is useful evidence for adoption priority; it does not
create a public plug-in API or disclose the native .af, .afdesign,
.afphoto or .afpub encodings.
Profile 0 is consequently a user-mediated SVG bridge:
canonical NUIF
-> nuif-svg-0 export + fidelity report
-> user imports SVG into a named Affinity version
-> user exports SVG
-> nuif-svg-0 import + host trial report
The bridge accepts only the existing nuif-svg-0 basic-shape and pinned-text
subset. An Affinity-exported SVG containing paths, transforms, effects, CSS,
external resources or other excluded SVG features is rejected or reported as
unsupported; it is never silently simplified. Native Affinity files remain
opaque evidence artifacts. Canva’s Connect API accepting Affinity file
extensions proves a supported Canva ingestion route, not that their schema is
public or suitable for a NUIF parser.
No headless automation, stable identity persistence, undo integration or exact native round trip is claimed. Promotion requires named desktop versions on each supported operating system, retained input/output files, screenshots or renders, a complete fidelity report and a second-person review. A future public Affinity scripting or document API would justify a separately versioned host profile; undocumented UI automation and native-format reverse engineering do not.
Canva path
The first programmable profile is adapters/canva/PROFILE-DRAFT.md and uses
the stable Apps SDK Design Editing API. It is deliberately limited to one
unlocked, fixed-dimension current_page session and the supported group, rect,
shape and rich-text element kinds. Images are media fills on rectangles in the
Canva model. Canva Docs, unbounded pages, tables, embeds, video, unsupported
elements, unavailable fonts and preview-only APIs are outside profile 0.
The app reads the page snapshot, translates it through a bounded normalized
schema, validates the complete candidate and then calls sync once. This makes
one confirmed NUIF import one host undo action and avoids partial mutation.
Sessions expire after one minute, locked pages and elements remain untouched,
and every unsupported property appears in the host report. The app never
replaces the entire design with an opaque app element or raster screenshot.
Canva Connect APIs are a secondary off-platform workflow. They support OAuth
imports of listed foreign formats and asynchronous exports to formats such as
PDF, PNG, JPG, PPTX, GIF, MP4, CSV and HTML. The current import list includes
Affinity files but not NUIF. Therefore SVG/PDF may be used as explicitly lossy
bridges, while exact NUIF import/export requires either the Apps SDK mapping or
future native application/nuif+zip support from Canva. Connect download URLs
are temporary and API scopes, rate limits, user authorization and server-side
privacy obligations remain outside the core.
The first public app uses only generally available APIs. Canva documents that preview APIs may change without versioning and prevent public review. CI can build and test a source bundle, but a public release still requires developer verification, source upload, listing and testing material, Canva review, and an explicit owner-triggered release. A team app is not the default open-source distribution path because Canva limits team apps to Enterprise teams.
Conformance gate
A host profile becomes integrated in adapters/index.json only when all of
these pass:
- checked-in host input and expected canonical NUIF output;
- exact repeated import/export results for the declared subset;
- a valid host report with non-empty host/API/profile identity;
- one fidelity entry for every mapped or excluded authored property;
- stable correspondence after reorder and reopen where the host supports it;
- duplicate/missing identity repair cases;
- maximum and limit-plus-one document, node, text and metadata inputs;
- cancellation and atomic-failure tests that leave the active host document unchanged;
- one native-host trial recording product version and package version.
Credential-free CI tests mapping functions, the compiled shell and snapshots. Live-host CI or manual certification supplies the final product/version evidence. Passing static shell tests does not justify a live integration claim.
Release operation
The native editor, nuif-wasm developer binding, Figma review shell, Affinity
interchange kit and Canva app are versioned independently. CI should build
review bundles, checksums, provenance, an SBOM where applicable and fixture
reports. Publication to a vendor marketplace is never inferred from a Git tag:
it requires the vendor account, assigned app identifier, identity and legal
disclosures, review forms, and an explicit authenticated release operation.
SDK and language-binding boundary
nuif-api is the ergonomic in-process SDK over the authoritative model,
codecs, package, operation, layout and render crates. It is intentionally a
façade rather than another implementation. The CLI, reference editor,
WebAssembly module and MCP adapter translate their environment at the edge and
delegate semantic work to this layer.
nuif-api::NuifDocument
load · validate · apply · export
│
canonical codecs · package · operations
│
model · layout · render · diagnostics
┌─────────────┼─────────────┐
native WASM process
Rust host browser/plugin CLI/MCP
Direct Rust use
Bare encodings are explicit; the SDK does not guess from an arbitrary byte
prefix. Package loading is a separate call because it structurally validates
the ZIP, manifest, document, resource descriptors, embedded bytes and policy
before a session is returned. Host support is negotiated after structural
decode or atomically through load_package_with_capabilities.
#![allow(unused)]
fn main() {
use nuif_api::{DocumentEncoding, NuifDocument};
use nuif_package::PackageMode;
let mut document = NuifDocument::load(bytes, DocumentEncoding::CanonicalText)?;
let report = document.validate()?;
let patch = document.apply_operations(transaction_id, operations)?;
let revision = document.canonical_hash()?;
let cbor = document.export(DocumentEncoding::DeterministicCbor)?;
let package = document.export_package(PackageMode::Portable)?;
}
load_package retains verified descriptors and shared immutable embedded bytes.
When the manifest has requirements, its session is structural and read-only:
validation, hashing, bare extraction and an unchanged same-mode package copy
remain available, while semantic mutation, undo/redo, evaluation and mode
conversion return the exact required set. Successful negotiation authorizes
those actions for that loaded session. export_package then replaces only the
package’s semantic document and requested mode and runs the ordinary package
policy; it cannot silently fetch linked resources. Applying operations uses the
same atomic transaction, revision and undo/redo implementation as the editor.
#![allow(unused)]
fn main() {
let mut structural = NuifDocument::load_package(package_bytes)?;
let report = structural.package_capability_report(&host_capabilities);
structural.require_package_capabilities(&host_capabilities)?;
let supported = NuifDocument::load_package_with_capabilities(
package_bytes,
&host_capabilities,
)?;
}
The report contains required, supported-required and missing-required sets in deterministic order. Extra host capabilities are ignored. Structural loading is appropriate for inert inspection, preservation and an explicit bare extraction before migration; it is not authorization to rewrite the package or a claim that the host can evaluate every required profile.
Structurally invalid but syntactically decodable bare documents can be loaded for diagnostics; canonical export, hashing, layout and rendering still fail closed. Package inputs are structurally valid or rejected atomically; full package support additionally requires explicit capability negotiation.
Binding rule
Wrappers contain transport and ownership conversion only:
- WebAssembly accepts byte arrays, bounds JSON patch and capability-set
transport, and delegates bare/package loading, validation, hashing,
capability negotiation, canonical export and history to
NuifDocument. - MCP bounds newline-delimited protocol messages and maps stateless tool calls to the same API.
- The CLI owns files and stdout; the editor owns window and interaction state.
- The CLI declares an empty package-capability support set. It may inspect,
validate, hash, extract or copy a capability-bearing package, but rejects
layout, render, snapshot, external-format export, changed package saves and
package-mode conversion with
PACKAGE_CAPABILITIES_REQUIRED. - The reference editor opens unsupported capability-bearing packages only for structural read-only inspection and exact copying; it rejects semantic edits at both the session and package-save boundaries.
- A host plug-in owns vendor objects, permissions and undo grouping. WASM does not become the Figma or Canva adapter merely because it runs in a plug-in; Affinity currently has no documented plug-in boundary for this project and uses a user-mediated interchange profile instead.
The cross-surface rule is exact: the same input and patch must produce the same
canonical hash, canonical bytes and diagnostics. For package-aware surfaces it
also requires the same deterministic archive bytes, retained resources and
missing-capability set. cargo xtask gate-wasm and cargo xtask gate-mcp
compare wrappers with native output; the Criterion sdk/direct_document group
measures direct text, CBOR and package loading plus canonical export.
C, C++, Swift and Kotlin decision
No stable foreign ABI is declared during the 0.0.x semantic-API phase. Rust’s
native ABI has no stability guarantee. A C-compatible ABI is possible, but it
adds an unsafe ownership boundary whose handle lifetime, buffer allocation and
release, panic containment, error representation, thread rules, symbol set and
calling convention become a compatibility promise independent of Rust source
compatibility. Generating a header does not decide those rules.
The promotion path is:
- Freeze a byte-oriented
nuif-ffi-1contract overNuifDocument, not over internal model structs. - Put all unsafe code in a separately reviewed
nuif-fficrate; keep the model, codec, operation and SDK crates underunsafe_code = "forbid". - Catch panics before the ABI boundary, return stable numeric error classes plus owned diagnostic bytes, and provide one allocator-matched buffer-free function and one idempotent handle-free function.
- Generate C and C++ headers with a pinned cbindgen release, diff the exported symbol/header surface in CI and run C consumers under sanitizers on every supported target.
- Generate Swift and Kotlin wrappers with a pinned UniFFI release after its generated ownership/checksum behavior passes native tests. Package an XCFramework/Swift package and Android AAR separately; UniFFI generates bindings but does not ship those platform artifacts.
- Apply semantic-version and ABI-compatibility checks to the FFI profile independently from the editor, WASM and MCP versions.
Promotion requires a reviewed error-code registry, a declared threading model, an API compatibility baseline, consumer fixtures in C/Swift/Kotlin, sanitizer evidence and release packages for their actual target triples. Until then, Swift or Kotlin desktop/mobile experiments should use the WASM package where their host embeds an appropriate runtime, or call a local CLI/process adapter; neither path is described as a native production binding.
For Node.js the WebAssembly package remains the default because it already has exact native parity and no native addon installation matrix. A Node-API addon is justified only if the benchmark suite shows a material workload that WASM cannot meet. The WebAssembly Component Model remains a possible future plug-in ABI, not a substitute for the currently deployed browser module.
Release boundary
The editor, CLI, WASM binding, MCP service and eventual FFI packages have
independent versions. A tag for the editor may attach tested developer
artifacts, but it does not promote nuif-api or promise a stable ABI. The CLI
archive is the explicit no-store process integration for automation that does
not need MCP; its package smoke report exercises real generation, validation,
canonicalization and inspection through the release binary. Publishing a crate,
npm package, Swift package, AAR or vendor plug-in requires an explicit policy
and authenticated release operation for that ecosystem.
Diagnostic code registry
This is the canonical public registry for structured profile-zero diagnostics
emitted through nuif_core::Diagnostic and for failure codes retained by the
reference conformance trial. Automation must branch on code, not on the
human-readable message. Messages may gain detail without changing a code.
The severity below is the default for the owning profile. A code is never
repurposed; a materially different condition receives a new code. Adding a
code is compatible within an alpha profile, while removing a code or changing
its meaning requires a profile-version decision. entity, pointer and
fidelity remain the attribution fields defined by the diagnostic record.
Transport and command failures such as malformed CLI arguments, unavailable package capabilities and WASM input limits are error classes rather than document diagnostics. Their promotion boundary is tracked separately in SDK and language-binding boundary.
cargo xtask diagnostic-audit compares this bytewise-sorted table with every
code literal owned by model validation, layout evaluation and the conformance
trial. It writes target/diagnostic-registry-report.json and fails when a code
is undocumented, duplicated, stale, malformed or out of order.
| Code | Default severity | Category | Producer | Stable meaning |
|---|---|---|---|---|
ASSET_RESOURCE_REQUIRED | error | asset | model validation | A portable or substituted asset lacks an exact resource digest. |
ASSET_UNAVAILABLE_HAS_RESOURCE | error | asset | model validation | An unavailable asset incorrectly binds resource bytes. |
COLOR_CHANNEL_OUT_OF_RANGE | error | paint | model validation | An encoded-sRGB fill channel is outside the inclusive zero-to-one range. |
EXTENSION_FALLBACK_NOT_USED | error | extension | model validation | A fallback is declared for a namespace absent from extensions_used. |
EXTENSION_NAMESPACE_INVALID | error | extension | model validation | An extension namespace is not a valid lowercase NUIF identifier. |
EXTENSION_REQUIRED_NOT_USED | error | extension | model validation | A required namespace is absent from extensions_used. |
EXTENSION_REQUIRED_UNSUPPORTED | warning | extension | model validation | The implementation preserves but does not interpret a required namespace. |
EXTENSION_UNDECLARED | error | extension | model validation | An attached extension namespace is absent from extensions_used. |
EXTENSION_UNSUPPORTED | information | extension | model validation | The implementation preserves but does not interpret a used namespace. |
GRID_EXPLICIT_AREA_EXHAUSTED | error | grid | model validation | Sparse placement cannot fit another child inside the explicit grid. |
GRID_PLACEMENT_OUT_OF_BOUNDS | error | grid | model validation | An explicit grid position or span exceeds the declared tracks. |
GRID_PLACEMENT_OVERLAP | error | grid | model validation | Two children occupy an overlapping explicit grid area. |
GRID_PLACEMENT_PARTIAL | error | grid | model validation | A child declares only one coordinate of a required row or column pair. |
GRID_PLACEMENT_WITHOUT_GRID_PARENT | error | grid | model validation | Grid placement is authored outside a direct grid child. |
GRID_SPAN_INVALID | error | grid | model validation | A grid row or column span is zero. |
GRID_STYLE_WITHOUT_GRID_FAMILY | error | grid | model validation | Grid tracks or flow are authored on a non-grid container. |
GRID_TRACKS_REQUIRED | error | grid | model validation | A grid container lacks an explicit row or column axis. |
GRID_TRACK_INVALID | error | grid | model validation | A fixed track size or fractional weight is not finite and positive. |
GRID_TRACK_LIMIT_EXCEEDED | error | grid | model validation | A grid axis exceeds the profile-zero track limit. |
IMAGE_ASSET_INVALID | error | image | model validation | Image metadata has zero dimensions or an invalid decoder profile. |
IMAGE_ASSET_MISSING | error | image | model validation | Image paint references an absent or non-image asset. |
IMAGE_CROP_INVALID | error | image | model validation | The crop is not a finite positive normalized rectangle inside the source. |
IMAGE_PAINT_INVALID | error | image | model validation | Image transform, opacity or color-conversion identity is invalid. |
IMAGE_PAINT_KIND_INVALID | error | image | model validation | Image paint is authored on an entity outside the image kind. |
LAYOUT_CONSTRAINT_FALLBACK | warning | layout | layout evaluation | Profile zero evaluates a constraint family through its declared freeform fallback. |
LAYOUT_FAMILY_PROFILE0_FALLBACK | warning | layout | layout evaluation | Profile zero evaluates a flex family through its declared stack fallback. |
LAYOUT_UNKNOWN_KIND_FALLBACK | information | layout | layout evaluation | An unknown kind uses its declared container or leaf fallback geometry. |
MODEL_ASSET_KEY_MISMATCH | error | model | model validation | An asset map key differs from the asset’s embedded identity. |
MODEL_ASSET_VERSION_NOT_OPAQUE | error | model | model validation | A newer asset schema is represented as a known rather than unknown kind. |
MODEL_CHILD_MISSING | error | model | model validation | A child identity does not exist in the entity map. |
MODEL_COMPONENT_MISSING | error | model | model validation | An instance references an absent or non-component entity. |
MODEL_CONTAINMENT_CYCLE | error | model | model validation | Entity containment contains a cycle. |
MODEL_DOCUMENT_VERSION_UNSUPPORTED | error | model | model validation | The document schema version is newer than the implementation. |
MODEL_DUPLICATE_CHILD | error | model | model validation | One parent lists the same child more than once. |
MODEL_DUPLICATE_ROOT | error | model | model validation | The root list contains the same entity more than once. |
MODEL_ENTITY_KEY_MISMATCH | error | model | model validation | An entity map key differs from the entity’s embedded identity. |
MODEL_ENTITY_UNREACHABLE | error | model | model validation | An entity is unreachable from every root. |
MODEL_ENTITY_VERSION_NOT_OPAQUE | error | model | model validation | A newer entity schema is represented as a known rather than unknown kind. |
MODEL_IDENTIFIER_INVALID | error | model | model validation | A semantic identifier violates the lowercase NUIF grammar. |
MODEL_MULTIPLE_PARENTS | error | model | model validation | An entity is listed under more than one parent. |
MODEL_NON_FINITE_NUMBER | error | model | model validation | Authored semantic state contains a non-finite number. |
MODEL_RELATION_TARGET_MISSING | error | model | model validation | A relation endpoint is absent from the entity map. |
MODEL_RESOURCE_LIMIT_EXCEEDED | error | security | model validation | The decoded semantic document exceeds a profile-zero resource limit. |
MODEL_RESPONSIVE_RANGE_INVALID | error | model | model validation | A responsive minimum width exceeds its maximum width. |
MODEL_ROOT_HAS_PARENT | error | model | model validation | A root entity is also listed as a child. |
MODEL_ROOT_MISSING | error | model | model validation | A root identity does not exist in the entity map. |
MODEL_TOKEN_KEY_MISMATCH | error | model | model validation | A token map key differs from the token’s embedded identity. |
MODEL_TOKEN_MISSING | error | model | model validation | A property references an absent token. |
RESOURCE_DIGEST_INVALID | error | resource | model validation | An asset resource digest is not canonical SHA-256 identity text. |
SNAPSHOT_FAILED | error | harness | conformance trial | A requested trial snapshot could not be evaluated or rendered. |
TEXT_FONT_BINDING_INVALID | error | text | model validation | Requested, replacement and asset font identities are inconsistent. |
TEXT_FONT_HASH_INVALID | error | text | model validation | A text font hash is not 64 lowercase hexadecimal digits. |
TEXT_FONT_NOT_PINNED | warning | text | layout evaluation | The requested exact font is absent from the evaluation context. |
TEXT_FONT_SUBSTITUTED | warning | text | layout evaluation | Layout uses an explicit replacement font present in the evaluation context. |
TEXT_FONT_SUBSTITUTE_NOT_PINNED | warning | text | layout evaluation | The declared replacement font is absent from the evaluation context. |
TEXT_FONT_UNAVAILABLE | warning | text | layout evaluation | The bound font asset explicitly declares the resource unavailable. |
TEXT_METRICS_INVALID | error | text | model validation | Text size or line height is not finite and positive. |
TRIAL_APPLY_FAILED | error | harness | conformance trial | A generated semantic patch failed atomic application. |
TRIAL_CBOR_ENCODE_FAILED | error | harness | conformance trial | The trial document could not be encoded as deterministic CBOR. |
TRIAL_CBOR_FIXPOINT_FAILED | error | harness | conformance trial | Deterministic CBOR did not reach an exact decode-encode fixpoint. |
TRIAL_CHOICE_LIMIT_EXCEEDED | error | harness | conformance trial | A generated choice stream exceeded its explicit decision budget. |
TRIAL_INVERSE_FAILED | error | harness | conformance trial | The inverse of an applied patch could not be applied. |
TRIAL_INVERSE_MISMATCH | error | harness | conformance trial | Applying a patch and its inverse did not restore the exact base document. |
TRIAL_RASTER_NONDETERMINISTIC | error | harness | conformance trial | Repeated CPU rasterization produced different bytes. |
TRIAL_REPLAY_FAILED | error | harness | conformance trial | Replaying a generated patch from the same base failed. |
TRIAL_REPLAY_HASH_MISMATCH | error | harness | conformance trial | Direct application and replay produced different canonical hashes. |
TRIAL_RERENDER_FAILED | error | harness | conformance trial | Repeated layout or scene construction failed. |
TRIAL_SNAPSHOT_FAILED | error | harness | conformance trial | The trial’s failure-reproduction snapshot could not be produced. |
TRIAL_TEXT_ENCODE_FAILED | error | harness | conformance trial | The trial document could not be encoded as canonical text. |
TRIAL_TEXT_FIXPOINT_FAILED | error | harness | conformance trial | Canonical text did not reach an exact decode-encode fixpoint. |
UNKNOWN_NAMESPACE_INVALID | error | extension | model validation | An unknown kind names an invalid extension namespace. |
UNKNOWN_NAMESPACE_UNDECLARED | error | extension | model validation | An unknown kind names a namespace absent from extensions_used. |
VALIDATION_DIAGNOSTICS_TRUNCATED | error | security | model validation | The validator reached its retained-diagnostic limit and stopped recording ordinary issues. |
Versioning and release operation
The native editor uses Semantic Versioning independently from the draft
specification profiles and unpublished library crates. The first editor release
is 0.1.0-alpha.1; the corresponding Git tag is v0.1.0-alpha.1. The decision
and source evidence are recorded in ADR 0007 and
nuif:research:github-release-delivery-and-provenance.
Version contract
apps/editor/Cargo.tomlcontains the editor version.- A release tag equals
vfollowed by the exact editor version. - Prerelease corrections increment the numeric suffix. A published tag is not moved or reused.
- The draft specification’s profile-zero identifier is not an application version and does not change when the editor prerelease increments.
- Editor alpha maturity does not transfer to proposed package, resource, capture, reconstruction or model artifacts. Each of those requires its own profile identifier, conformance evidence and release record.
- Library crate versions remain independent until a crate publication policy is adopted.
nuif-apiis the direct Rust SDK façade during this policy-free0.0.xphase; its source API is usable from a reviewed checkout but no stable Rust or C ABI is promised. The FFI promotion gate is defined in ADR 0011 anddocs/SDK-AND-BINDINGS.md.
Release sequence
- Update the editor version and
docs/releases/<version>.mdin one reviewed commit. - Run
cargo xtask all, the native editor tests,cargo deny check, andcargo xtask editor-packageon a clean tree. - Create and push the exact version tag.
.github/workflows/release.ymlchecks out the tag and runscargo xtask release-check <tag>followed by the complete verification harness.- Native jobs build, test, package, and attest five editor host architectures; a separate job builds, cross-checks, packages and attests the browser binding, and two five-host matrices do the same for the MCP service and standalone CLI developer binaries.
- The publication job writes checksums and a combined release manifest, creates a draft release, uploads all assets, and publishes the prerelease.
The workflow can be rerun manually for an existing unpublished tag. It refuses
to replace a published release. GitHub’s immutable-release setting is compatible
with the draft-attach-publish sequence but remains a user-managed repository
setting. Locator: ADR 0007; .github/workflows/release.yml; GitHub Immutable
releases, retrieved 2026-08-30:
https://docs.github.com/en/code-security/concepts/supply-chain-security/immutable-releases.
Artifact contract
Release archives use this form:
nuif-editor-<version>-<os>-<architecture>.<tar.gz|zip>
Each archive has a sibling .manifest.json. The release also contains
SHA256SUMS, nuif-editor-<version>.cdx.json, and
release-manifest.json. The CycloneDX document inventories the editor’s Cargo
dependency graph for all release targets. GitHub provenance can be checked with:
gh attestation verify <archive> --repo refpath/nuif
Tagged prereleases also attach nuif-wasm-0.0.1-web.tar.gz and its
.binding.json manifest as a separately versioned developer binding. The
binding is listed under bindings in the release manifest and does not count
among the five native editor packages used by the source updater. Its
package.json is private: GitHub download is automated, npm publication is not
authorized until an independent library-version policy is adopted.
The same prerelease attaches five nuif-mcp-0.0.1-<os>-<architecture>
archives, sibling manifests and a separate nuif-mcp-0.0.1.cdx.json SBOM.
These independently versioned developer-service records appear under
services in the release manifest. Every host binary is exercised through the
live stdio conformance gate before packaging. The crate remains unpublished;
developers may either use an attested archive or build it from a reviewed
checkout with cargo install --path crates/nuif-mcp --locked.
Five nuif-cli-0.0.1-<os>-<architecture> archives, sibling manifests and a
separate CLI SBOM provide the no-store command-line path. Before packaging,
each release binary reports its version and capabilities, creates the profile-0
fixture, validates and canonicalizes it, and inspects the canonical result.
The records appear under tools in the release manifest. The CLI has only
caller-selected path and standard-stream authority; it has no background
service or implicit network access. Developers may instead build the same
binary from a reviewed checkout with
cargo install --path crates/nuif-cli --locked.
SHA-256 verification on Linux uses sha256sum -c SHA256SUMS. macOS uses
shasum -a 256 -c SHA256SUMS. PowerShell users can compare
Get-FileHash -Algorithm SHA256 with the corresponding line in
SHA256SUMS.
Developer channel contract
Release archives are evidence and an expert opt-in path. The primary developer
installation builds locally from a retained source checkout according to ADR
0009. source installs the current clean revision. alpha additionally
requires the exact release tag and rejects a dirty tree.
An explicit editor-update --channel alpha operation queries published
prereleases and selects the greatest numeric MAJOR.MINOR.PATCH-alpha.N
version. It accepts a release only when release-manifest.json exists, its five
package records passed from one clean source revision, and its GitHub
attestation verifies the repository, release workflow, tag, source digest and
GitHub-hosted runner. The updater fetches the attested tag with Git hooks
disabled and requires the checked-out commit to equal that digest before
building with Cargo.lock.
The lifecycle retains an active and previous immutable install. It never moves
a release tag, installs from a mutable branch, performs a silent update or
changes an operating-system trust policy. See apps/editor/INSTALLING.md for
commands and paths.
Signing boundary
The alpha artifacts are unsigned and record that status in each manifest.
Checksums and GitHub attestations verify bytes and build origin; they do not
provide an operating-system publisher identity. Apple distribution requires a
Developer ID signature, hardened runtime, timestamp, notarization, and ticket
stapling. Windows direct distribution requires a trusted publisher signature
for an identified publisher and reduced SmartScreen friction. Locator:
nuif:research:github-release-delivery-and-provenance, Evidence.
Signing credentials are never stored in the repository. Adding a signing stage requires a separate review of identity ownership, secret storage, rotation, fork behavior, and release recovery.
NUIF foundation
Document status:
draft. Canonical source.
NUIF investigates a portable, vendor-neutral draft specification for authored user-interface documents. The candidate model is intended to preserve meaning across editors and implementation targets rather than treating a rendered bitmap, a vendor scene graph, or source-language AST as the universal truth.
Thesis
A useful portable interface specification must coordinate several representations instead of collapsing them into one:
- semantic/document containment;
- component and instance identity;
- authored layout and responsive constraints;
- resolved geometry at explicit evaluation contexts;
- geometry, paint, typography, and assets;
- design-token references and themes;
- interaction/state and data-binding graphs;
- source/tool provenance and correspondence;
- extension payloads that can survive unknown intermediaries;
- deterministic operations, diff, patch, and reconciliation.
Portable resources add a second identity boundary: editable semantic assets retain stable IDs, while exact image/font bytes use immutable content digests. Package paths and source URLs are locators/provenance, not identity.
NUIF therefore treats portability as a synchronization problem as much as a serialization problem.
Architectural hypothesis
The working model is a small canonical core plus coordinated graphs and extension dialects. The containment tree answers ownership and order. Typed relationship graphs express constraints, components, tokens, interactions, provenance, dependencies, and other relationships that do not belong in a tree.
The reference implementation will preserve both authored and resolved state. Resolved state is always scoped to an evaluation context and is never allowed to silently replace authored intent.
Fidelity model
Every adapter and transformation must classify material mappings:
lossless— semantics are preserved exactly;representable— equivalent target semantics exist, even if encoded differently;approximated— a declared approximation is produced;preserved_unrenderable— data survives as an extension but the target cannot render/edit it;unsupported— data cannot currently be represented or preserved safely.
Silent loss is a conformance failure.
Explicit non-goals
NUIF does not promise to infer the unique original source program from pixels, reproduce arbitrary JavaScript execution, make every platform text renderer bit-identical, or force every target to support every capability. The draft specification should make such boundaries inspectable and machine-readable.
Screenshot reconstruction is therefore an optional inference client, not a new canonical truth. It may propose a validated editable hypothesis and calibrated alternatives, but screenshot-only evidence cannot be classified as lossless authored source.
Reference implementation role
The Rust implementation and editor are executable research instruments and conformance references. They do not define semantics by accident; normative behavior belongs in spec/ and must be testable independently.
NUIF architecture thesis
NUIF is a specification-first authored-interface model. Its center is neither a vendor editor nor a source framework.
Core thesis
A portable interface document must retain intent, structure, relationships and evaluated results simultaneously. A single flattened scene tree cannot preserve enough information for loss-minimizing round trips across editors and runtime frameworks.
The recommended architecture is a layered hybrid:
Document containment tree
│ stable IDs
├── component / instance graph
├── token / theme graph
├── layout constraint graph
├── interaction / state graph
├── provenance / correspondence graph
└── asset dependency graph
Authored model ──evaluate/lower──► resolved model ──► render scene
▲ │
└──────── reconcile / lift ◄────┘
Borrowed foundations
- MLIR: dialects, explicit lowering, partial legality and multiple abstraction levels.
- OpenUSD: non-destructive composition, references, layers and variants.
- glTF: small core, extension registry, used/required capabilities.
- DTCG: token interchange.
- SVG/Unicode/OpenType: geometry and text foundations.
- Retentive/symmetric lenses: synchronization with preserved source regions.
New work required
NUIF must define the missing combination: authored UI semantics + resolved state + cross-tool provenance + structural loss accounting + source patch synchronization.
Canonical layers
- Document layer — identity, containment, semantics, accessibility.
- Component layer — definitions, instances, slots, parameters, variants and overrides.
- Layout layer — authored sizing/layout intent independent of resolved geometry.
- Visual layer — geometry, paint, text and effects.
- Behavior layer — interactions, states, animation and data bindings.
- Resolved layer — computed layout, shaped text, flattened paint/effect plans for a declared evaluation context.
- Provenance layer — source/destination correspondence and fidelity diagnostics.
- Resource layer — stable semantic assets bound to content-addressed bytes, package/resolver locators and derivation records.
No lower layer is permitted to silently erase a higher-level authored construct. Lowerings that cannot represent a construct must emit fidelity records.
Stable identity
Identity is semantic and independent of path, order and display name. Moving an entity does not change its ID. Content hashes identify immutable resources and canonical snapshots, not editable semantic entities.
Compiler and reconstruction ports
Deterministic source adapters and probabilistic screenshot reconstruction meet at the operation boundary:
retained source + resolved host observations ─┐
├─> typed operations -> core
pixels + OCR/CV/model hypotheses ─────────────┘ -> render/evaluate
Source-backed and screenshot-only inputs retain distinct evidence classes. A model/provider is replaceable and cannot redefine the operation grammar, validator, layout semantics, resource identity or fidelity ceilings.
Falsifiability
The architecture fails if the v0 experiment cannot preserve a non-trivial responsive component through editor→HTML→NUIF→editor while retaining component identity, token bindings, layout intent, an opaque foreign extension and a minimal source patch after an edit.
The resource/reconstruction extension fails if independent package writers cannot reproduce the proposed bytes, if browser capture cannot be pinned without secret leakage, if visual objectives reward flat screenshot copies, or if adaptation fails to beat the untuned tool-assisted baseline on a frozen holdout.
Layout and rendering research synthesis
Layout is not geometry
NUIF separates authored constraints from resolved boxes. Fixed x/y/width/height are valid authored values for freeform content, but they are not the universal layout representation.
The initial layout vocabulary contains families rather than one universal algorithm:
freeform— transforms/anchors and explicit geometry.stack— one-dimensional flow with intrinsic sizing, distribution, alignment and gaps.flex— web-compatible flexible layout semantics.grid— bounded explicit fixed/frtracks, spans and deterministic no-implicit-track placement in profile 0; broader CSS Grid features remain capability-reported adapter input.constraint— relational linear constraints for editor/native-layout cases.custom— extension/dialect-defined evaluator with declared fallback/resolved geometry.
Common sizing primitives are normalized across families: fixed, intrinsic-min, intrinsic-max, fit-content, fill/available, percentage, min/max clamps, aspect ratio and content measurement.
Taffy is the recommended first evaluator for CSS-compatible block/flex/grid behavior because it implements web algorithms in Rust. SwiftUI’s proposal-response model and Cassowary-style constraints demonstrate why the canonical schema must remain a superset rather than serializing Taffy’s Style directly.
Evaluation context
Resolved layout is keyed by an explicit context including viewport/container size, pixel ratio, locale, writing direction, font set, token/theme selection and feature/dialect capabilities. Multiple resolved snapshots may coexist as caches or conformance fixtures.
Rendering semantics
The draft specification defines the visual meaning of paths, fills, strokes, transforms, clipping, masks, gradients, compositing, images, text and supported effects. It does not specify GPU command buffers or a renderer implementation.
The reference renderer uses a backend trait. Vello/wgpu is the leading interactive experiment, but conformance requires deterministic raster comparisons and must permit CPU reference rendering where GPU differences would make tests unstable.
Text
Canonical text remains Unicode text + style runs + semantic annotations + font references. Shaping produces resolved glyph IDs, clusters, advances and offsets using pinned font data and a declared Unicode/shaping version. A glyph cache never replaces semantic text.
Portability reports must distinguish font substitution, missing glyphs, line-break differences and rasterization differences from document-model loss.
Protocol, portability and synchronization
NUIF treats portability as an ongoing synchronization problem.
Operations
The protocol operates on stable entities and semantic properties. Operations include create/delete/move, set/unset property, list/set relation edits, component/instance overrides, token bindings, extension edits and transactions. Editor gestures lower to these operations.
A drag inside a stack should usually become a reorder or layout-property edit; a drag in freeform space may become a transform edit. GUI coordinates are input data, not the protocol abstraction.
Patch model
A patch is a deterministic ordered set of operations with base snapshot identity, optional preconditions, transaction metadata and provenance. Patches can be replayed headlessly.
Three-way merge uses stable identity first and structural matching only when identity is absent. Conflicts are typed: property, delete/edit, ordering, relationship, extension and semantic-lowering conflicts.
Correspondence
Adapters maintain correspondence records between NUIF entities/properties and foreign constructs such as DOM nodes, CSS declarations, Svelte component props or design-tool node IDs. Correspondence is separable from the canonical design so source-specific metadata can be detached when unnecessary.
Fidelity classes
Every adapter/evaluator may report:
lossless— semantics preserved and reconstructable.representable— equivalent semantics represented through different constructs.approximated— visible/behavioral approximation with known semantic loss.preserved_unrenderable— data retained opaquely but not understood/rendered.unsupported— data could not be safely preserved.
Silent degradation is a conformance failure.
Serialization, collaboration and governance
Logical model before encoding
NUIF defines one logical model with multiple conforming encodings.
Text form
A canonical, reviewable representation is implemented for examples, fixtures,
diffs and Git workflows. nuif-text-0 fixes number formatting, UTF-8 key order,
layout and strict decode/canonicalize behavior; later text profiles may evolve
only through explicit versioning.
Binary form
Deterministic CBOR is the profile-0 binary form because the NUIF profile closes the choices left by RFC 8949 without coupling the logical model to generated code. The executable codec gate finds it near 41% of canonical-text size at 4,096 entities on an Apple M5 Pro run, while its typed decode path is slower than text. That result supports CBOR as a compact canonical form, not as a universal latency winner.
A candidate is timed only after complete-model round trip, canonical fixpoint and unknown-data preservation through a neighboring edit. Protobuf does not specify canonical binary output. FlatBuffers deliberately permits different byte layouts and old readers ignore new fields, so a rebuilding editor needs a separate retention strategy. Cap’n Proto specifies a schema-agnostic canonical form and is the preferred next experiment, but it still needs a complete NUIF mapping, bounded old-reader edit trial and two agreeing canonical writers. Compiled zero-copy runtime caches remain separate, explicitly noncanonical profiles rather than replacements for authoring interchange.
The experimental package form separates manifest/document records from
content-addressed resources. RFC 0010 selects a candidate deterministic ZIP
profile with fixed mimetype, canonical manifest/document records and
SHA-256-addressed blobs. Bare encodings use explicit .nuif.json and
.nuif.cbor names. Exact ZIP header fixtures, two independent local writers and
bounded image/font segments now exist. Cross-platform and externally authored
writer evidence remains required before package-profile acceptance.
Semantic document, resource and package hashes have different scopes. Stable asset identity is not content addressing. Unknown extension payloads remain explicit typed bytes/values and must not depend on accidental codec unknown- field behavior.
Collaboration
Canonical documents do not require CRDT tombstones, clocks or replica metadata. A collaboration profile maps NUIF operations to an append-only/change structure and can use Automerge, Yjs or another convergent transport. Checkpoints serialize back to canonical NUIF.
This keeps offline files simple and permits multiple collaboration engines.
The executable register profile uses causal multi-value registers. The separate existing-tree profile replays uniquely ordered moves, rejects cycles, models deletion as profile trash and orders siblings through stable RGA-style origins. Semantic move and deletion conflicts remain visible even when the profile can choose a deterministic checkpoint. Automerge is presently tested as an operation-set transport, not claimed as an implementation of the tree algorithm.
Governance
Early development occurs in refpath/nuif, but the architecture assumes eventual neutral stewardship. A plausible progression is:
- OSS research/reference implementation under Refpath.
- public RFC process + implementer registry.
- independent community/working group once two independent implementations exist.
- investigate W3C Community Group for UI/document semantics and/or Khronos-style governance if renderer/asset vendors become primary stakeholders.
Specification text, schemas and conformance tests need clear royalty-free contribution/IP terms before claiming standards-track stability.
Prior art and competitive map
No surveyed system currently combines the whole NUIF thesis. Several solve important subsets.
| System | Strongest reusable idea | Gap relative to NUIF |
|---|---|---|
| Penpot | open inspectable design document; SVG mapping | shape-centric; not a cross-runtime synchronization standard |
| OpenPencil | programmable editor, Figma codec, DOM/CSS, CLI/MCP | editor ecosystem, not neutral standards governance |
| Figma | mature component/layout authoring semantics | proprietary canonical model and evolving vendor format |
| W3C UI Specification Schema CG | implementation-agnostic UI field/schema goal | closed 2026; schema approach lacked executable renderer/protocol proof |
| Open UI | component anatomy/states/accessibility research | web-control scope, not authored visual document exchange |
| SVG | vector geometry/paint interoperability | lacks high-level components/responsive authored layout |
| Lottie/Rive | portable animation and state-machine runtimes | animation/runtime focus rather than general UI authoring |
| DTCG | neutral token semantics | intentionally only tokens |
| OpenUSD | non-destructive layers/references/variants | 3D scene domain, not UI semantics |
| glTF | compact core + extension governance | delivery/runtime asset rather than authoring model |
| MaterialX | renderer-independent typed graph | material domain |
| MLIR | dialects/multi-level lowering | compiler infrastructure rather than document semantics |
| CSS | rigorous layout families and authored→formatting pipeline | web-specific cascade/DOM/runtime semantics |
| IFC/STEP | long-lived semantic interchange and profiles | complexity warns against over-generalizing the core |
Directly borrow
Stable standards concepts: SVG geometry, DTCG token values, Unicode/OpenType text foundations, CSS-compatible algorithms for matching profiles, glTF-style capability declarations, OpenUSD-style composition principles, MLIR-style dialect/lowering discipline.
Adapt
Retentive lenses → property/source correspondence; CRDTs → collaboration profile; WebRender/Vello/Skia → renderer boundary and conformance strategy; Tree-sitter → source-preserving adapter infrastructure.
Invent/prove
The proposed integration combines authored and resolved UI state, stable cross-tool semantic identity, opaque extension retention, fidelity accounting, bidirectional semantic patches and a native open editor whose internal state is the draft model itself.
Implementation language and runtime choice
Decision: Rust reference core
Rust is the strongest default for the reference implementation because the project simultaneously requires untrusted binary parsing, graph/document transforms, geometry, text shaping, native/WASM embedding, GPU access, fuzzing and stable C-compatible boundaries.
Alternatives
- C++ has the deepest graphics ecosystem and mature Skia/Yoga integration, but expands memory-safety risk in parsers/plugins and makes a browser/WASM-safe reference core less attractive.
- Zig offers excellent systems control and C interoperability but has a smaller mature graphics/text/schema ecosystem and less API stability for a standards reference implementation.
- Go is strong for services/tooling but weaker for low-level rendering/WASM/native GUI integration and deterministic allocation-sensitive engines.
- TypeScript is ideal at web/editor adapter boundaries but unsuitable as the only renderer/codec/reference-core implementation.
Stack
- Rust: document model, operations, layout abstraction, codec, renderer scene, conformance, WASM bindings.
- Taffy: initial CSS-compatible evaluator behind NUIF types.
- Vello/wgpu: interactive renderer experiment behind a NUIF renderer trait.
- HarfBuzz-compatible shaping: text experiment with pinned font inputs.
- Masonry + AccessKit: reference editor shell (ADR 0006, accepted; toolchain 1.98.0, MSRV 1.96); Svelte 5 + TypeScript for the later browser demonstration over the WASM bindings.
- Tree-sitter/language-native parsers: source adapters where concrete syntax retention is required.
Adapters MAY be written in the ecosystem-native language; conformance is against behavior/protocol, not implementation language.
Risk register and impossibility boundaries
Fundamental boundaries
- Rendered output is underdetermined. Pixels/boxes cannot uniquely reveal whether layout came from flex, grid, constraints, absolute positioning or runtime code. Imported foreign content must mark inferred intent.
- Arbitrary program behavior is not serializable as UI structure. NUIF does not promise to recover arbitrary JavaScript/Swift/Dart application logic.
- Text is environment-sensitive. Font files, shaping versions, fallback and rasterization can differ. Exact conformance requires pinned inputs; portability must classify substitution separately.
- Platform-native controls differ. Semantic equivalence may be possible while exact visuals/behavior are platform-specific.
- Effects/shaders can exceed a portable core. Extensions may be preserved without being renderable.
- Standard complexity can kill adoption. IFC/STEP demonstrate the cost of excessive semantic scope.
- A single reference implementation can accidentally become the spec. Independent implementation is a standards gate.
- Resource bytes carry legal and security constraints. Exact font/image preservation does not imply permission to redistribute or safe decoding.
- Visual metrics are gameable. A flat screenshot can look exact while discarding editability, semantics, accessibility and responsive behavior.
- Model confidence can be misleading. Raw probability is not calibrated correctness and cannot upgrade inferred evidence into source truth.
- Capture can leak secrets. Browser/network/accessibility observations may expose credentials, personal data or proprietary content unless collection, export, retention and training are separately bounded.
Containment strategies
- explicit fidelity reports and inference confidence;
- authored + resolved snapshots instead of pretending either alone is canonical truth;
- opaque extension preservation;
- deterministic capability/evaluation contexts;
- small core + profiles;
- reference implementation backed by normative conformance fixtures;
- fuzzing/resource budgets for all untrusted inputs;
- early independent implementation and adapter experiments.
- stable asset IDs separated from resource digests, locators and provenance;
- verify resource size/digest before decoding and never fetch implicitly;
- typed operation output for models, atomic validation and finite correction loops;
- structural/text/resource/edit-task metrics alongside visual diagnostics;
- calibrated decision-level confidence, alternatives and abstention;
- private captures default to local processing, no retention and no training.
Current implementation risks
- The macOS graphics fork is a maintenance boundary. The editor pins
refpath/xilemcommit1b96eb8; its parenteabfe0amoves the active renderer from wgpu 28 and metal-rs to wgpu 29 and the objc2 Metal bindings. The fork changes public API call sites and does not patch the Objective-C blocks ABI. Each fork update requires the editor tests, reverse dependency trace, and macOS Metal window smoke test recorded innuif:research:macos-metal-block-future-incompatibility. A separaterefpath/metal-rsmove-to-block2branch exists only for review; NUIF does not depend on the deprecated binding or that experimental patch. - Release signing is credential-bound. The editor packaging gate builds, archives and smoke-tests an unsigned host package. Platform signing and notarisation require release credentials and remain separate from source conformance.
- Portable resources are only narrowly implemented. The deterministic package, RGBA8 PNG and static single-face TrueType subsets have executable gates, while CPU profile 0 remains unchanged. RFC 0010 cannot be accepted until broad media/font matrices, the configured Linux/Windows/macOS jobs produce passing hosted evidence, external reproduction, calibrated aggregate budgets and interoperability review pass.
- Capture and reconstruction accuracy remains unestablished. RFC 0011 and specification 14 now have bounded fixed-input contracts, a local pinned browser fixture and a typed synthetic evaluation report. They establish interfaces, refusal rules and metric consistency, not portable browser-capture or screenshot-reconstruction accuracy. No current release makes those accuracy claims.
Thesis falsifiers
The project should rethink its architecture if ordinary source round trips require broad regeneration, if unknown extension preservation cannot survive routine edits, if the layout vocabulary becomes a vendor-property dump, or if a second implementation cannot reproduce v0 behavior from specification + fixtures alone. The resource/reconstruction path should additionally narrow if package bytes cannot reproduce across writers, correction loops improve pixels by deleting semantics, confidence cannot support useful risk/coverage, or tuned models do not beat the untuned tool-assisted baseline.
Governance and standardization strategy
NUIF starts under Refpath because research and implementation need a concrete home, but the target is neutral stewardship.
Repository governance now
- Public RFCs for semantic changes.
- ADRs for reference-implementation choices.
- Research evidence is distinct from normative requirements.
- Extension registry changes require examples and conformance fixtures.
- No vendor adapter can redefine core semantics.
Standardization path
A W3C Community Group is a plausible early venue for document/component semantics and coordination with Open UI/DTCG, but NUIF should not enter formal standards work before an executable v0 and at least one external implementer exist. Khronos/ASWF-style governance offers useful precedent for graphics/rendering and extension registries. A neutral foundation can become appropriate once multiple vendors/projects depend on the format.
IP/licensing goals
- reference code: permissive dual MIT/Apache-2.0;
- specification/schema/conformance text: permissive terms compatible with standards adoption;
- contributions: explicit patent/IP policy before standards-track claims;
- trademarks/conformance branding: separate from implementation copyright.
The project must not call itself an industry standard merely because the repository is public. Conformance, multiple implementations and neutral governance are prerequisites.
Naming and project identity
NUIF is the current working name and repository slug, not yet a cleared standards trademark.
A public reconnaissance found prior acronym use in an old Nexus User Input Framework and in a 2024 computer-vision paper, plus unrelated uses. None currently appears to occupy the same open UI-authoring interchange category, but collision risk is non-zero.
The architecture therefore separates human branding from protocol identity. Stable schema namespaces, extension IDs and version identifiers must not depend on a product trademark remaining unchanged.
Before v0.1 branding is promoted outside the research project:
- perform repository/package/domain/trademark clearance in key jurisdictions;
- decide whether
NUIFis an acronym or simply a proper project name; - reserve crate/package/extension namespaces;
- define conformance-mark governance separately from the open specification;
- make any rename before third-party persisted documents become common.
Research coverage and continuous completeness
A research repository cannot truthfully claim to contain every paper that will ever be relevant. NUIF instead defines operational completeness: every planned architectural front must have an explicit status, evidence links, unresolved questions and an experiment/decision path.
research/coverage.yaml is the machine-readable coverage contract. It maps the founding research plan to research IDs, specification modules, RFCs/ADRs, code seams and experiments. The project can therefore identify gaps structurally rather than relying on prose search to infer that a topic was forgotten.
Current state
All founding fronts are represented. The resource, browser-capture,
reconstruction-evaluation, adaptation/distillation and AI artifact-governance
fronts are now explicit rather than hidden inside “serialization” or
“inference.” Decisions that can safely be made from mature prior art are marked
covered. Questions whose answer would be premature without an implementation
are marked experiment-required. Areas whose evidence base will continuously
evolve—prior art, adapters, reconstruction and data/model governance—remain
ongoing by design.
This distinction is important: marking an open research problem as finished would be less rigorous than preserving it as a first-class graph node.
Additional boundaries from the final sweep
- WAI-ARIA and accessibility API mappings support a semantic-role/state layer
distinct from platform-specific accessibility trees. The bounded
nuif-web-accessibility-0lowering now proves computed role/name/state agreement for one eleven-node fixture across pinned Chromium, Firefox and WebKit while retaining native-platform and behavior non-claims. - KHR_interactivity provides contemporary precedent for portable,
capability-aware behavior graphs rather than arbitrary scripts embedded in
visual nodes. The smaller
nuif-behavior-state-machine-0sidecar now has exact Rust/Node traces for ordered guards, state, effects and explicit required/optional capability handling without claiming a final semantic schema. Its first wire experiment is one inert canonical-CBOR, content-addressed package resource: a Rust gate validates document binding and hostile cases while an independent Python ZIP reader checks exact container bytes. The attachment remains outside the canonicalDocumentand never grants execution authority. A separate one-way web lowering maps the bounded effects through native activation,hiddenand an ARIA status region; five events agree across pinned Chromium, Firefox and WebKit under one exact CSP-hash-authorized runtime without extending that evidence to native UI or screen-reader speech. - ReverseORC and related layout-inference work show that multiple viewport observations materially improve recovery of responsive intent.
- Screenshot-to-code research continues to show that visual reconstruction is not equivalent to recovering authored layout or behavior.
- Merkle/content addressing is appropriate for immutable assets and snapshots but not for editable semantic identity.
- EPUB OCF and OCI descriptors support a narrow manifest-driven package with size/digest verification; NUIF now has an in-repository independent-writer fixture. A three-OS CI matrix now exercises the package, image and font gates, while successful hosted evidence and external reproduction remain open.
- OpenType and Fontations evidence now supports one executable static TrueType
package baseline with a pinned HarfBuzz metadata oracle. The retired
ttf-parserdecision remains documented; broad font formats, portability outcomes and shaping/raster integration remain experiment-required. Warmed parser and packaged-validation allocation ceilings cover every accepted fixture. - Browser source capture and screenshot reconstruction are different evidence
lanes and cannot share a blanket
losslessclaim. - Current screenshot-to-code work supports OCR/region/hierarchical and render- correction experiments, not a claim that authored UI recovery is solved.
- LoRA, quantized adaptation and distillation are conditional experiment techniques; evaluation, rights-cleared traces and artifact governance precede training.
The continuous-research process should periodically re-run topic searches, append or supersede research records, and update research/coverage.yaml only when new evidence or experiments change the status of a front.
Cross-industry patterns: evidence, adoption and rejection
Document status:
draft. Canonical source.
This document synthesizes 52 research records added on 2026-08-29 from visual-effects interchange, game-engine asset systems, programming-language research on bidirectional transformation and layout verification, distributed-systems testing, and 2D-rendering conformance practice. Each pattern below is classified as borrowed (adopted as is), adapted (adopted with a stated change) or rejected (ruled out with the reason). Record identifiers (nuif:research:*) carry the locators; this document does not repeat them.
Method
The 52 records synthesized here were reviewed from primary sources (specifications, source code at a named commit and papers with DOI). reviewed does not mean every material claim has completed locator-level verification; research/AUDIT.md defines the stricter verified state. Claims resting on one record inherit its evidence status as well as its confidence. Source conflicts remain explicit until an RFC or experiment resolves them.
Patterns
Document model and composition
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
| Opinion strength ordering over composition arcs (LIVERPS) | openusd-composition-and-crate | Adapt: NUIF needs a total, documented resolution order for library, theme, variant and instance-override opinions; six arc kinds are more than a UI document needs | spec/03-components-and-composition.md, question composition-strength |
| Flatten as an explicit, named lowering that discards composition | openusd-composition-and-crate, alembic | Borrow: flattening is a lowering with a fidelity record, never the save format | spec/00-conformance.md, docs/whitepaper/01-architecture.md |
| Authored network versus cooked output with pull-based, memoized evaluation and push-based dirtying | houdini-pdg-and-hda, hydra-render-delegate | Borrow for the evaluator: resolved snapshots are pull-evaluated per context and invalidated by hierarchical locator sets rather than global dirty bits | crates/nuif-layout, ADR 0002 |
| Prefab override as a sparse modification set against a source definition | unity-prefabs-and-yaml-merge | Borrow: instance overrides are sparse property sets keyed by stable identity and property path | spec/03-components-and-composition.md |
| Resolved-only interchange (baked samples) | alembic | Reject as a canonical form; accept as an explicit cache profile | spec/08-serialization.md |
Identity and ordering
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
| File-local numeric identity with global identity through a second key | unity-prefabs-and-yaml-merge, godot-tscn-scene-format | Reject: file-local identities orphan cross-file references on replacement; NUIF identities are global from creation | spec/02-identity-and-properties.md |
| Path-based addressing in patches | json-patch-rfc6902-and-merge-patch | Reject for entities; retain for property paths inside an identity-addressed operation | spec/06-operations-and-patches.md |
| Parent link and fractional position as one atomic property | figma-multiplayer-and-rendering-engineering | Adapt: parent and anchor move together; collaboration profiles may use list identifiers internally, while canonical operations use Start/After(id) anchors | RFC 0006, crates/nuif-protocol |
| Tree move with undo/redo of concurrent operations and cycle rejection, mechanized proof | crdt-tree-move-operation | Adapt for the collaboration profile; the canonical document keeps a totally ordered log and needs no replica metadata | spec/10-collaboration-profile.md |
| Random resource identifiers with path fallback and warnings | godot-tscn-scene-format | Borrow the fallback discipline for asset references; a resolved-by-path reference must be diagnosed | spec/09-provenance-and-fidelity.md |
Unknown data preservation and schema evolution
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
UnknownSchema keeps name, version and raw payload and re-emits it verbatim; round trip asserted by a test | opentimelineio | Borrow verbatim: this is the executable form of nuif:claim:opaque-preservation | rfcs/0002-extension-preservation.md, experiment unknown-extension-roundtrip |
| Unknown node class preserved as a placeholder that records its original class and properties, written back on save | godot-tscn-scene-format | Borrow: entities of unknown kind are preserved, not dropped, and their original kind is restored on export | spec/07-extensions-and-dialects.md |
| Unknown data ignored and not re-saved | blender-dna-rna-and-headless | Reject: this is the failure mode NUIF exists to prevent | risk register |
| Per-schema version numbers with gap-tolerant upgrade functions and a generated version manifest | opentimelineio, unreal-asset-versioning-and-automation | Adapt: each core record kind carries a version; migrations are pure functions registered per kind; reading a newer version than known is an error, not silent loss | spec/08-serialization.md, migrate command |
| Self-describing struct layout embedded in the file | blender-dna-rna-and-headless | Reject: NUIF encodings are schema-versioned, not struct-layout-described; the deterministic CBOR profile already carries structure | ADR 0004 |
| Extension prefix registry with a status ladder that requires validator support before release | gltf-validator-and-sample-assets | Borrow: EXT namespaces are promoted only with a conformance fixture and validator rule | question extension-governance |
Operations, undo and merge
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
| Every editor action is an operator with typed parameters, invocable from scripts | blender-dna-rna-and-headless | Borrow: every editor gesture lowers to a protocol operation that a script can invoke | RFC 0004 |
| Memento-based transaction snapshots | unreal-asset-versioning-and-automation, blender-dna-rna-and-headless | Reject for the canonical log; inverse operations are recorded instead, because snapshots do not commute and cannot be merged | spec/06-operations-and-patches.md |
| Undo restores expected user state under concurrent edits; undo rewrites redo history | figma-multiplayer-and-rendering-engineering, command-pattern-undo-and-event-sourcing | Adapt: the invariant “undo, copy, redo leaves the document unchanged” becomes a metamorphic relation in the operations suite | conformance/HARNESS.md |
| Structural three-way merge keyed by class and identity with declared set-valued fields and float epsilons | unity-prefabs-and-yaml-merge | Borrow: a merge-rules declaration per property kind (ordered list, identity set, scalar with tolerance) | spec/06-operations-and-patches.md |
| Tree matching becomes the identity map when stable identifiers exist; the residual problem is move and order conflicts | ast-diff-gumtree-and-structural-merge | Borrow: no heuristic matching in NUIF-native merges; GumTree-style matching is reserved for adapters without identity | docs/whitepaper/03-protocol-and-portability.md |
| Conflicts as first-class states rather than failures | patch-theory-darcs-pijul | Borrow: typed conflict objects are document state until resolved | spec/06-operations-and-patches.md |
| Operational transformation with server serialization | operational-transformation-vs-crdt | Reject as a canonical model; permissible as a collaboration profile | ADR 0005 |
Canonical encoding
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
| RFC 8949 §4.2 core deterministic encoding rules | canonicalization-rfc8785-and-cbor-deterministic | Borrow | nuif-cbor-0 |
| Float handling: CDE keeps numeric kinds while dCBOR reduces integral floats and zero | same, cbor-data-model-and-key-order-correction | Preserve NUIF’s declared integer/real kinds; canonicalize both real zeros to positive floating zero | RFC 0008 |
RFC 8785 number serialization via shortest round-trip and -0 to 0 | same | Adapt for nuif-text-0 | spec/08-serialization.md |
| Content-addressed deduplication of array values | openusd-composition-and-crate, alembic | Borrow for the package asset store; reject for editable entities | ADR 0004 |
Layout
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
Fixture tests generated from browser layout through WebDriver, compared at < 0.1 px, with structural (not numeric) handling of known divergences | taffy-and-yoga-browser-generated-tests, differential-testing | Borrow: the layout differential suite is generated, never edited by hand; divergences are classified per case | experiment layout-differential |
| Layout as SMT-solvable constraints with a visual assertion logic | cassius-web-layout-verification | Adapt: the assertion vocabulary (no overlap, containment, alignment, text fits) becomes a fixture-level oracle; the SMT encoding itself is out of scope because no formalization covers flex or grid | conformance/HARNESS.md |
| Relational constraint synthesis from multi-device examples | inferui-and-layout-synthesis | Adapt for import inference only; results are marked inferred with confidence | experiment layout-inference |
| Flexbox §9.9.1.2 placeholder and grid intrinsic-sizing divergences | css-flexbox-grid-algorithm-specs | Record: NUIF conformance cannot claim exact agreement where the CSS specification is implementation-defined; such cases are tolerance-tiered | spec/04-layout.md |
Rendering determinism
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
| Renderer as a pluggable backend behind a stable scene abstraction | hydra-render-delegate | Borrow (already ADR 0003) | crates/nuif-render |
| GPU shaders as ground truth compared by a perceptual mean | vello-testing-and-cpu-reference | Reject as the conformance oracle; borrow the threshold values for the interactive backend | conformance/HARNESS.md |
CPU f32 pipeline with tolerance 0 inside Vello’s own harness | vello-testing-and-cpu-reference, resvg-test-suite | Treat as candidate evidence, not proof of cross-platform identity; calibrate NUIF’s path with pinned assets and a CI matrix | render-tolerance experiment, ADR 0003 |
Per-test numeric and perceptual thresholds (idiff, oiiotool, WPT fuzzy, WebRender fuzzy(max,count)) | hydra-render-delegate, blender-dna-rna-and-headless, skia-gold-and-gm-tests, webrender-reftests | Adapt into three declared tiers: exact, bounded per-channel delta with pixel count, perceptual (ꟻLIP mean) | conformance/HARNESS.md |
| Reftests (two documents that must render identically) over pixel baselines | skia-gold-and-gm-tests | Borrow for equivalence-preserving rewrites | metamorphic relation class 1 |
| Scene capture to a text serialization for deterministic replay | webrender-reftests | Borrow: render scenes are serializable fixtures | crates/nuif-render |
| WGSL and WebGPU leave rounding, reassociation, sample locations and edge inclusion implementation-defined | gpu-rendering-nondeterminism | Record: GPU output is never normative | spec/05-geometry-paint-text.md |
Text
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
Shaping fixtures as glyph strings with font hash, options and expected glyph=cluster@dx,dy+adv output | text-rendering-reproducibility | Borrow the format for the text suite | experiment text-pinning |
| Hinting off, grayscale coverage, declared subpixel quantum, font SHA-256, Unicode and shaper versions pinned | same | Borrow | spec/05-geometry-paint-text.md |
Testing methodology
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
| Deterministic simulation: single-threaded scheduler, seeded PRNG, all nondeterminism behind injectable interfaces, reproduction by seed | deterministic-simulation-testing | Borrow: the trial loop is seed-driven and prints the seed on failure | conformance/HARNESS.md |
| Swarm testing (random feature subsets per run) | same | Borrow for operation generators | same |
| Metamorphic relations with tolerant equality; reduction by reversing recorded transformations | metamorphic-testing-graphics | Borrow: nine relation classes are defined in the record | same |
| ddmin over operation sequences, hierarchical reduction over the document, choice-sequence shrinking over generated values | delta-debugging-and-test-case-reduction | Borrow: three-level reducer | QA contract item 9 |
| Model-based testing with a small reference model and precondition-preserving shrinking | property-based-testing-state-machines | Borrow: proptest-state-machine over an ordered-forest model | same |
| Structure-aware fuzzing with explicit depth and allocation budgets | fuzzing-structured-inputs | Borrow: arbitrary does not bound value depth; NUIF bounds depth and node count explicitly | spec/11-security.md |
| Snapshot testing with redactions, sorted output and a single update variable | golden-master-and-snapshot-testing, libtest-mimic-and-data-driven-fixtures | Borrow: NUIF_UPDATE_EXPECT is the only regeneration switch | conformance/HARNESS.md |
| Machine-readable validation report with severity codes, pointers and per-code policy | gltf-validator-and-sample-assets | Borrow as the report schema for validate, import, export | spec/12-cli-api-and-automation.md |
| Sample-asset corpus with per-asset metadata, tags and CI validation | same | Borrow for conformance/fixtures | conformance/HARNESS.md |
Editor automation
| Pattern | Source | Decision | NUIF artifact |
|---|---|---|---|
Headless execution with a script (--background --python, hython, commandlets, -nullrhi) | blender-dna-rna-and-headless, houdini-pdg-and-hda, unreal-asset-versioning-and-automation | Borrow: the editor binary runs a session script without a window | apps/editor/UI-SPEC.md |
| Plugin API as a programmable surface, but no headless mode and read-mostly REST | figma-plugin-and-rest-api-as-automation-surface | Record as the gap NUIF closes; borrow pluginData-style opaque per-entity stores as adapter evidence | adapters/README.md |
| Accessibility tree as the semantic query and action surface for UI tests | accesskit-semantic-ui-testing, egui-and-egui-kittest, masonry-xilem-and-linebender-test-harness | Borrow: entity identifiers are carried in the accessibility tree; tests query by role and label and dispatch actions without pointer synthesis | ADR 0006 |
| Same-frame scene and accessibility outputs with virtual time and CPU rasterization | masonry-xilem-and-linebender-test-harness | Borrow | ADR 0006 |
| Pixel-based UI screenshot tests with per-OS thresholds | egui-and-egui-kittest | Reject as the primary editor oracle; permitted only for shell wiring | apps/editor/QA.md |
Ruled out
The following were examined and excluded from the architecture; the reason is recorded so the question is not reopened without new evidence.
- A self-describing binary struct layout (Blender DNA): solves version drift for one implementation but does not preserve data it cannot re-save and does not compose with a schema-versioned interchange model.
- Memento undo as the canonical history: does not commute, cannot be merged, and bloats logs; inverse operations are required by
spec/06. - Integer child indices in
MoveandInsertas the only order representation: non-commutative under concurrency; a list identifier is required for the collaboration profile and harmless for the canonical form. - GPU rendering as a normative oracle: implementation-defined by the WebGPU and WGSL specifications.
- Perceptual UI screenshot tests as the primary editor test: platform-dependent text rendering makes them a shell-wiring check only.
- Heuristic tree matching for NUIF-native merges: unnecessary with stable identity and a source of spurious moves.
- Whole-project regeneration as the synchronization model: contradicted by the lens and delta-lens laws that NUIF’s patch model must satisfy (
lenses-foster-boomerang,bidirectional-evaluation-direct-manipulation).
Consequences for the specification
The records imply the following changes. Items 1–3 were decided by follow-up research on 2026-08-29 and are recorded as accepted RFCs; items 4–6 remain proposals.
- Sibling order is a canonical array without keys;
InsertandMoveuse anchors (Start,After(id)); the collaboration profile maps anchors onto a Fugue-family list CRDT (RFC 0006). nuif-cbor-0follows deterministic preferred serialization while preserving integer/real data-model identity; real zero is positive floating zero, text and CBOR key orders are distinct, strict decoders reject non-canonical input, text hashes through CBOR and strings remain verbatim (RFCs 0005 and 0008).- Entities of unknown kind load as
Unknownwith typed core fields and an opaque payload; ignorant implementations preserve bytes, knowing ones may re-encode; validation severities follow the glTF pattern (RFC 0007). - Every serialized record kind carries a schema version; migrations are registered pure functions; newer-than-known versions load as
Unknownfor entities (RFC 0007) and are diagnosed for other records. - Validation, import and export reports follow one schema with stable codes, severities and pointers.
- Layout conformance declares tolerance tiers per case and classifies every divergence from a browser reference as schema loss, evaluator defect or implementation-defined behavior.
Open questions raised by this synthesis
Recorded in research/questions.yaml: cbor-float-zero (decided, RFC 0005), sibling-order-identifier (decided, RFC 0006), unknown-kind-preservation (decided, RFC 0007), editor-toolchain-msrv (ADR 0006), layout-assertion-vocabulary, render-tolerance-tiers.
Resources, capture and model-neutral reconstruction
Document status:
draft. Canonical source.
NUIF’s next research front is not “add an AI converter.” It is a coordinated resource, capture and reconstruction architecture with explicit truth boundaries. Images and fonts must survive as verified resources. Source-backed browser imports must retain authored/resolved evidence. Screenshot-only imports must remain honest probabilistic hypotheses. Every path converges through one core operation, validation, rendering and fidelity contract.
Decision summary
The recommended direction is:
- define stable assets separately from immutable byte resources;
- make
.nuifa deterministic portable package after cross-writer proof; - grow the executable narrow PNG and static TrueType resource baselines only through named profiles and measured hostile-input budgets;
- add a pinned browser-capture adapter separate from static source sync;
- build screenshot reconstruction as a replaceable observation/proposal loop;
- freeze a structural and visual evaluation suite before training;
- consider adaptation or distillation only after the untuned loop exposes a repeatable learnable error distribution.
RFC 0010 and RFC 0011 remain proposed contracts. Their bounded package, narrow PNG/static-font and capture/reconstruction experiments are implementation evidence only for the named subsets; they are not published conformance or standards claims.
One core, two import lanes
Source-backed lane
HTML/CSS + browser execution + resource responses
-> retained source + resolved observations
-> deterministic adapter/lowering + explicit inference where needed
|
v
typed NUIF operations
|
v
core validation/apply
|
v
resource-aware NUIF
^
|
Screenshot-only lane |
pixels + context |
-> OCR/CV/grounding observations
-> hierarchy/layout/resource hypotheses
-> typed operations -> render/diff/correct
The lanes differ in evidence, not in their mutation authority. Both use the same typed operations. Neither provider can write core structs directly. A browser observation may support an exact resolved value under one pinned context, but it does not automatically reveal authored intent. A screenshot cannot establish source equivalence regardless of visual score.
Portable resource model
Four identities must remain distinct:
| Concern | Identity | Change behavior |
|---|---|---|
| editable semantic asset | AssetId | stable when its content is replaced |
| immutable encoded bytes | ResourceDigest | changes for any byte change |
| package/resolver location | locator | may change without changing bytes |
| source/derivation history | provenance record | may grow without renaming asset/bytes |
An asset points to a content descriptor containing media type, SHA-256 and byte length. A package path or external URL only locates candidate bytes; size and digest are checked before decoding. External resolution is opt-in. Opening a document never triggers network access.
Resource roles clarify hash and retention behavior:
source: original encoded bytes;authoring: exact bytes needed to evaluate/edit semantics;derived: crop, trace, selected frame, conversion or generated result with input digests and transformation identity;cache: decoded pixels, GPU textures or acceleration state that can be deleted without changing the semantic document.
This avoids a common failure: storing only a decoded bitmap and calling the source image preserved, or hashing an editable asset by its current bytes and therefore breaking every reference after replacement.
Candidate .nuif package
The proposed first package is a deliberately small deterministic ZIP profile:
mimetype
manifest.cbor
document.cbor
blobs/sha256/<digest>
mimetype is first and stored. Manifest and document are deterministic CBOR.
Every embedded blob is addressed by exact SHA-256 bytes. The first profile uses
stored members only, fixed metadata and sorted ASCII paths to make independent
writer byte equality attainable and to avoid compression-version variability.
There are three hashes:
- semantic document hash over canonical
document.cbor; - resource digest over each exact blob;
- package hash over the exact ZIP artifact.
Cache or report changes may change the package hash while leaving the semantic
document hash untouched. Bare canonical forms remain .nuif.json and
.nuif.cbor. Historical alpha .nuif raw files need read-only detection during
migration; new .nuif output becomes the package only after RFC acceptance.
The package reader rejects duplicate or unsafe paths, symlinks, directories, encryption, split archives, unsupported compression, inconsistent headers, undeclared/missing blobs and digest mismatch. It does not extract to a filesystem. Exact byte fixtures, member/resource limits and two independent writers are acceptance gates.
Package-to-session handoff uses shared immutable buffers. The release gate passes an 8 MiB resource through package, handle map and session with the same allocation pointer while keeping handoff allocator traffic and retained bookkeeping below 1 MiB. This prevents a host from paying one full resource copy merely to enter the core.
Images
The original encoded image is authoritative. A semantic image asset records its resource digest and intrinsic interpretation; each image paint records fit, crop, transform, sampling, opacity and color conversion. Derived decoded pixels and GPU textures are caches.
PNG is the correct first format because its current W3C specification covers
lossless encoded pixels, alpha and explicit colour metadata. The executable
nuif-png-rgba8-0 baseline chooses an intentionally smaller contract:
non-interlaced RGBA8, no ancillary metadata or one valid sRGB chunk, encoded
samples interpreted as sRGB, straight decoded alpha, identity encoded orientation,
declared fit/crop/sampling/opacity and bounded integer CPU composition. png
0.18.1 and zune-png 0.5.2 must emit identical RGBA bytes for the accepted
fixtures; encoded resources remain digest-identical through package edits.
This avoids pretending that decoder agreement on simple images settles PNG
Third Edition. The compatible nuif-png-basic-rgba8-1 profile now admits the
lossless-to-RGBA8 subset: 1/2/4/8-bit greyscale and indexed images, RGB8,
greyscale-alpha8, RGBA8 and valid palette/colour-key transparency. It preserves
encoded bytes and requires exact normalized RGBA agreement between both
decoders. It is separately named so profile zero never changes meaning.
Image-paint affine semantics are orthogonal to decoder choice. The executable
matrix [a c tx; b d ty; 0 0 1] maps crop-local source coordinates forward
into the fitted rectangle. The CPU reference inverse-maps destination pixel
centers, clips to the entity, and rejects singular or numerically unbounded
matrices. Flip, rotation and translation fixtures make composition order
observable; live host trials are still required for vendor interoperability.
Decoded pixels are interned once per digest/profile in the renderer-independent scene and commands carry compact deterministic handles. A 64 MiB total is preflighted before each new inflation. The release gate retains one 1 MiB surface for 1,024 image instances under 8 MiB allocated and 4 MiB retained, and rejects a 64 MiB plus 16 byte declared total before the second decode.
Any broader profile still has to pin:
- accepted chunks and metadata conflicts;
- color-space precedence and output space;
- Exif orientation;
- conversion and premultiplication points for every accepted colour signal;
- sampling and compositing;
- encoded, pixel, decoded-byte, chunk and metadata limits;
- independent decoder and malformed-input fixtures.
16-bit/interlaced PNG, CICP/ICC/gamma/chromaticity, Exif, animation, perspective/tiling and host-specific affine equivalence are not claimed. A Linux/Windows/macOS CI matrix runs the profile, but the cross-platform claim remains withheld until its hosted artifacts pass. JPEG, WebP, AVIF, video and SVG follow as separate profiles. Freezing a frame or tracing a screenshot crop is a derived approximation, not recovery of the original asset. Generative upscaling/inpainting requires an explicit user policy and cannot silently become canonical source evidence.
Fonts
Exact typography depends on exact font bytes, face/collection index, variation axes, features, coverage, shaping inputs and renderer parameters. Packaging also depends on redistribution policy.
The proposed policy states are portable, private_authoring, linked,
substituted and unavailable. OpenType fsType is preserved as machine-
readable evidence, including restricted/preview/editable/no-subsetting/bitmap
flags, but is not treated as a complete legal license decision.
The executable nuif-opentype-static-single-0 baseline accepts only one
canonically packed, checksummed TrueType-outline sfnt face at index zero.
Skrifa 0.46.2 supplies package-facing metadata after NUIF validates the
directory, ranges, packing and checksums and directly checks required sfnt and
OS/2 fields. A committed hb-info 14.4.0 capture independently checks Ahem
metrics, family, tables and Unicode coverage. Exact bytes, family names,
coverage, fsType, license expression and explicit embedding review must
agree. Package encode/decode and caller-resolved linked bytes run the same
validation. Four static TrueType fixtures are accepted, while six package
trials distinguish portable, private-authoring, linked, substituted and
unavailable outcomes. Each accepted inspection and packaged-font validation is
also measured after warmup against a 4 MiB allocator-traffic and 2 MiB retained
reference ceiling; these are implementation regressions, not format semantics.
Six additional trials retain requested identity separately from a stable font
asset, render with an available declared replacement as approximated, and
emit no text command with item-level unsupported fidelity when replacement
bytes or the font are unavailable.
This is intentionally not general OpenType support. TTC, CFF/CFF2, variable,
color, bitmap, SVG and WOFF/WOFF2 sources, historic ambiguous permission
combinations, subsetting, cluster-level fallback, arbitrary packaged-font
shaping and cross-platform raster behavior remain separate fixtures and
profiles. The configured three-OS parser/package matrix does not establish
cross-platform raster behavior. Parser acceptance and fsType do not grant
redistribution rights.
Browser capture can identify platform fonts used for a node and capture downloaded web-font response bodies. It generally cannot retrieve arbitrary local font bytes. A family/PostScript name is therefore never exact resource identity. Missing bytes produce a link, substitution or unavailable fidelity record instead of a false portable-font claim.
Source-backed browser capture
Static Tree-sitter source synchronization and live browser capture solve different problems. The former preserves source spans for a bounded authored subset. The latter runs a pinned browser to observe actual cascade, layout, fonts, resource responses, accessibility and pixels. They should correlate through provenance, not become one oversized adapter.
The first browser-capture profile records browser/protocol build, OS, viewport, DPR, page scale, locale, timezone, color/reduced-motion preferences, font environment, scroll/pseudo state, navigation identity, readiness/network policy and animation freeze. It collects:
- original HTML/CSS and stylesheet text where accessible;
- DOM snapshots including available frame/template/shadow content;
- boxes, inline text boxes, paint order and a bounded style set;
- downloaded image/font/style response bodies and hashes;
- platform-font usage and font readiness;
- accessibility tree;
- reference screenshots with exact parameters.
Canvas, WebGL, video and worklets are bounded observation surfaces; a frame can be preserved without pretending its generating program was reconstructed. Cookies, authorization headers, credentials, storage and secret form values are not exported. Scripts remain inert.
Multiple viewports and states are more valuable than one oversized capture: they constrain layout hypotheses and permit held-out responsive evaluation.
The automated nuif-cdp-live-0 segment now implements that boundary for one
loopback fixture and exact Chrome for Testing 152.0.7977.64. It starts fresh
temporary profiles, retains a structured runtime context, waits for the exact
navigation loader and declared freeze/readiness point, captures bounded
DOM/layout/background/font/accessibility/resource/PNG evidence, and replaces
opaque browser node IDs with deterministic preorder identities. Four runs at
360, 768, held-out 900 and repeated 360 px retain exactly the expected five
response bodies and repeat the narrow capture bytes. Five exercised
query/cookie/storage/authorization/header canaries are absent from serializable
capture, observations, proposals and package bytes. The two fitted viewports
beat the one-viewport freeform baseline on the held-out fixture.
The separate bounded nuif-layout-inference-0 artifact ranks row stack,
column stack, Grid, linear constraint and fixed freeform candidates using only
the 360/768 px observations. It then evaluates the untouched 900 px holdout,
where the selected constraint records 0.0626 normalized error versus 0.2918
for freeform. All alternatives and exact geometry observation identities are
retained; confidence is raw and uncalibrated, and the result remains
inferred. The trial tests a mechanism on one fixture, not general accuracy or
recovery of original authored intent.
This is a falsifiable local baseline, not the entire profile described above. Cross-browser/OS reproduction, opaque frames and response bodies, full matched-style/source correlation, canvas/video frame capture, authenticated sites and licensed real-page evaluation remain open. WebDriver BiDi is the standards-track transport to revisit as its implemented evidence surface grows; Playwright is the higher-level candidate when NUIF owns a real multi-engine matrix and can make one tool the browser-version authority.
Screenshot reconstruction
A screenshot supplies visible samples but not the unique scene graph or layout program. The recommended pipeline is:
screenshots + contexts
-> OCR and baselines
-> deterministic regions, colors, edges, repetitions and asset candidates
-> optional replaceable UI grounding
-> typed observation graph with confidence and evidence regions
-> replaceable reasoner proposes hierarchy/layout and NUIF operations
-> core validates and applies atomically
-> deterministic layout/render
-> text/structure/geometry/resource/visual differences
-> bounded corrective operations
The model emits typed operations, not an unconstrained full document or code to execute. Invalid and stale transactions fail without partial state. High- resolution full views, overlapping tiles and semantic crops share explicit coordinate transforms; duplicate or conflicting observations remain visible.
The result contains a valid document or no-result, accepted operation log, observations, derived resources, item fidelity, alternatives/abstentions, evaluation report and exact pipeline artifact identities.
Evaluation before training
The benchmark has separate synthetic-exact, licensed real screenshot and source-backed suites. Synthetic NUIF rendering provides exact entities, properties, operations and resources. Real images need human-reviewed visible targets and must preserve ambiguity. Source-backed cases evaluate retained bytes and observations unavailable to the screenshot-only route.
Required metrics include:
- valid operation/document rate;
- OCR region recall, character/word error and baselines;
- element precision/recall and hierarchy error;
- property/geometry accuracy;
- held-out viewport behavior;
- exact resource digest only when bytes exist;
- provenance/fidelity honesty;
- accessibility evidence where justified;
- raw pixels, FLIP, SSIM and pinned LPIPS diagnostics;
- calibrated confidence, abstention and risk/coverage;
- latency, peak RAM/VRAM, iterations and cost.
No pixel score is sufficient. A page-sized screenshot can be visually perfect and semantically useless. Structural/text/resource/edit-task metrics prevent that reward shortcut. Dataset splits group by origin, template, component, font, resource and generator; near duplicates cannot cross splits.
The executable nuif-reconstruction-corpus-manifest-0 turns that rule into a
bounded audit. It pins the data snapshot, dataset card, evaluator and every
input/target by digest; records public/restricted/withheld disclosure and
evaluation/calibration/adaptation/redistribution permission independently; and rejects
exact artifact or declared family reuse across adaptation, calibration,
validation and test. Private/authenticated records also require explicit
authorization and a withdrawal-policy artifact. This is declaration integrity,
not automated legal review or duplicate discovery; real records and their group
assignments still need independent human/tool review.
The ablation ladder is deterministic OCR/CV, one-shot reasoner, observation- assisted reasoner, hierarchical crops, multi-viewport ranking, correction loop, then any tuned or distilled student. Every addition uses the same frozen holdout and budget.
Adaptation and distillation
Training is justified only after the untuned loop and error taxonomy are reproducible, rights-cleared traces exist, and the remaining errors appear learnable. Training examples contain input hashes, observation versions, proposals, diagnostics, accepted operations, intermediate renders/differences and final package/fidelity reports. Positive sequence targets are validated accepted transitions, not raw model transcripts.
Compare prompt/schema/tool improvements and retrieval before fine-tuning. If adaptation remains justified, compare ordinary supervised tuning, low-rank adaptation and quantized low-rank adaptation under equal data and evaluation. Quantized adaptation is a memory technique, not an accuracy claim.
Sequence-level distillation may train a smaller student from the best evaluated teacher pipeline. The teacher is a measured system of tools plus a model, not a provider name. Distillation transfers errors too, so render validation and held-out evaluation remain mandatory.
Models, processors, adapters and datasets are separately versioned optional
artifacts with digests, model cards, dataset datasheets, license lineage and
training manifests. They never redefine nuif-core or travel as ordinary
document resources.
The executable provider boundary uses a deliberately small canonical wrapper, not a new AI bill-of-materials vocabulary. It binds NUIF capabilities, execution modes and observation/proposal profiles to exact implementation, model, processor, adapter, quantization, prompt and tool artifacts. Observation bundles carry the complete manifest registry, so a digest cannot dangle and a proposal cannot substitute an unpublished provider before mutation. Released or learned providers point to content-addressed SPDX 3.0.1 or CycloneDX 1.7 inventory; learned providers also point to a model card. This complements runtime packaging such as MLflow or ONNX external data without making either a required NUIF dependency.
Private/authenticated captures default to local processing, no retention and no training. Remote transfer, telemetry, retention and training are independent consent/policy decisions.
Maturity boundary
The current 0.1.0-alpha.3 label belongs to the developer editor application.
It provides no evidence that the broad image/font resource, portable browser
capture or screenshot reconstruction accuracy profiles are complete. A
deterministic package, narrow PNG/static-font segments, fixed provider-input
contracts and one pinned local live-browser segment are implemented, but their
deliberately narrow evidence does not promote the broader profiles.
Promotion requires the package/resource cross-writer fixtures, pinned capture reproduction, baseline/closed-loop/calibration harness, leak-resistant licensed evaluation data, independent result reproduction and at least one real edit workflow that benefits from the inferred semantics. Until then the work is research and proposed specification text, not a standard or production reconstruction promise.
Primary research records
nuif:research:resource-packaging-and-source-capture-synthesisnuif:research:model-agnostic-screenshot-reconstruction-and-trainingnuif:research:provider-artifact-manifests-and-ai-bomsnuif:research:epub-ocf-package-containernuif:research:oci-resource-descriptorsnuif:research:opentype-font-embedding-and-portabilitynuif:research:ttf-parsernuif:research:fontationsnuif:research:chromium-source-backed-ui-capturenuif:research:live-chromium-cdp-capturenuif:research:design2code-real-world-benchmarknuif:research:pix2struct-screenshot-parsing-pretrainingnuif:research:screenai-ui-annotationnuif:research:confidence-calibration-and-selective-predictionnuif:research:lora-low-rank-adaptationnuif:research:qlora-quantized-adaptationnuif:research:sequence-level-knowledge-distillation
NUIF specification
Document status: pre-draft. No module or conformance profile is published as an. Canonical source.
accredited standard.
This directory contains the candidate normative text. Research documents can motivate semantics but cannot silently define them. The current modules remain draft inputs until their status, licensing terms and corresponding conformance requirements are published through the governance process.
Planned modules
- Core document and identity model
- Components, instances, parameters, variants, and slots
- Layout and responsive evaluation
- Geometry, paint, color, typography, and assets
- Tokens and themes
- Interaction/state and data binding
- Operations, transactions, diff, patch, and reconciliation
- Extensions/dialects and capability negotiation
- Serialization and package format
- Observation, capture, reconstruction, confidence, and inference provenance
- Conformance, diagnostics, fidelity, and security
A specification statement becomes normative only when its module explicitly declares normative status and the conformance suite contains corresponding executable tests.
Conformance and fidelity
Document status:
draft. Canonical source.
A conforming implementation MUST NOT silently discard semantically relevant information. Import, lowering, migration, and export operations MUST be able to emit structured diagnostics.
Initial fidelity classes are:
losslessrepresentableapproximatedpreserved_unrenderableunsupported
Capability profiles will identify which optional modules and extensions an implementation can evaluate, render, mutate, and preserve.
Unknown extensions MUST be preserved byte-for-byte or canonical-value-for-canonical-value when the package/encoding permits preservation and when doing so does not violate security policy.
Logical model
Document status:
draft. Canonical source.
The working NUIF model is a layered hybrid rather than a universal AST.
Identity
Every durable authored entity has a stable EntityId. Identity is independent from display names, containment position, serialization offsets, and vendor IDs. External correspondences are recorded separately as provenance.
Containment
Documents contain ordered authored entities. Containment expresses lifetime/ownership and author-facing hierarchy only.
Coordinated relations
Relationships that are not ownership belong in typed relation sets/graphs: component-instance links, token references, constraints, interactions, state transitions, dependencies, provenance, and extension-defined relations.
Authored and resolved data
Authored data expresses intent. Resolved data is derived for a named evaluation context such as viewport, scale, font environment, capability profile, and theme. Resolved data is cacheable/reproducible output and MUST NOT overwrite authored intent.
Extensions
The core admits namespaced extension values. An implementation may understand, preserve without understanding, approximate, or reject an extension according to declared capability and security rules.
02 — Identity and properties
Document status: draft. Canonical source.
Entity identity
Every authored entity MUST have a stable 128-bit-or-greater identifier. Identity MUST NOT depend on name, path, child index, geometry or serialized byte offset. Implementations MAY use UUIDv7/UUIDv4-compatible identifiers; the normative requirement is uniqueness and stability, not one generation algorithm.
Immutable assets MAY additionally use cryptographic content IDs.
Property model
A property is addressed by (entity_id, namespace, key). Core properties use the nuif namespace. Published extensions use registered namespaces and vendor extensions use registered vendor identifiers.
Properties distinguish:
- authored value;
- token/expression binding when present;
- resolved value for an evaluation context;
- provenance/correspondence metadata.
Resolved values MUST NOT overwrite authored values.
03 — Components and composition
Document status: draft. Canonical source.
A Component is a reusable authored definition with typed parameters and named slots. An Instance references a component and carries parameter values and non-destructive overrides.
Core parameter classes: boolean, number, string/text, enum, token reference, asset reference and content/slot value.
Variants are named parameter configurations, not duplicate component definitions.
Composition supports references to external NUIF libraries and ordered override layers. Themes and brands SHOULD be expressed as token/layer opinions rather than destructive copies.
DTCG-compatible design tokens are the default token interchange representation. NUIF bindings add stable token identity and resolved context values.
04 — Layout
Document status: draft. Canonical source.
NUIF defines authored layout separately from resolved layout.
Core families
freeform, stack, flex, grid, constraint and extension-defined custom.
Shared sizing
Axes support fixed, auto/intrinsic, min-content, max-content, fit-content, percentage and fill/available sizing plus min/max clamps and aspect ratio.
For the CSS-compatible stack/flex subset, stretch applies only when the item’s cross-axis size is auto; an explicit fixed, intrinsic, percentage or fit-content cross size wins. fill is an explicit request for the available cross size regardless of the container’s alignment. This distinction is covered by the Gate C foreign-reference regression.
Responsive rules
Conditional authored values use predicates over declared evaluation features: viewport/container dimensions, orientation, input capability, theme/media preference and named application conditions. Predicates MUST be deterministic for a supplied evaluation context.
Resolved layout
An evaluator emits boxes/transforms plus diagnostics and the context fingerprint. Resolved geometry may be cached/serialized but is derived unless the authored family is freeform.
Portability
Lowering to a target layout model MUST return fidelity records for every rule that is approximated, preserved-unrenderable or unsupported.
Bounded explicit Grid
Profile 0 defines a deliberately finite Grid subset. It is a portable authored layout primitive, not an alias for the complete CSS Grid algorithm.
A grid container MUST declare one or more columns and one or more rows in
its layout.grid value. Each track is either:
fixed(n), wherenis a positive finite pixel length; orfraction(n), wherenis a positive finite flexible weight.
Profile 0 permits at most 256 tracks on either axis and 4,096 grid tracks in a document. It has no intrinsic, percentage, auto, repeated, named, subgrid, masonry or implicit tracks. Importers MUST report those foreign features as a loss, approximation or preserved extension rather than silently lowering them to this subset.
The one container gap is used for both axes. For one axis, let inner be the
non-negative content-box length after padding, gaps be gap * (track_count - 1), fixed be the sum of fixed tracks, and weight be the sum of fractional
weights. Track sizes are:
remaining = max(0, inner - gaps - fixed)
fr = remaining / max(1, weight)
fixed(n) = n
fraction(n) = n * fr
The max(1, weight) rule intentionally follows CSS Grid’s fractional sizing
rule for a total flex factor below one. Fixed tracks can overflow the content
box. Fractional tracks never receive negative space, and a fractional weight
sum below one can leave unused space at the end of the axis.
Grid-item column and row are zero-based explicit track indices. An item MUST
provide both or neither. column_span and row_span default to one and MUST be
positive. Explicitly positioned items reserve their complete rectangular areas
before any auto placement, independent of child order. Overlap and
out-of-bounds areas are invalid.
Items with neither index are placed in child order. auto_flow: row scans rows
then columns; auto_flow: column scans columns then rows. Scanning begins at the
first cell and a cursor advances past the last placed item’s span. Each item
occupies the first unoccupied rectangle at or after that cursor that fits its
spans. The cursor does not move backwards to fill earlier holes; this is the
sparse CSS grid-auto-flow behaviour, not dense. The explicit grid is
exhausted when no such rectangle exists; an evaluator MUST NOT create implicit
tracks.
An item’s grid area includes the gaps crossed by its spans. fill consumes the
area on that axis. auto consumes the area when container alignment is
stretch, otherwise it resolves to intrinsic size. Other size intents resolve
against the grid-area length. start, center and end place the resulting
box on both axes; stretch places it at the area’s start and only stretches
auto or fill. Authored freeform position is ignored for an in-flow grid
item. Descendants are evaluated within the resulting item box.
Using layout.grid on another layout family, using non-default grid placement
without a direct Grid parent, or using the grid family without valid explicit
tracks is invalid. This removes the former stack-flow fallback: a conforming
implementation either implements these semantics or reports the Grid feature
as unsupported.
Differential context
CSS-compatible layout claims are checked against exact pinned Taffy and browser versions. The machine report MUST retain the generator source revision, browser executable/version, seed, viewport, all compared boxes, per-fixture measured tolerance and a typed classification for every value outside that tolerance. A global unexplained tolerance is not conforming evidence.
05 — Geometry, paint and text
Document status: draft. Canonical source.
Geometry follows established 2D vector mathematics: affine transforms, rectangles/rounded rectangles, ellipses, lines and Bézier paths. Path semantics SHOULD align with SVG where possible.
Paint supports solid colors, gradients, images, strokes, opacity, clipping/masks and compositing/blend modes. Color values MUST declare a color space; conversions are evaluator responsibilities.
Asset and resource boundary
An asset is a semantic entity with stable AssetId. A resource is an immutable
byte sequence identified by ResourceDigest. Package paths and external
locators are resolution hints and MUST NOT be used as either semantic or byte
identity.
Replacing the bytes bound to an asset MUST be expressed as a semantic operation
that preserves AssetId and changes its ResourceDigest. Every resource
descriptor declares media type and byte length. An implementation MUST verify
the declared size and digest before media-specific decoding.
Resource roles are:
source— exact bytes received from an origin;authoring— bytes required to evaluate or edit the semantic document;derived— bytes produced from named inputs by a declared transformation;cache— deletable acceleration data that cannot affect semantic hashes.
Derived resources MUST identify all source digests and the transformation
artifact/profile. A crop or trace created from a screenshot is derived; it
MUST NOT be presented as the original source asset.
Image assets
ImageAsset records AssetId, current encoded-resource digest, intrinsic pixel
dimensions and decoder profile. ImagePaint refers to the asset and records
fit, crop, affine transform, sampling, opacity and color-conversion policy.
Decoded pixels and GPU textures are caches keyed by encoded digest plus decoder
profile.
Renderer-independent scenes MUST store a unique decoded surface once per resource-digest/decoder-profile pair and reference it from image commands. The reference scene budget is 64 MiB of decoded RGBA surfaces and MUST be checked from bounded metadata before allocating the next decode.
The affine fields use [a c tx; b d ty; 0 0 1] and map the selected crop’s
normalized source coordinates forward into normalized coordinates of the
fitted paint rectangle. Fit is calculated first. Rasterizers inverse-map
destination pixel centers, apply crop selection after that inverse, and clip to
the entity rectangle. The origin is the fitted rectangle’s top-left; callers
encode any center-origin adjustment in the translation. Executable matrices
MUST be finite and invertible. The reference bound rejects components or
inverse components above 1,000,000 in magnitude and determinant magnitudes
below 1e-12 as unsupported fidelity.
The first executable image profile, nuif-png-rgba8-0, is PNG-only and
deliberately narrow. It accepts non-interlaced RGBA8 with no ancillary chunk or
one valid pre-image sRGB chunk, interprets the encoded samples as sRGB and
keeps alpha straight through decoding. It rejects palette, grayscale, RGB-only,
16-bit, CICP, ICC, gamma/chromaticity, Exif, animation, arbitrary ancillary
chunks and trailing bytes. Its exact chunk sequence, dimensions, pixel/byte
budgets, fit/crop, bounded affine transform, nearest/fixed-bilinear sampling, opacity
and encoded-sRGB integer composition contract is in
crates/nuif-media/PROFILE.md. JPEG, WebP, AVIF, animation, video and SVG are
unsupported by that profile rather than silently decoded through host defaults.
The separately named nuif-png-basic-rgba8-1 profile accepts the
non-interlaced PNG colour/depth combinations that normalize to RGBA8 without
sample-precision loss, including required palettes and valid tRNS
transparency. It does not change profile zero and still rejects 16-bit,
interlaced and colour-managed inputs. Its exact matrix is also in
crates/nuif-media/PROFILE.md.
An animation or video adapter MAY create a derived still resource when it records source digest, selected frame/time, decoder profile and item-level loss. SVG is evaluated only through a declared safe adapter profile; otherwise its bytes remain inert and preserved.
Font assets and portability
Text stores Unicode scalar content, style runs, paragraph attributes,
direction/language hints and a requested content-addressed font identity. A
font SHA-256 reference MUST contain 64 lowercase hexadecimal digits. Font size
and line height MUST be finite and positive. An optional font_asset binds the
text item to a stable font asset; a family or PostScript name is never a
substitute for that identity.
A font asset additionally records media type, face or collection index, names
used for matching, variation axes, feature selections, coverage and portability
policy. The policy is portable, private_authoring, linked, substituted
or unavailable. OpenType embedding flags and explicit license metadata are
policy evidence; the format does not claim to make a complete legal decision.
A portable package MUST NOT embed a font whose effective export policy forbids that embedding. Linked fonts retain expected digest and explicit resolver hint; resolution is opt-in and digest-checked. Substitution and unavailability MUST produce item-level fidelity. Family/PostScript names alone MUST NOT satisfy an exact-font profile.
For an exact binding, the asset resource SHA-256 MUST equal the requested text
hash. For a substituted binding, the text retains the requested hash and the
asset resource identifies the exact replacement bytes. For an unavailable
binding, the asset MUST carry no resource. Layout and rendering MUST use an
available declared replacement with approximated fidelity; if replacement
bytes are absent, or the asset is unavailable, rendering MUST emit no text
command and MUST report item-level unsupported fidelity. Resolution MUST NOT
query a platform font database or perform I/O.
The first executable resource subset, nuif-opentype-static-single-0, accepts
only one canonically packed, checksummed TrueType-outline sfnt face at index
zero. It requires exact font/ttf bytes, matching family names and Unicode
coverage, no variation axes, matching fsType evidence, a non-empty license
expression and an explicit embedding review. It rejects TTC, CFF/CFF2,
variable, color, bitmap, SVG and WOFF/WOFF2 sources. Exact limits and non-claims
are versioned in crates/nuif-font/PROFILE.md. This narrow package resource
profile does not establish shaping or raster equivalence.
A conformance profile that compares resolved text MUST declare the exact font bytes and hash, shaper and Unicode-data versions, direction, language, script-selection rule, feature set, cluster level, cluster coordinate unit, positioning unit and resource limits. Resolved runs contain source text plus ordered glyph identifiers, clusters, advances and offsets; they MUST NOT depend on system font discovery. Profile 0 uses Unicode-scalar indices for cluster coordinates and unscaled font units for advances and offsets.
Shaping and rasterization are distinct conformance stages. A shaping pass does not imply raster conformance. A raster profile MUST additionally declare outline extraction, hinting, stem darkening, anti-aliasing, subpixel quantization, color/blend space and compositing rules. Until those parameters and their foreign/cross-platform trials exist, an implementation MUST classify a glyph-ID bitmap proxy as approximated rather than exact text rendering.
Implementations MUST preserve source text even when resolved glyph information is present.
CPU render profile 0
Profile 0 is deliberately narrower than the complete model. Its supported visual operations are encoded-sRGB solid fills of rectangles and ellipses, plus the text subset below. Color channels are finite numbers in the inclusive range 0 through 1. The reference raster starts as opaque white RGBA8.
Logical geometry is multiplied by the target scale factor before rasterization. A rectangle covers every pixel in floor(x)..ceil(x + width) and floor(y)..ceil(y + height), clipped to the target; it does not compute fractional edge coverage. Float color channels become bytes with round(clamp(channel, 0, 1) × 255). For mask coverage coverage, effective source alpha is (source_alpha × coverage + 127) / 255 using integer division. Each encoded-sRGB destination color channel becomes (source_channel × alpha + destination_channel × (255 - alpha) + 127) / 255; output alpha remains 255.
An ellipse is the closed four-cubic path inscribed in its bounds using control coefficient 0.551915024494. It is rasterized with nonzero fill into an 8-bit grayscale mask by Zeno 0.3.3, crates.io checksum 6df3dc4292935e51816d896edcd52aa30bc297907c26167fec31e2b0c6a32524, then composited by the integer rule above. conformance/render/profile-zero-v1.json fixes rectangle and ellipse scene/PNG hashes on the recorded platform matrix.
Profile-0 text uses Ahem 1.50, HarfRust 0.13.3 with Unicode 17.0.0, unhinted Skrifa 0.46.2 outlines in signed 26.6 font units, and Zeno 0.3.3 grayscale masks. CRLF is one hard break; CR, LF, NEL, LINE SEPARATOR and PARAGRAPH SEPARATOR are individual hard breaks. Each hard line is shaped independently. Intrinsic width is the greatest shaped line advance; intrinsic height is the number of hard lines times line_height. The first baseline is 800 Ahem font units below the line top, subsequent baselines differ by line_height, LTR starts at the left edge, RTL starts at the right edge, and output is clipped to the text box. Profile 0 performs no automatic soft wrapping. Because wrapping is not an authored property in this profile, absence of soft wrapping is exact profile behavior rather than an approximation.
Path geometry, image assets, component-instance materialization and extension-defined paint/effects are not supported by CPU render profile 0. Lowering MUST emit unsupported or preserved_unrenderable fidelity with the originating entity and property pointer; it MUST NOT substitute bounds rectangles or silently omit the data. nuif-png-rgba8-0 is an orthogonal experimental image segment and does not change profile-0 results. Future profiles may compose accepted segments explicitly.
The asset and broad-font requirements above remain draft inputs for broader profiles. The narrow executable image and static-font resource segments do not implicitly enter CPU render profile 0.
06 — Operations, patches and merge
Document status: draft. Canonical source.
Operations are stable, serializable semantic mutations. Core operations include entity insertion/removal/move/rename; typed size, position, layout, fill and text edits; generic authored-value edits; token and extension edits; and transaction grouping. SetPosition, SetFill and SetText replace their complete typed property and therefore have exact inverse operations. An editor that applies several related controls together MUST place them in one transaction so validation and undo remain atomic.
Position (RFC 0006)
Sibling order in the canonical document is an ordered array of entity identifiers without keys, tombstones or replica metadata. Insert and Move specify position as an anchor, Start or After(entity), never as an integer index. An anchor MUST refer to a current child of the target parent; otherwise the operation fails with AnchorMissing. A move into the moved entity or its descendants fails with CycleRejected. Insertions at the same anchor within one patch apply in patch order.
Patches
A patch’s optional base_revision, when present, is the profile-qualified canonical content hash of the document to which it applies. An implementation MUST reject a mismatch before applying any transaction. Transactions and operations are ordered. Preconditions MAY guard expected prior values, including ParentIs and Follows.
Undo is represented as inverse semantic operations or transaction history; it is not part of canonical document state. A generated inverse patch declares the hash of the post-apply document as its base_revision. The invariant “undo, copy, redo leaves the document unchanged” is a conformance relation.
Merge
Three-way merge MUST prefer stable identity; structural matching is reserved for adapters without identity. Implementations MUST surface typed conflicts rather than selecting arbitrary winners: property, delete/edit, ordering (OrderAmbiguous, informational, ordered by declared branch precedence), move (MoveConflict), relationship, extension and semantic-lowering conflicts. Conflict objects are document state until resolved.
Merge rules per property kind (ordered list, identity set, scalar with tolerance) are declared by the schema so that structural merges are deterministic.
Unknown entities (RFC 0007)
Remove, Move, Rename, SetExtension, RemoveExtension and core-property operations apply to entities of unknown kind unchanged. SetUnknownPayload is valid only for implementations that declare the payload’s namespace.
07 — Extensions and dialects
Document status: draft. Canonical source.
NUIF uses a small core and namespaced extensions.
Documents declare extensions_used and extensions_required, and MAY declare a fallback_kind per namespace. A required extension means correct interpretation/rendering cannot be guaranteed without it.
Extension lifecycle namespaces use the lowercase identifier grammar from RFC 0005:
nuif.*— ratified NUIF extension;ext.*— multi-implementation experimental extension;- a collision-resistant vendor/project namespace such as a reversed DNS name — owner-specific extension.
Preservation (RFC 0002, RFC 0007)
Unknown extension payloads MUST be preserved at their attachment point unless an operation explicitly removes the owning entity/property. An implementation that does not declare a namespace preserves its payloads byte-for-byte; one that declares it MAY re-encode deterministically. Payloads are opaque byte strings with a declared encoding (Cbor or Octets); a malformed payload yields a diagnostic on its owner and does not invalidate the document.
An entity of unknown kind, or of a known kind with a newer schema version than supported, loads as Unknown with namespace, kind name, schema version and payload retained; its core fields stay typed and editable; layout uses the declared fallback_kind or Container; rendering reports preserved_unrenderable.
Validation
- namespace present but not in
extensions_used: error; - namespace in
extensions_usedand unsupported: information; - namespace in
extensions_requiredand unsupported: blocks faithful-rendering claims, never structural editing.
Promotion of an ext.* namespace to nuif.* requires a conformance fixture and a validator rule (adapted from the glTF extension status ladder).
Dialects may define higher-level authored constructs and lowering rules. A dialect cannot redefine core semantics.
08 — Serialization and package format
Document status: draft. Canonical source.
The NUIF logical model is encoding-independent.
Initial profiles:
nuif-text-0— deterministic human-readable canonical form for fixtures/review.nuif-cbor-0— deterministic CBOR following draft-ietf-cbor-serialization §4.1 (preferred serialization) and §5.1 (bytewise-lexicographic map key order), with the narrowing rules of RFC 0005 stated by value.
cargo xtask codec-benchmark is the non-normative decision harness for these
profiles. It records size, latency and allocation only after exact semantic,
canonical and unknown-data edit fixpoints pass. Native partial loading is a
separate capability: profile-0 decoders currently load the complete document.
Schema-generated candidates MUST NOT be compared using a partial logical model;
the active next-candidate investigation is Cap’n Proto because its encoding
specification defines a canonical form. No schema-generated NUIF profile is
currently accepted.
The experimental nuif-package-0 profile assigns .nuif to a deterministic ZIP
container. Bare encodings use .nuif.json and .nuif.cbor. Historical alpha
files that used .nuif for bare bytes MAY be recognized read-only through
content detection, but new .nuif output MUST be a package once this profile is
accepted.
nuif-package-0 is proposed by RFC 0010. Its reference codec, cross-writer byte
fixture, package/resource identity relations, explicit resolver and hostile
archive/one-over suite are executable through cargo xtask gate-i-package.
The separate nuif-png-rgba8-0 segment is executable through
cargo xtask gate-i-image; nuif-opentype-static-single-0 is executable
through cargo xtask gate-i-font. This is not full Gate I: broader PNG and
OpenType interpretation plus cross-platform/external package and media evidence
remain incomplete.
Numeric and string rules (RFC 0005)
- Numeric kinds are
integer(signed 64-bit) andreal(binary64). Authored reals MUST be finite. Negative zero is not distinct from zero. - In
nuif-cbor-0, integers use major type 0 or 1 and reals use the shortest IEEE 754 floating-point width that round-trips, including integral reals. Integer and real are distinct logical values. Both real zeros use positive floating-point zero; integer zero remains distinct. Integer heads MUST be shortest; lengths MUST be definite; map keys MUST be strictly increasing in bytewise order of their complete deterministic encoding; no tags and no simple values other thanfalse,trueandnullappear; extension and unknown-kind payloads are byte strings hashed verbatim (RFC 0008). - Decoders used for hashing and conformance MUST reject non-canonical input rather than re-canonicalize it.
- In
nuif-text-0, reals print as the shortest round-trip decimal in the fixed layout of RFC 0005 rule 15;NaNand infinities are parse errors; keys are in UTF-8 byte order; layout is not significant. Text key order is intentionally independent from CBOR encoded-key order (RFC 0008). - Identifiers (namespaces, keys, kind names, extension names) match
[a-z0-9][a-z0-9_.:-]*. String values are stored verbatim as valid UTF-8 and are never normalized by canonicalization.
Hash
The canonical hash of a document is SHA-256 over its nuif-cbor-0 bytes. The text profile has no separate hash: hash(text) = hash(cbor(parse(text))). Published content identifiers carry the profile identifier. Canonical hashes MUST exclude transport-only compression differences.
Package, resource and semantic hashes are distinct:
document_hashis SHA-256 of canonicaldocument.cborand covers semantic asset bindings;resource_digestis SHA-256 of exact resource bytes;package_hashis SHA-256 of the complete deterministic package bytes.
Package-only caches or reports may change package_hash without changing
document_hash. Replacing a resource bound to an asset changes the resource
digest and semantic document hash while preserving stable AssetId.
Proposed package profile 0
The package member set is:
mimetype
manifest.cbor
document.cbor
blobs/sha256/<digest-hex>
The first member is stored mimetype with exact ASCII value
application/nuif+zip. This media type remains provisional until registration.
manifest.cbor and document.cbor are canonical nuif-cbor-0.
The manifest declares package profile/version, the canonical document descriptor, required capabilities, stable assets and every immutable resource descriptor. A descriptor includes media type, SHA-256 digest, size, role and an embedded or explicit linked locator. The manifest is not self-addressed.
Required capabilities are a set of at most 256 identifiers, each at most 128
ASCII bytes and matching [a-z0-9][a-z0-9_.:-]*. Structural package decode
MUST validate and preserve this set without pretending the host implements it.
Before claiming full package support, a host MUST compare the complete set with
capabilities it explicitly declares. Missing requirements MUST be returned as
an exact deterministic set. Inspection, preservation and migration tools MAY
operate after structural decode without claiming full semantic or behavioral
support and MUST NOT execute a resource merely because its capability is
declared.
Profile 0 uses stored ZIP members only; mimetype is first and other names
are bytewise sorted. Names are exact ASCII registered paths. Writers use fixed
timestamps/header attributes, no comments/extra fields/data descriptors,
encryption, directories, ZIP64 or split archives. The manual reference writer
and zip 8.6.0 independently reproduce the exact header fixture.
Readers MUST reject duplicate decoded names, non-ASCII/backslash/absolute/dot paths, directories, symlinks, encryption, unsupported compression, inconsistent headers, unknown members, undeclared blobs, missing required blobs and size/digest mismatches. Readers MUST NOT extract package members to a filesystem.
Portable packages embed every resource required by their declared profile. Linked resources are explicit and never fetched implicitly; a caller-supplied resolver verifies expected size and digest before use. Credentials MUST NOT be stored in resource locators.
Schema versions
Every serialized record kind carries a schema version. Migrations are registered pure functions per kind; reading a record whose version is newer than the implementation knows is an error with a diagnostic, never silent loss.
Parsers MUST enforce resource limits and reject cycles where the relevant graph is specified acyclic. The experimental package limits are 80 MiB per archive, 32 MiB per resource, 64 MiB total embedded resources, 8,192 descriptors and 256 required capabilities of at most 128 bytes each. nuif-png-rgba8-0 additionally limits encoded input to 32 MiB, each dimension to 8,192, pixels to 16,777,216 and chunks to 4,096. nuif-opentype-static-single-0 limits encoded input to 32 MiB, tables to 256, family names to 256, coverage ranges to 65,536 and feature settings to 64. Broader image interpretation and general font-format/policy limits remain experiment-required and MUST be accepted through later media profiles before implementations claim those capabilities.
09 — Provenance, correspondence and fidelity
Document status: draft. Canonical source.
ProvenanceRecord identifies an origin system, source artifact revision and optional source path/range/node/property identity.
CorrespondenceRecord maps NUIF identities/properties to one or more foreign identities/properties and can retain adapter-specific reconstruction hints.
Correspondence is optional canonical-adjacent metadata: it may be stored in a package/profile without changing the semantic document.
Every import/export/lowering returns a FidelityReport with item-level status: lossless, representable, approximated, preserved_unrenderable or unsupported. Diagnostics identify the entity/property and transformation pass responsible.
This mechanism is informed by symmetric and retentive lens research.
Evidence classes
Provenance for capture/reconstruction additionally declares one evidence class:
authored_sourceresolved_sourceobserved_pixelsinferreduser_confirmedderivedunavailable
The evidence class and fidelity status answer different questions. Evidence states what was available; fidelity states what was preserved or represented. Confidence states predicted correctness and MUST NOT promote a weaker evidence class to a stronger fidelity claim.
authored_source MAY be lossless only under a declared adapter round-trip
profile. resolved_source can be exact for one pinned resolved context without
proving authored intent. observed_pixels and inferred MUST NOT be lossless
for authored structure, original resources, responsive rules, accessibility or
behavior. User confirmation is an additional record and does not erase the
original inference history.
A derived record identifies its input digests and exact deterministic or
generative transformation. Screenshot crops, traces, inpainting and upscaling
are derived resources, never recovered originals.
Confidence and alternatives
Raw provider confidence and calibrated confidence are separate fields. Calibrated confidence identifies a versioned calibration profile and typed correctness event. Whole-document confidence cannot substitute for property- or decision-level confidence.
When evidence is ambiguous, an implementation SHOULD preserve ranked alternatives or abstain. Automatic application MAY be governed by a declared risk/coverage policy; below-threshold values require review or remain unresolved.
Observation identity
An observation records source artifact digest, source region/locator, coordinate space, evaluation context, provider artifact/version, candidate values, evidence class, confidence and privacy/retention class. Coordinate transforms between source pixels, device pixels, viewport pixels, crop-local pixels and NUIF units are explicit.
10 — Collaboration profile
Document status: executable bounded register and existing-tree structural profiles. Canonical source.
Collaboration is operation-based and layered above canonical NUIF.
A collaboration engine MUST be able to materialize a canonical NUIF snapshot without collaboration metadata. Replica IDs, clocks, tombstones and sync-state are profile data.
The profile defines convergence requirements, causal/change identifiers, transaction grouping, awareness/presence separation and checkpoint materialization. It does not mandate Automerge or Yjs.
Semantic conflicts that cannot be merged safely remain explicit conflict objects even if the underlying CRDT converges structurally.
Executable register profile 0
nuif-collab-registers-0 represents each collaboration change as a replica/counter dot, a transitive version-vector context and one semantic operation. The metadata lives in the profile state and is stripped from the materialized Document.
Register-like operations use one multi-value register per entity/property pointer. Causally superseded values leave the frontier. Concurrent distinct values remain in an explicit SemanticConflict; a deterministic selected dot permits a provisional canonical checkpoint without discarding the candidates from the checkpoint report. The operation-set join is commutative, associative and idempotent, and incomplete causal histories fail closed.
Profile 0 supports rename, size, container layout, grid-item placement, token, authored-value, extension-declaration, entity-extension and unknown-payload registers. It rejects insert, remove, move and restore-subtree. Structural collaboration requires a declared tree move/list algorithm, cycle handling and tombstone policy and MUST NOT be inferred from register convergence.
cargo xtask gate-h compares an operation-set maximality materializer with an incremental replica-log frontier materializer over every delivery permutation of the bounded conflict fixture. These are algorithmically separate in-repository implementations, not foreign-engine interoperability evidence.
Executable existing-tree structural profile 0
nuif-collab-tree-0 accepts only Move and Delete for entity identities
already present in one validated canonical base. Creation, subtree payloads,
relations, property changes and mixed structural/property transactions are not
part of this profile. A structural change uses the same dot and transitive
version-vector requirements as the register profile. Every engine and joined
operation set is bound to the canonical hash of exactly one base; merging
different bases is a typed failure. The dot’s total order is (counter, replica).
A move carries (entity, new_parent, anchor). anchor is either Start or a
stable position identifier. A base position is Base(entity_id); every move
creates Change(dot). An authoring surface MUST resolve a canonical
After(entity_id) against its current checkpoint and persist the resulting
stable position. It MUST NOT reconstruct a stale anchor from the entity’s
current position after synchronization. A Change(dot) anchor MUST name a
received change included in the author’s transitive causal context; a missing
or non-causal anchor change fails the checkpoint as incomplete history.
The materializer MUST behave as if all changes were applied in ascending dot order:
- An unknown entity or parent is a typed failure and produces no checkpoint.
- Moving an entity below itself or its current descendant has no structural
effect, remains in the applied history and emits
CycleRejected. - A missing anchor, an anchor from another parent, or the entity’s own position has no structural effect and emits its typed anchor conflict.
- A valid move deactivates the entity’s prior position and creates an active position under the target parent. Inactive positions remain as sibling origins.
- Delete deactivates the prior position and assigns the entity to synthetic profile trash. Descendant relationships are retained in profile state.
- A later valid move may restore a trashed entity or rescue one of its descendants.
Sibling order is an RGA-style origin traversal. Positions with the same origin sort by descending position identifier, followed recursively by their own descendants. Base sibling order is represented as an origin chain. Position IDs, inactive positions, clocks and trash are profile metadata. The canonical checkpoint stores only ordinary ordered child arrays and removes every entity not reachable from a visible root; it MUST validate and hash as canonical NUIF.
Concurrent distinct moves of one entity, delete/move of one entity, deletion of a destination parent and deletion of a moved entity’s base ancestor MUST remain typed semantic conflicts. Total ordering selects a provisional tree but does not erase those candidates.
Gate H compares full sorted replay with an incremental local/rollback-replay engine for all 5,040 deliveries of the bounded fixture, multiple joins, duplicate delivery and a 4,096-change scaling case. Pinned Automerge 3.4.1 must reproduce the exact immutable operation set through different merge orders and save/load. That foreign check covers convergent transport only; it is not an independent implementation of these tree semantics.
11 — Security and resource limits
Document status: draft. Canonical source.
NUIF documents and extensions are untrusted input.
Implementations MUST bound decoded sizes, nesting depth, entity/relation counts, path segment counts, image/font sizes, decompression ratios and renderer resource allocations. Cyclic references MUST be detected where forbidden.
Executable profile-0 limits
An implementation claiming executable profile-0 conformance MUST accept values at these boundaries and MUST reject the first value above them as a resource-limit error:
| Resource | Limit |
|---|---|
| encoded document bytes | 16 MiB |
| text or CBOR syntax depth | 64 |
| entities | 8,192 |
| roots | 4,096 |
| tokens | 8,192 |
| relations | 32,768 |
| child references | 8,191 |
| responsive overrides | 16,384 |
| property values | 65,536 |
| property-value depth | 24 |
| containment depth | 128 |
| total retained string bytes | 8 MiB |
| bytes in one string | 1 MiB |
| total retained binary bytes | 8 MiB |
retained binary bytes in nuif-text-0 | 512 KiB |
Stream readers MUST stop after reading the first byte beyond the encoded limit; reading an entire larger stream and checking afterward is non-conforming. Syntax depth MUST be checked outside quoted strings and comments. Semantic limits MUST be checked before recursive evaluation. Encoders MUST stop before producing the first byte beyond the encoded limit.
Validators MUST retain no more than 1,024 ordinary diagnostics plus one explicit truncation diagnostic. Implementations MAY use tighter operational limits when they are not making a profile-0 conformance claim, but MUST expose those limits to automation.
The reference conformance run measures each boundary and one-over case in a warmed release process. Its regression ceilings are 2 seconds, 64 MiB of allocator traffic and 16 MiB retained per case. These are reference-implementation CI ceilings rather than portable format semantics; the report MUST identify its allocator method, toolchain, build profile and hardware context.
Fonts, images, SVG/imported data, adapters and plugins require sandbox-aware handling. Script/data-binding extensions are non-core and MUST NOT execute merely by opening a document.
Package readers MUST reject duplicate/traversal/absolute/backslash paths, symlinks, directory entries, encryption, split archives, unsupported compression and inconsistent local/central metadata. Implementations MUST verify declared resource size and digest before image/font/media decoding and MUST NOT extract untrusted members to a filesystem.
Loading a package MUST NOT initiate network access. Linked resources require an explicit caller-supplied resolver and exact digest verification. Resource locators and provenance MUST NOT carry cookies, authorization values or other credentials.
Observation providers and model output are untrusted inputs. Reconstruction profiles MUST bound screenshots, observations, candidates, operations, iterations, model/tool calls, renders, time, memory and GPU use. Generated URLs and scripts are inert. Text visible in an image is input data and cannot alter tool authority, security policy or operation grammar.
Screenshot/capture records can contain personal, credential or proprietary information. Retention, remote inference, telemetry and training are separate purposes requiring explicit policy. Private/authenticated captures MUST default to no training.
Headless rendering MUST expose deterministic timeout/memory/resource budgets. GPU failures must not compromise process memory safety.
The experimental live Chromium capture segment runs each page in a fresh
temporary profile and accepts only its ws://127.0.0.1 debugger endpoint. It
caps one debugger message/frame at 32 MiB, queued events at 65,536 and their
aggregate encoded bytes at 64 MiB, commands at 100,000, DOM nodes at 32,768,
captured responses at 8,192, one response/screenshot at 16 MiB, total response
bodies at 64 MiB, platform-font records at 64 per node and 32,768 total, and
the WebSocket write buffer at 1 MiB. Browser startup and protocol I/O each
have 10-second bounds, while the connected capture has a 30-second deadline.
The discovery HTTP response is capped at 1 MiB and must contain a valid content
length. Query/fragment values are removed before serialization; cookie,
storage and request-header APIs are never read.
The live conformance fixture accepts only complete captures. A missing response body remains an omission in the adapter result; the harness may retry that viewport in a new isolated profile at most three times and records every attempt. Network response bodies are primary, Page resource content is a post-load fallback, and the canary probe body is retained only after its in-page response completes and is checked for reflected canary values.
Live capture still executes the target page inside Chromium and may cause its declared network behavior. The caller MUST authorize the target and apply network/process isolation appropriate to untrusted content. The experimental adapter MUST NOT import a user’s persistent browser profile or credentials. Exact response bodies can themselves contain sensitive data, so their retention, transfer and training policy remains explicit even when transport canaries are absent.
Image, font, compressed-package, path-segment and GPU budgets are not part of executable CPU profile 0. Orthogonal resource profiles MUST calibrate and publish their own limits before claiming those resource classes.
The orthogonal experimental nuif-png-rgba8-0 image segment publishes a
32 MiB encoded-byte limit, 8,192-pixel limit per dimension, 16,777,216-pixel
decoded limit and 4,096-chunk limit. Its inspector applies these bounds before
inflation. Render scenes additionally cap unique decoded RGBA surfaces at
64 MiB, preflight each new surface, and deduplicate repeated digest/profile
uses. Its two decoders verify datastream integrity. These limits do not
authorize other PNG forms, non-PNG media, GPU allocation or inclusion in CPU
render profile 0.
The orthogonal experimental nuif-opentype-static-single-0 segment publishes
a 32 MiB encoded-byte limit, 256-table limit, 256-family-name limit,
65,536-coverage-range limit and 64-feature-setting limit. It validates sfnt
search fields, sorted unique table records, exact contiguous zero-padded
packing, per-table checksums and the whole-font checksum before accepting face
metadata. Its warmed reference implementation additionally caps one inspection
or packaged-font validation at 4 MiB allocator traffic and 2 MiB retained
memory. The allocation ceilings are CI regressions rather than portable format
semantics. These limits do not authorize other font containers/outlines,
native rasterizer execution, redistribution or inclusion in CPU render profile
0.
12 — CLI, API and automation surface
Document status: draft. Canonical source.
A conforming reference implementation MUST expose semantic operations without GUI automation.
Required command classes: inspect, query, validate, canonicalize, diff, patch, render, layout, snapshot, migrate, capabilities, replay, import and export.
Every command MUST support machine-readable output and stable diagnostic codes. Headless commands SHOULD accept stdin/stdout for pipeline use.
The editor MUST route mutations through the same operation layer available to CLI/API clients. AI/MCP adapters are optional clients of this interface and are never the canonical protocol.
The reference in-process SDK profile is a byte-oriented façade over the canonical codecs, package and operation implementation. Bare text/CBOR loading MUST name its encoding; package loading MUST run the complete package/resource validation path. A loaded package MUST retain verified embedded resources and descriptors across neighboring semantic edits. Export to a portable mode MUST rerun the target mode’s resource policy.
Language and process wrappers MAY add transport limits, ownership conversion and host authorization. They MUST NOT copy the semantic model or independently implement validation, canonicalization, hashing, package policy or operation application. Equivalent wrapper calls over the declared common subset MUST produce the same canonical bytes, hash and diagnostics as the direct SDK.
A package-aware wrapper MUST distinguish structural load from full-support negotiation. Structural load MAY preserve or inspect unknown required capabilities, but MUST NOT execute them or claim support. Before evaluation it MUST compare the manifest requirements with an explicit bounded host set and fail with the exact unavailable identifiers. A package-preserving wrapper MUST produce the same deterministic archive bytes as the direct SDK for the same document, resources, capabilities and target mode.
When any requirement is unavailable, a package-aware SDK or wrapper MUST also reject semantic mutation, undo/redo, changed package saves and package-mode conversion atomically. It MAY validate, hash, extract a bare document or copy the unchanged same-mode package. Successful complete-set negotiation MAY authorize mutation and evaluation for that loaded session; a failed partial negotiation MUST NOT do so.
An editor that lacks any required package capability MUST treat the package as read-only unless a capability-specific authoring profile defines how every affected resource is updated or explicitly detached. Structural selection, inspection and exact package copying MAY remain available. A semantic mutation or changed save MUST fail atomically with the unavailable requirement set; silently carrying opaque resources onto a new document revision is forbidden.
A future C ABI is a separate versioned profile. It MUST define opaque-handle lifetime, byte-buffer ownership and release, panic containment, stable error classes, threading, calling convention and exported-symbol compatibility. C, Swift or Kotlin bindings are not claimed merely because a shared library or generated header compiles. Native consumer tests and platform packages are required before integration status.
The experimental nuif-mcp-tools-0 profile is a stateless, stdio-only process
adapter for MCP 2026-07-28. It exposes validate, inspect, canonicalize
and atomic apply_patch as pure inline-text transforms over the authoritative
core. It MUST NOT infer filesystem paths, retain a hidden document session, or
gain network, credential, package-resource or host-product authority. Every
request is independently bounded and carries current protocol metadata; a
client does not perform the retired initialization handshake. MCP tool
annotations describe side effects but do not grant authority.
Capture and reconstruction systems are also optional clients. They MUST submit bounded typed transactions through the ordinary operation API, and MUST receive the same validation, stale-revision, atomicity and diagnostic behavior as the CLI/editor. A model/provider cannot gain direct mutation access to internal document structs.
An automation surface supporting reconstruction SHOULD expose distinct commands
or calls for observe, propose, evaluate and correct. Every call records
input/output hashes, budgets, provider artifact identity and machine-readable
diagnostics. Model weights, low-rank adapters, processors and training data are
operational artifacts outside the NUIF document and core conformance profile.
13 — Semantics, accessibility and behavior
Document status: exploratory draft. Canonical source.
Semantic role layer
NUIF entities MAY carry semantic roles independently of visual entity kind. The
current wire model carries a portable role identifier, one direct accessible
name and Boolean state keys. Document relationships can express
labelled-by, described-by, controls, owns and flow-to. A direct
description string, non-Boolean value/state data and semantic relationship
cardinality rules require a future schema revision; adapters MUST NOT invent
them from visual geometry.
Adapters MUST map portable semantics to host accessibility facilities where supported and MUST report unsupported or approximated semantics. Visual appearance MUST NOT be treated as sufficient evidence of semantic role.
Web accessibility projection profile 0
nuif-web-accessibility-0 is an experimental bounded lowering to inert HTML
and ARIA. It admits at most 4,096 entities and 8,192 relationships. Its role set
is button, checkbox, group, img, main, navigation, paragraph,
radio, region and switch. Role-specific required/prohibited naming and
Boolean-state rules are fixed by the profile. A switch MUST carry checked;
unsupported or misplaced states MUST fail closed. A direct accessible name and
labelled-by MUST NOT compete on one entity. Direct and referenced names MUST
be whitespace-normalized for computed-name comparison and MUST NOT be empty
after normalization.
The five relationship kinds above lower to their corresponding ARIA IDREF
attributes using stable NUIF entity identifiers. Relationship order is retained
and duplicate targets fail closed. The owns graph MUST be acyclic and each
owned target MUST have at most one ARIA owner. Native HTML semantics are used
where the profile has an exact element; explicit ARIA is used only for group,
img and switch. Output contains no script, external URL, event handler or
synthesized host behavior.
The foreign oracle MUST record the exact test-engine and host versions, compare computed role/name/state rather than source attributes alone and classify required-subset loss separately from other host-tree differences. Browser-tree agreement does not establish native platform API, keyboard interaction or application behavior equivalence.
Behavior graph
Portable interaction is represented as a separate bounded graph referencing stable entity/property identities. It MUST NOT embed arbitrary general-purpose code. The initial executable research profile is deliberately smaller than the eventual vocabulary of events, state, value transforms, property writes, navigation and animation triggers.
Behavior capabilities are negotiated like extensions. Missing optional capabilities may degrade according to declared fallback; missing required capabilities prevent a claim of behavioral conformance.
Behavior state-machine profile 0
nuif-behavior-state-machine-0 is an experimental sidecar and is not part of
the canonical semantic Document model. It defines a flat deterministic state machine with a
single active state. Its only external event is activate, addressed to a
stable entity carrying role button, checkbox, radio or switch.
Transitions in the active state are evaluated in authored order; the first
exact event and equality-guard match executes its actions sequentially and
selects its target state. An unmatched event is a no-op. An event MUST complete
before the next external event is accepted.
Profile values are Boolean or bounded string. Actions may set a value, toggle a
Boolean or emit an abstract visibility(Boolean) or announcement(String)
effect to a stable entity. The reference runtime MUST NOT directly mutate the
document or invoke host APIs. Target adapters consume effects under separately
declared capability and fidelity contracts.
Every used effect capability MUST be declared required or optional_noop.
A missing required capability MUST reject runtime construction before actions
execute. An unavailable optional capability MUST emit no host effect and MUST
be recorded as skipped in the trace. Unknown or incompatible entities, states,
variables, value types, capabilities, unreachable states and over-limit graphs
MUST fail closed before execution.
The profile admits at most 128 states, 1,024 transitions, 4,096 total actions, 64 actions per transition, 128 variables, 64 capabilities and 4,096 external events per run. Timers, internal events, parallel states, numeric computation, navigation, animation, filesystem/network effects and scripts are excluded. Conformance compares complete event, selected-transition, state, variable, effect and skipped-capability traces. Final-state agreement alone is insufficient.
Behavior package resource profile 0
nuif-behavior-package-resource-0 is the experimental transport defined by
RFC 0012. It does not change nuif-package-0 or the canonical Document.
Exactly one embedded resource MAY carry a behavior program. Its descriptor
MUST use role source, no derivation and provisional media type
application/nuif-behavior+cbor; its bytes MUST be canonical nuif-cbor-0
for one nuif-behavior-state-machine-0 program. The package manifest MUST also
declare nuif-behavior-state-machine-0 as a required capability.
A behavior capability without its resource, a behavior resource without its capability, multiple behavior resources, a linked resource, non-canonical bytes or a program invalid for the package document MUST fail the attachment profile. The program MUST validate against the actual package document both before attachment and after decode.
The resource digest identifies exact behavior bytes; it does not include the document hash. The deterministic package manifest and complete package hash bind the behavior descriptor and document descriptor together. Attachment therefore changes the package hash without changing the semantic document hash. A transplanted resource MUST be revalidated against its new package document.
Generic package decoding MAY verify, preserve and re-encode the inert resource without interpreting it. Full behavioral conformance requires explicit attachment decoding and a separately authorized runtime capability set. Opening, inspecting or preserving a package MUST NOT execute behavior or grant filesystem, network, script or host-mutation authority.
Web behavior projection profile 0
nuif-web-behavior-0 is an experimental one-way composition of
nuif-behavior-state-machine-0 and nuif-web-accessibility-0. The complete
source profiles MUST validate before host output. The web projection admits
activate only for enabled native button elements or the button-backed
switch role. Checkbox and radio event sources and disabled transition sources
MUST fail closed. Every enabled button/switch in the document MUST be bound so
an activation with no matching transition retains the source profile’s no-op
semantics. A visibility effect MUST NOT hide an admitted event source or its
containing ancestor because later native activation would become impossible.
visibility(Boolean) MUST map to the target HTML element’s hidden property.
A non-empty announcement(String) MUST map to the text of one unfocused
status live region with polite, atomic semantics and retain its stable target
identity as observation metadata. A transition MUST NOT emit more than one
announcement or repeat the same effect-kind/target pair because a single host
task may collapse those abstract effects into one observation.
The generated output MUST contain only a fixed finite interpreter plus the validated program as escaped data. Program strings MUST NOT terminate the script element. Arbitrary authored script, handler attributes, evaluation, dynamic import, timers, network and filesystem authority are prohibited. A self-contained document MUST restrict the exact runtime by a content hash and deny every unneeded resource class. A serving host SHOULD deliver its own response-header Content Security Policy and MUST NOT assume a nested document’s policy transfers to the host page.
Conformance MUST drive native activation and compare selected transition, target state and retained host effects after every event through each declared engine. DOM and browser accessibility-tree observations do not establish assistive-technology speech, focus behavior or native platform UI equivalence.
Host logic boundary
Application business logic, arbitrary network effects and unrestricted scripts are outside the core document model. Adapters may preserve references/bindings to host logic using extensions and provenance, but another implementation is not required to execute unknown host code. The state-machine sidecar MUST NOT be interpreted as authority to execute such bindings.
Inference
Behavior inferred from screenshots or static design states is never lossless solely because the generated result looks equivalent. Inference records MUST identify evidence, confidence and unresolved alternatives.
14 — Observation, capture and reconstruction
Document status: draft. This module specifies candidate contracts from RFC 0011. No. Canonical source.
screenshot reconstruction profile is currently conforming. The fixed-input
contract baseline exercises observation/proposal encoding, evidence ceilings,
manifest-bound providers, flat-copy rejection and bounded correction stops. The separate
nuif-cdp-live-0 baseline exercises one pinned, local Chromium fixture; it is
not a cross-browser capture or reconstruction-accuracy conformance profile.
Scope
This module covers evidence captured from a runtime or image and the production of a validated NUIF hypothesis. It does not standardize a model architecture, provider, training library, dataset or inference service.
An implementation MUST distinguish:
- deterministic parsing of retained authored source;
- resolved observations from a pinned runtime context;
- measurements of pixels;
- inferred semantic/layout/resource hypotheses;
- explicit user confirmations;
- derived resources/values;
- unavailable evidence.
Observation record
An observation contains:
| Field | Requirement |
|---|---|
id | stable within its evidence bundle |
evidence_class | one class from specification 09 |
subject | optional entity/property/resource target |
source_digest | exact source artifact or screenshot digest |
source_locator | path/node/range or pixel region |
coordinate_space | named space and dimensions |
context | evaluation/capture context identifier |
provider | provider kind plus canonical provider-manifest digest |
candidates | typed value(s) and alternatives |
raw_confidence | optional provider score |
calibrated_confidence | optional calibrated score plus profile |
privacy_class | retention/transfer/training policy input |
Every observation bundle MUST carry the exact canonical provider manifest for each referenced identity. Missing, duplicate, malformed or digest-mismatched manifests fail the complete bundle. A proposal provider MUST resolve through the same registry before any operation is applied. An observation MUST NOT imply that its candidate is already a semantic document value. Applying a candidate requires an operation and validation.
Coordinate spaces include source pixels, device pixels, viewport CSS pixels, crop-local pixels and NUIF logical units. A conversion MUST record source and target spaces, matrix/scale/offset and rounding behavior.
Source-backed browser capture
A browser capture profile records browser/protocol build, operating system, viewport, device-pixel ratio, page scale, locale, timezone, media preferences, font environment, scroll/pseudo state, navigation identity, settling policy and animation/time freeze.
The first proposed Web capture observes, where available:
- retained HTML/CSS response bytes and stylesheet text;
- DOM including iframes/templates/shadow content visible to the protocol;
- resolved layout boxes, inline text boxes and paint order;
- declared computed and matched styles;
- downloaded resource bodies, final URLs, response media types and hashes;
- platform-font usage and font-readiness state;
- accessibility tree;
- reference screenshot and capture parameters.
Unavailable cross-origin responses, local font bytes, canvas/WebGL semantics, video state, worklet output and arbitrary script behavior MUST be explicit. Canvas/video output MAY be frozen as a bounded derived image/frame. Captured scripts remain inert resources.
The capture MUST exclude cookies, authorization headers, credentials, storage values and secret form fields from its output. Redaction is a recorded transformation, not silent source equivalence.
The implemented live baseline records a bounded string map as
nuif-browser-runtime-context-0 in the observation bundle. It fixes exact
browser and reported protocol, operating system/architecture, viewport/DPR,
locale, timezone, screen media, color scheme, reduced-motion preference, scroll
origin, settling policy and animation policy. Context keys are identifiers,
the map is limited to 64 entries and all names/values share the observation
string-byte limits. A consumer MUST NOT assume equivalence between bundles with
different contexts.
nuif-cdp-live-0 uses an isolated temporary browser profile, accepts only a
loopback debugger socket, waits for the lifecycle load event belonging to the
exact navigation loader, fixes animation/transition and scroll state, waits two
animation frames, awaits font readiness and waits two final frames. It normalizes opaque browser node
identifiers before export and strips URL query/fragment data twice: before live
capture serialization and again during observation normalization. The gate
exercises query, cookie, storage, authorization and custom-header canaries; this
proves those declared ingress paths are not retained, not that arbitrary
response bodies contain no sensitive application content.
The first implementation observes element layout rectangles, a bounded background-style subset, text, containment/order, downloaded response bodies, actual platform-font use, accessibility role/name and a viewport screenshot. Inline text boxes, complete paint ordering, matched-rule/source-map correspondence, opaque cross-origin bodies, canvas/video derived-frame capture and general interaction states remain outside this automated segment and MUST NOT be inferred from its passing report.
Screenshot-only reconstruction
A screenshot-only implementation receives one or more image/context pairs and MAY run replaceable OCR, computer-vision, grounding and proposal providers. It MUST normalize their output into observations before semantic application.
The proposal engine emits only operations permitted by its declared operation grammar and profile. It MUST NOT emit executable code, implicit network actions or direct core-memory mutations.
Every proposal is applied transactionally:
- parse under operation/resource budgets;
- validate operation kinds and expected revision;
- apply atomically;
- validate the complete document;
- evaluate declared layout/render contexts;
- return a result or leave the prior document unchanged.
A valid screenshot reconstruction includes its accepted operation log, observations, fidelity report, alternatives/abstentions, evaluation report and pipeline artifact manifest.
Screenshot evidence cannot establish original font/image bytes, hidden entities, authored layout constraints, responsive rules, accessibility or behavior. These values remain inferred, substituted, derived or unavailable.
Multi-context inference
When multiple viewports/states are supplied, proposed semantic identity links the corresponding observations. Candidate layout families/constraints SHOULD be ranked by prediction of a held-out context. Fit to every supplied pixel does not prove original authored intent.
An implementation claiming responsive reconstruction MUST evaluate at least one context not used to fit the candidate and MUST report its error separately.
The bounded nuif-layout-inference-0 experimental profile accepts two through
sixteen strictly increasing training viewports, one untouched held-out viewport
and no more than 4,096 ordered items with stable identities. It ranks row stack,
column stack, Grid, linear constraint and fixed-parent-relative freeform
candidates by training observations only. The held-out observation is a
falsification oracle and MUST NOT influence selection. Its report retains all
five alternatives, training score, held-out error, raw confidence and the exact
geometry observation identities used for fitting. Confidence is not calibrated
and the selected value remains inferred; neither a low error nor the selected
family establishes original authored intent. Geometry and identity drift fail
closed. The profile is a deterministic geometric baseline, not a general
breakpoint, intrinsic-sizing, wrapping or layout-program synthesizer.
Correction loop
A profile MAY iteratively render and correct. Each iteration records:
- input document hash and revision;
- proposed transaction and diagnostics;
- resulting document hash;
- render context and raster hash;
- property-level and visual differences;
- objective vector and protected metrics;
- accept/reject reason.
The loop MUST bound iterations, provider/tool calls, time, memory and resources, and MUST stop on repeated state. Accepted corrections preserve validity and declared semantic non-regression constraints.
An editable reconstruction objective MUST reject a viewport-sized copy of the source screenshot as success unless a flat image document was explicitly requested.
Evaluation
Reports separate source-backed capture from screenshot-only reconstruction. Required metric families for an editable screenshot profile are:
- valid transaction/document rate;
- text-region precision/recall, character/word error and baseline geometry;
- visible-element precision/recall;
- tree/parent/sibling correctness where a target is justified;
- property and geometry error;
- held-out-context layout error;
- exact resource-digest recall only where source bytes are supplied;
- provenance/fidelity honesty;
- accessibility evidence accuracy where a target is justified;
- raw pixels plus declared perceptual diagnostics;
- calibrated confidence, abstention and risk/coverage;
- latency, peak RAM/VRAM, iterations and external cost.
The executable nuif-reconstruction-evaluation-0 report implements these as
typed, bounded per-example fields in
crates/nuif-reconstruct/src/evaluation.rs. Rates retain their integer
numerator and denominator; a zero denominator serializes with no value and is
never treated as perfect. Unavailable RAM/VRAM or latency measurements remain
null rather than zero. Screenshot-only suites MUST leave exact source-resource
recall unscored. cargo xtask reconstruction-evaluation exercises the schema,
derived-value validation, resource-claim boundary, local-error visibility and
hostile work limits. Its typed corpus aggregate refuses mixed evidence suites,
duplicate examples, incompatible calibration thresholds, perceptual evaluator
identity drift and mixed currencies. It reports pooled integer-evidence rates
beside per-example scored/unscored counts, mean and nearest-rank p50/p95. This
synthetic contract fixture is not an accuracy corpus or a confidence interval.
Every perceptual entry also carries a bounded parameter map and optional
content digest identifying the evaluator artifact. Aggregation is permitted
only when method, direction, parameters and artifact identity agree. The
current diagnostic is LDR-FLIP mean at a declared 67 PPD over opaque sRGB8;
transparent input MUST be composited against a declared background before
evaluation, never silently discarded.
No visual metric alone establishes conformance. Metrics are reported per example and as distributions; local/small-element errors MUST NOT be hidden by a large background average.
Synthetic exact fixtures and licensed/human-reviewed real fixtures are separate corpora. Splits MUST prevent origin, template, component, font, resource, generator and near-duplicate leakage. Benchmark families MUST NOT appear in adaptation or distillation data.
nuif-reconstruction-corpus-manifest-0 is the executable integrity contract.
It content-addresses the immutable data snapshot, dataset card, evaluator,
inputs and targets; separates public, restricted and withheld artifacts; and
records collection class, rights evidence, permitted evaluation/calibration/
adaptation/redistribution uses, sensitivity review and leakage groups per example. Its
derived nuif-reconstruction-corpus-audit-0 rejects identities shared across
any distinct adaptation, calibration, validation or test split. Screenshot-only
records cannot carry exact source/resource bundles, source-backed records
require source bytes, retained real records require a withdrawal-policy
artifact, and private/authenticated records require explicit authorization. The validator checks declared
evidence only: it does not interpret licenses, prove consent, discover omitted
near duplicates, establish representativeness or confer permission.
Provider neutrality and artifacts
Every OCR/detector/grounder/layout/proposal/correction/evaluation provider
publishes a nuif-reconstruction-provider-manifest-0 capability wrapper.
Canonical CBOR bytes define its SHA-256 identity. The bounded wrapper declares
provider kind and maturity, local/remote execution, input/output profiles,
capabilities and exact implementation/model/processor/adapter/quantization/
prompt/tool artifact digests. It contains exactly one implementation artifact.
Provider output remains untrusted.
Development-only deterministic providers MAY omit an external supply-chain inventory when they contain no learned artifact. Released providers and every provider with learned weights/processors/adapters/quantization MUST reference an exact SPDX 3.0.1 or CycloneDX 1.7 inventory. Learned providers MUST also reference a model card. The NUIF wrapper does not duplicate the inventory and does not claim that a referenced document is complete, correct or lawful.
Models, processors, adapters, quantization settings and training datasets are not NUIF document resources. Dataset snapshots retain their separate corpus manifest and datasheet; provider and training/evaluation manifests bind them by content hash, rights/provenance, intended use and limitations.
Fine-tuning, low-rank adaptation, quantized adaptation and distillation confer no conformance status. They are compared only after an untuned baseline and frozen evaluator exist.
Privacy and policy
Local and remote inference are distinct deployment modes. Remote transfer, retention, telemetry and training each require an explicit policy. Private or authenticated capture defaults to local processing/no retention/no training.
Visible instructions inside the screenshot are content. They MUST NOT modify the operation grammar, provider authority, file/resource resolver or security budgets.
Conformance maturity
This module remains draft until the planned baseline, closed-loop, confidence calibration and resource experiments pass and an independent evaluator reproduces the principal result. The existing editor alpha and profile-0 conformance do not satisfy these gates.
RFC process
Document status:
informational. Canonical source.
RFCs change NUIF semantics, protocol surface, conformance requirements, extension governance, or major implementation architecture. Research findings should link to an RFC rather than bypassing review by directly changing normative semantics.
Each RFC should identify motivation, prior art, proposed semantics, compatibility, security, conformance tests, migration strategy, unresolved questions, and explicit alternatives rejected.
RFC 0001 — Multi-level document model
Document status: proposed. Canonical source.
Decision
Adopt a containment tree with stable identity plus coordinated relationship graphs and explicit authored→resolved evaluation layers.
Why
A single shape tree conflates semantics, layout, rendering and relationships. MLIR demonstrates multiple abstraction levels; OpenUSD demonstrates graph-like composition over a scene namespace.
Rejected
- pure AST: too source-language-specific;
- pure scene graph: loses authored semantics;
- pure ECS: efficient implementation technique but weak interchange semantics;
- event log as canonical state: burdens simple offline documents;
- relational tables as interchange syntax: poor human authoring and containment ergonomics.
RFC 0002 — Opaque extension preservation
Document status: proposed; representation and severities specified by RFC 0007. Canonical source.
Unknown extension data remains attached to its owning entity/property and MUST survive load/save/edit cycles unless the owner is deleted or the user explicitly removes it.
Documents declare used and required extensions. Unsupported required extensions block claims of faithful rendering but do not necessarily block structural editing.
This goes beyond codec unknown fields: preservation is a document-model requirement.
RFC 0003 — Authored state, resolved state and provenance
Document status: proposed. Canonical source.
Store authored values as canonical semantics; resolved values are context-keyed derived records. Provenance/correspondence records map foreign constructs to NUIF constructs for retentive synchronization.
The same entity may have multiple resolved records for different viewport/theme/font contexts.
Source adapters should patch original syntax trees using correspondence and structured edits instead of regenerating whole files.
RFC 0004 — Headless QA contract
Document status: proposed. Canonical source.
Every meaningful reference-editor action must be expressible through semantic operations and inspectable via CLI/API. Tests must be able to create fixtures, execute operations, compute layouts, render deterministic snapshots, query semantic/accessibility trees and replay failures without mouse-coordinate automation.
An AI QA agent is therefore just one client of a stable automation contract.
RFC 0005 — Deterministic numeric and string canonicalization
Document status: accepted (decision delegated to research on 2026-08-29; evidence in
nuif:research:deterministic-cbor-profiles-and-numeric-canonicalization,nuif:research:ipld-dag-cbor-strictness,nuif:research:canonicalization-rfc8785-and-cbor-deterministic). Canonical source.
Correction: RFC 0008 supersedes rules 6 and 8 and the key-order coincidence claim in rule 17. Integer and real values remain distinct in nuif-cbor-0; readers must apply RFC 0008 when implementing this RFC.
Motivation
spec/08-serialization.md requires byte-stable canonical hashes for nuif-cbor-0 and nuif-text-0 but leaves numeric normalization, map ordering and string rules undefined. The IETF CBOR working group’s Common Deterministic Encoding draft (draft-ietf-cbor-cde) was parked on 2025-10-19 for lack of consensus and expired on 2026-04-16 without an RFC; its successor, draft-ietf-cbor-serialization (revision 08, 2026-07-29, Standards Track, in working-group last call), preserves the data model and is silent on negative zero and duplicate keys. The individual draft dCBOR (revision 18, 2026-08-10) narrows the data model (integral floats become integers, all zeros become 0x00, one NaN, NFC strings, strict decoders) and is not working-group adopted. NUIF must state its rules by value so that neither draft’s future changes alter NUIF hashes.
Prior art
RFC 8949 §4.2 (core deterministic encoding requirements and the list of decisions left to protocols); RFC 8949 erratum 8589 (NaN sign bit in key equivalence); RFC 8785 (JSON Canonicalization Scheme: ECMAScript shortest number serialization, -0 to 0, NaN and Infinity rejected); IPLD DAG-CBOR (strict codec: shortest floats, no NaN or Infinity, no indefinite lengths, content identifiers carry the codec identifier); Automerge’s binary change format (hash over a defined byte layout); glTF JSON and GLB (same document, two containers, no cross-container hash equivalence claimed).
Decision
Logical numeric model
- The logical model has two numeric kinds:
integer(signed 64-bit) andreal(IEEE 754 binary64). There is no 32-bit real kind in the model; adapters convert. - An authored
realproperty MUST NOT hold NaN, positive infinity or negative infinity. Validation rejects such values at set time with a diagnostic. Resolved snapshots MAY carry non-finite values only inside diagnostics, never as resolved geometry. - Negative zero has no distinct identity in the logical model:
-0.0and+0.0are the same value and canonicalize identically.
nuif-cbor-0
- The base profile is draft-ietf-cbor-serialization §4.1 (preferred serialization: shortest integer heads, definite lengths only, shortest float width that preserves the value including subnormals, the single NaN encoding
0xf97e00, no bignum tags for values within the 64-bit range) and §5.1 (map keys sorted by bytewise lexicographic order of their encoded form). NUIF cites these sections by name and restates every rule it depends on below so that a change in the draft does not change NUIF. - Integers encode as major type 0 or 1 with the shortest argument. Values outside
[-2^63, 2^63-1]are invalid forintegerproperties; the wire range beyond that is never produced. - Superseded by RFC 0008: a
realremains a floating-point data item even when integral. Decoding never depends on an external property schema to restore numeric kind. - A
realthat is not integral MUST be encoded as the shortest of half, single or double precision that round-trips the value exactly. Subnormals are preserved. - Superseded by RFC 0008: both real zeros encode as positive half-precision floating zero; integer zero remains
0x00. - NaN and infinities never occur in canonical documents (rule 2). If a future property type admits them, the encoding is
0xf97e00for NaN and the shortest-width infinity. - Simple values other than
false,trueandnullare not used. No tags appear in the canonical body. - Map keys MUST be in strictly increasing bytewise lexicographic order of their encoded form; a duplicate key is invalid.
- Decoders used for hashing or conformance MUST be strict: any deviation from rules 4–11 is rejected with a diagnostic. Decoders MUST NOT re-canonicalize accepted input silently; a lenient import path MAY exist for foreign data and MUST report
approximatedwhen it rewrote bytes. - Extension payloads and unknown-kind payloads are CBOR byte strings and are hashed verbatim; the container decoder never inspects their content. Tag 24 (embedded CBOR) is not used because it requires well-formed content.
nuif-text-0
- The text profile is a lossless surface syntax over the same value set. Its canonical hash is defined as the hash of the
nuif-cbor-0encoding of the parsed document:hash(text) = hash(cbor(parse(text))). No separate text hash exists. - Reals print as the shortest decimal digit string that round-trips to the same binary64 value (Rust
core::num::flt2decshortest mode is a conforming implementation), laid out as plain decimal when10^-6 <= |v| < 10^21and otherwise asd.ddde±x. Integral reals print without a fraction.-0is never printed.NaN,infand-infare parse errors. - Integers print in decimal without leading zeros or a plus sign.
- Map keys in
nuif-text-0are written in UTF-8 byte order. RFC 0008 corrects the earlier claim of coincidence: CBOR encoded-key order differs when key lengths differ. - Whitespace, comments and key quoting styles are not significant and are not preserved; the text encoder emits one fixed layout.
Strings and identifiers
- Namespaces, property keys, extension names and entity kind names are identifiers restricted to
[a-z0-9][a-z0-9_.:-]*. They are compared bytewise. - String property values (names, text content, token names) are stored verbatim as valid UTF-8 and are never normalized by canonicalization. Rationale: retentive synchronization and minimal source patches require byte-exact preservation of authored text; Unicode normalization is idempotent but not the identity, and applying it would rewrite user text. Adapters that must compare strings semantically compare NFC forms without altering stored values. The validator emits an informational diagnostic for identifiers or token names that are not in NFC.
- Invalid UTF-8 is a decoding error, not a repairable condition.
Hash
- The canonical hash of a document is SHA-256 over the
nuif-cbor-0bytes of the document record. Published content identifiers carry the profile identifier (nuif-cbor-0) so that a future profile cannot collide silently. - Resolved snapshots and correspondence records are hashed separately under the same rules and are never part of the document hash.
Compatibility
No documents exist yet. Decoders implementing RFC 8949 §4.2 accept the structural subset of nuif-cbor-0 output. dCBOR is not wire-compatible with RFC 0008 integral real values or with NUIF’s verbatim-string rule and is not a NUIF decoder.
Security
Strict decoding removes canonicalization ambiguity as an attack surface (two byte sequences for one value). Extension payloads are opaque bytes with a declared size limit enforced by the parser (spec/11-security.md). RFC 0008 keeps numeric kind self-describing, so generic decoding does not depend on a property schema.
Conformance tests
- canonicalization suite: encode, decode, encode fixpoint; hash stability across platforms; every rule 4–13 has a positive and a negative fixture (non-canonical inputs rejected by the strict decoder).
- numeric fixtures: subnormals,
2^53 ± 1, integral reals remaining floats at the2^63boundary, both real zeros canonicalizing to positive floating zero, and shortest-width selection for half and single precision (RFC 0008). - text fixtures: layout switch at
10^-6and10^21;5e-324prints as5e-324; parse rejection ofNaNandinf. - dCBOR §7 test vectors for the rules NUIF shares, with the divergent cases (NFC) marked as intentionally not shared.
Implementation
nuif-codec converts Serde values to ciborium::Value, recursively rejects forbidden values, sorts map entries by encoded key bytes and uses Ciborium’s shortest-width writer. Strict decoding retains the value tree, re-encodes it and compares the original bytes before deserializing the document. This supplies the ordering/checking layer Ciborium intentionally omits while preserving numeric kind under RFC 0008.
Rejected alternatives
- Adopt dCBOR by reference: an individual draft with strict-reject semantics on NFC that would rewrite user text and can change under NUIF.
- Adopt draft-ietf-cbor-serialization by reference alone: silent on negative zero and duplicate keys; NUIF must state both.
- Preserve negative zero as distinct: rejected because the logical model equates the two real zeros. Preserve float/integer distinction: accepted by RFC 0008 because the logical model distinguishes those kinds.
- Separate text hash: two hashes for one document invite inconsistency; the text profile is a view.
- NFC-normalize string values: violates verbatim preservation required by RFC 0003 and the minimal-patch requirement.
- Use JSON with RFC 8785 as the binary profile: no byte strings, no integer/real distinction, larger payloads.
Unresolved
- Whether Rust’s shortest-digit algorithm and ECMAScript
Number::toStringever differ in tie-breaking (no counterexample found; irrelevant to NUIF hashes because the text profile hashes through CBOR). - Whether the text surface remains canonical JSON or adopts a purpose-built syntax after reviewability trials; hashes are unaffected because text hashes through CBOR.
RFC 0006 — Sibling order: canonical arrays and anchored operations
Document status: accepted (decision delegated to research on 2026-08-29; evidence in
nuif:research:list-ordering-fractional-indexing-vs-list-crdts,nuif:research:crdt-tree-move-operation,nuif:research:figma-multiplayer-and-rendering-engineering). Canonical source.
Motivation
nuif-protocol expresses Insert and Move with an integer index. Integer positions are not commutative under independent edits, make replay order-sensitive, and cannot be merged without renumbering. The collaboration profile (spec/10) needs positions that converge; the canonical document (ADR 0005) must not carry replica metadata.
Prior art
OpenUSD stores resolved child order as an array and expresses authored reordering as sparse reorder list operations composed at index time (pcp/composeSite.cpp). Figma uses fractional string keys per child, deduplicated server-side, and reports key growth and interleaving as known costs. Automerge and RGA insert relative to a neighbour identifier. Penpot’s change log names an after-shape. Fugue (Weidner and Kleppmann 2023) is the list CRDT with proved forward and backward non-interleaving; Logoot and LSEQ interleave; the PaPoC 2019 interleaving anomaly definition is unsatisfiable per the Fugue paper.
Decision
Canonical form
- A canonical document MUST represent sibling order as an ordered array of entity identifiers (
Entity.children) with no duplicates. It MUST NOT carry order keys, tombstones, replica identifiers or timestamps. - Integer indices are a resolved view computed from the array; they never appear in canonical state or in operations.
Operations
InsertandMoveMUST specify position as an anchor:Anchor::StartorAnchor::After(EntityId).- An anchor MUST refer to a current child of the target parent at application time; otherwise the operation fails with the typed conflict
AnchorMissing, and patch application reports it. - A
Movewhose target parent is the moved entity or one of its descendants MUST fail withCycleRejected. - Within one patch, insertions at the same anchor are applied in patch order.
- Preconditions MAY assert
ParentIs { entity, parent }andFollows { entity, anchor }.
Merge
- In three-way merge, insertions from different branches at the same anchor are ordered by declared branch precedence and emit the informational conflict
OrderAmbiguouslisting the entities. - Concurrent moves of one entity to different positions produce
MoveConflictcarrying both targets; a profile MAY converge on one, but the conflict object MUST be retained until resolved.
Collaboration profile
- A collaboration profile maps anchored operations onto a list CRDT that satisfies forward non-interleaving (Fugue Definition 2) and SHOULD satisfy maximal non-interleaving. Tree moves follow the undo/redo algorithm of Kleppmann et al. 2021 with cycle rejection.
- Checkpoint materialization MUST emit the plain array of rule 1.
- Fractional keys MAY be used as profile-internal transport and MUST NOT appear in
nuif-coretypes or canonical hashes.
Type changes (nuif-protocol, signatures only)
#![allow(unused)]
fn main() {
pub enum Anchor { Start, After(EntityId) }
pub enum Operation {
Insert { parent: Option<EntityId>, anchor: Anchor, entity: Entity },
Move { entity: EntityId, new_parent: Option<EntityId>, anchor: Anchor },
// Remove, Rename, SetExtension unchanged
}
pub enum Precondition {
ParentIs { entity: EntityId, parent: Option<EntityId> },
Follows { entity: EntityId, anchor: Anchor },
}
pub enum Conflict {
AnchorMissing { entity: EntityId, parent: Option<EntityId>, anchor: Anchor },
CycleRejected { entity: EntityId, new_parent: EntityId },
MoveConflict { entity: EntityId, targets: [(Option<EntityId>, Anchor); 2] },
OrderAmbiguous { parent: Option<EntityId>, anchor: Anchor, entities: Vec<EntityId> },
}
}
Entity.children: Vec<EntityId> is unchanged and documented as the canonical order with a no-duplicates invariant.
Compatibility
No persisted documents exist. Adapters that import index-based formats compute anchors from the imported order.
Security
Anchor resolution is O(children); cycle rejection is a parent-chain walk bounded by the depth limit in spec/11.
Conformance tests
- operations suite: independent inserts under different parents commute to identical hashes;
AnchorMissingandCycleRejectedfixtures; replay of anchored logs is deterministic. - merge suite: same-anchor concurrent inserts produce
OrderAmbiguouswith branch-precedence order; concurrent moves produceMoveConflict; Kleppmann’s concurrent-move test cases (cycle-forming pair) converge without cycles. - collaboration experiment: two profile engines converge to the same checkpoint array and no interleaving of two concurrent runs of sequential inserts (Fugue forward non-interleaving check).
Rejected alternatives
- Integer index in operations: non-commutative; order-sensitive replay.
- Fractional keys in the canonical form: history-dependent hashes, unbounded key growth at one gap, jitter randomness, interleaving, no rebalancing procedure documented by Figma or rocicorp.
- List-CRDT identifiers in the canonical form: require tombstones and replica metadata, contradicting ADR 0005.
- Logoot or LSEQ positions in the profile: interleave under concurrent sequential inserts.
Unresolved
- Branch precedence for rule 8 is a merge-policy parameter, not derivable from sources.
- Whether the profile adopts Fugue or FugueMax (and how Yjs YATA’s ordering compares) is settled by the collaboration experiment.
RFC 0007 — Unknown entity kinds and opaque payloads
Document status: accepted (decision delegated to research on 2026-08-29; evidence in
nuif:research:unknown-schema-preservation-strategies,nuif:research:opentimelineio,nuif:research:godot-tscn-scene-format,nuif:research:openusd-composition-and-crate,nuif:research:gltf-validator-and-sample-assets). Extends RFC 0002. Canonical source.
Motivation
RFC 0002 requires unknown extension data to survive load, save and edit. It does not define how an entity whose kind is unknown is represented, which operations remain valid on it, how it lays out and renders, or how validation severities are assigned. Godot preserves unknown node classes as placeholders that record the original class and properties and write them back; OpenTimelineIO preserves unknown schemas with name, version and raw payload while still decoding nested known objects; Blender ignores and cannot re-save unknown data; OpenUSD keeps unknown typeName as metadata with fallbackPrimTypes; glTF gates by extensionsUsed and extensionsRequired with validator codes UNDECLARED_EXTENSION (error) and UNSUPPORTED_EXTENSION (information).
Decision
Representation
- An entity whose kind namespace, kind name or schema version is not supported MUST be loaded as
EntityKind::Unknown, retaining namespace, kind name, schema version and the kind-specific payload bytes. A known kind whoseschema_versionexceeds the implementation’s support MUST also load asUnknown, never fail. - Core fields (identifier, name, children,
nuif-namespace properties, relations, extensions) of an unknown entity remain typed and editable. Only the kind-specific payload is opaque. - Every containment slot MUST admit
Unknown.
Operations
Remove,Move,Rename,SetExtension,RemoveExtensionand core-property operations MUST apply to unknown entities unchanged.SetUnknownPayloadis valid only when the applying implementation declares the payload’s namespace; other implementations MUST reject it with a diagnostic.
Preservation
- An implementation that does not declare a namespace MUST preserve that namespace’s payloads byte-for-byte. An implementation that declares the namespace MAY re-encode deterministically (value-for-value). RFC 0002’s “byte/value-for-byte” language is read as these two cases.
- In
nuif-cbor-0, opaque payloads are CBOR byte strings; the declared encoding (CbororOctets) is a sibling field, not a CBOR tag (RFC 0005 rule 13 forbids tags in the canonical body). Hashing covers the bytes.nuif-text-0encodes the bytes losslessly. - A malformed opaque payload yields a diagnostic on the owning entity and MUST NOT invalidate the document.
- Lowering, flattening and codec conversion MUST carry
Unknownentities and extension payloads through; a conformance fixture asserts byte identity after an edit cycle through an implementation that declares neither the kind nor the namespace.
Evaluation
- Layout treats an unknown entity as the kind named by the document’s
fallback_kinddeclaration for its namespace, else asContainerwith its authored size intents. - Rendering of an unknown entity reports
PreservedUnrenderable { namespace, entity }and draws nothing for the kind-specific payload; children render normally.
Validation severities
- A namespace present in the document but absent from
extensions_used: error. - A namespace declared in
extensions_usedand unsupported by the implementation: information. - A namespace declared in
extensions_requiredand unsupported: blocks claims of faithful rendering and export fidelity abovepreserved_unrenderable; MUST NOT block structural editing.
Type changes (nuif-core, signatures only)
#![allow(unused)]
fn main() {
pub enum OpaqueEncoding { Cbor, Octets }
pub struct OpaquePayload { pub encoding: OpaqueEncoding, pub bytes: Vec<u8> }
pub struct UnknownKind { pub namespace: String, pub kind: String, pub schema_version: u32, pub payload: OpaquePayload }
pub enum EntityKind { /* existing variants */ Unknown(UnknownKind) }
pub struct Extensions(pub BTreeMap<String, OpaquePayload>);
pub struct ExtensionDeclarations { pub used: BTreeSet<String>, pub required: BTreeSet<String>, pub fallback_kind: BTreeMap<String, EntityKind> }
pub enum Fidelity { /* existing */ PreservedUnrenderable { namespace: String, entity: Option<EntityId> } }
}
nuif-protocol: SetExtension { entity, namespace, payload: OpaquePayload }, RemoveExtension { entity, namespace }, SetUnknownPayload { entity, payload: OpaquePayload }.
Compatibility
Existing Extensions(BTreeMap<String, Vec<u8>>) gains the encoding field; no documents exist.
Security
Payload size and count are bounded by parser limits (spec/11); payloads are never interpreted by implementations that do not declare the namespace, so untrusted content in them cannot reach an interpreter.
Conformance tests
- extensions suite: decode with an ignorant implementation, apply
Rename,Move,SetExtensionon neighbours and on the unknown entity itself, encode, assert byte identity of the unknown payload and restoration of the original kind name and version by a knowing implementation. - validation suite: fixtures for severities 12–14.
- layout suite: unknown entity with
fallback_kindand without.
Rejected alternatives
- Silent drop (pre-3.5 proto3 behaviour, Blender): the failure NUIF exists to prevent.
- Host-typed re-encoding of unknown properties (Godot
Variant): changes bytes in an ignorant implementation. - Hard failure on newer schema versions (OpenTimelineIO): prevents structural editing of otherwise valid documents.
- CBOR tag 24 for declared-CBOR payloads: tags are forbidden in the canonical body by RFC 0005, and tag 24 requires well-formed content, which rule 8 does not.
Unresolved
- Whether
fallback_kindmay name a kind from another extension namespace (transitive fallback) is deferred until a second dialect exists.
RFC 0008 — Preserve numeric kinds and distinguish text/CBOR key order
Document status: accepted (corrective; primary evidence in
nuif:research:cbor-data-model-and-key-order-correction). Supersedes RFC 0005 rules 6 and 8 and the coincidence claim in rule 17. All other RFC 0005 rules remain in force. Canonical source.
Motivation
RFC 0005 made integral real values use CBOR integer encodings and required a property schema to restore the type. The NUIF logical value model, extensible properties and unknown data do not guarantee that such a schema is available. RFC 8949 §2 also defines integer and floating-point data items as distinct. The previous rule therefore made a generic lossless decoder impossible and caused integer(1) and real(1.0) to collide.
RFC 0005 also stated that UTF-8 key order coincides with CBOR encoded-key order for NUIF identifiers. It does not: complete CBOR key encodings include a length head, so "z" (61 7a) sorts before "aa" (62 61 61) in deterministic CBOR while UTF-8 sorts "aa" first.
Decision
integerandrealremain distinct logical values in every encoding and in canonical hashes.- An
integeruses CBOR major type 0 or 1 with the shortest argument. - A finite
real, including an integral real, uses the shortest IEEE 754 half, single or double CBOR floating-point encoding that round-trips exactly. - Negative real zero has no distinct NUIF identity and encodes as positive half-precision floating-point zero (
f9 00 00). Integer zero encodes as00. This replaces RFC 0005 rule 8. - Maps in
nuif-cbor-0sort by bytewise lexicographic order of each key’s complete deterministic CBOR encoding. - Objects in
nuif-text-0sort string keys by UTF-8 byte order. Text and CBOR key order need not coincide because the canonical document hash is computed from parsednuif-cbor-0, not text bytes. - A decoder used for conformance or hashing rejects an integral real encoded as an integer when the surrounding NUIF value discriminant says
real; it does not infer or repair the lost kind.
Compatibility
No published documents exist. The profile identifier remains nuif-cbor-0 because the prior rules had no implementation or fixtures; the first executable codec implements this RFC.
Security
Keeping numeric kinds self-describing removes schema confusion at extension boundaries. Strict duplicate-key and finite-number checks from RFC 0005 remain required.
Conformance tests
integer(1)andreal(1.0)encode differently, decode to their original variants and have different canonical hashes.- positive and negative real zero both encode as the same floating-point data item; integer zero remains distinct.
- a map containing
"aa"and"z"emits"z"first in CBOR and"aa"first in canonical text. - encode/decode/encode is a byte fixpoint for both profiles.
Implementation
nuif-codec converts Serde values to ciborium::Value, recursively rejects non-finite values and tags, sorts map entries by their encoded key bytes, and then uses Ciborium’s shortest-width numeric writer. Strict decoding re-encodes the retained value tree and compares bytes before deserializing the NUIF document.
Rejected alternatives
- Restore the type only from a property schema: unavailable to generic tools and unknown extensions.
- Treat integers and integral reals as one logical type: contradicts the declared NUIF value variants and creates surprising adapter behavior.
- Give text and CBOR the same key order: either violates UTF-8 lexical text order or deterministic CBOR encoded-key order.
RFC 0009 — Bound profile-0 ingestion and semantic resources
Document status: accepted (primary evidence and calibration in
nuif:research:resource-bounded-serde-and-ciborium). Canonical source.
Motivation
The draft security chapter required bounds but carried unmeasured depth-1,024 and one-million-node examples from unrelated parsers. A byte limit alone does not prevent recursive stack exhaustion, large decoded collections, diagnostic amplification or repeated canonical-key work. Applying a codec limit only after read_to_end also permits the input allocation the limit is intended to prevent.
Decision
Profile-0 decoders and encoders enforce these limits:
| Resource | Limit |
|---|---|
| encoded document | 16 MiB |
| text/CBOR syntax depth | 64 |
| entities / tokens | 8,192 each |
| roots | 4,096 |
| relations | 32,768 |
| responsive overrides | 16,384 |
| child references | 8,191 |
| property values | 65,536 |
| property-value depth | 24 |
| containment depth | 128 |
| total retained strings | 8 MiB |
| one retained string | 1 MiB |
| total retained binary data | 8 MiB |
binary data in nuif-text-0 | 512 KiB |
Readers consume at most 16 MiB plus the first excess byte and reject that byte without reading the remaining stream. Text nesting is counted outside strings and JSON5 comments before deserialization. CBOR uses an explicit recursion limit. The decoded semantic walk is iterative for property values and runs before recursive validation or evaluation. A canonical writer stops before appending the first byte beyond its profile limit.
Validation retains at most 1,024 ordinary diagnostics and one VALIDATION_DIAGNOSTICS_TRUNCATED issue. It may continue bounded structural work after the retention cap, but it cannot amplify malformed input into an unbounded report.
The reference hostile-input trial is a release build after one fixed warmup. Each enumerated case must produce its expected acceptance/error class within 2 seconds, no more than 64 MiB of allocator traffic and no more than 16 MiB retained at observation time. Those three values are reference-CI regression ceilings rather than cross-implementation format semantics; foreign implementations report their own allocation method while accepting the normative boundary fixtures.
Compatibility
No published profile-0 documents exist. The limits become part of the first executable profile. nuif-text-0 has a lower binary ceiling because its canonical JSON byte arrays expand substantially; the same semantic document remains representable in nuif-cbor-0 up to the 8 MiB model ceiling.
Security
Bounded bytes, syntax and semantic cardinality convert parsing and validation work into finite functions of declared profile limits. They do not replace process-level cancellation, tenant quotas or sandboxing in servers. Future images, fonts, compressed packages, paths and GPU resources require separately measured limits before their profiles can be accepted.
Conformance tests
- exactly-at-limit entity, property, containment, string and CBOR-binary cases are accepted;
- the first byte, syntax level or semantic item over each tested limit returns
ResourceLimitnaming the exceeded resource; - strings, escapes and JSON5 comments do not affect the text depth count;
- a wide canonical CBOR map completes within the allocator/time ceiling and malformed document shape remains classified separately;
- CLI and editor readers retain at most the selected limit plus one byte;
- validation output is capped and ends with the truncation diagnostic;
cargo xtask hostile-inputswrites a machine report with the cases, limits, measurements, toolchain, warmup, allocator method and platform.
Implementation
nuif-core::resource_usage owns semantic accounting. nuif-codec owns encoded, syntax and encoding-specific bounds. The CLI and headless editor bound reads before passing bytes to the codecs. nuif-testing uses an instrumented system allocator in an isolated single-threaded binary, and CI uploads its JSON report.
Rejected alternatives
- Keep depth 1,024 and one million nodes: unsupported by measurements and unsafe for recursive downstream layout.
- Enforce only encoded bytes: shallow compact inputs can still create excessive semantic nodes or diagnostic output.
- Apply the limit after
fs::read/read_to_end: too late to bound ingestion allocation. - Use wall-clock timeout as the sole guard: nondeterministic across machines and incapable of preventing memory amplification.
- Share an 8 MiB binary limit between text and CBOR: canonical text expansion can exceed the encoded limit and allocate a very large generic JSON value tree.
RFC 0010 — Portable resource and package profile
Document status: proposed, with an executable experimental container subset. The. Canonical source.
reference implementation now provides stable assets, deterministic
nuif-package-0, explicit verified resolution and the package segment of
nuif:experiment:portable-package-resources. This RFC does not add image
rendering or general packaged-font conformance to profile 0. The orthogonal
nuif-png-rgba8-0 experiment now implements a narrow cross-decoder and CPU
image path. The nuif-opentype-static-single-0 experiment likewise implements
one narrow package/parser/policy baseline; broader image and font profiles still
have to satisfy their acceptance criteria.
Motivation
The current implementation serializes a semantic document as canonical text or CBOR. That is sufficient for profile-0 fixtures because images are unsupported and the one text font is an external pinned conformance input. It is not a portable authoring package: images, fonts, source correspondence and other resources have no common descriptor or delivery contract.
The draft serialization module previously called .nuif a package while
leaving its archive technology experimental. The roadmap then described
“package/assets” as complete for profile 0. Those statements conflict. This RFC
defines a candidate profile and restores the implementation gate.
The design must distinguish:
- stable semantic asset identity;
- identity of immutable resource bytes;
- a locator inside or outside a package;
- source and derivation provenance;
- semantic document hash;
- exact delivered-package hash.
Using one path, URL or hash for all six roles would break editing, offline portability or reproducibility.
Prior art and evidence
nuif:research:epub-ocf-package-container: OCF separates an abstract container from a constrained ZIP form, requires a manifest and identifies the media type in a fixed first uncompressed member.nuif:research:oci-resource-descriptors: OCI binds media type, size and digest, verifying cheap limits and bytes before content interpretation.nuif:research:content-addressed-versioning: immutable byte identity is not stable editable entity identity.nuif:research:png-image-preservation-and-decoding: original encoded image bytes and declared decode parameters are distinct from decoded caches.nuif:research:opentype-font-embedding-and-portability: reproducible text needs exact bytes, but redistribution policy must be explicit.nuif:research:penpot: the existing bounded adapter proves defensive ZIP handling but not this package layout or cross-writer determinism.
Proposed semantics
1. Encoding and extension names
.nuifidentifiesnuif-package-0, the portable package profile..nuif.cboridentifies bare canonicalnuif-cbor-0document bytes..nuif.jsonidentifies bare canonicalnuif-text-0document bytes.
During the alpha migration, readers MAY recognize historical bare documents
whose name ends in .nuif. Writers MUST emit the package for .nuif and MUST
use an explicit bare extension for new bare documents. Legacy recognition is
read-only compatibility and MUST NOT weaken canonical codec validation.
2. Resource identities
AssetId is the stable semantic identity referenced by document entities.
Replacing an asset’s bytes does not change AssetId; it changes the asset’s
bound ResourceDigest through a semantic operation.
ResourceDigest in package profile 0 is:
sha256:<64 lowercase hexadecimal digits>
It identifies exact encoded bytes. Implementations MUST verify declared size and digest before decoding the resource.
ResourceDescriptor contains:
digest ResourceDigest
size unsigned byte length
media_type normalized ASCII media type without a retrieval-dependent value
role source | authoring | derived | cache
locator embedded path, or explicit linked locator plus expected digest
derivation required for role=derived; absent for source/authoring/cache
The descriptor is immutable by digest. Metadata that changes interpretation of the bytes belongs in the semantic asset or a versioned decoder profile, not in an untracked package field.
3. Asset semantics
An Asset contains stable identity, kind, current resource digest, portability
policy and kind-specific semantic metadata. Initial kinds are image and
font; unknown future kinds follow extension-preservation rules.
An image asset records intrinsic dimensions and the decoder profile. An
ImagePaint refers to AssetId and records fit, crop, transform, sampling,
opacity and declared color conversion. Decoded RGBA and GPU textures are
deletable caches, never source resources.
The reference scene interns decoded RGBA by resource digest plus decoder profile and gives commands deterministic numeric surface handles. Its 64 MiB unique decoded-surface total is preflighted from bounded image metadata before inflation. Repeated asset instances therefore do not duplicate decoded pixels or descriptor strings.
A font asset records exact byte digest when available, media type, face or
collection index, names used for matching, axes, features, character coverage
and policy evidence. Text shaping continues to pin its execution inputs. An
optional stable text-to-font AssetId keeps the requested text hash distinct
from the effective resource hash: exact bindings require equality, substituted
bindings retain the request and select the asset resource, and unavailable
bindings select an asset with no resource.
Font portability policy is one of:
portable: exact bytes can be embedded for the declared package use;private_authoring: bytes may exist in a private workspace package but not a distributable portable package;linked: bytes are absent; locator and expected digest are explicit;substituted: another exact resource is used and fidelity identifies the substitution;unavailable: no usable bytes; fidelity identifies the consequence.
OpenType fsType is recorded policy evidence, not a complete legal conclusion.
4. Package members
nuif-package-0 is a ZIP archive with these members:
mimetype
manifest.cbor
document.cbor
blobs/sha256/<64 lowercase hexadecimal digits>
Optional correspondence, capture, report and cache records MAY be added only at paths registered by a later profile. Profile 0 package readers reject unregistered member paths instead of guessing their meaning.
mimetype is the first local-file member, stored without compression,
encryption or extra fields. Its exact ASCII bytes are:
application/nuif+zip
This media type is provisional until registration and MUST NOT be represented as IANA-registered.
manifest.cbor and document.cbor are canonical nuif-cbor-0. The manifest
declares profile/version, the semantic document descriptor, required
capabilities, assets, all resource descriptors and their roles. It does not
contain its own digest.
The required-capability set contains at most 256 identifiers of at most 128
ASCII bytes each. Structural readers validate and preserve those requirements.
Hosts use capability_report or require_capabilities with an explicitly
declared supported set before claiming full package support; missing
requirements are reported exactly. Structural decode alone is intentionally
available to inert inspection, preservation and extraction tools and is not a
semantic-support claim.
A structural SDK, CLI or editor that does not support every required capability
MUST keep the package read-only. It MAY validate, hash, extract the bare
document and copy the unchanged same-mode package, but it MUST NOT evaluate,
change document.cbor or change package mode while carrying capability
resources forward unless complete-set negotiation succeeds or a
capability-specific authoring profile explicitly detaches those resources. A
failed partial negotiation grants no authority. This prevents a
content-addressed sidecar from being silently bound to a document revision it
never validated.
Every embedded resource is stored at the path derived from its SHA-256 digest. The path is a locator. The manifest digest remains the identity and MUST match the bytes even if a future profile permits another physical layout.
5. Deterministic ZIP profile
The first profile uses stored members only. This deliberately trades archive size for cross-writer byte determinism and simple expansion limits. Images and fonts are usually already compressed; a later measured profile may add a fixed compression method without changing semantic hashes.
Writers MUST:
- emit
mimetypefirst, then every other member in bytewise ASCII path order; - use only the exact registered ASCII paths above;
- use ZIP method 0 (stored), no encryption, no data descriptors, no ZIP64 unless a later profile permits it, no archive/member comments and no extra fields;
- set the DOS timestamp to 1980-01-01 00:00:00;
- set a fixed creator/version and regular-file external attributes defined by the conformance fixture;
- precompute CRC-32 and sizes so local and central headers agree;
- emit no directory entries;
- emit one central-directory entry for each local member in the same order.
The manual writer and the independently implemented zip 8.6.0 writer now
produce identical fixture bytes with creator/version 0x030a, version-needed
10, regular-file attributes 0x81a40000 and the rules above. These values are
fixed for the experimental nuif-package-0 implementation. Standards-track
stability still requires cross-platform and externally maintained reproduction.
Readers MUST reject duplicate decoded names, backslashes, absolute paths, dot-segments, empty segments, non-ASCII paths, symlinks, directories, split or spanned archives, encryption, unsupported compression, inconsistent headers, undeclared blobs and declared embedded blobs that are absent.
6. External resolution
A portable package embeds every resource required to evaluate its declared profile. A linked/private authoring package may contain a linked locator, but:
- loading a document MUST NOT cause implicit network access;
- a resolver is an explicit caller-supplied capability;
- resolved bytes MUST match declared size and digest;
- redirects and authentication are resolver policy, never package authority;
- credentials, cookies and bearer tokens MUST NOT be stored in locators or provenance records;
- failure to resolve produces a typed fidelity/availability result.
Original URLs are provenance or retrieval hints, not resource identity.
7. Hashes
The semantic document hash remains SHA-256 of canonical document.cbor bytes.
It changes only when semantic document content changes.
The package hash is SHA-256 of the complete deterministic ZIP bytes. It proves the exact delivered artifact and changes when package-only records or caches change.
A resource digest is SHA-256 of exact resource bytes. An asset binding is semantic and therefore participates in the semantic document hash. Package locations and deletable caches do not.
Executable narrow image segment and broader proposal
nuif-png-rgba8-0 now executes the smallest unambiguous subset: bounded,
non-interlaced RGBA8; no ancillary metadata or one valid pre-image sRGB
intent; encoded samples interpreted as sRGB; straight decoded alpha; identity
decoder orientation; declared fit/crop, bounded forward affine transform,
nearest or fixed-bilinear sampling, opacity
and encoded-sRGB integer source-over. It rejects every other colour type,
bit-depth, colour signal, Exif/animation chunk and arbitrary metadata. Two
independent decoder libraries must agree on exact RGBA bytes. Exact rules and
non-claims are versioned in crates/nuif-media/PROFILE.md, and
cargo xtask gate-i-image emits its machine evidence.
The separately named nuif-png-basic-rgba8-1 decoder profile adds every
non-interlaced PNG colour/depth combination that can normalize to RGBA8 without
sample-precision loss. Image-paint transforms use
[a c tx; b d ty; 0 0 1] from crop-local source coordinates into the fitted
paint rectangle; the CPU reference inverse-samples pixel centers and rejects
singular or numerically unbounded matrices. Decoder and paint semantics remain
separate contracts even though Gate I exercises them together.
The broader PNG experiment remains separate. It must pin:
- accepted PNG conformance and ancillary chunks;
- decoder library/version or independent semantic rules;
- CICP/ICC/sRGB/gamma/chromaticity precedence and conflict policy;
- Exif orientation behavior;
- straight-alpha decoded representation and explicit premultiplication point;
- output color space, sampling and compositing;
- encoded bytes, dimensions, pixel count, decoded bytes, chunks and metadata limits;
- malformed-image and independent-decoder fixtures.
Animated PNG, JPEG, WebP, AVIF, video and SVG do not enter this profile by fallback. An adapter may freeze a selected animation/video frame as a derived resource and must report the transformation. SVG is imported through a safe declared adapter subset or retained as inert source; scripts and external resources never execute when a package opens.
Executable narrow font segment and broader proposal
nuif-opentype-static-single-0 accepts exact static TrueType-outline sfnt bytes
only under a declared portability policy. It requires face index zero,
canonically packed and checksummed tables, matching family names and exact
Unicode coverage, no variation axes, a matching fsType value, a non-empty
license expression and explicit embedding review. Package encode/decode and
resolved linked bytes run the same validation. The exact rules and non-claims
are in crates/nuif-font/PROFILE.md; cargo xtask gate-i-font compares Skrifa
0.46.2 behind NUIF-owned sfnt validation with a committed HarfBuzz 14.4.0
metadata capture for the pinned Ahem fixture.
This executable slice deliberately rejects TTC, CFF/CFF2, variable, color, bitmap, SVG and WOFF/WOFF2 fonts. The broader font-resource profile must pin:
- parser and table/resource budgets;
- face/collection selection;
- axes, named instances and feature selection;
- coverage and shaping inputs;
- malformed-table fixtures;
- handling of
fsType, no-subsetting and bitmap-only flags; - policy outcomes for portable/private/linked/substituted/unavailable resources.
No profile may infer exact font identity from a family name or screenshot.
Compatibility and migration
Existing canonical document bytes and hashes remain valid. Packaging them does
not alter document.cbor or its semantic hash. Current .nuif raw fixtures are
readable during alpha through content detection; tools should rewrite them only
on explicit save/export and should report the transition.
The package layer belongs above nuif-codec: codecs own canonical bare
encodings, while the package implementation owns manifest/resource/ZIP rules.
The core owns asset/resource semantics. CLI, editor, WASM, FFI and process
adapters call the same package API and MUST NOT carry independent ZIP policy.
The reference nuif-wasm-api-0 binding exposes structural package load,
explicit manifest-capability negotiation and deterministic package export over
byte arrays. Its cross-surface gate requires both no-op and edited package bytes
to match the native SDK exactly and preserves an embedded capability resource
without interpreting it. The same gate rejects a semantic edit through a
structurally loaded requirement-bearing package before complete-set
authorization. This is the browser/plugin package transport; host object access
and capability execution remain separate adapters.
Security
The package is untrusted. Before acceptance, the profile requires calibrated limits for total archive bytes, member count, per-member bytes, total expanded bytes, descriptor count, image/font bytes and decoder allocations. Resource size and digest are verified before media parsing. Readers do not extract to a filesystem.
The executable allocation gate additionally requires package-to-session handoff to share immutable resource buffers. Its 8 MiB trial preserves the allocation pointer and permits at most 1 MiB of handoff allocator traffic and retained bookkeeping. A 1,024-instance image trial permits at most 8 MiB of scene-build allocator traffic and 4 MiB retained for one 1 MiB decoded surface. These are reference-CI regression ceilings rather than wire-format limits.
Package resources never execute by being present. Scripts, shaders, links and embedded metadata are inert unless a separately authorized sandboxed capability interprets them.
Conformance tests
nuif:experiment:portable-package-resources must prove:
- two independent writers produce identical bytes on the normative fixture;
- package write/read/write reaches a byte fixpoint;
- required capabilities are bounded identifiers and missing host support is returned as the exact deterministic requirement set;
- document hash is unchanged by package creation and deletable-cache changes;
- asset identity survives byte replacement while the resource/document hashes change as specified;
- missing, extra, duplicate, traversal, symlink, directory, encrypted, unsupported, mismatched-size and mismatched-digest cases fail atomically;
- no implicit external resolution occurs;
- boundary and one-over archive/resource cases pass measured limits.
The narrow PNG experiment and package experiment must pass before claiming
nuif-png-rgba8-0. The broader PNG and font experiments must pass before those
resource classes are claimed generally.
Rejected alternatives
- Keep
.nuifas ambiguous raw JSON/CBOR indefinitely: prevents reliable media identification and resource delivery. - Use paths or URLs as identity: moving a package or changing a CDN URL would change identity without changing bytes.
- Use content hashes as editable asset IDs: every image replacement would break semantic references and operation history.
- Store decoded RGBA instead of original images: loses source encoding, color metadata and efficient distribution.
- Embed every font found by a browser: local bytes may be inaccessible and redistribution may not be permitted.
- Permit ordinary ZIP/ZIP64/compression options: expands attack surface and prevents a simple first cross-writer byte profile.
- Use TAR: weak random access and no widely deployed first-member media-type convention for this use.
- Use OCI artifacts directly: descriptor ideas are valuable, but registry/image layering semantics are unnecessary for a single design package.
Unresolved questions
- Whether future packages add deterministic compression or rely on outer transport compression.
- Whether correspondence/capture/report records are canonical-adjacent members or separate linked artifacts.
- Media-type registration timing and final name.
- License-expression vocabulary beyond preserved evidence and user/admin policy.
RFC 0011 — Observation, reconstruction and inference provenance
Document status: proposed. This RFC refines RFC 0003 for imported observations and. Canonical source.
probabilistic reconstruction. It does not make any model, capture provider or screenshot profile normative.
Implementation note: nuif-capture, nuif-reconstruct, cargo xtask capture-baselines and cargo xtask reconstruction-provider-manifest exercise
a bounded fixed-input subset of these contracts. Every encoded observation
bundle carries the canonical manifests behind its provider identities and
proposal application rejects an identity that is absent from that registry.
cargo xtask gate-j-live separately exercises one pinned local Chromium
fixture, structured runtime context, exact response retention, secret canaries
and held-out viewport measurement. That automation does not change this RFC’s
proposed status or establish a portable capture/reconstruction accuracy
profile.
Motivation
NUIF supports deterministic source adapters and is researching browser capture and screenshot reconstruction. These routes expose different evidence:
- a retentive source adapter can preserve exact source bytes and correspondences for its declared subset;
- a browser capture can observe source responses, resources, resolved layout, styles, accessibility and pixels under a pinned execution context;
- a screenshot can observe only visible pixels plus supplied metadata.
Without typed evidence classes, a visually similar generated document could be misreported as a lossless import. This RFC defines the records and fidelity ceilings needed to prevent that category error while keeping models replaceable.
Prior art and evidence
nuif:research:chromium-source-backed-ui-captureidentifies the independent DOM/layout/style/network/font/accessibility/screenshot observations available from a pinned browser protocol.nuif:research:live-chromium-cdp-capturecompares CDP, Playwright and WebDriver BiDi boundaries and records the bounded transport plus first live executable result.nuif:research:design2code-real-world-benchmarkrecords real-page element recall and layout failures in one-shot screenshot-to-code systems.nuif:research:reverse-layout-inferenceandnuif:research:inferui-and-layout-synthesisshow why inferred layout needs multiple contexts, alternatives and held-out evaluation.nuif:research:confidence-calibration-and-selective-predictionmotivates empirical calibration and abstention.nuif:research:model-agnostic-screenshot-reconstruction-and-trainingsynthesizes the replaceable-provider, typed-operation and closed-loop plan.
Evidence classes
Every imported or reconstructed property has one or more evidence links whose class is:
authored_source: exact retained source or host semantic value with stable correspondence;resolved_source: value observed from a pinned evaluator/runtime context;observed_pixels: value directly measured from identified image pixels;inferred: hypothesis produced from observations or heuristics;user_confirmed: value explicitly accepted or supplied by a user;derived: value produced by a declared deterministic or generative transformation from other evidence;unavailable: expected evidence could not be obtained.
user_confirmed does not rewrite the history of an inferred value; both records
remain linked. derived names its inputs and transformation identity.
Observation records
An ObservationRecord contains:
observation_id
evidence_class
subject: optional entity/property/resource reference
source_artifact_digest
source_region_or_locator
coordinate_space_and_context
provider: kind + canonical provider-manifest digest
value_or_candidates
raw_confidence: optional
calibrated_confidence: optional
calibration_profile: optional digest
alternatives: ordered candidates with evidence
privacy_and_retention_class
Coordinates MUST name their space: source pixels, device pixels, viewport CSS pixels, crop-local pixels or NUIF logical units. A transformation between spaces is an explicit record.
Raw and calibrated confidence MUST NOT share one field. A calibrated confidence is valid only for the decision type and evaluation distribution identified by its calibration profile.
Capture contexts
A resolved browser observation context records at least browser/protocol build, operating system, viewport, device-pixel ratio, page scale, locale, timezone, color/media preferences, font environment, scroll/pseudo state, navigation identity, readiness/settling policy and animation/time freeze.
Repeated capture under an unspecified context cannot support a reproducibility claim. Multiple contexts are separate records linked by proposed semantic identity.
Cookies, authorization headers, credentials, storage values and secret form content are excluded from export. If exclusion changes an observation, the capture report records the unavailable/redacted evidence without retaining the secret.
Reconstruction interface
Probabilistic systems are optional clients of the normative semantic operation interface. They MUST propose a bounded transaction; they MUST NOT mutate core document structures directly or emit executable code as an implicit operation.
A reconstruction attempt returns:
ReconstructionResult {
status: valid_result | no_result | budget_exceeded | policy_rejected,
document_hash: optional,
accepted_transaction_log,
observations,
fidelity_report,
alternatives_and_abstentions,
evaluation_report,
pipeline_artifact_manifest,
}
Every proposed transaction passes syntax/resource limits, operation validation, atomic application and complete document validation. Rejection leaves the prior document unchanged and returns stable diagnostic codes.
Fidelity ceilings
Fidelity describes evidence, not confidence or visual quality.
authored_sourceMAY belosslessonly inside a declared adapter profile whose correspondence and round-trip laws pass conformance.resolved_sourceMAY belosslessonly for the declared resolved observation under its exact context; it cannot prove authored intent.observed_pixelsandinferredMUST NOT belosslessfor authored semantics, source resources, responsive rules, accessibility or behavior.- screenshot-derived crops, traces or generated resources are
derivedand at bestapproximatedrelative to an unavailable source resource. - a visually matching flat screenshot cannot satisfy an editable reconstruction profile unless the requested target is explicitly a flat image document.
- behavior inferred from a static screenshot remains inferred even when an icon or label strongly suggests an action.
Confidence MUST NOT promote an item above its evidence-class fidelity ceiling.
Closed-loop correction
A reconstruction profile MAY render and correct a candidate iteratively. Each iteration records input document hash, proposed transaction, validation result, render context/hash, property and visual differences, objective vector, acceptance decision and next document hash.
The loop MUST have finite iteration, model-call, time, memory and resource budgets. It MUST stop on repeated state. Acceptance MUST preserve validity and declared protected metrics. The profile MUST reject objectives that can be satisfied by deleting semantics or covering the viewport with the source screenshot.
No single perceptual metric is a conformance oracle. A reconstruction report uses typed structure/text/geometry/resource/responsive/accessibility measures plus declared visual diagnostics.
Provider and model neutrality
OCR, region detection, UI grounding, proposal engines and correction engines are replaceable providers with capability/artifact manifests. The specification does not name a required model, vendor, training library, parameter count or deployment service.
The implemented nuif-reconstruction-provider-manifest-0 wrapper is canonical
CBOR. It binds capabilities, local/remote execution modes and input/output wire
profiles to one exact implementation plus optional model weights, processor,
adapter, quantization, prompt and tool-configuration artifacts. Released or
learned providers require a content-addressed SPDX 3.0.1 or CycloneDX 1.7
inventory; learned providers additionally require a model card. NUIF points to
that external inventory instead of defining another SBOM vocabulary. The
current browser and screenshot baselines are explicitly development-only,
source-bundle-identified providers with no learned-artifact or accuracy claim.
Model weights, processors, low-rank adapters, quantization configurations and
training datasets are not NUIF document resources. They are separately
versioned operational artifacts. A .nuif package can record which artifact
produced an inference without carrying or requiring that artifact for ordinary
document use.
Training and distillation boundary
Training is non-normative, but any released reconstruction artifact claiming a NUIF benchmark result has a digest-pinned manifest, model card, dataset datasheets and reproducible evaluation report.
Training examples derived from reconstruction runs include only validated accepted operation transitions as positive targets. Private/authenticated captures are excluded by default and require explicit training consent. Frozen benchmark families are excluded from training and distillation.
LoRA, quantized low-rank adaptation and knowledge distillation are experiment choices. Their use confers no format or conformance status.
Compatibility
Existing provenance/fidelity records remain valid. Implementations may migrate
an untyped provenance record to authored_source only when its retained source
and adapter profile prove that classification. Otherwise migration uses
unavailable or inferred; it does not guess a stronger evidence class.
The current deterministic HTML/SVG/DTCG/Penpot adapter results are unaffected. Their existing profile laws determine losslessness. A future browser capture is a separate adapter rather than a hidden mode of the Tree-sitter HTML adapter.
Security and privacy
Model and provider outputs are untrusted inputs. Parsers enforce operation, observation, string, binary, entity, resource and iteration budgets before application. URLs/scripts in model output are inert. External fetch and code execution require explicit caller capabilities and remain outside reconstruction conformance.
Screenshot text may contain personal, credential or proprietary information. Inference can run locally; remote transfer is an explicit deployment policy. Retention, telemetry and training are separately consented purposes.
Visible prompt injection is data in the screenshot. It cannot alter tool policy, operation grammar, package resolution or security limits.
Conformance tests
The planned baseline, loop and calibration experiments must prove:
- every provider output records artifact identity, source region/context and evidence class;
- invalid, stale or over-budget model operations fail atomically;
- screenshot-only results never emit authored
losslessclassifications; - exact source bytes, derived crops and unavailable resources are distinguishable;
- coordinate transforms reproduce observation locations;
- the loop stops on success, no improvement, repeat and budget exhaustion;
- the editable profile rejects flat-image reward gaming;
- raw/calibrated confidence and calibration identity remain distinct;
- automatic/review/abstain decisions reproduce the declared risk threshold;
- secret canaries do not appear in exported observations or training traces.
Rejected alternatives
- End-to-end screenshot-to-document text with no typed observation/operation boundary: invalid outputs and hallucinated semantics become hard to audit.
- Put model calls in
nuif-core: makes conformance provider-dependent and the deterministic core non-reproducible. - Treat one high visual score as lossless: rewards flat screenshots and hides text, structure, resource and responsive failures.
- Store only a whole-document confidence: cannot express one uncertain font, parent or behavior inside an otherwise strong result.
- Require a particular OCR, detector or VLM: confuses an evolving implementation choice with interchange semantics.
- Train before a frozen evaluator exists: no stable evidence that the tuning improves the intended task rather than the training distribution.
Unresolved questions
- Final observation and calibration-profile schemas.
- Whether observations live inside a portable package, a sidecar evidence bundle or both under separate profiles.
- Minimum edit-task suite for an “editable reconstruction” claim.
- Rules for equivalent alternative structures when pixels do not distinguish them.
- Risk/coverage thresholds for automatic application by profile.
RFC 0012 — Behavior as a content-addressed package resource
Document status: proposed, with executable experimental profile. Canonical source.
nuif-behavior-package-resource-0. This RFC chooses the first transport for
the bounded behavior experiment. It does not add behavior to the canonical
semantic Document, standardize a media type, or authorize execution when a
package is opened.
Motivation
The independent Rust/Node behavior traces and the three-engine web lowering show that one finite state-machine subset is executable. Keeping it only as an out-of-band JSON fixture, however, cannot test offline delivery, package identity, corruption handling or document-reference validation after a package round trip. Moving it directly into the canonical document would make a much larger and less-tested compatibility commitment.
The narrow requirement is therefore: transport one exact behavior program with one exact document using the current deterministic package, while keeping behavior modular and inert until an explicit capability-aware runtime accepts it.
Prior art and evidence
nuif:research:behavior-portability-state-machinesdefines the finite trace contract and its current exclusions.nuif:research:behavior-package-resource-bindingcompares OCI descriptors, EPUB container processing and KHR_interactivity, then records the executable package decision.- RFC 0010 already defines deterministic package manifests, digest-addressed blobs, resource roles, explicit resolution and the rule that resources do not execute merely because they are present.
Profile
nuif-behavior-package-resource-0 uses nuif-package-0 unchanged. A conforming
attachment has all of these properties:
- exactly one resource descriptor has media type
application/nuif-behavior+cbor; - that media type is provisional and is not an IANA-registration claim;
- the descriptor role is
source, its locator is embedded, and it has no derivation record; - the embedded path is the normal
blobs/sha256/<digest>path; - the resource bytes are canonical
nuif-cbor-0encoding of exactly onenuif-behavior-state-machine-0BehaviorProgram; required_capabilitiescontainsnuif-behavior-state-machine-0;- the complete program validates against
document.cborbefore attachment and after package decode.
Zero behavior resources with no behavior capability is a valid package without behavior. A capability with no resource, a resource without its capability, more than one behavior resource, a linked behavior resource, non-canonical bytes or invalid document references fail the attachment profile.
Identity and binding
The behavior resource digest identifies its exact canonical program bytes. The document canonical hash identifies exact semantic document bytes. Neither is redefined to include the other.
manifest.cbor carries both descriptors, and the deterministic complete
package hash binds their delivered pairing. Consequently:
- attaching behavior changes the package hash but not the document hash;
- changing the document changes both its hash and the complete package hash;
- the same behavior resource can be deduplicated by digest without claiming it is valid for every document;
- every explicit load revalidates stable entity references against the actual package document.
Processing and authority
Package processing is layered:
NuifPackage::decodevalidates the deterministic ZIP, canonical manifest, document, descriptors, sizes, hashes and embedded bytes.capability_reportorrequire_capabilitiescompares every package requirement with a caller-declared host capability set without executing it.attached_behavioropts into this RFC, checks descriptor/capability agreement, decodes canonical behavior bytes and validates the program against the document.- A caller creates a behavior runtime only with an explicit set of available abstract-effect capabilities.
- A host adapter separately maps admitted effects under its own profile and security policy.
Step 1 never implies steps 2–5. Generic tools may inspect, copy and deterministically re-encode the package without executing the resource. Tools that claim full behavioral conformance must understand the required behavior capability; a structural package decode alone is not such a claim.
Security and limits
The package verifies declared byte length and SHA-256 before canonical CBOR or behavior validation. Existing package byte/resource/member limits and behavior state/transition/action/string limits both apply. Linked behavior is rejected so the profile neither invokes a resolver nor grants network authority.
The program remains data. It contains no scripts, dynamic imports, filesystem paths, URLs, timers or host calls. Runtime creation and every host lowering are separate authorization decisions.
Conformance
nuif:experiment:behavior-package-resource requires:
- canonical program bytes and exact attach/decode/encode fixpoint;
- unchanged document hash and changed package hash after attachment;
- exact required capability, source role, media type and digest-derived path;
- refusal of missing/duplicate/linked/malformed/rebound resources;
- rejection of package-byte corruption before behavior decoding;
- an independent standard-library ZIP reader checking exact package bytes, member order/metadata, CRC and the content-addressed behavior resource.
cargo xtask gate-behavior includes this gate before the independent behavior
trace gate. CI archives all generated package and report artifacts.
Rejected alternatives
- Canonical
Document.behaviorin the first experiment: premature semantic commitment and forced support in all readers. - A special
behavior.cborZIP path: duplicates the resource manifest and requires a new container path profile. - An opaque extension inside an arbitrary entity: gives behavior no unique package-level discovery or cardinality rule and couples it to one visual node.
- JSON attachment: creates a competing canonical encoding and larger parsing surface.
- Automatic execution on package open: violates RFC 0010’s inert-resource boundary and prevents safe inspection tools.
Unresolved questions
- Which next event/effect types remain finite and portable across web, native and presentation hosts.
- When independently implemented adapters provide enough evidence to propose a canonical behavior model.
- Final media-type naming and registration timing.
ADR 0001: Rust for the reference core
Document status:
accepted. Canonical source.
Decision
The initial reference implementation uses Rust for the canonical in-memory model, operations protocol, validation, codecs, layout/renderer integration, headless tooling, and WASM boundary.
Rationale
The project needs memory safety for hostile document parsing, deterministic systems-oriented behavior, strong enum/type modeling, high-performance geometry/text/GPU ecosystems, native and WebAssembly targets, fuzz/property testing, and stable FFI boundaries. Rust gives the strongest combined fit among the evaluated mainstream implementation languages.
The editor UI and source-specific adapters may use TypeScript or target-native languages where that produces a cleaner integration. Those layers must communicate through stable protocol/API boundaries and must not redefine the canonical model.
ADR 0002 — Use Taffy as initial CSS-family layout evaluator
Document status: accepted for prototype. Canonical source.
Taffy provides Rust implementations of Block, Flexbox and Grid and is already embedded by multiple UI systems. NUIF will wrap it behind its own evaluator interface. Taffy types MUST NOT become canonical schema types.
Constraint/freeform/proposal-response semantics remain separate evaluators/lowerings.
Gate C verification
The first executable use is an independent test lowering, not canonical-schema coupling: nuif-testing pins Taffy 0.14.0 and compares it with Chrome for Testing 152.0.7977.64 and the profile-0 reference evaluator. This exposed and corrected definite cross sizes being overwritten by stretch. The v0 and generated stack/flex subset now agree within measured fixture-local bounds.
Grid remains intentionally unwired in the reference evaluator because the current authored model has no track-sizing or item-placement fields. The differential report classifies those differences as schema loss. A follow-up schema decision is required before a Taffy Grid lowering can be called representable.
ADR 0003 — Renderer abstraction with Vello/wgpu experiment
Document status: accepted for prototype. Canonical source.
Use a NUIF-owned render-scene boundary. Implement an interactive Vello/wgpu backend experiment and retain the ability to add CPU/reference backends for deterministic conformance.
No Vello internal data type is normative NUIF state.
ADR 0004 — Encoding strategy
Document status: provisional. Canonical source.
Define serialization independently of the logical model. Prototype a canonical text form and deterministic CBOR binary form. Benchmark schema-based alternatives before ratification.
Opaque extensions are explicit NUIF values/bytes so preservation does not depend on a codec’s unknown-field implementation.
Current experimental outcome
nuif-text-0 and nuif-cbor-0 are executable profile-0 encodings. The codec
decision gate measures both only after semantic, canonical and opaque-data
fixpoints pass. Deterministic CBOR materially reduces file size, but its current
typed decoder is not faster than canonical text, so performance alone
does not ratify it as the only future binary profile.
Schema codecs must present a complete NUIF mapping before benchmark admission. Protobuf is presently excluded because its own specification says deterministic binary serialization is not canonical. FlatBuffers offers direct access but permits multiple byte layouts for equal values and does not prove retention through an old-schema edit. Cap’n Proto is the next candidate because it specifies schema-agnostic canonicalization and traversal limits; it remains unadmitted until cross-version retentive editing and two canonical writers pass.
The decision report is generated by cargo xtask codec-benchmark; transient
host timings are evidence artifacts, not normative limits.
ADR 0005 — Keep CRDT state out of canonical documents
Document status: accepted. Canonical source.
Collaboration engines operate over NUIF semantic operations and materialize canonical snapshots. Replica clocks/tombstones/history belong to a collaboration profile or sidecar, not every .nuif document.
This permits Automerge/Yjs experiments and independent non-collaborative implementations.
ADR 0006: Rust-native reference editor on Masonry, Vello and AccessKit; toolchain policy
Document status:
accepted. Canonical source.
Decision delegated to research on 2026-08-29. Evidence: nuif:research:masonry-editor-stack-decision, nuif:research:rust-toolchain-and-msrv-policy, nuif:research:masonry-xilem-and-linebender-test-harness, nuif:research:egui-and-egui-kittest, nuif:research:iced-slint-gpui-makepad-floem, nuif:research:accesskit-semantic-ui-testing, nuif:research:vello-testing-and-cpu-reference, nuif:research:wasm-headless-execution.
Context
docs/whitepaper/06-language-and-runtime-choice.md and apps/editor/ARCHITECTURE.md proposed a Svelte 5 and TypeScript shell over a Rust/WASM core. The editor’s role (apps/editor/README.md, RFC 0004) is a test instrument: headless execution, deterministic snapshots through a CPU reference path, an accessibility tree that carries entity identifiers, and the same harness as the CLI.
Decision
Stack
- The reference editor is a Rust binary crate (
apps/editor) built directly on Masonry (retained widget tree) with Vello rendering and AccessKit. Xilem is not used: its view layer lags the widget set (xilem issue 1710) and entity-to-widget identity must be explicit in the editor. - Masonry is pinned by full SHA to the reviewed
refpath/xilemfork at1b96eb8; its parenteabfe0acarries the wgpu 29 migration, and it descends from the evaluated upstream main revisionb81d8d7(2026-08-28). The 0.4.0 release of 2025-10-29 lacks theCanvaswidget, paints into a Vello 0.6Scene, and screenshots only through wgpu. The pinned line providesWidget::paintover animaging::Painter,Canvas::update_scenerecording animaging::record::Scene, andmasonry_testingrasterization throughimaging_vello_cpuwithout a GPU. - The editor canvas is a lowering from NUIF
RenderScenetoimagingcommands. NUIF’s own CPU reference renderer remains the conformance oracle; Masonry’svello_cpupath rasterizes shell chrome for headless screenshot tests only. - For every type that crosses the widget boundary (Vello, Parley, AccessKit, wgpu), the editor crate follows Masonry’s pinned versions. A
cargo metadataprobe of Masonry main with independently chosen Vello 0.10, Parley 0.11 and AccessKit 0.25 produced duplicate Vello (0.8/0.10), wgpu (28/29), Parley (0.8/0.11.1), AccessKit (0.24.1/0.25) and fouraccesskit_consumerversions; the editor build therefore disables NUIF’s standalone Vello backend and exchanges pixel buffers with the reference path as bytes. - The Svelte shell becomes a later browser demonstration of the WASM bindings. Engine parity in browsers is tested through
wasm-bindgen-testand Playwright layout oracles; in-browser WebGPU pixel oracles are not used.
Toolchain and MSRV
rust-toolchain.tomlpins the current stable, 1.98.0 (released 2026-08-20). The pin is raised in one dedicated commit within each release cycle.rust-versionis 1.96:max(toolchain − 2, highest dependency MSRV), where Masonry main requires 1.96,imaging1.92, Vello and Parley 1.88, wgpu 1.87, AccessKit, HarfRust and proptest 1.85. This coincides with Masonry’s own N-0..N-2 practice and with Bevy (1.96), Zed and Servo (1.97.1) and rust-analyzer (1.98).- The workspace uses
resolver = "3"so that dependency resolution is MSRV-aware. - CI runs the main job on the pinned toolchain and an
msrvjob that checks the workspace on 1.96.0.
Licensing
- Masonry is licensed Apache-2.0 only; Vello, Parley and
imagingareApache-2.0 OR MIT; AccessKit isMIT OR Apache-2.0;accesskit_winit,winit,tree_arena,instaandciboriumare Apache-2.0 only. Depending on Apache-2.0-only crates does not constrain NUIF’sApache-2.0 OR MIT(Apache-2.0 §4 obliges notice retention on redistribution and permits different terms for one’s own work); the only downstream effect is on GPLv2-only consumers. deny.tomlallows exactly:MIT,Apache-2.0,Apache-2.0 WITH LLVM-exception,BSD-2-Clause,BSD-2-Clause-Patent,BSD-3-Clause,ISC,Zlib,Unicode-3.0,BSL-1.0,CC0-1.0,0BSD,Unlicense.BlueOak-1.0.0is not allowed;minicboris therefore excluded (RFC 0005 selectsdcbor, BSD-2-Clause-Patent).
Rationale
- Masonry main is the only candidate whose harness returns the frame’s visual layer plan and AccessKit
TreeUpdatefrom oneredraw(), acceptsActionRequests directly, controls time, hosts a custom-painted canvas, and rasterizes without a GPU. - egui:
egui-wgpucallbacks paint inside egui’s own render pass, while Vello requires compute passes; egui would duplicate the text and vector stack;egui_kittest’s query API remains the model for NUIF’s harness surface. - Floem: no AccessKit (issues 8 and 973 open). Blitz: custom paint sources exist but the harness is unpublished. GPUI: builds an AccessKit tree per frame but exposes no query in test contexts and requires latest stable. iced: no accessibility tree. Slint: licence and DSL-owned element tree.
Widget inventory
Masonry main ships 37 widgets including Split (draggable), CollapsePanel, Selector, StepInput, TextInput, VirtualScroll, Canvas and a tooltip layer. Missing for apps/editor/UI-SPEC.md and composed in the app crate: tree view (layers panel), drag-and-drop reparenting (emits protocol Move from canvas pointer events), menus and tab strip, colour picker, keyboard-shortcut table, multi-line text input (newlines unsupported in TextInput), text undo/redo (xilem issue 1417).
Risks and mitigations
- API churn: paint signature, layout system and renderer changed between 0.4.0 and main without a changelog; 304 days without a release. Mitigation: git pin, single bump commits, harness behind NUIF’s session-driver trait, no Masonry types in
crates/. Splitpointer defect (xilem issue 1581);unsafeNodeId workaround inaccess_node(AccessKit issue 701, fixed inaccesskit_consumer0.36 while Masonry pins 0.35). Mitigation: isolated in the app crate;unsafe_code = "forbid"stays incrates/.vello_cpuregressions affect shell screenshots only (tier 2 tolerance); the render suite uses NUIF’s CPU reference.
Consequences
rust-toolchain.toml,Cargo.toml(rust-version,resolver),.github/workflows/ci.ymlanddeny.tomlare updated with this ADR.apps/editor/ARCHITECTURE.md,apps/editor/UI-SPEC.mdanddocs/whitepaper/06-language-and-runtime-choice.mdreference this ADR.- The editor exposes entity identifiers through AccessKit
author_id; the harness queries by role, label and identifier and dispatches actions without pointer synthesis.
Unverified
Linebender Zulip release announcements were not retrieved; the anyrender CustomPaintSource trait was not located; the 1.99 release date (2026-10-01) is computed from the cadence, not announced.
ADR 0007: Tag-driven native editor prereleases
Document status:
accepted. Canonical source.
Decision delegated to research on 2026-08-30. Evidence:
nuif:research:github-release-delivery-and-provenance and
nuif:research:cargo-workspace-xtask-and-ci-layout.
Context
The native editor packaging task produces host-specific archives and verifies their executable entry points. CI retained those archives as temporary workflow artifacts. It did not provide a version contract, a durable download location, cross-platform checksums, or build provenance. Application delivery requires a tag-to-version invariant and a publication transaction that does not expose a partially assembled release.
Decision
- The editor uses Semantic Versioning independently from unpublished library
crates. The first application version is
0.1.0-alpha.1and its release tag isv0.1.0-alpha.1. - A release tag identifies one immutable source revision. Tags are not moved; a correction increments the prerelease number.
.github/workflows/release.ymlvalidates that the tag equalsvfollowed by the editor package version and that the checkout is clean.- The release workflow runs the complete repository harness before building native packages on Linux x86-64, Linux Arm64, Windows x86-64, macOS Arm64, and macOS x86-64 hosts.
- Each archive name contains the editor version, operating system, and architecture. Each package manifest records its source revision, version, platform, architecture, executable and archive SHA-256 digests, smoke tests, and signing status.
- GitHub artifact attestations cover every archive and platform manifest. The
publication job creates and attests
SHA256SUMS, a CycloneDX software bill of materials, andrelease-manifest.json. - Publication creates a draft, attaches every asset, and then publishes the prerelease. A failed build or upload leaves no published partial release.
- Alpha packages are unsigned. macOS Developer ID signing and notarization, Windows publisher signing, installer publication, and automatic updates are credentialed stages that require separate review.
- cargo-dist is not introduced for the alpha because the existing xtask owns native application layouts, manifests, trials, and smoke tests.
Consequences
- GitHub Releases becomes the durable download surface for tagged editor builds. CI artifacts remain diagnostic evidence for ordinary commits.
- ADR 0009 makes locally built, user-scoped source installation the primary developer path. Release archives remain durable evidence and an expert opt-in path rather than the default installation mechanism.
- Application consumers can verify SHA-256 checksums and GitHub provenance without building the repository.
- Unsigned alpha packages may trigger Gatekeeper or SmartScreen warnings. The release notes state this limitation.
- The stable-release gate remains open until macOS and Windows signing identities, credential rotation, and update policy are approved.
ADR 0008: Vendor products integrate through host adapters
Document status:
accepted. Canonical source.
Decision delegated to research on 2026-08-30. Evidence:
nuif:research:figma-plugin-and-rest-api-as-automation-surface,
nuif:research:figma, and nuif:research:adobe-uxp-host-integration.
ADR 0012 amends clauses 5 and 8 for the active vendor priority: Affinity is the desktop interchange target and Canva is the programmable adoption target. The Adobe clauses below remain the historical decision and prior-art record.
Context
The native NUIF editor is a reference implementation and conformance tool. It is not a plug-in runtime for every design product. Figma and Adobe applications already own their document stores, undo systems, permissions, extension runtimes and distribution channels. Embedding the NUIF editor executable would duplicate the host shell and would not provide access to the host document.
Source adapters use byte spans because they synchronize retained source. Host APIs expose objects and properties instead of source bytes, so they require a different correspondence record and evidence report.
Decision
- A vendor product integrates NUIF through a host adapter that maps its public object API to canonical NUIF documents and operations. It does not embed the NUIF editor application.
nuif-adapter::HostAdapterReportis the common evidence envelope for API hosts. It records profile, direction, host/API versions, optional host revision, canonical hash, host-object correspondence, fidelity and unmapped-data preservation.- Every host and product has a separately versioned bounded profile. “Figma” and “Adobe” are not fidelity claims by themselves.
- Figma uses a user-run TypeScript/JavaScript Plugin API bridge with dynamic page loading. The default package has no network access. UI-frame file transfer and host-object mutation are separate message channels.
- Adobe UXP delivery is one
.ccxpackage per supported host. The initial profile targets InDesign. Photoshop is separate and performs all mutations inside oneexecuteAsModaltransaction. Illustrator remains unclaimed until its current public SDK and package contract are researched and tested. - Host correspondence uses host object identifiers plus namespaced NUIF metadata when the host documents such persistence. Duplicate or missing persisted identifiers are repaired with new NUIF identifiers and reported; silent identity reuse is forbidden.
- Vendor plug-in versions are independent of the native editor version. A plug-in declares the NUIF profile/spec revisions it supports and runs the same checked-in fixtures before publication.
- GitHub Actions may build and retain review artifacts. Publication into Figma Community or Adobe Marketplace remains a separately authenticated, host-governed release step.
Consequences
- Vendors can adopt NUIF without adopting the reference editor UI or Rust.
- API and source adapters share fidelity semantics while using correspondence appropriate to their medium.
- A credential-free repository can test pure mapping logic and checked-in host snapshots, but live-host claims require the named host/version evidence.
- Marketplace identifiers, approvals and publisher accounts remain outside the repository and cannot be inferred from a GitHub release.
ADR 0009: Source-built developer installation
Document status:
accepted. Canonical source.
Decision delegated to research on 2026-08-30. Evidence:
nuif:research:developer-source-installation-and-os-trust and
nuif:research:github-release-delivery-and-provenance.
Context
NUIF Editor will remain a developer-facing reference and conformance tool. Unsigned native archives are useful build evidence, but making them the normal installation path couples routine research use to Apple and Microsoft publisher-reputation systems. Marketplace publication and paid signing identities are not architectural requirements for a developer tool.
A different path must still preserve source identity, dependency locking, platform integration, updates, rollback and safe removal. It must not claim that local compilation bypasses every managed-device policy or weaken an operating system’s security controls.
Decision
- The primary editor installation is a local build from a clean, pinned source
revision using the checked-in
Cargo.lockand repository Rust toolchain. cargo xtaskowns user-scoped install, update, doctor, rollback and uninstall commands. There is no system-wide installation mode.- Source installs record the version, source revision and cleanliness, lockfile digest, toolchain, platform, architecture and installed binary digest in a machine-readable receipt.
- The
alphachannel resolves only published prereleases. Its release manifest must pass GitHub attestation verification for the NUIF release workflow, tag and source revision before the revision is built. - Updates are explicit. The active and previous immutable installations are retained so activation can be rolled back without rebuilding.
- macOS installs a locally built, ad-hoc-signed application under the user’s
Applications directory. Windows installs under the user’s local application
directories and may create a user Start-menu shortcut. Linux installs under
user XDG and
~/.localpaths. - The lifecycle never disables Gatekeeper, System Integrity Protection, Defender, SmartScreen or Smart App Control and never adds a certificate to a trust store. Managed-device exceptions require organization approval.
- GitHub release archives remain CI evidence, reproducibility material and an expert opt-in path. They are not described as trusted publisher-signed applications.
- Homebrew source formulae and Nix flakes are compatible future convenience layers. A Scoop bucket downloads the unsigned Windows archive and therefore cannot replace the source-built trust path.
Consequences
- A developer can install and update the editor without Apple Developer Program membership, Microsoft Store publication or administrator access on an ordinary development machine.
- The default path takes compilation time and requires the platform Rust/native build prerequisites.
- A locked-down machine may still reject locally built unsigned code. Its administrator must supply an approved certificate or policy; NUIF does not work around that control.
- Release automation and source installation have separate responsibilities: releases attest the channel input, while the local lifecycle owns the executable installed on the developer’s machine.
ADR 0010: Bounded retentive Svelte source adapter
Document status:
accepted. Canonical source.
Decision delegated to research on 2026-08-30. Evidence:
nuif:research:svelte-source-adapter-surface,
nuif:research:tree-sitter, and
nuif:research:dependency-and-subsystem-audit.
Context
Svelte source combines declarative markup with JavaScript, TypeScript, preprocessing, directives, blocks and component-scoped CSS. NUIF needs useful source interchange without claiming that static parsing reproduces an application runtime or rewriting unrelated developer source.
Decision
nuif-svelteimplements only the versionednuif-svelte-static-0profile. It maps regulardivcontainers andspanliteral text through explicitdata-nuif-*markers and a fixed inline-style vocabulary.tree-sitter-svelte-next0.1.1 supplies the production concrete syntax tree and UTF-8 byte ranges. It does not define Svelte semantics.- Exact official
svelte5.57.0 is the foreign parser/compiler oracle. It is test tooling only and never enters a NUIF deliverable. - Synchronization edits only recorded scalar spans, rejects stale or structural changes atomically, reparses the result, and requires canonical NUIF equality.
- Scripts, stylesheets, preprocessors, expressions, blocks, directives, components, special elements and dynamic attributes are outside profile zero. Top-level comments and whitespace may be retained but have no claimed semantic mapping. Other top-level nodes are rejected.
- Component CSS is deferred. A later profile must define selector ownership, cascade, specificity, Svelte scope hashing and source-locality rules before it can be accepted.
Rationale
The official compiler is the only suitable semantic authority, but its printer may normalize whitespace and quoting. A concrete syntax tree is therefore the right production mechanism for retained spans, while live official compilation prevents that community grammar from becoming a self-oracle. The broader unofficial Rust compiler adds a second semantic implementation and a much larger dependency graph without improving the deliberately static boundary.
Consequences
The adapter is useful for generated components and controlled developer tools, not arbitrary Svelte applications. Its conformance claim includes exact round-trip and byte-complement preservation plus pinned official parse/compile acceptance. Runtime rendering equivalence and general CSS equivalence remain unclaimed.
ADR 0011: One byte-oriented SDK precedes foreign ABI stabilization
Document status:
accepted. Canonical source.
Decision delegated to research on 2026-08-31. Evidence:
nuif:research:rust-sdk-and-foreign-language-bindings and
nuif:research:wasm-headless-execution.
Context
The reference engine already has a headless nuif-api session and working CLI,
WASM, MCP and editor clients. The WASM crate nevertheless duplicated bare
document decoding, validation, canonical export, hashing and session history
composition. Adding C, Swift and Kotlin wrappers before removing that
duplication would create more places for semantics to escape the core.
Rust does not promise stability for its native ABI. A public C ABI can be stable only after the project specifies ownership, buffers, errors, panics, threads and symbol compatibility. Swift and Kotlin additionally need generated language wrappers and platform packaging; binding generation alone is not a distribution system.
Decision
nuif-api::NuifDocumentis the single package-aware, byte-oriented Rust SDK façade. It owns no filesystem, network, plug-in or process authority.- The façade accepts explicit canonical text/CBOR profiles, separately loads fully verified packages, applies typed semantic operations and exports bare encodings or deterministic packages through existing implementations.
- WASM, MCP, CLI and editor wrappers may add transport limits and host policy, but must not implement document semantics independently.
- Cross-surface conformance requires exact canonical bytes, hash, diagnostics and patch behavior for the shared subset.
- No stable C ABI is declared while the semantic SDK is
0.0.x. The eventualnuif-ffiis a small separately reviewed crate over SDK byte records, not a C representation of internal Rust structs. - C/C++ headers should be generated with pinned cbindgen. Swift/Kotlin wrappers should prefer pinned UniFFI over handwritten per-language ownership glue, but only after ABI, sanitizer and native-consumer gates are defined.
- Each foreign binding and platform package has an independent profile, version, compatibility report and release stream.
Consequences
- Direct Rust and browser clients use one operation and serialization path.
- Packages and embedded resources survive SDK edits without giving wrappers implicit resolution authority.
- The repository avoids advertising a stable ABI whose memory or error contract has not been reviewed.
- Native mobile bindings remain planned, with explicit promotion evidence instead of placeholder exports that appear production-ready.
ADR 0012: Prioritize Affinity interchange and Canva host adoption
Document status:
accepted. Canonical source.
Decision delegated to research on 2026-08-31. Evidence:
nuif:research:affinity-interchange-and-adoption,
nuif:research:canva-apps-and-connect-adoption,
nuif:research:svg and
nuif:research:figma-plugin-and-rest-api-as-automation-surface.
This ADR amends the active vendor priorities in ADR 0008. It does not erase the historical Adobe UXP research or change ADR 0008’s common host-report contract.
Context
The initial vendor plan selected Figma plus host-specific Adobe UXP packages. The project needs an accessible desktop product for human interchange trials and a supported programmable product with a realistic public distribution path. The all-new Affinity is available at no cost and spans vector, photo and layout work, but the reviewed official surface does not expose a stable public document API or native file schema. Canva exposes both an in-editor Apps SDK and off-platform Connect APIs, with explicit capability and review constraints.
Treating these products as equivalent would either overclaim Affinity automation or reduce Canva to lossy file conversion. They require different profiles and evidence.
Decision
- Affinity replaces Adobe as the active desktop vendor-adoption priority. Its
first profile is a user-mediated SVG interchange trial over the already
executable
nuif-svg-0subset, not a native plug-in or.af*parser. - Native Affinity files remain opaque provenance. NUIF will not reverse- engineer them or use pointer/keyboard UI automation as conformance evidence. A future public scripting/document API requires a new profile.
- Canva is the primary programmable vendor-adoption path. Its first profile uses stable Apps SDK Design Editing APIs on one fixed-dimension current page and only the documented supported element subset.
- A Canva import validates a complete bounded plan and applies it with one
confirmed
sync. Locked, unsupported, conflicting or expired sessions fail before mutation and every loss is represented inHostAdapterReport. nuif-wasmmay be bundled in the Canva iframe under the documented CSP, but only the Apps SDK owns host objects and mutation. Remote executable code, workers and preview APIs are absent from the public profile.- Canva Connect APIs are a secondary OAuth workflow. Because their current format lists omit NUIF, SVG/PDF use is explicitly lossy and no native NUIF Connect round trip is claimed.
- The Affinity kit and Canva app each have independent profile and package versions. CI may build evidence and review artifacts. Marketplace submission, developer verification, approval and release remain explicit authenticated human/organization operations.
- Adobe UXP research remains available as historical prior art and a possible contributor-maintained future adapter, but it is removed from the active target inventory and roadmap queue.
Consequences
- Contributors can run the desktop interchange experiment without purchasing a separate design application.
- The project distinguishes evidence from a file-interchange host and an API host instead of implying that one adapter architecture fits both.
- Affinity coverage initially inherits the narrow SVG exclusions and cannot claim native identity, undo or round-trip fidelity.
- Canva offers a realistic source-bundle and marketplace adoption route, but API stability, review, identity disclosure, OAuth and live-host evidence are external gates.
- Historical research remains auditable without steering implementation toward a vendor path that is no longer prioritized.
Conformance
The profile-0 baseline is executable through the nuif-conformance package and xtask gates. It covers structural validation, canonicalization, parser/serializer round trips, unknown-extension preservation, operations/replay/inversion, responsive stack/flex layout, bounded explicit fixed/fr Grid tracks and placement, pinned shaping and hard-line text, deterministic solid rectangle/ellipse and bounded RGBA8 image CPU rendering, measured codec/model/resource limits, pinned browser/Taffy differential layout, editor-driver parity, seven bounded retentive adapter profiles, one pure Figma snapshot/mutation-plan mapping profile and machine reports. Broader CSS Grid, paths/instances, broader image and paint profiles, live vendor adapters and perceptual tiers remain outside the implemented profile.
Install the locked Chrome for Testing build once with cargo xtask browser-install, then run cargo xtask gate-c. The report at target/layout-differential-report.json contains the raw NUIF, Taffy and browser boxes, engine versions, source revision, fixture-local calibration and every classified divergence. A schema-loss classification may describe input outside a declared profile, but cannot excuse a mismatch inside the bounded Grid profile; unclassified or evaluator differences fail the command.
Run cargo xtask gate-d for both text and paint. It writes target/text-pinning-report.json and target/render-profile-report.json; committed hashes, missing font failures, color validation and property-attributed unsupported/preserved fidelity are blocking checks.
Automated and conventional QA run the same headless tests. GUI automation is supplementary; semantic API operations are the primary test interface. HARNESS.md specifies the workspace layout, fixture format, determinism controls, oracles, trial loop, reducer and report schema.
Run cargo xtask editor-hostile-inputs for the release-mode semantic editor
boundary. It rejects excessive snapshot allocations before rendering, checks
finite parsing across every accessibility numeric family, and proves failed
transactions and history boundaries preserve the document and replay log.
Run cargo xtask performance for the portable release-mode smoke profile. It measures validation, both codecs, protocol apply, layout, scene lowering, CPU rasterization, end-to-end snapshots, bounded PNG and font inspection, and real embedded-resource package paths; records median/p95 latency and warmed allocation counts; enforces deliberately broad catastrophic-regression budgets; writes target/performance-profile-report.json; and compiles the statistical suite. Run cargo bench -p nuif-conformance --bench profile_zero -- --noplot and cargo bench -p nuif-conformance --bench system_surfaces -- --noplot on controlled hardware for scaling and subsystem comparisons. Shared CI timing is evidence, not a fine-grained cross-machine baseline. See PERFORMANCE.md for the workload contract, comparison workflow and interpretation rules.
Run cargo xtask gate-svg for the bounded nuif-svg-0 adapter. It checks
exact model round trips, seven byte-local semantic edits, preservation of the
complete unchanged-byte complement, typed hostile cases and a public-CLI
export/sync/import bridge. The machine report and synchronized sources are
written under target/svg-sync-*; the declared subset and its exclusions are
specified in adapters/svg/PROFILE.md.
Run cargo xtask gate-dtcg for the bounded nuif-dtcg-scalar-0 adapter. It
checks exact scalar-token round trips, integer/real discrimination, eight
byte-local edits, root and token extension preservation, duplicate/depth/count
and source limits, and a public-CLI bridge. Reports and retained sources are
written under target/dtcg-sync-*; groups, aliases and composite types remain
outside the declared profile.
Run cargo xtask gate-penpot for the bounded nuif-penpot-v3-0 package
adapter. It imports a fixture from the official JavaScript library, proves
deterministic export and exact no-op archive retention, applies eight mapped
JSON edits while preserving opaque members, exercises hostile ZIP boundaries
and completes an export/sync/import bridge through the public CLI. Reports and
packages are written under target/penpot-sync-*; components, libraries,
interactions, media, paths, layout and compact pages remain outside the profile.
Run cargo xtask gate-react for the bounded nuif-react-jsx-0 source profile.
It extracts one directly returned marked intrinsic subtree without executing
JavaScript, applies 11 byte-local scalar edits, preserves unrelated module
source, rejects eleven dynamic/hostile cases and exercises the public CLI bridge.
Reports and synchronized JSX are written under target/react-sync-*;
components, hooks, spreads, handlers, runtime expressions, TSX and browser
runtime equivalence remain outside the profile.
Run cargo xtask gate-svelte for the bounded nuif-svelte-static-0 source
profile. It applies 11 byte-local edits, preserves the complete unchanged-byte
complement, rejects 13 executable or hostile inputs, completes the public CLI
bridge, and then parses and compiles both synchronized outputs with exact
official svelte/compiler 5.57.0. Reports and components are written under
target/svelte-*; scripts, blocks, directives, components, component CSS and
runtime rendering equivalence remain outside the profile.
Run cargo xtask gate-figma for the credential-free
nuif-figma-plugin-snapshot-0 mapping. It repeats normalized snapshot bytes,
round-trips the declared subset through the CLI, repairs portable identity and
exercises unsupported-property and hostile-input paths. It does not run inside
Figma or certify host mutation behavior.
Run cargo xtask gate-wasm for nuif-wasm-api-0. It compiles the same core
for wasm32-unknown-unknown, generates Node and direct-browser JavaScript plus
TypeScript surfaces, initializes the web target in pinned headless Chrome,
exercises load/validate/hash/text/CBOR/patch/undo/redo in Node and requires the
edited canonical bytes to equal the native CLI output.
The report and direct-browser developer package are written to
target/wasm-conformance-report.json and target/nuif-wasm-web. This gate does
not claim browser-layout, host plug-in or WASI CLI conformance.
Run cargo xtask adapter-audit to validate the complete advertised adapter
inventory independently of executable profile tests. It requires research and
explicit boundaries for twelve targets, checks crate/profile/gate references for
the ten integrated profiles and prevents researched or externally blocked
targets from claiming executable directions.
Run cargo xtask diagnostic-audit to require every model, layout and trial
diagnostic code to appear exactly once in the public registry with a stable
severity, category, producer and meaning.
v0 hard experiment — responsive card
This fixture is the first architecture falsification test.
Required authored features
- component
Cardwith nestedButtoncomponent; - enum variant and boolean state;
- color, spacing and radius token bindings in the profile-0 token model;
- responsive layout changing from one-column to split layout;
- intrinsic text and a vector icon;
- hover/pressed state metadata;
- one opaque
vendor.probeextension unknown to an intermediate implementation.
Round-trip path
NUIF reference editor → HTML/CSS adapter → source edit synchronization → NUIF → neutral intermediate editor → NUIF.
Figma/Penpot adapters may be inserted as additional targets, but no vendor tool is required for the core proof.
Success criteria
- 100% stable semantic entity/component IDs on NUIF-native cycles;
- 100% token-reference preservation where target can represent tokens;
- opaque extension bytes survive unchanged;
- resolved boxes at 360/768/1440 px remain within declared tolerance after representable round trips;
- no silent fidelity downgrade;
- a padding/token/text edit patches only the corresponding source region plus unavoidable formatter changes;
- canonical encoding is byte-stable after decode/encode;
- replaying the same operation log from the same base yields the same canonical hash.
Automated baseline
Generate the deterministic package fixture with nuif fixture v0-responsive-card <output.nuif>. Run editor-trial.jsonl through nuif-editor --headless --script ... --document <output.nuif> to exercise entity-bound selection, semantic edits, undo/redo, complete mutation-log replay, package-preserving save and a deterministic snapshot. Snapshot directories contain the real input.nuif package and a generated input.nuif.json inspection projection. nuif trial <seed> <iterations> [snapshot-interval] drives replay, inversion and text/CBOR fixpoints on every patch, with responsive-layout/CPU-rerender checks at the requested interval. cargo xtask gate-b runs 10,000 patches with a raster interval of 100.
After cargo xtask browser-install, cargo xtask gate-c lowers this fixture independently to Taffy and DOM/CSS at 360 × 640, 768 × 768 and 1,440 × 900. The strict 2026-08-29 report records exact agreement for every box across all three engines. cargo xtask gate-g separately compares canonical text, opaque preservation, layout, decoded RGBA and fidelity with the standard-library-only Python implementation.
cargo xtask gate-f-v0 closes the retentive source segment. The model trial retains 181 source correspondences and changes only eight spans for token, padding, text and responsive edits while preserving every other byte and the unknown payload. Its second path edits name and width through the headless semantic editor, synchronizes through the public CLI and requires the imported canonical NUIF bytes to equal the editor output. The profile preserves path and instance identity but does not claim browser path rendering or instance materialization; those limitations remain explicit fidelity rather than weakening the experiment’s model round trip.
Penpot foreign fixture
fixture.penpot is generated by the official @penpot/library 1.1.0 package
from generate.mjs. It supplies foreign-producer evidence for the bounded
nuif-penpot-v3-0 package reader. The fixture contains one file, one page, one
board, one rectangle, one circle and one literal text shape. The org-nuif
plug-in data on the text shape carries the pinned font identity that Penpot’s
literal package member does not otherwise express.
Regeneration requires Node.js and network access for npm ci:
cd conformance/foreign/penpot
npm ci --ignore-scripts
npm run generate
The generator fixes JavaScript time before loading the ZIP writer and supplies all mapped UUIDs. The gate compares decoded member semantics and exact retained payloads; it does not treat the ZIP envelope timestamp or compression stream as a semantic format guarantee.
Primary locator: Penpot repository library/README.md,
library/src/lib/builder.cljs and library/src/lib/export.cljs, retrieved
2026-08-30: https://github.com/penpot/penpot/tree/develop/library.
Profile-0 render fixtures
profile-zero-v1.json defines the exact solid-paint CPU subset by value: encoded-sRGB color, scaled rectangle edge inclusion, Zeno 0.3.3 grayscale ellipse masks and integer source-over composition on an opaque-white RGBA8 target.
cargo xtask gate-d-render repeats every scene, raw-RGBA raster and PNG artifact; compares committed scene/pixel SHA-256 baselines; rejects out-of-range colors; and verifies property-attributed fidelity for path, image, instance, unknown-kind and document/entity extension data. PNG hashes are reported but are not blocking because a lossless compressor can change bytes without changing pixels. The recorded rectangle and ellipse scene/pixel hashes reproduce on macOS/aarch64, Linux/aarch64 and Linux/x86_64. This matrix does not imply equality on untested platforms.
The fixture does not claim support for gradients, strokes, arbitrary paths, images, masks, effects or component-instance materialization. Those inputs are outside render profile 0 and remain explicit in fidelity reports.
Profile-0 text shaping fixtures
harfbuzz-14.4.0-ahem.json is an independent shaping and outline oracle captured with HarfBuzz 14.4.0 and the exact Ahem 1.50 bytes embedded by font-test-data 0.7.0. The font SHA-256 is f0a92cd0cc45735591c9b5b1fa8aecd5194e8dc518895ca22af94a46c23550dc.
The fixture compares glyph identifiers, Unicode-scalar cluster indices, offsets and advances with glyph names disabled. It also compares unhinted Skrifa 0.46.2 outlines in signed 26.6 font units with five normalized hb-vector paths; normalization removes hb-vector’s redundant explicit line to the contour start before close. cargo xtask gate-d-text repeats both stages, exercises both writing directions, checks typed font failures and writes target/text-pinning-report.json.
The CPU rasterizer uses the same unhinted outlines, pinned Zeno 0.3.3 8-bit grayscale nonzero-fill masks, a fixed 800-font-unit first baseline and alpha composition over encoded sRGB channels. Three scene/raw-RGBA context hashes reproduce on macOS/aarch64, Linux/aarch64 and Linux/x86_64. PNG hashes diagnose deterministic artifact encoding but do not gate pixel conformance. Profile 0 shapes CR/LF/CRLF/NEL/LS/PS hard lines independently, positions baselines by the authored line height, aligns to the inline-start edge and clips without automatic soft wrapping. The report classifies this deliberately bounded text profile as lossless; full UAX #14 soft wrapping is not claimed.
Benchmark suite
NUIF separates portable release budgets from controlled-hardware statistical
measurements. cargo xtask performance always runs the portable release
profile, then executes every Criterion path once with Criterion’s test mode.
That smoke execution detects stale fixtures, panics and dependency drift without
pretending that a shared CI runner provides stable throughput measurements.
Successful execution writes target/criterion-smoke-report.json; portable
latency and allocation budgets remain in target/performance-profile-report.json.
For controlled measurements, run:
cargo bench --locked -p nuif-conformance --bench profile_zero
cargo bench --locked -p nuif-conformance --bench system_surfaces
Keep the machine idle, record its CPU, operating system, power mode, toolchain and source revision, and compare saved Criterion baselines only on equivalent hardware. Do not turn one noisy percentage into a merge gate. A proposed optimization must preserve the conformance gates first and then improve a predeclared workload over repeated samples.
Coverage
profile_zero measures validation, canonical text/CBOR, patch application,
local transactions, undo/redo, layout, scene construction, CPU rasterization
and complete SDK snapshots over bounded fixture scales.
system_surfaces measures direct SDK text/CBOR/package calls, structural and
authorized package-capability paths, entity queries, both collaboration
materializers, package/resource profiles and every integrated adapter profile:
- HTML/CSS profile 0 and full-v0 import/export/synchronization;
- SVG, DTCG, React and Svelte import/export/synchronization;
- Penpot native/foreign package import, export and no-op/edited synchronization;
- Figma snapshot import and mutation-plan generation;
- web accessibility and finite behavior projection.
Figma and browser profiles measure pure mapping only; live host latency belongs to a separately versioned host trial. WASM, MCP and CLI process startup belong to their cross-surface package gates, because mixing process launch with in-process Criterion samples would obscure both costs. Affinity, Canva, SwiftUI, Compose and Flutter have no integrated executable profile and therefore no benchmark claim. The Affinity draft composes the already measured SVG adapter with a separately timed human/live-host trial; the Canva draft must gain a pure mapping gate before it enters Criterion.
Optimization rule
Benchmark setup constructs and validates fixtures before timing. Measured code
must consume inputs through black_box, and mutation/history measurements use
batched clones so state does not leak between samples. A faster result is not
accepted if it changes canonical bytes, diagnostics, fidelity, resource policy
or operation atomicity.
Test-harness architecture
Status: profile-0 baseline, deterministic nuif-package-0, narrow cross-decoder nuif-png-rgba8-0, Gate C browser/Taffy, Gate D text/render, Gate E complete editor authoring, bounded and full-v0 Gate F HTML/CSS synchronization, SVG/DTCG/Penpot/React/Svelte retentive adapter gates, the pure Figma snapshot mapping gate, the three-engine bounded web-accessibility projection, the Rust/Node bounded behavior state-machine differential, nuif-wasm-api-0, Gate G independent v0 reproduction, Gate H property-register and existing-tree convergence, and a bounded five-target sanitizer fuzz suite are implemented. Bounded browser/screenshot capture and reconstruction contracts, typed per-example/corpus reconstruction evaluation, a group-isolated corpus-manifest auditor, canonical provider manifests, pinned LDR-FLIP diagnostics, the typed confidence-calibration evaluator and the pinned local live-Chromium segment have executable evidence; their portable cross-provider accuracy corpus is not yet a release gate. Empirical perceptual thresholds and distributional reconstruction comparison, concurrent entity creation and broader foreign-runtime trials remain planned. This document specifies how round-trip trials run unattended, fail reproducibly, minimize themselves and report in machine-readable form. Evidence is cited by research record identifier.
Goals
- Every conformance suite in
conformance/PLAN.mdruns fromcargo test --workspaceand fromnuifCLI commands without a display or GPU. - A failing seeded trial is reproducible from its seed, iteration, sampled viewport, snapshot decision, source revision and minimized semantic operation sequence. When a report path is supplied, the CLI also reduces the base document and atomically emits a sibling regression-fixture directory.
- Oracles are explicit: reference implementation, alternative implementation, self-consistency (metamorphic relations) and declared assertions. No oracle is a human.
- The editor is a client of the same engine and harness; nothing is testable only through the GUI.
Workspace layout
Cargo.toml workspace; [workspace.lints]; resolver = "3"; Cargo.lock committed
rust-toolchain.toml pinned toolchain (see ADR 0006 for the MSRV decision)
.cargo/config.toml [alias] xtask = "run --package xtask --"
crates/
nuif-core canonical model
nuif-protocol operations, transactions, patches, inverses
nuif-layout evaluation context and profile-0 reference evaluator
nuif-render render scene, backends (CPU reference; Vello interactive)
nuif-codec nuif-text-0, nuif-cbor-0, canonicalizer, migrations
nuif-package deterministic bounded .nuif ZIP package and resource policy
nuif-media bounded declared media decoders; narrow PNG RGBA8 profile
nuif-font bounded static OpenType inspection and package policy
nuif-reconstruct typed observations/proposals and finite correction loop
nuif-capture browser-source and strict screenshot capture baselines
nuif-query semantic queries
nuif-api Engine trait, report types, session driver
nuif-wasm byte-oriented browser/Node binding over nuif-api
nuif-cli command surface; JSON output; stable exit codes
nuif-testing seeded trials, hostile-input measurement, v0 fixture, direct Taffy/Chrome oracles, reducer and reports
apps/
editor headless driver plus tested Masonry GUI shell; package-preserving I/O
conformance/
Cargo.toml executable profile-0 conformance package
src/lib.rs v0 responsive, extension, seeded-trial and parity assertions
fixtures/<suite>/<id>/ input.nuif, context.toml, expected.*, meta.toml
fonts/ pinned fonts referenced by fixtures; no system fonts
generated/ planned persisted browser cases; runtime Gate C cases are seed-derived
fuzz/ pinned cargo-fuzz package; five bounded production-core targets
xtask/ implemented research/verify/Gate B/Gate C/hostile-input/editor loop
tools/
research/ record validator
git/ commit lint
Rationale: harness = false suites enumerate fixture directories at run time and remain compatible with cargo nextest (libtest-mimic-and-data-driven-fixtures); shared test support lives in a normal crate so the editor, the CLI and the suites use one generator and one reducer; xtask replaces shell scripts so generation is reproducible under --locked (cargo-workspace-xtask-and-ci-layout).
Fixture format
One directory per case. Files:
| File | Content |
|---|---|
input.nuif | deterministic nuif-package-0 archive; resources remain content-addressed and policy-checked |
input.nuif.json | generated canonical nuif-text-0 projection for transparent inspection or an independent implementation that intentionally tests only the document profile |
input.cbor | bare nuif-cbor-0 input for codec-only binary cases; never labelled .nuif |
context.toml | evaluation context: viewport, scale factor, locale, writing direction, theme, font set (by hash), capability profile, determinism tier and tolerances |
expected.canonical.nuif | deterministic package form after decode and re-encode (package suite) |
expected.canonical.nuif.json | canonical bare-document form after decode and re-encode (document codec suite) |
expected.layout.json | resolved boxes and diagnostics keyed by entity identifier |
expected.scene.json | render scene serialization (render suite) |
expected.png | reference rasterization from the CPU path (render suite only) |
expected.report.json | fidelity and validation report (adapter, provenance, security suites) |
ops.nuif-log | operation log for operations, merge and replay cases |
meta.toml | unique title, issue reference, tags, `disabled = true |
Persisted expectation regeneration remains a planned extension and will use one variable, NUIF_UPDATE_EXPECT=1: a missing expectation fails the case and writes expected.*.new, and generated suites are replaced wholesale rather than hand-edited (taffy-and-yoga-browser-generated-tests). The implemented Gate C runner instead derives cases from a recorded seed, measures all three engines in one run and stores raw observations in its report; it has no stale golden file to update. Per-asset metadata for future persisted suites follows the sample-asset corpus model (gltf-validator-and-sample-assets).
Determinism controls
- Seed: every implemented generator draws from one xorshift PRNG seeded per trial;
nuif trial <seed> <iterations> [snapshot-interval] [report-path]records the seed and failing iteration and optionally persists the JSON report (deterministic-simulation-testing). - Time and randomness: profile 0 has no time-dependent engine semantics; the editor uses monotonically assigned transaction identifiers, and generated values come only from the trial PRNG. Virtual time is required before behavior/animation work begins (
masonry-xilem-and-linebender-test-harness). - Floating point: canonical text follows RFC 0005’s stated shortest-digit layout and canonical CBOR follows RFCs 0005/0008. Gate C records each fixture’s observed Taffy/browser maximum and rounds it upward to 0.01 px, capped by the 0.1 px foreign-engine safety bound. Exact agreement retains zero tolerance; no aggregate bound silently replaces the fixture values.
- Fonts: the fixture uses the content-addressed Ahem 1.50 bytes (
f0a92c…550dc), HarfRust 0.13.3, Unicode 17.0.0 and Unicode-scalar cluster indices. Eight LTR/RTL goldens were independently captured with HarfBuzz 14.4.0. Five unhinted Skrifa 0.46.2 outlines match normalizedhb-vectorpaths in signed 26.6 font units. Pinned Zeno 0.3.3 8-bit grayscale nonzero-fill coverage produces the same three scene and raw-RGBA hashes on macOS/aarch64, Linux/aarch64 and Linux/x86_64. CR/LF/CRLF/NEL/LS/PS hard-line layout is exact; profile 0 does not request automatic soft wrapping (text-rendering-reproducibility). - Threads: the current suites have no shared mutable global state and pass under libtest’s normal parallel execution.
- Environment: CI uses
--lockedand pinned Rust 1.98.0; the separate MSRV job checks 1.96.0.
Oracles by suite
| Suite | Oracle class | Comparison |
|---|---|---|
| model | assertions | identity uniqueness, containment acyclicity, relation target existence |
| canonicalization | self-consistency | E(D(E(d))) = E(d); hash stability; idempotent canonicalize |
| extensions | self-consistency through an ignorant implementation | byte identity of unknown payloads after decode, edit, encode (opentimelineio, godot-tscn-scene-format) |
| layout | implemented metamorphic relations plus pinned Taffy 0.14.0 and Chrome for Testing 152.0.7977.64 | responsive v0 at 360/768/1440 px and 12 seeded stack/flex/grid cases; raw three-engine boxes, measured fixture bounds and typed divergences (differential-testing, css-flexbox-grid-algorithm-specs) |
| text shaping/outlines | pinned HarfRust/Skrifa plus independently captured HarfBuzz 14.4.0 goldens | exact glyph IDs, Unicode-scalar clusters, font-unit advances and direction over eight Ahem cases; five normalized hb-vector outline paths; repeated scene runs and typed missing-font failures (text-rendering-reproducibility) |
| render | reference rasterization | profile-0 exact for scaled rectangle inclusion, Zeno ellipse coverage, pinned text masks and integer-composited encoded-sRGB solid color; unsupported path/image/instance/extension semantics remain property-attributed; proposed tier 2 bounded and tier 3 perceptual thresholds remain non-normative (vello-testing-and-cpu-reference, flip-perceptual-difference-metric, webrender-reftests) |
| operations | self-consistency and reference model | replay to identical hash; apply(t⁻¹, apply(t, d)) ≡ d; commutation of independent operations; undo-copy-redo invariance (command-pattern-undo-and-event-sourcing) |
| merge | assertions | three-way merges produce typed conflicts, never arbitrary winners; move and order cases from crdt-tree-move-operation |
| provenance | assertions | correspondence records survive representable round trips; minimal-patch locality measured as changed source spans |
| adapter | round trip and fidelity report | canon(Y(X(d))) = canon(d) on the representable subset; every deviation explained by a report entry |
| package | independent ZIP writer plus fixpoint and corruption trials | fixed member order/metadata, exact package bytes, manifest/document/resource hashes, no traversal/symlink/encryption/compression/ZIP64 ambiguity, explicit linked-resource resolver |
| image resource | independent decoder plus package/render metamorphic checks | exact RGBA agreement across all PNG row filters, encoded-byte preservation, repeatable fit/crop/sampling/opacity CPU raster, fail-closed metadata and one-over inputs |
| font resource | independent parser plus package/policy metamorphic checks | exact static TrueType metrics/coverage agreement, byte preservation, explicit embedding review and fail-closed sfnt/policy/one-over inputs |
| capture/reconstruction contracts | metamorphic and policy assertions over fixed provider inputs | repeated normalized observations/packages, exact retained source resources, secret-query absence, evidence/omission truthfulness, observation codec fixpoint, typed proposal application, flat-copy rejection and finite-loop stop states; no live-capture or accuracy claim |
| security | measured boundary and one-over cases | bare readers stop at 16 MiB plus one byte; packages stop at 80 MiB with 32 MiB per resource, 64 MiB total embedded resources and 8,192 resources; syntax depth 64 and the RFC 0009 semantic limits are enforced; release bare-codec cases fail above 2 s, 64 MiB allocated or 16 MiB retained; CPU targets remain capped at 16,777,216 pixels (resource-bounded-serde-and-ciborium) |
Trial loop
The target loop is shared conceptually by CLI, CI and editor automation. Profile 0 currently implements generation, replay, inverse, canonical encodings, responsive layout, CPU rerender, operation ddmin, document-aware subtree/scalar reduction, choice-stream shrinking, atomic minimized-fixture writing, adapter-specific round trips, fuzz choice streams and the Gate C foreign layout matrix.
trial(seed, profile):
d0 := load(fixture) | generate(seed, profile) # swarm-selected feature subset
log := generate_ops(seed, d0, reference_model) # preconditions checked against the model
d1 := apply(log, d0) # engine
assert replay(log, d0) == d1 # determinism
assert apply(inverse(log), d1) ≡ d0 # inversion
for ctx in context_matrix: L[ctx] := layout(d1, ctx) # resolved snapshots
for R in metamorphic_relations: assert R(d1, L) # nine relation classes
for A in adapters: assert roundtrip(A, d1) with report # fidelity accounting
bytes := encode(d1); assert canonical(bytes) # fixpoint
scene := lower(d1, L[ctx0]); img := raster_cpu(scene) # tier 1 reference
compare(img, expected | reftest_pair) # oracle by tier
on failure: reduce(seed, d0, log) -> fixture; write report
The implemented reducer first runs complement-based ddmin over semantic operations. Its document pass removes entity subtrees in progressively finer chunks, prunes containment and relation edges, and relies on full model validation to reject dangling component, token or asset references before the interestingness predicate runs. It then reduces relations, tokens, assets, extension namespaces and known scalar fields; unknown-kind opaque bytes remain fixed. The choice-stream pass deletes contiguous regions, exhaustively lowers bytes and redistributes adjacent numeric choices toward shortlex order. Accepted candidates are content-hash memoized. A fixture writer atomically creates input.nuif.json, operations.json, reduction.json and fixture.json and refuses an existing destination. cargo xtask reduction-profile exercises all of these paths and archives the report and emitted fixture (delta-debugging-and-test-case-reduction, property-based-testing-state-machines).
Report schema
One JSON document per run, modelled on the glTF Validator report (gltf-validator-and-sample-assets) and required by apps/editor/QA.md item 10:
{
"schema_version": 1,
"engine": {"version": "0.0.1", "toolchain": "rustc 1.98.0 (…) ", "source_revision": "…", "dirty": false},
"profile": {"capabilities": ["model", "operations", "layout-profile-0", "render-cpu-profile-0"], "encodings": ["nuif-text-0", "nuif-cbor-0"]},
"trial": {"seed": 42, "iterations": 10000, "operations_per_iteration": 16, "snapshot_interval": 100},
"contexts": [{"viewport": [360, 640], "canonical_hash": "nuif-cbor-0:sha256:…", "layout_boxes": 8, "render_commands": 4}],
"issues": {"errors": 0, "warnings": 0, "information": 4, "hints": 0, "messages": ["…"]},
"fidelity": [{"context": [360, 640], "entity": "…", "status": {"class": "approximated", "reason": "…"}}],
"artifacts": [],
"reproduction": null
}
Diagnostic codes are stable strings emitted in machine reports; severities
serialize as error, warning, information or hint, and command exit
status depends only on errors. The canonical diagnostic code
registry records every model, layout and trial code,
default severity, category, producer and stable meaning. cargo xtask diagnostic-audit blocks undocumented, duplicated, stale or reordered entries.
The hostile-input experiment writes a separate target/hostile-input-report.json because allocator and elapsed-time measurements are process-level rather than document fidelity entries. It records every input size, expected/observed error class, allocation counters, retained bytes, elapsed microseconds, limits, warmup, allocator method, toolchain and platform. cargo xtask hostile-inputs regenerates it and CI uploads it as hostile-input-report.
The editor hostile-interaction experiment writes target/editor-hostile-input-report.json. Its release runner rejects zero, one-over-edge and maximal snapshot requests before raster allocation; accepts the exact one-dimensional edge boundary; rejects non-finite size, position and spacing values plus malformed paint without mutation; and checks missing selection/node errors, atomic multi-operation failure, empty history, redo invalidation and exact patch-log replay. Unit tests additionally exercise bounded script reads, per-line limits, command limits and malformed-line attribution. cargo xtask editor-hostile-inputs is blocking and CI archives the report.
The reduction experiment writes target/reduction-profile-report.json plus
target/reduction-profile-fixture/. Its fixed interestingness predicate reduces
the complete responsive card to the valid three-entity ancestor path, proves
component references cannot dangle, reduces a byte choice stream, records every
accepted transformation and confirms that the atomic writer refuses overwrite.
On a real nuif trial failure with a report path, the same machinery emits
<report-path>.reproduction/ using the recorded failure code, viewport and
snapshot decision.
The standalone fuzz/ workspace pins nightly 2026-08-28, cargo-fuzz 0.13.2
and libfuzzer-sys 0.4.13 without adding sanitizer dependencies to release
packages. cargo xtask fuzz-smoke regenerates target-specific valid seeds from
production fixtures, then runs raw codec, package/archive, resource-decoder,
static-source-adapter and typed-operation targets with explicit input, timeout,
allocation and RSS limits. The operation target maps bytes to valid production
operations rather than maintaining a second document grammar; parser targets
retain malformed bytes. CI runs 512 inputs per target under AddressSanitizer
and archives target/fuzz-smoke-report.json. Crash bytes remain local until
reduced and promoted to a named regression fixture.
The 10,000-patch Gate B run writes target/gate-b-report.json. cargo xtask all also installs or reuses the locked browser oracle and writes target/verification-manifest.json on success or at the first failing step. The manifest records revision, dirty state, toolchain, completed steps and the presence of every expected evidence artifact, so CI and autonomous research controllers can make a decision without parsing console output. cargo xtask manifest performs the narrower presence audit on an already-generated evidence set; its manifest is labelled artifact-index, does not claim that it executed any trial, and fails after writing the index when an artifact is absent.
The adapter inventory audit writes target/adapter-coverage-report.json before the executable gates. cargo xtask adapter-audit requires all twelve advertised targets to have a reviewed or verified research-record path, next bounded profile and explicit exclusion boundary. Integrated entries must resolve crate, profile and routed gate paths; researched and externally constrained entries must not claim executable directions. This is coverage and claim-boundary evidence, not foreign-runtime conformance.
The layout-differential experiment writes target/layout-differential-report.json. It records the source revision, dirty state, toolchain, exact Taffy and browser pins, launch flags, seed, case source, viewport, raw box maps, observed foreign delta, fixture-local assertion value and every typed divergence. Missing browsers, version drift, evaluator defects, Taffy/browser differences beyond the measured bound and unclassified differences fail cargo xtask gate-c. Schema-loss records remain available for inputs outside a declared profile, but the bounded explicit-Grid cases permit no schema-loss exemption.
The text-pinning experiment writes target/text-pinning-report.json. It records the exact font, shaper, Unicode, outline extractor, rasterizer and independent HarfBuzz oracle pins; expected and observed glyph/outline strings; source/toolchain/platform identity; hard-break/no-soft-wrap semantic trials; repeatability and committed scene/raw-RGBA baselines at three evaluation contexts; PNG artifact hashes; and negative missing/malformed-font cases. cargo xtask gate-d-text fails on any pin, golden, semantic, scene/pixel baseline, repeatability or negative-case mismatch. A PNG-reference mismatch is diagnostic because the lossless compressor is outside the pixel boundary. The bounded text profile is lossless and its scene/pixel hashes agree on macOS/aarch64, Linux/aarch64 and Linux/x86_64.
The independent-reproduction experiment writes target/gate-g-report.json plus canonical text, layout and PNG artifacts under target/gate-g-independent. cargo xtask gate-g generates a real package and reference artifacts at three viewports, exports one generated input.nuif.json projection, runs the standard-library-only Python document-profile implementation’s unit suite, then compares independently computed canonical document bytes, opaque preservation, boxes, decoded RGBA and fidelity. The Python implementation deliberately does not claim package parsing and does not import, link or invoke any Rust workspace package; only the outer differential harness invokes both implementations.
The render-profile experiment writes target/render-profile-report.json. It fixes every supported paint input by value, repeats rectangle and ellipse scenes/raw-RGBA rasters/PNG artifacts, rejects out-of-range sRGB channels, and requires entity/property pointers for unsupported path, image and instance kinds plus preserved document/entity extensions. cargo xtask gate-d-render fails on any scene/pixel baseline, repeatability, validation or fidelity-attribution mismatch; cargo xtask gate-d runs both Gate D reports.
The narrow image-resource experiment writes
target/image-resources-report.json. cargo xtask gate-i-image compares exact
RGBA output from png 0.18.1 and independently implemented zune-png 0.5.2
across all row filters with absent/valid sRGB, preserves exact encoded bytes
through package fixpoint and an unrelated edit, repeats resource-aware scene
and CPU raster output, and rejects unsupported, corrupt and one-over cases. The
report explicitly excludes broad PNG colour/metadata support, non-identity
transforms, GPU/cross-platform image reproduction and non-PNG formats.
The narrow font-resource experiment writes
target/font-resources-report.json. cargo xtask gate-i-font compares the
profile’s Skrifa interpretation with a committed hb-info 14.4.0 capture of
metrics, family, tables and Unicode coverage for the exact pinned Ahem bytes,
while three more static TrueType fixtures exercise acceptance. NUIF-owned sfnt,
checksum and OS/2 validation remains ahead of Skrifa. The gate proves package
byte fixpoint and resource retention, mutates metadata and embedding evidence,
distinguishes six portability outcomes, and rejects synthetic malformed cases
plus real TTC, CFF, variable, COLR, bitmap, CBDT and sbix inputs. The report
explicitly excludes TTC, CFF/CFF2, variable, color, bitmap, SVG and WOFF/WOFF2
fonts; it does not claim shaping/raster equivalence or that technical flags
grant redistribution rights.
The HTML/CSS retentive experiment writes target/html-sync-report.json and target/html-sync-output.html. It pins Tree-sitter and both grammars, exactly re-imports the declared subset, repeats synchronization, checks the complete unchanged-byte complement of six text/token/padding edits, preserves injected comments/unmapped markup and requires typed stale-span, unsupported-property and one-over-size failures. cargo xtask gate-f is blocking; the bounded profile and its non-claims are specified in adapters/html-css/PROFILE.md.
The full-v0 follow-on writes target/html-sync-v0-report.json, target/html-sync-v0-output.html, target/html-sync-v0-editor-report.json and target/html-sync-v0-editor-output.html. cargo xtask gate-f-v0 checks 181 source correspondences, the unchanged-byte complement of eight model edits, exact opaque preservation and typed negative cases, then drives a semantic editor name/width edit through CLI synchronization and CLI import to byte-identical canonical NUIF. Target visual limits and arbitrary-CSS non-claims are specified in adapters/html-css/V0-PROFILE.md.
The SVG retentive experiment writes target/svg-sync-report.json, a direct synchronized SVG and edited canonical document at target/svg-sync-edited.nuif.json, plus separate public-CLI synchronization report and SVG. cargo xtask gate-svg checks exact import/export, repeatability, the unchanged-byte complement of seven accessibility, paint, geometry and text edits, preserved comments and metadata, and typed unsupported-property, structural, stale-span, derived-geometry, DTD, XML-node and byte-limit cases. The CLI bridge exports a package fixture, synchronizes from the explicit bare-document projection and requires byte-identical canonical document re-import. The mapped SVG 2 subset and arbitrary-SVG non-claims are specified in adapters/svg/PROFILE.md.
The DTCG scalar-token experiment writes target/dtcg-sync-report.json, a direct synchronized token file and edited canonical document at target/dtcg-sync-edited.nuif.json, plus separate public-CLI synchronization report and token file. cargo xtask gate-dtcg checks exact import/export, NUIF Integer/Real discrimination inside DTCG number, repeatability, the unchanged-byte complement of eight name/type/value/metadata edits, and root/token extension retention. Duplicate members, aliases, undeclared standard members, excessive JSON depth, one-over token count, one-over source bytes, unsupported values, structural changes and stale spans are typed failures. The CLI bridge requires byte-identical canonical document re-import. The mapped DTCG 2025.10 subset and token-model limitations are specified in adapters/dtcg/PROFILE.md.
The Penpot v3 package experiment writes target/penpot-sync-report.json, a synchronized Penpot package and edited canonical NUIF document at target/penpot-sync-edited.nuif.json, plus separate public-CLI synchronization report and Penpot package. cargo xtask gate-penpot imports the fixture produced by official @penpot/library 1.1.0, checks deterministic export and byte-exact no-op archive retention, applies eight mapped JSON scalar edits, preserves untouched member payloads plus injected opaque binary/JSON data, and requires exact canonical document re-import. Unsafe paths and one-over package/member limits are typed failures. The library importer additionally rejects excessive count/expansion/ratio/depth/value cases, duplicate names, directories, symlinks, encryption and unsupported compression. The mapped package subset and compact/components/libraries/interactions non-claims are specified in adapters/penpot/PROFILE.md.
The static React JSX experiment writes target/react-sync-report.json, a
synchronized JSX module, an edited canonical document and separate CLI bridge
artifacts. cargo xtask gate-react uses Tree-sitter JavaScript byte ranges but
never evaluates JavaScript. It checks 21 correspondences, 11 mapped edits, the
exact unchanged-byte complement, repeated output, typed stale/structural/profile
failures and eleven excluded or hostile sources, including the one-over mapped
JSX depth case. The intrinsic-only mapping and
runtime non-claims are specified in adapters/react/PROFILE.md.
The static Svelte experiment writes target/svelte-sync-report.json, a
synchronized component, an edited canonical document, separate CLI bridge
artifacts and target/svelte-compiler-oracle-report.json. cargo xtask gate-svelte checks 21 correspondences, 11 mapped edits, exact unchanged-byte
complement preservation, repeated output, typed stale/structural/profile
failures and 13 excluded or hostile sources. It then uses the exact npm
lockfile with lifecycle scripts disabled and requires official
svelte/compiler 5.57.0 to parse in modern-AST mode and compile both direct and
CLI output without warnings. Tree-sitter owns retained byte ranges; the
official compiler remains a separate semantic oracle. Runtime rendering,
component CSS and executable template semantics are explicit non-claims in
adapters/svelte/PROFILE.md.
The Figma pure-mapping experiment writes
target/figma-snapshot-report.json. cargo xtask gate-figma repeats the
normalized snapshot bytes, maps the exact visible/opaque/fixed-size subset in
both directions through the public CLI, records deterministic repair for
portable identity, reports hidden/transparent/effect/variable properties and
rejects duplicate host IDs plus the byte limit plus one. It does not load a
page, create a node, mutate a host document or test undo inside Figma; those
remain live-host requirements in adapters/figma/PROFILE-DRAFT.md.
The WebAssembly cross-surface experiment writes
target/wasm-conformance-report.json and generates Node and direct-browser
packages. cargo xtask gate-wasm pins wasm-bindgen 0.2.127, initializes the
direct-browser target in pinned Chrome, runs the generated Node ABI through
canonical text/CBOR, validation, atomic patch and history paths, and requires
the output bytes to equal the native CLI after the same patch. It also checks
stale, malformed and one-over-byte failure atomicity and an empty authority
declaration. Browser layout, WASI and vendor plug-in behavior remain separate
trials.
The MCP cross-surface experiment writes
target/mcp-conformance-report.json. cargo xtask gate-mcp launches the real
stdio binary, opens the 2026-07-28 stateless lifecycle with server/discover,
and sends complete metadata on every valid request. An independent Python
driver checks the exact four-tool set, JSON input/output schemas, side-effect
annotations, typed errors, connection survival after a rejected request and a
one-over 4 MiB frame. Canonicalization and atomic patch output must be
byte-identical to the native CLI. Twenty-five repeated validation calls record
wire median, p95 and maximum latency with a catastrophic two-second p95 budget;
this smoke distribution is not a controlled throughput benchmark.
The performance gate follows the same distinction. cargo xtask performance
records portable release-mode latency/allocation budgets for catastrophic
regressions and executes both Criterion suites once in test mode. The
controlled-hardware suites cover core scaling, resources, package-capability
negotiation and every declared direction of all ten integrated adapter
profiles. Statistical before/after measurements, machine controls and explicit
exclusions are defined in BENCHMARKS.md; shared-runner timing
noise is not a merge threshold.
The collaboration register experiment writes target/collaboration-report.json. cargo xtask gate-h exhausts all 5,040 deliveries through operation-set and replica-log materializers, checks multiple merge orders and duplicate delivery, requires property-attributed multi-value conflicts and inspects canonical text for leaked replica state. Structural operations still fail before register-profile ingestion.
The separate existing-tree experiment writes target/collaboration-structure-report.json, exhausts 5,040 deliveries of seven move/delete/cycle/stable-anchor changes through sorted-set and incremental rollback/replay materializers, requires one-parent/acyclic checkpoints plus explicit move, deletion, cycle and anchor conflicts, and runs a 4,096-change release scaling guard. tools/automerge-oracle uses pinned @automerge/automerge 3.4.1 to merge the seven immutable operation records in different orders and through save/load, writing target/collaboration-automerge-report.json. This is foreign transport evidence; Automerge does not provide the NUIF tree materializer. Both executable boundaries are specified in crates/nuif-collab/README.md and spec/10-collaboration-profile.md.
The capture/reconstruction contract experiment writes
target/capture-reconstruction-report.json. cargo xtask capture-baselines
uses fixed browser-provider and strict PNG inputs to exercise repeatability,
resource identity, query-secret redaction, evidence classes and omissions,
typed atomic proposal application, flat-copy rejection, observation codec
fixpoints, calibration interpolation/selective review and finite loop stops.
The correction-loop fixture now exercises an explicit successful objective
threshold; the library also reports no_improvement when a fresh candidate is
rejected, while repeated canonical state remains a separate terminal result.
The report carries explicit non-claims for live browser capture, OCR/model
accuracy, a broad or held-out corpus, independent evaluation and training.
The reconstruction-evaluation contract writes
target/reconstruction-evaluation-report.json. cargo xtask reconstruction-evaluation computes and validates the complete typed
nuif-reconstruction-evaluation-0 per-example family over one deterministic
synthetic fixture. It keeps exact rate evidence, nullable unavailable resource
measurements and suite-specific resource claims, and asserts that one local
pixel difference and one missed element remain independently visible. It also
rejects inconsistent derived rates and edit-distance work beyond its bound. A
three-example nuif-reconstruction-evaluation-aggregate-0 fixture proves
input-order independence, pooled rate arithmetic, nearest-rank p50/p95 and
explicit missingness. Aggregation rejects mixed suites, duplicate identities,
incompatible calibration/perceptual configurations and mixed currencies. The
same gate runs pinned nv-flip 0.1.2/nv-flip-sys 0.1.1 LDR-FLIP at 67 PPD
over explicitly opaque sRGB8 input. It requires zero mean for identity and a
nonzero bounded mean for one local error, records the implementation, PPD,
pooling and platform-sensitivity parameters and refuses transparency or shape
ambiguity. FLIP remains diagnostic: the report explicitly disclaims OCR/model
accuracy, a real or held-out corpus, empirical perceptual thresholds,
statistically calibrated uncertainty and independent evaluation.
The separate confidence-calibration contract writes
target/confidence-calibration-report.json. cargo xtask confidence-calibration
evaluates typed decision events for text and geometry over disjoint calibration
and test groups, includes normal and font-shifted holdouts, computes reliability
bins, Brier score, ECE and risk/coverage/AURC, and validates a JSON round trip.
Equal-confidence cases are admitted as a group and calibration mappings are
monotonic, so input order cannot change selection. The report is a synthetic
evaluator smoke test: it does not claim production calibration, model quality,
distributional coverage or a rights-cleared corpus.
The separate corpus-integrity contract writes
target/reconstruction-corpus-audit-report.json. cargo xtask reconstruction-corpus-audit round-trips a bounded typed manifest and exactly
derived audit over adaptation, calibration, validation and test fixtures. The
gate injects exact-artifact and origin/template/component/font/resource/
generator/near-duplicate collisions across splits; forbidden adaptation,
calibration or evaluation use, screenshot-only source claims, absent
near-duplicate assignments, unauditable private captures and derived-count
drift must fail. Artifact disclosure and permitted use remain separate. These
synthetic declarations prove the checker only: they are not licensed real data,
a duplicate detector, a legal interpretation, a representativeness analysis or
an independent evaluator.
The provider identity contract writes
target/reconstruction-provider-manifest-report.json. cargo xtask reconstruction-provider-manifest canonicalizes a bounded
nuif-reconstruction-provider-manifest-0 fixture, requires an exact identity
fixpoint and proves that changing model bytes changes the provider identity.
Every observation bundle carries the manifests behind its observation and
proposal identities; duplicate, malformed, missing and dangling registry
entries fail before document mutation. The gate also rejects ambiguous
implementation identity, duplicate artifact identifiers, a learned provider
without a model card and a released/learned provider without external SPDX
3.0.1 or CycloneDX 1.7 inventory identity. Its learned fixture uses synthetic
digests, and its browser/screenshot fixtures identify development source
bundles; this is supply-chain contract evidence, not an inventory audit,
released model, inference run or accuracy result.
The separate live capture experiment writes
target/live-browser-capture-report.json. cargo xtask gate-j-live installs or
reuses exact Chrome for Testing 152.0.7977.64 and accepts isolated 360, 768,
held-out 900 and repeated 360 px captures through bounded loopback CDP. It
allows at most three recorded fresh-profile attempts per viewport and accepts
only the exact resource/font fixture. It requires loader-specific load plus
image/font readiness, a bounded event-quiet point, stable consecutive
screenshots, structured context, exact
HTML/CSS/PNG/font/probe body set, actual downloaded-font and accessibility
evidence, repeat-identical capture/normalization/screenshot bytes, absence of
five exercised transport/storage secret canaries and lower held-out aggregate
geometry error from two viewports than the one-viewport freeform baseline. The
adapter also enforces aggregate event-byte, command, node, font-use, resource,
decode, write-buffer and connected-capture limits; the gate requires the four
accepted captures and any recorded retries to finish within 120 seconds. The report explicitly excludes
cross-browser/OS, opaque-frame, authenticated-site, canvas/video semantic and
reconstruction-accuracy claims.
Editor participation
The editor exposes an in-process session driver (nuif-api) that the harness calls without a window: create/open, apply operation, query accessibility tree, dispatch accessibility action, redraw to a CPU frame and snapshot. The accessibility tree carries entity identifiers (accesskit-semantic-ui-testing), so a test asserts “the selected entity is X” by role and identifier rather than by pixel position. cargo xtask editor-trial authors the complete v0 fixture from an empty document, demands byte identity with the direct generator and replay, and emits target/editor-authoring-report.json plus canonical document/context/layout/scene/CPU-PNG/fidelity artifacts under target/editor-authoring-snapshot. GUI screenshot comparison is limited to shell wiring and uses the same tiers as the render suite with per-OS baselines avoided by CPU rasterization. Gesture tests assert the emitted protocol operations, not canvas pixels.
CI matrix
| Job | Content |
|---|---|
| commit-lint | subject rules |
| research | record validation |
| rust | fmt, check, clippy pedantic, cargo test --workspace --locked, 10,000-patch Gate B trial, hostile-input release measurement, pinned Gate C three-way layout trial, both Gate D text/render trials and all report uploads (all render suites CPU only) |
| reduction-profile | validity-preserving subtree/scalar and choice-stream reduction plus atomic regression-fixture emission (currently part of the rust complete gate) |
| wasm | generated Node/direct-browser binding, Node/native byte differential, typed limit failures and downloadable developer artifact (currently part of the rust complete gate) |
| fuzz-smoke | five cargo-fuzz/libFuzzer targets, regenerated production seeds, AddressSanitizer, explicit resource limits and an archived campaign report |
| layout-differential | cargo xtask browser-install plus cargo xtask gate-c; seed-derived cases run in headless Chrome and fail on pin drift or blocking/unclassified divergence |
| editor-headless | editor session scripts through nuif-api; accessibility-tree assertions; CPU snapshots |
| gpu-optional | interactive backend under tier 3 on a GPU runner; failures are reported, not blocking |
Non-goals
No GUI pointer automation as an oracle; no per-OS pixel baselines; no hand-edited generated fixtures; no test that depends on network access.
Profile-zero performance methodology
NUIF uses two complementary performance gates. The portable smoke profile is a release binary that runs on developer machines and shared CI, records latency and allocation evidence, and rejects only catastrophic regressions. The Criterion suite is the statistical comparison tool for controlled hardware. Results from different machines, operating-system states or Rust toolchains are not treated as comparable baselines.
Workloads
| Group | Scales | Work measured |
|---|---|---|
| Model validation | 8, 128, 1,024, 4,096 entities | Complete profile-zero structural and resource validation |
| Canonical text and deterministic CBOR | 8, 128, 1,024 entities | Encode and decode independently |
| Protocol/session | 8, 128, 1,024, 4,096 entities | Clone a document and apply one rename transaction; local sessions cover cold and revision-cached edits plus undo/redo |
| Layout | 8, 128, 1,024, 4,096 entities | Evaluate a flat mixed shape/text document |
| Scene lowering | 8, 128, 1,024 entities | Lower evaluated entities to deterministic render commands |
| CPU raster and API snapshot | 360x640, 768x640, 1,440x900 | Raster an interactive card fixture; snapshot includes hash, layout, scene and raster |
| Embedded image and font resources | RGBA8 image plus pinned static TrueType font | Inspect and decode the media profiles; encode and decode exact-resource packages; lower a resolved image scene and raster it at 256x256 |
| Semantic query | 128, 1,024, 4,096, 8,192 entities | Stable-ID lookup and kind scan over the authored model |
| Collaboration | 2, 32, 256, 1,024 concurrent register writers | Materialize identical conflict checkpoints through the operation-set and replica-log algorithms |
| Integrated adapters | Both declared HTML/CSS profiles, SVG, DTCG, Penpot, static React JSX, static Svelte and normalized Figma snapshot fixtures | Every declared import/export direction is measured separately; retentive profiles also measure synchronization, Penpot measures official-foreign import and byte-exact no-op synchronization, and Figma measures pure mutation-plan construction and snapshot import |
nuif_testing::performance_fixture is deterministic, valid, bounded by the
profile-zero 8,192-entity resource limit and uses the repository-pinned font.
Every sixteenth child is text in mixed workloads. Fixture construction is
outside timed sections. Protocol clone cost remains deliberately included
because atomic application currently requires an isolated candidate document.
Commands
Run the portable release-mode smoke gate and compile every statistical benchmark:
cargo xtask performance
Run the complete Criterion suites without plotting:
cargo bench -p nuif-conformance --bench profile_zero -- --noplot
cargo bench -p nuif-conformance --bench system_surfaces -- --noplot
On an otherwise idle controlled machine, save a baseline before a change and compare against it afterward:
cargo bench -p nuif-conformance --bench profile_zero -- --save-baseline before --noplot
cargo bench -p nuif-conformance --bench profile_zero -- --baseline before --noplot
cargo bench -p nuif-conformance --bench system_surfaces -- --save-baseline before --noplot
cargo bench -p nuif-conformance --bench system_surfaces -- --baseline before --noplot
Criterion accepts a filter after --, such as -- codec --noplot. Use
--profile-time 15 to collect a longer, non-statistical profiling workload for
an external profiler. Keep the same power mode, foreground load, toolchain and
build inputs for both sides of a comparison.
Local calibration
The first system_surfaces calibration ran on macOS/aarch64, Apple M5 Pro,
rustc 1.98.0, with 20 Criterion samples, one second of warm-up and two seconds
of measurement per case on 2026-08-30. These values establish workload
plausibility and one optimization comparison; they are not portable release
budgets.
- Stable-ID lookup at 8,192 entities measured 18.5–19.2 ns; a complete kind scan measured 19.2–19.5 µs.
- HTML/CSS, SVG and DTCG synchronization measured 226.7–228.0 µs, 189.7–190.5 µs and 55.5–55.8 µs respectively for their declared fixtures.
- Penpot native export measured 108.3–109.2 µs, native import 79.3–80.2 µs, official-library import 98.3–99.1 µs, a two-scalar synchronized rebuild/re-import 93.9–94.6 µs, and byte-exact no-op synchronization 2.85–2.87 µs. These figures use the 7,855-byte native and 5,439-byte foreign fixtures rather than a large production design.
- The matching allocation-instrumented smoke run now covers 50 cases. The nine resource-path cases add PNG structure/decode, static-font inspection, image and font package encode/decode, resolved scene lowering and a 256x256 image raster. On the same machine, font inspection measured a 0.67 ms median and about 1.12 MiB allocated; font package encode/decode measured 1.05/0.89 ms and about 1.41/1.38 MiB. The 96-byte image fixture is deliberately a boundary smoke case rather than a throughput claim; Criterion owns controlled comparisons and larger media corpus work remains explicit.
- Package encode originally serialized the same 1,024-entity document five times while constructing and then validating its manifest. Reusing the one already-validated canonical CBOR buffer reduced measured encode allocation from 56,840,802 to 11,952,738 bytes and invocation allocations from 634,633 to 125,459. Decode now verifies the descriptor against the canonical archive member already accepted by the codec, reducing allocation from 35,525,804 to 13,081,772 bytes and invocation allocations from 384,447 to 129,860. A consecutive smoke run observed encode/decode medians fall from 27.3/18.5 ms to 8.8/9.3 ms, but those latency figures are diagnostic rather than a statistical cross-run claim. Package byte fixpoints, independent ZIP output, canonical hashes and hostile-input checks remained gating postconditions. A follow-on exact-size ZIP preflight reduced encode allocation by a further 509,614 bytes, from 11,952,738 to 11,443,124, without changing the archive; moving decoded blobs into shared ownership instead of cloning them removed exactly the 22,572-byte font and 96-byte image payloads from their respective decode allocation counts.
- Static React JSX export/import/synchronization measured 0.078/0.077/0.235 ms median for the 778-byte declared fixture, allocating about 71/67/244 KiB per invocation and retaining zero. These are parser-boundary calibration values, not React runtime or browser performance claims.
- Static Svelte export/import/synchronization measured 0.053/0.052/0.166 ms median for the 761-byte declared fixture, allocating about 68/64/236 KiB per invocation and retaining zero. Official compiler execution is intentionally outside these production-parser timings and remains a conformance-oracle cost, not an application-runtime benchmark.
- HTML/CSS v0 and the normalized Figma snapshot profile are included in both
performance layers. The smoke report derives the complete integrated-profile
inventory from
adapters/index.jsonand fails if its benchmark inventory drifts, so a newly integrated adapter cannot silently omit performance evidence. Figma timing covers only the deterministic mapper; plug-in-host responsiveness remains a named live-host boundary. - Penpot native export and edited synchronization allocated 291 KiB and 246 KiB per invocation; native and foreign imports allocated 215 KiB and 665 KiB, and the no-op path allocated 35 KiB. Every adapter case retained zero bytes after the measured invocation.
- An initial all-Deflate native export was 4,688 bytes and allocated about 4.04 MiB; edited synchronization allocated about 3.99 MiB. Storing native JSON members below 4 KiB increased this small package by 3,167 bytes while reducing those allocations by about 93% and the two Criterion times by about 49% and 52%. Imported compression methods remain retentive, so foreign-package behaviour did not change. The threshold is a profile workload decision, not a general claim that ZIP storage is preferable to Deflate.
- The 1,024-writer operation-set checkpoint measured 4.08–4.15 ms before
replacing an all-pairs causal-maximality search with per-replica maximum
observed vector contexts. The algorithmically independent replica-log
frontier remained separate. A same-process saved-baseline comparison measured
2.51–2.81 ms afterward and Criterion classified the change as a 36.1–40.3%
reduction in time (
p < 0.05). Exact checkpoint equality, 5,040 delivery permutations and both materializers remain Gate H postconditions.
Interpretation
- Treat the portable budgets as availability limits, not optimization targets. Its JSON artifact is useful for trend inspection and allocation diagnosis, but shared-runner timing noise is expected.
- Require a repeatable Criterion change on the same machine before claiming a small speedup or regression. Inspect both time and throughput across scales; one small fixture can hide an algorithmic regression.
- Preserve canonical bytes, hashes, diagnostics, fidelity records and operation atomicity while optimizing. Performance never relaxes conformance.
- Record allocation changes alongside latency. A time win that creates unbounded intermediate data is not accepted.
- Add a workload only when it has a stable fixture and a clear user-visible operation. Avoid benchmarks that measure fixture setup or debug assertions.
The command design follows the official Cargo benchmark
documentation and
Criterion’s command-line
and configuration
guidance. iai-callgrind remains a possible Linux-only instruction-count layer,
but is not a portable gate because it requires Valgrind; Criterion and the
allocation-instrumented smoke profile cover the current cross-platform need.
Conformance plan
NUIF conformance is split into independently testable profiles.
Required suites
model— stable IDs, containment, graph references, cycle rules.canonicalization— deterministic text/binary representation and hashes.extensions— used/required negotiation and opaque preservation.layout— authored→resolved fixtures across viewport/context matrices.render— normative geometry/paint/text behavior with declared tolerances.operations— patch replay, inversion, preconditions and deterministic results.merge— three-way semantic conflicts and move/reorder cases.provenance— correspondence retention and fidelity diagnostics.adapter— import/export loss reports and foreign-extension preservation.security— parser depth/size limits, malicious assets and renderer budgets.independent-reproduction— a non-reference package parses/writes the fixture and independently reproduces resolved layout, raster and fidelity.
Test techniques
- golden structural fixtures;
- property-based tests for operation sequences;
- fuzz parsers/codecs and path geometry;
- differential layout checks against browser/Taffy where semantics match;
- metamorphic tests such as encode→decode→encode stability;
- deterministic operation replay;
- visual snapshots with perceptual thresholds only where exact pixels are not normative.
A test result must include implementation version, capability profile, fixture ID and evaluation context. Foreign-reference results additionally include exact oracle versions, generator source revision, raw per-engine observations, a fixture-local measured bound and typed classifications for every divergence.
The implemented adapter suite covers nuif-html-css-0, the separately declared nuif-html-css-v0 responsive-card profile, nuif-svg-0 and nuif-dtcg-scalar-0 with exact export/import, byte-local synchronization and property-attributed rejection. Arbitrary web or SVG source, non-scalar DTCG and other adapter targets remain non-conformant until separately declared profiles pass.
The implemented independent-reproduction suite covers the complete v0 fixture in canonical text and the declared profile-0 layout/render subset. The Python standard-library implementation computes its own boxes and pixels, and the harness compares decoded RGBA so PNG encoder behavior is not mistaken for render divergence. This is an in-repository mechanical reproduction, not evidence of external implementation provenance or standards adoption.
Reference editor research preview
The editor is an executable conformance and research instrument, not the owner
of the NUIF data model. Its 0.1.0-alpha.N versions describe application
maturity and do not assign a maturity level to the draft specification. The
native Masonry shell and headless driver edit NUIF through the same semantic
operation API available to the CLI and automated clients.
The executable profile-zero shell provides a file menu for native document import/save, PNG export, and the repository’s declared SVG, HTML/CSS, DTCG, Penpot, static React JSX and static Svelte profile adapters. A foreign import is bounded before parsing, presents its fidelity summary before opening as a new unsaved NUIF document, and leaves the active document untouched when parsing or confirmation fails. A foreign export writes a sibling .report.json fidelity record. The shell also provides page creation, layer and component browsing, identity-backed canvas selection, frame/rectangle/ellipse/path/text insertion, subtree duplication and deletion, undo/redo, evaluation-width presets, zoom, panel visibility and a command palette. The canvas opens with a document-aligned background grid, pixel rulers and explicit px measurement labels; grid and rulers can be toggled independently. Move-tool drags author whole-pixel-snapped freeform positions or same-parent Stack/Flex order, while resolved selection handles author bounded fixed sizes and anchored freeform positions in one transaction. Its inspector authors names, positions, sizing intents, stack/flex layout, gaps, four-edge padding, alignment, bounded explicit Grid tracks/flow/placement/spans, solid fills and pinned-font text. Multi-field Apply is one atomic transaction.
Run cargo run --locked -p nuif-editor to open the native editor from a checkout. The canonical persistent path is cargo xtask editor-install --user --channel source; tagged source uses the alpha channel. Update, doctor, rollback and uninstall remain explicit checkout-owned operations. See INSTALLING.md for the verified source lifecycle and user-owned platform paths. The same editor binary accepts --headless --script <jsonl> with either direct command records or entity-bound action records. Its accessibility surface supports selection plus name, size intent, position, layout spacing, solid fill and text edits; undo/redo patches are logged too, and every run replays the complete mutation log from the opening document and requires the final hashes to match. Document reads share the 16 MiB profile limit; scripts are capped at 8 MiB, 100,000 commands and 64 KiB per line before JSON parsing.
cargo xtask editor-gui-trial drives the real AccessKit nodes of the native shell and derives pointer paths from resolved entity geometry rather than hard-coded screen coordinates. It exercises freeform move, north-west resize, responsive Stack reorder, undo and redo; independently replays the operation log; renders the complete 1280×800 Masonry tree through the CPU harness twice; and requires identical canonical, replay, shell, menu and document hashes. It emits the screenshot, canonical output, semantic-node inventory and machine-readable report under target/editor-gui-trial/. The full shell is specified in UI-SPEC.md; a Svelte 5 shell over WASM remains a later browser demonstration.
Snapshots reject zero dimensions, an edge above 4,096 pixels or more than
16,777,216 pixels before layout or raster allocation. Accessibility and
inspector numeric inputs reject non-finite values for fixed, percentage and
fit-content sizes, positions, spacing and text metrics. cargo xtask editor-hostile-inputs checks those boundaries together with missing semantic
nodes, atomic multi-operation failure, empty history, redo invalidation and
complete operation-log replay; it writes
target/editor-hostile-input-report.json.
Opening a package is not a capability grant. Packages with no required
capabilities remain editable and preserve their verified embedded resources.
A package requiring behavior or another capability the reference editor does
not implement opens structurally in persistent read-only mode. Selection,
inspection, static snapshots and byte-exact Save As remain available; semantic
commands, accessibility mutations, undo/redo mutation and a changed package
save fail with the exact missing set. cargo xtask editor-hostile-inputs
exercises both the driver and save boundary so opaque resources cannot be
silently rebound to an unvalidated document revision.
The draft UI-SPEC.md is broader than executable profile zero. Multi-selection, cross-parent/tree drag, Grid/Constraint reorder, persisted aspect-ratio constraints, object smart guides, managed leading-edge resize, in-editor token editing, component authoring, advanced paint/effects, arbitrary foreign formats and non-PNG rendering export remain gated on corresponding model, protocol, layout, adapter or renderer profiles. The shell does not present inert controls for those features.
Use cargo xtask editor-package to build and verify the native package for the host platform, or cargo xtask editor-launch to package and open it without installation. macOS produces NUIF Editor.app, Windows produces a GUI-subsystem executable, and Linux produces a relocatable desktop application directory. Version tags produce five GitHub prerelease archives with checksums and provenance attestations. Those archives are release evidence and an expert opt-in path; developer installation builds locally according to ADR 0009. See INSTALLING.md, PACKAGING.md and docs/VERSIONING.md.
Reference editor architecture
The editor is a client of the same semantic engine used by CLI/API tooling. The shell technology is decided in ADR 0006 (accepted): a Rust-native shell on Masonry, Vello and AccessKit; the Svelte 5 shell below is retained as the browser demonstration path. The user-interface specification is UI-SPEC.md.
Rust shell (Masonry widgets, AccessKit tree) — or Svelte 5 shell over WASM for the browser demonstration
│ typed commands/events
▼
Rust core (in-process; WASM boundary only in the browser build)
├── document store
├── protocol/transactions
├── layout evaluators
├── render-scene builder
├── query/diagnostics
└── codecs
│
▼
renderer backend (WebGPU/Vello experiment)
The UI shell may keep ephemeral selection/viewport/panel state, but authored document state is NUIF state. Canvas gestures MUST translate into semantic protocol operations before mutation.
Package loading has a separate capability boundary. The editor structurally verifies and preserves every package resource, but its declared package capability set is empty until a complete capability-specific authoring and evaluation profile is implemented. A package with any unsupported required capability opens inspectable and copyable but read-only. Both the shared editor driver and package-save boundary reject semantic document changes, preventing an opaque resource from remaining attached to a document revision it was not validated against.
The editor must expose a local automation endpoint or in-process API that mirrors CLI semantics. MCP may be added as an adapter, never as the canonical automation contract.
Developer installation
NUIF Editor is installed as a user-scoped developer tool built from reviewed source. Apple notarization, Microsoft Store publication and administrator access are not prerequisites for this path. The source checkout is the control plane for install, update, diagnosis, rollback and removal; retain it after the initial installation.
Prerequisites
- Git, Rustup and the Rust toolchain selected by
rust-toolchain.toml; - GitHub CLI (
gh) for verified alpha-channel updates; - the native build prerequisites for the host;
- Linux Fontconfig development files when building on Linux.
On Debian or Ubuntu, install the Linux native dependency with:
sudo apt-get install libfontconfig1-dev
Do not pipe a remote installation script into a shell. Clone an exact release tag, inspect it when required, and build through the checked-in xtask:
git clone --branch <release-tag> --depth 1 https://github.com/refpath/nuif.git
cd nuif
git rev-parse HEAD
cargo xtask editor-install --user --channel alpha
The alpha install rejects a dirty checkout, requires the exact
v<editor-version> tag at HEAD, uses Cargo.lock, builds the native package,
installs it, and immediately runs editor-doctor. A local research branch uses
the source channel:
cargo xtask editor-install --user --channel source
Dirty source is rejected by default. An intentional local experiment can opt
in with --allow-dirty; its receipt contains a deterministic working-tree
digest as well as the commit revision.
Lifecycle
Run lifecycle commands from any retained NUIF checkout that contains ADR 0009 support:
cargo xtask editor-update --user --channel alpha --check
cargo xtask editor-update --user --channel alpha
cargo xtask editor-doctor --user
cargo xtask editor-rollback --user
cargo xtask editor-uninstall --user
editor-update selects the highest published numeric alpha version. Before it
executes release source, it downloads release-manifest.json, verifies the
GitHub attestation against refpath/nuif, the release workflow, the tag, the
source revision and a GitHub-hosted runner, then fetches that exact tag with Git
hooks disabled. The fetched commit must equal the attested revision and remain
clean. Updates are explicit; the editor never updates itself while it is
running.
The active and previous immutable installations are retained. Rollback only changes the platform integration point and state file; it does not rebuild or contact the network. A later rollback swaps the two versions again.
editor-doctor verifies the managed marker, state and receipt schemas, source
and lockfile identities, installed binary digest and reported version,
platform integration, and the local macOS ad-hoc signature. Its JSON report
also states whether Git, GitHub CLI, Cargo and Rustc are available for a source
update.
User-owned paths
| Host | Immutable state | Active application integration |
|---|---|---|
| macOS | ~/Library/Application Support/org.nuif.Editor/dev/versions/ | ~/Applications/NUIF Editor Dev.app |
| Windows | %LOCALAPPDATA%\NUIF Editor Dev\versions\ | %LOCALAPPDATA%\Programs\NUIF Editor Dev\ and a user Start-menu shortcut |
| Linux | ${XDG_DATA_HOME:-~/.local/share}/nuif-editor-dev/versions/ | ~/.local/bin/nuif-editor-dev plus user XDG desktop and icon entries |
Every removable state root and Windows program directory carries a product
marker. Existing unrelated files, directories, symbolic links, desktop entries
or shortcuts are rejected rather than claimed. --root <absolute-path> places
state and integration below an isolated root for CI and disposable testing;
filesystem roots are rejected.
Trust boundary
The macOS source install applies and verifies a free local ad-hoc signature. That signature is not Developer ID, notarization or a publisher identity. The Windows source install does not add a certificate or change Defender, SmartScreen or Smart App Control. No lifecycle command disables Gatekeeper, System Integrity Protection or another operating-system security control.
A managed machine may still require an organization-approved signing certificate or device policy. That is an administrator-owned trust decision, not an installation workaround and not a requirement to publish through an Apple or Microsoft marketplace.
Secondary distribution
GitHub archives, their manifests, checksums, SBOM and attestations remain release evidence, reproducibility material and an expert opt-in download path. They are intentionally not the primary developer installation.
A future Homebrew tap should use a source formula. A Nix flake can provide a locked macOS/Linux development environment. A Scoop bucket may provide a convenient user-scoped Windows archive installation, but it still runs the downloaded unsigned executable and cannot be presented as a SmartScreen trust solution.
Native editor packaging
cargo xtask editor-package builds the release application wrapper, creates the native package for the host platform, runs the packaged executable with --help and --version, hashes the executable and archive, and writes a package-specific manifest plus target/dist/editor-package-manifest.json.
| Host | Package | Archive | Application entry point |
|---|---|---|---|
| macOS | target/dist/nuif-editor-<version>-macos-<arch>/NUIF Editor.app | .tar.gz | Finder, open, or the executable under Contents/MacOS |
| Windows | target/dist/nuif-editor-<version>-windows-<arch>/ | .zip | NUIF Editor.exe |
| Linux | target/dist/nuif-editor-<version>-linux-<arch>/ | .tar.gz | bin/nuif-editor; freedesktop entry and scalable icon are under share/ |
cargo xtask editor-launch rebuilds and verifies the host package and then opens it. On macOS this calls open -n on the .app; on Windows and Linux it starts the packaged application executable.
Packages contain the Apache-2.0 and MIT license files, a scope notice and a package-local manifest. Versioned archives preserve the executable layout and are CI artifacts. Tag builds also become GitHub prerelease assets with package manifests, SHA256SUMS, a CycloneDX software bill of materials, release-manifest.json, and GitHub artifact attestations. The tag and publication contract is defined in docs/VERSIONING.md and ADR 0007.
Downloaded packages are not the canonical developer installation. cargo xtask editor-install --user builds the same package from the retained source
checkout and installs an immutable version with a source/build receipt. The
explicit update, doctor, rollback and uninstall lifecycle is documented in
INSTALLING.md and ADR 0009.
Current alpha archives are unsigned. Code signing, notarization and installer or store publication require platform credentials and remain separate release operations. The archive manifest records the unsigned status. A locally built macOS developer installation is ad-hoc signed after copying and records adhoc-local; that is neither Developer ID nor notarization. Apple and Windows signing requirements are recorded in nuif:research:github-release-delivery-and-provenance and the source-install boundary in nuif:research:developer-source-installation-and-os-trust.
The Linux package expects a graphical Wayland or X11 session, Fontconfig at runtime, and a graphics driver supported by wgpu. Building it on Debian or Ubuntu requires libfontconfig1-dev; CI installs that package before every workspace or editor build. The archive is a relocatable application directory, not a distribution-specific system package. The Windows application wrapper uses the GUI subsystem so a console window is not opened; the separate nuif-editor binary retains its headless/JSONL interface.
The native-editor CI matrix runs the editor check, tests, semantic and visual trial, complete sandboxed install/doctor/uninstall lifecycle, and packaging command on GitHub-hosted macOS, Windows and Linux systems. Each job uploads its archive, package manifest, install receipt evidence, semantic-node inventory and shell screenshot. This is native-host evidence; a successful cross-compilation alone is not treated as platform verification.
The release workflow adds Linux Arm64 and separate macOS Arm64 and x86-64 hosts. It requires five archives and five package manifests before publication. Each release job builds at the tagged source revision, runs the sandboxed lifecycle in strict alpha mode and uses the pinned Rust 1.98.0 toolchain.
AI/headless QA contract
Status: items 1–8 and 10 have a profile-0 implementation through nuif-api, nuif-testing, nuif and nuif-editor; automatic minimized failure-fixture writing remains partial. The native-shell wiring has a deterministic AccessKit and CPU-render trial.
An automated QA client must be able to perform the following without synthetic mouse input:
- create/open/save/canonicalize documents;
- query entities by identity/type/name/relationship;
- execute semantic transactions and capture inverse/replay logs;
- evaluate layout at explicit contexts;
- inspect resolved boxes, text diagnostics and accessibility semantics;
- render deterministic snapshots;
- diff canonical documents and resolved snapshots;
- validate fidelity and extension-preservation assertions;
- minimize a failing operation sequence into a reproducible fixture;
- emit one machine-readable report containing inputs, versions, capabilities and artifacts.
GUI automation is reserved for testing shell wiring, focus, pointer/keyboard interactions and browser integration.
The headless client MUST apply the same bounded document reader as the CLI and MUST bound script bytes, line bytes and command count before retaining an operation log. cargo xtask hostile-inputs verifies document-ingestion boundaries; editor unit tests verify the limit-plus-one reader.
Headless and GUI sessions report the package capability negotiation result. If
any required capability is unavailable, selection and inspection may proceed,
but every semantic transaction and changed package save MUST fail atomically.
The no-op package copy remains byte-exact. This boundary is part of
cargo xtask editor-hostile-inputs.
cargo xtask editor-trial is the required complete-authoring trial. It starts from an explicit empty document identifier, dispatches only semantic/identity actions, compares canonical output bytes with the direct fixture generator, independently replays the logged patches, validates the result, and archives the full snapshot evidence used by automated and AI-driven iteration.
cargo xtask editor-gui-trial is the supplementary shell trial. It selects an entity and edits name, sizing, position, layout spacing and fill controls through AccessKit Click and SetValue requests, then locates resolved entities through the canvas widget and performs captured freeform move, north-west resize and responsive Stack reorder pointer drags in document coordinates. Movement commits one position operation; the leading-corner resize commits its anchored position and both fixed-size axes in one transaction; Stack/Flex reorder commits one same-parent Move based on resolved sibling centres. All are whole-pixel snapped where applicable, the expected final child order is asserted, and the reorder is exercised by the following undo/redo pair. The runner independently replays every patch, validates the output document, records the semantic tree, captures both the normal shell and the open File menu, requires every native and profile import/export route to be visible, and repeats the run to require identical canonical and pixel hashes. The shell unit tests additionally cover all eight handle geometries, Shift-proportional corner geometry, managed no-op/rejection boundaries, inactive-tool rejection, atomic resize validation, pixel grid/ruler defaults and exact export/import round trips for the declared SVG, HTML/CSS, DTCG, Penpot, static React JSX and static Svelte fixtures. No hard-coded screen coordinate identifies a document entity or handle; the runner derives pointer paths from resolved layout and the canvas transform.
Reference test editor: user-interface specification
Status: draft specification for the reference test editor. The editor replicates the spatial layout, tool set, property sections and keyboard bindings of the Figma design editor (UI3) as documented in nuif:research:figma-ui3-editor-layout and nuif:research:figma-tools-and-keyboard-shortcuts, without branding, icons, logos, typography or colour values taken from that product. The feature set is limited to what conformance testing, import and export require. Anything not listed here is out of scope; additions require an RFC.
Implementation status: the native application implements the complete interactive surface for currently executable profile-zero properties: native document import/save and PNG export; bounded import and reported export through the declared SVG, HTML/CSS, DTCG, Penpot, static React JSX and static Svelte adapter profiles; history; pages/layers/components browsing; canvas selection, insertion, captured freeform-child movement, Stack/Flex reorder and bounded resizing; subtree duplicate/delete; responsive evaluation-width presets; zoom and interface visibility; a default pixel grid and rulers; command routing; and atomic name, position, sizing-intent, stack/flex spacing/alignment, bounded explicit Grid, solid-fill and pinned-text inspection. Freeform movement previews the selection outline, commits one position operation on release, snaps to whole pixels by default and suspends snapping while Control is held. A Stack/Flex child drag derives the effective axis from resolved sibling geometry, chooses a same-parent insertion point by sibling centres and commits one Move; unchanged order creates no history. Grid, Constraint, cross-parent and instance-child reorder fail closed instead of guessing semantics. Freeform children expose eight resize handles; managed-layout children expose only east, south and south-east handles because leading-edge anchoring would require an ineffective position edit. A resize previews resolved geometry and commits the changed fixed-size axes plus any freeform leading-edge position as one atomic transaction. Shift preserves the starting aspect ratio for corner drags, Control suspends whole-pixel snapping, and roots and non-finite, non-positive or greater-than-1,000,000 px dimensions fail without mutation. Canvas transforms require the Move tool. Grid authoring exposes positive fixed-pixel and fractional track lists, row/column sparse auto-flow, atomic one-based item position and positive spans; it cannot create implicit tracks or exceed profile-zero resource bounds. Each document edit lowers to an invertible protocol operation and is covered by the pointer/AccessKit/replay trial. Foreign imports open a new session after confirmation rather than mutating the current document. Sections below whose data is not in profile zero remain specification targets, not inert or simulated controls—most notably multi-selection, persisted aspect-ratio constraints, managed-layout leading-edge resize, cross-parent/tree drag and Grid/Constraint reorder, component authoring, in-editor token editing, advanced paints/effects, arbitrary foreign formats and non-PNG rendering export.
Native packages whose required capability set is not fully supported open structurally in a persistent read-only mode. Selection, inspection, static snapshot and byte-exact copying remain available, while driver and save-boundary mutations fail with the exact missing identifiers. This avoids silently carrying opaque resources onto a document revision they were not validated against.
Purpose and constraints
- The editor is a client of
nuif-api. Every gesture lowers to protocol operations before any document mutation (apps/editor/ARCHITECTURE.md). - The editor is a test instrument. Its state is NUIF state plus ephemeral selection, viewport and panel state.
- Automated test iterations use the CLI and the in-process session driver, not the GUI. The GUI exists to author fixtures by hand, to inspect import and export results, and to prove that a human-authored fixture and a replayed operation log converge (roadmap phase 5 exit).
- Layout conventions replicated here are user-interface conventions, not protected expression; names, marks, icons and visual assets are not reproduced (
nuif:research:design-editor-ui-conventions-synthesis).
Regions
┌──────────────────────────────────────────────────────────────────────────────┐
│ [A] Top bar: document name · page selector · Minimize UI │
├───────────────┬───────────────────────────────────────────┬──────────────────┤
│ [B] Left │ [C] Canvas │ [D] Right panel │
│ panel │ rulers · infinite canvas · zoom │ zoom % · Export │
│ Pages │ marquee · snapping · smart guides │ ┌ Design ────┐ │
│ Layers │ measurement overlay │ │ sections │ │
│ Components │ ┌───────────────────────────────┐ │ │ (resizable)│ │
│ (resizable) │ │ [E] Toolbar (floating, bottom) │ │ └────────────┘ │
│ │ │ Move Hand Frame Shapes Pen Text│ │ Diagnostics │
│ │ └───────────────────────────────┘ │ │
└───────────────┴───────────────────────────────────────────┴──────────────────┘
Minimize UI collapses A and B; D reappears while a selection exists. Hide UI hides A, B, D and E. Panels B and D are resizable; widths persist per session only. Panel pixel widths are not specified by evidence and are chosen by implementation.
Left panel [B]
- Pages: ordered list of surfaces (NUIF
Surfaceentities); add, rename, reorder, delete. - Layers: containment tree of the current page; expand and collapse; drag to reparent and reorder; rename inline; visibility and lock toggles; multi-selection synchronized with the canvas.
- Components: local component definitions of the document; drag to instantiate.
Excluded: team libraries, asset search, remote components.
Canvas [C]
- Infinite canvas with independently toggled pixel rulers and a document-aligned background grid. The profile-zero authoring unit is
pxand is shown in the top bar, status bar and numeric inspector labels. The page background colour comes from the surface. - Interaction grammar (bindings in the table below): marquee selection from empty space; additive and subtractive selection; hierarchical traversal (child, parent, next and previous sibling); deep selection; duplicate by modifier drag; pan by Space drag or Hand tool; zoom by modifier scroll and by shortcuts (fit, selection, 100 %).
- Snapping to objects and pixel grid with smart guides; suspended while the declared modifier is held. Measurement overlay to the hovered entity while the declared modifier is held.
- Every canvas gesture emits one transaction: move in freeform becomes a transform edit; drag inside a stack becomes a reorder; resize becomes a sizing-intent edit or a fixed size according to the parent family (
docs/whitepaper/03-protocol-and-portability.md).
Excluded: comments, cursors of other users, presentation mode, prototype links.
Toolbar [E]
Floating strip centred at the bottom of the canvas. Tools, in order, with their group menus:
| Group | Tools | Emits |
|---|---|---|
| Move | Move (V), Hand (H) | selection and transform transactions; Hand emits nothing |
| Region | Frame (F) | Insert of a Container |
| Shape | Rectangle (R), Ellipse (O), Line (L) | Insert of a Shape |
| Vector | Pen (P) | Insert of a Shape(Path) and path edits |
| Text | Text (T) | Insert of a Text entity |
| Actions | command palette (Cmd/Ctrl K) listing every operation and command by name | the selected operation |
The command palette is the keyboard route to every operation and to Import, Export, Validate and Snapshot. Excluded tools: Scale, Section, Slice, Polygon, Star, Arrow, Pencil, Comment, Dev Mode toggle, drawing and illustration tools, AI actions.
Right panel [D]
One tab, Design. Sections appear in this order for a container with a stack or flex family; sections absent for the selected kind are hidden.
- Header: entity name; Create component; Detach instance (for instances).
- Position: alignment row (six alignments and two distributions for multi-selection); X, Y; rotation; flip horizontal and vertical; constraints (freeform children only).
- Layout family: family selector (freeform, stack, flex, grid, constraint); flow direction and wrap; gap; padding uniform or per side; alignment grid. Grid containers additionally expose row/column sparse auto-flow and explicit column/row track lists using compact
120px 1frsyntax. Their children expose one atomic position field (autoor one-basedcolumn row) and positive column/row spans. This section replaces the product’s auto-layout section and exposes NUIF families directly. - Sizing: W, H; per-axis intent (fixed, intrinsic, fill, fit-content, percentage); min and max; aspect ratio; clip content. Values are plain numbers or token references (see Tokens).
- Appearance: opacity; corner radius uniform or per corner; blend mode; visibility.
- Fill: ordered list of solid colour and linear or radial gradient paints; image fill for
Imageentities. - Stroke: colour, weight, alignment, per-side, dash pattern, cap, join.
- Effects: drop shadow, inner shadow, layer blur, background blur.
- Typography (Text entities): font family from the pinned font set, weight, size, line height, letter spacing, alignment, text sizing behaviour.
- Component (definitions): parameters of kind boolean, enum variant, text, instance swap. Instance: parameter values and override reset.
- States: named interaction states (default, hover, pressed, focused, disabled) with per-state property overrides. This section exists because the v0 fixture requires state metadata; no prototype player is included.
- Tokens: every numeric or colour control accepts a token reference chosen from the document’s token set; a bound control shows the token name and a detach action. Token sets are edited in a dialog opened from the command palette (DTCG-compatible).
- Export: format (PNG, SVG, PDF,
nuif-text-0,nuif-cbor-0, adapter targets), scale, suffix; Export opens the fidelity report.
Diagnostics: a collapsible list under the sections showing validation, fidelity and layout diagnostics for the selection, produced by nuif-api; each entry links to the entity.
Excluded: Prototype and Inspect tabs, selection colours, styles library, variables modes UI, glass, noise and texture effects, video and pattern fills, layout guides, image cropping, boolean operations, masks, vector networks beyond path editing, version history, branching.
Dialogs
- Import: the File menu selects NUIF, SVG, HTML/CSS, DTCG, Penpot, static React JSX or static Svelte. External input is size-bounded before parsing; text profiles require UTF-8 while the Penpot profile validates bounded ZIP members in memory. A native NUIF package is structurally verified before opening; unsupported required capabilities are listed in the status bar and window title and force read-only inspection. A confirmation dialog shows fidelity-class and correspondence counts before the external imported document replaces the session as an unsaved document. Merge-into-document import is not implemented.
- Export: the File menu selects PNG, SVG, HTML/CSS, DTCG, Penpot, static React JSX or static Svelte. Each external adapter export writes the artefact and a sibling
.report.json; a profile mismatch fails before the destination chooser and writes nothing. - Tokens: token set editor.
- Evaluation context: viewport size presets (360, 768, 1440 px and custom), scale factor, locale, writing direction, theme; the canvas renders the selected context; multiple contexts can be shown side by side for a surface.
- Snapshot: writes the canonical document, the resolved snapshot for the current context and the CPU rasterization to a fixture directory in the harness format (
conformance/HARNESS.md).
Keyboard bindings
Bindings reproduce the documented product bindings where verified; entries marked U in the source record are chosen to match common expectation and are not claimed to be product-accurate.
| Action | macOS | Windows and Linux |
|---|---|---|
| Move, Hand, Frame, Rectangle, Ellipse, Line, Pen, Text | V, H, F, R, O, L, P, T | same |
| Command palette | Cmd K | Ctrl K |
| Add or remove stack layout | Shift A / Option Shift A | Shift A / Alt Shift A |
| Group / Ungroup / Frame selection | Cmd G / Shift Cmd G / Option Cmd G | Ctrl G / Shift Ctrl G / Ctrl Alt G |
| Create component / Detach instance | Option Cmd K / Option Cmd B | Ctrl Alt K / Ctrl Alt B |
| Duplicate / duplicate by drag | Cmd D / Option drag | Ctrl D / Alt drag |
| Copy / Paste | Cmd C / Cmd V | Ctrl C / Ctrl V |
| Undo / Redo | Cmd Z / Cmd Shift Z | Ctrl Z / Ctrl Shift Z, Ctrl Y |
| Select all / inverse | Cmd A / Cmd Shift A | Ctrl A / Ctrl Shift A |
| Select child / parent; next / previous sibling | Enter / Shift Enter; Tab / Shift Tab | same |
| Deep select; nested marquee; subtractive marquee | Cmd click; Cmd drag; Shift drag | Ctrl click; Ctrl drag; Shift drag |
| Nudge 1 px / 10 px | Arrow / Shift Arrow | same |
| Align left, right, top, bottom, centre horizontal, centre vertical | Option A, D, W, S, H, V | Alt A, D, W, S, H, V |
| Flip horizontal / vertical | Shift H / Shift V | same |
| Zoom in / out; fit; selection; 100 % | Cmd + / Cmd −; Shift 1; Shift 2; Shift 0 | Ctrl + / Ctrl −; Shift 1; Shift 2; Shift 0 |
| Rulers; pixel grid | Shift R; Cmd ’ | Shift R; Ctrl ’ |
| Minimize UI / Hide UI | Cmd Shift \ / Cmd \ | Ctrl Shift \ / Ctrl \ |
| Export | Shift Cmd E | Shift Ctrl E |
| Temporarily disable snapping; measure to hovered | hold Control; hold Option | hold Control; hold Alt |
| Pan | hold Space and drag | same |
Automation surface
The editor binary accepts --headless --script <file> and either --document <file> or --new-document <id>, then runs a session script against the same nuif-api engine without creating a window (nuif:research:blender-dna-rna-and-headless, nuif:research:unreal-asset-versioning-and-automation). --expect-document makes byte-exact parity blocking; --report and --snapshot-dir write the operation log and canonical/context/layout/scene/CPU-raster artifacts. The JSONL script language contains editor commands and semantic accessibility actions sharing one session.
The nuif-editor-automation feature-gated binary drives the native Masonry tree in process. It dispatches AccessKit actions, captures the matching accessibility tree and CPU-rendered shell frame, replays the protocol log independently and emits a machine-readable artifact set. cargo xtask editor-gui-trial repeats that run and requires identical canonical and pixel hashes. No socket transport is implemented.
Widget identity: every widget bound to a document entity exposes the entity identifier in its accessibility node (author_id), and every control exposes a role and label, so a harness locates “the width control of entity X” by query and sets it through an accessibility SetValue action (nuif:research:accesskit-semantic-ui-testing). No test depends on pixel coordinates of widgets.
Rendering and text
The canvas renders through nuif-render. Interactive rendering uses the Vello backend; snapshots and headless runs use the CPU reference backend so that editor snapshots and conformance references are produced by the same path. Fonts are limited to the pinned set shipped with fixtures; system fonts are not enumerated.
Implementation stack
Decided in ADR 0006 (accepted): Rust-native shell on Masonry (pinned by git revision; Xilem not used) with Vello rendering and AccessKit, replacing the earlier Svelte 5 shell proposal for the reference editor. The canvas lowers NUIF RenderScene to Masonry’s imaging command set; the CPU reference renderer in nuif-render remains the conformance oracle. Toolchain 1.98.0, MSRV 1.96. A browser build through WASM remains a later demonstration target, not the reference editor.
Adapter inventory
This page is generated from adapters/index.json. The inventory contains 12 host or format targets.
| Target | Status | Surface | Profiles |
|---|---|---|---|
html-css | integrated | DOM and CSS source | 4 |
svg | integrated | SVG 2 XML | 1 |
dtcg | integrated | Design Tokens Format Module 2025.10 JSON | 1 |
react | integrated | JSX source and React DOM properties | 1 |
svelte | integrated | .svelte source and compiler AST | 1 |
penpot | integrated | .penpot v3 ZIP and JSON package | 1 |
figma | integrated | normalized Plugin API snapshots and mutation plans; writable Plugin API host | 1 |
affinity | external_runtime | user-mediated SVG import/export in a named Affinity desktop runtime | 0 |
canva | external_runtime | Canva Apps SDK Design Editing API; Connect APIs as a separate OAuth workflow | 0 |
swiftui | external_runtime | Swift source and proposal-response layout runtime | 0 |
jetpack-compose | external_runtime | Kotlin source and constraint layout runtime | 0 |
flutter | external_runtime | Dart source and box-constraint runtime | 0 |
Adapters
External ecosystems are peers around NUIF, not parents of its canonical model.
The first executable adapter is the deliberately bounded nuif-html-css-0 retentive profile. It records concrete byte spans, round-trips its declared container/text/token subset exactly, applies mapped changes without regenerating comments or unmapped regions, and rejects every unsupported semantic change with target/property fidelity.
The follow-on nuif-html-css-v0 profile carries the complete responsive-card model, including responsive rules, component/instance identity, unknown kinds and opaque extensions. Its automated editor bridge proves semantic editor output can patch retained source and return through the CLI to byte-identical canonical NUIF. Path rendering, instance materialization, unknown visuals and arbitrary HTML/CSS remain explicitly outside the target profile.
The separate nuif-web-accessibility-0
projection lowers a ten-role, role-specific Boolean-state and five-relationship
subset into inert native HTML and ARIA. A pinned Playwright oracle compares the
computed roles, accessible names and supported states of the same eleven-node
fixture across Chromium, Firefox and WebKit while retaining host-tree
differences separately. It does not synthesize behavior or claim native
platform accessibility equivalence.
The one-way nuif-web-behavior-0 projection
composes that semantic mapping with the bounded state-machine sidecar. It maps
enabled native button/switch activation, hidden visibility and one polite
status announcement through a generated finite runtime authorized by its exact
CSP hash. Three Playwright engines reproduce every event’s transition, state
and retained host effects. Authored JavaScript, behavior import, focus,
checkbox/radio mutation and assistive-technology speech remain excluded.
The nuif-svg-0 profile maps one surface, freeform groups,
rectangles, ellipses and literal pinned-font text to SVG 2 XML. It retains
UTF-8 spans for identity, geometry, paint and accessibility scalars, preserves
unmarked XML byte-for-byte during edits and rejects unsupported geometry,
paint and structure with typed fidelity.
The nuif-dtcg-scalar-0 profile maps flat DTCG 2025.10
boolean, string and number tokens while preserving NUIF integer/real identity
through namespaced metadata. It retains unknown extension bytes through the
same CLI synchronization contract as the source adapters. Its deliberately
narrow boundary precedes the token-model RFC required for groups, aliases and
composite types.
The nuif-penpot-v3-0 profile maps one Penpot v3 package,
file, page and board with direct rectangle, ellipse and pinned literal-text
children. It retains unedited member payloads and unknown package data, returns
the original archive byte-for-byte on no-op synchronization, and applies ZIP,
expanded-data, member, compression and JSON resource limits before parsing.
The nuif-react-jsx-0 profile extracts one directly
returned, marked intrinsic JSX subtree without executing JavaScript. It maps
fixed flex containers and literal pinned-font text through byte spans, retains
unrelated module source and rejects components, spreads, handlers and runtime
expressions.
The nuif-svelte-static-0 profile maps one marked static
Svelte component made of regular containers and literal text. It patches the
same 21 semantic correspondences through the shared scalar planner, rejects
executable template constructs and checks every synchronized source against
the exact official Svelte compiler as a foreign oracle.
The nuif-figma-plugin-snapshot-0 profile now
implements the credential-free pure mapping between normalized Plugin API
objects, canonical NUIF and a host mutation-plan tree. The compiled no-network
review shell in figma/plugin consumes that exact schema,
requires confirmation before mutation and is checked against the Rust importer.
It covers a deliberately narrow visible/opaque fixed-size subset, repairs
portable identity deterministically and reports every declared Figma-only
property. The shell is static evidence, not a live Figma claim.
The remaining researched or externally bounded targets are Affinity, Canva, Flutter, SwiftUI and Jetpack Compose. Figma and Canva retain bounded API-host draft profiles and a serializable host-object correspondence report; Affinity has a separate user-mediated SVG bridge draft. None has a corresponding live host claim. Broader HTML/CSS, SVG and DTCG profiles remain separate future work beyond the ten executable profiles. Each adapter must emit structured fidelity diagnostics and record provenance/correspondence sufficient for later synchronization and minimal source patches where feasible.
STATUS.md records the current primary integration surface,
implementation status, next bounded profile and exclusion boundary for every
advertised target. Research coverage and executable conformance are listed
separately.
index.json is the machine-readable counterpart. cargo xtask adapter-audit requires all advertised targets to have a primary research
record, explicit target and per-profile directionality, a next bounded profile
and a non-empty boundary. Integrated entries additionally require each
profile’s directions to be a subset of the target union plus crate, profile and
routed gate paths; non-integrated entries cannot claim executable directions. The
audit writes target/adapter-coverage-report.json and blocks the complete gate.
Vendor-specific semantics belong in namespaced extensions or adapter-local logic; they must not leak into the core merely because a vendor is popular.
Source and file-interchange adapters use AdapterReport and byte-span or
artifact correspondence. Plug-in/API hosts use HostAdapterReport and stable
host-object identifiers because they do not expose retained source bytes. See
ADRs 0008 and 0012 and docs/HOST-INTEGRATION.md.
NUIF Figma review shell
This directory contains the compiled, no-network host shell for the bounded
nuif-figma-plugin-snapshot-0 profile. It is a development and review tool,
not a published Figma Community plug-in and not live-host conformance evidence.
The shell intentionally does not contain a made-up plug-in ID. Figma assigns that ID when a reviewer creates a development plug-in. The checked-in manifest is therefore a template and cannot be imported as-is.
Build and review
Requires the pinned Node range in package.json.
npm ci --ignore-scripts --no-audit --no-fund
npm run check
This compiles the main thread against the exact official Figma typings, builds
an inline UI, runs the pure protocol/normalizer tests and emits dist/ with a
deterministic build report. The manifest declares allowedDomains: ["none"];
the build also rejects remote URLs in generated files.
To create a locally importable manifest, first use Figma’s development plug-in flow to obtain an ID, then run:
FIGMA_PLUGIN_ID=your-assigned-id npm run package
Replace the illustrative value with the exact ID Figma assigned to your
development plug-in, then import dist/manifest.json. Do not commit that
generated manifest.
Export and import flow
Export requires exactly one selected frame and downloads a normalized snapshot:
nuif import figma-plugin-snapshot-0 selection.figma-snapshot.json selection.nuif.json fidelity.json
Import starts from canonical NUIF and creates a mutation plan outside Figma:
nuif export selection.nuif.json figma-plugin-snapshot-0 plan.json fidelity.json
Load plan.json in the plug-in UI. It checks the full bounded schema and keeps
Apply disabled until the user explicitly confirms. A successful plan is
committed as one Figma undo step. A failed creation removes every node created
by that attempt before returning the error.
The exact text subset requires Ahem Regular to be installed and its pinned SHA-256 marker in shared plug-in data. This is a conformance constraint, not a general font-import solution. General Figma fonts remain outside this profile.
Evidence boundary
Credential-free CI proves TypeScript checking, deterministic compilation, no-network packaging, message validation, a mock Plugin API snapshot and its successful import by the Rust core. Only a reviewer-run Figma trial can prove actual node creation, font availability, undo behavior, cancellation and plug-in-data persistence in a named Figma product version.
Adapter implementation status
The adapter program separates an ecosystem’s public interchange or source surface from the subset for which NUIF can provide executable round-trip laws. Research coverage does not imply an implemented conformance profile.
The inventory contains twelve targets and ten executable profiles across
seven target families. The remaining targets have no executable direction in
adapters/index.json.
This table is explanatory. adapters/index.json is the machine-audited target
inventory; cargo xtask adapter-audit checks its research, profile, crate and
gate references and writes target/adapter-coverage-report.json.
| Target | Primary integration surface | Executable status | Next bounded profile | Boundary |
|---|---|---|---|---|
| HTML/CSS | DOM and CSS source | nuif-html-css-0, nuif-html-css-v0, nuif-web-accessibility-0 and nuif-web-behavior-0 | Extend only with separately tested CSS/layout/focus/control-state features | Arbitrary cascade, authored scripts and unmarked DOM are not imported; behavior lowering is one-way and finite |
| SVG | SVG 2 XML | nuif-svg-0 | Add paths or transforms only under separately declared geometry laws | Paths, transforms, CSS cascade, paint servers, effects, animation, scripts and external resources are excluded |
| DTCG tokens | Design Tokens Format Module 2025.10 JSON | nuif-dtcg-scalar-0 | Expand only after a token-model RFC and separate profile | Core tokens lack declared type, groups, aliases, descriptions, deprecation and token-local extensions |
| React | JSX source and React DOM properties | nuif-react-jsx-0 | TSX or CSS-class support only under a separate grammar/runtime matrix | Components, hooks, spreads, control flow and runtime expressions require execution |
| Svelte | .svelte source and compiler AST | nuif-svelte-static-0 | Component CSS only after selector/cascade/scope-hash rules | Runes, scripts, blocks, directives, preprocessors, component CSS and dynamic components require execution or another profile |
| Penpot | .penpot v3 ZIP and JSON package | nuif-penpot-v3-0 | Add compact pages only after the opt-in representation stabilizes | Components, libraries, interactions, media, paths, layout and compact pages are excluded |
| Figma | Normalized Plugin API snapshot/plan plus writable host | nuif-figma-plugin-snapshot-0 mapping and compiled no-network review shell; no live host run | Assigned-ID reviewer run and live fixtures in adapters/figma/PROFILE-DRAFT.md | .fig is not a public contract; static evidence does not prove host writes, undo or persistence |
| Affinity | User-mediated SVG import/export in the desktop application | Research and existing nuif-svg-0 bridge only; no live Affinity trial | Retained two-way SVG trial in adapters/affinity/PROFILE-DRAFT.md | No public document API or native .af* schema is claimed; native files are opaque and UI automation is non-conformant |
| Canva | Apps SDK Design Editing API; Connect APIs are a separate OAuth workflow | Research and host report contract only; no reviewed app or live host run | Stable one-page snapshot and one-sync mutation mapping in adapters/canva/PROFILE-DRAFT.md | Current-page fixed documents and documented supported elements only; preview APIs, Docs, native NUIF Connect I/O and marketplace approval are excluded |
| SwiftUI | Swift source and proposal–response layout runtime | Research complete; no implementation | Generated stack/text/shape subset with a pinned Apple toolchain | Arbitrary Swift and custom layouts are executable programs |
| Jetpack Compose | Kotlin source and constraint layout runtime | Research complete; no implementation | Generated row/column/text/shape subset with a pinned Android toolchain | Arbitrary Kotlin, state, modifier chains and subcomposition are executable programs |
| Flutter | Dart source and box-constraint runtime | Research complete; no implementation | Generated row/column/text/shape subset with a pinned Flutter toolchain | Arbitrary Dart, state, inherited widgets and custom render objects are executable programs |
Implementation order
The SVG basic-shape, DTCG scalar-token and Penpot v3 profiles are implemented
because their bounded subsets map directly to the current model and run without
credentials or platform SDKs. Full DTCG coverage requires a token-model RFC.
Penpot’s package path enforces ZIP resource limits and unknown-member retention
through one shared test contract. Figma’s normalized snapshot and mutation-plan
mapping now has exact Rust/CLI trials, while its host execution remains
uncertified. Affinity has a bounded interchange draft over the existing SVG
profile; Canva has a bounded API-host draft and the shared HostAdapterReport
envelope. Neither has live-host evidence yet.
React and Svelte now use the common byte-span correspondence contract for one marked static subtree. Svelte additionally compiles direct and CLI output with the exact official compiler. Native declarative UI targets begin as one-way lowerings with foreign-runtime layout and screenshot comparisons. Bidirectional claims remain out of scope until a static, profile-owned source subset has exact import and edit-locality tests.
Conformance requirements
Every implemented adapter profile must provide:
- a versioned profile document and machine-readable capability identifier;
- input byte, syntax-depth, entity/member and retained-data limits;
- import/export fixtures with canonical NUIF expected outputs;
- exact round trips for the declared subset and typed fidelity for every excluded property;
- correspondence records with foreign identity and property or source span;
- unknown-data preservation tests when the foreign format permits retention;
- repeated-output determinism, stale-correspondence rejection and atomic failure;
- foreign-validator or runtime evidence pinned by version when one exists;
- CLI,
xtaskand CI integration before the profile is listed as executable.
The supporting primary-source records are svg, dtcg, accessibility-semantics,
react-jsx-adapter-surface, svelte-source-adapter-surface, penpot, figma,
affinity-interchange-and-adoption, canva-apps-and-connect-adoption,
swiftui-layout, jetpack-compose-layout and flutter-layout under
research/items/. The earlier adobe-uxp-host-integration record remains
historical prior art rather than an advertised target.
Draft Affinity SVG bridge profile 0
Status: researched user-mediated interchange specification; no native Affinity parser, plug-in, scripting API or live-host conformance claim.
Profile identifier: nuif-affinity-svg-bridge-0.
Primary evidence: nuif:research:affinity-interchange-and-adoption and ADR
0012.
Host and scope
- A named desktop build of the all-new Affinity on a recorded operating system.
- One document and one fixed-size artboard/page per trial.
- The exact shapes, groups, solid encoded-sRGB fills, opacity and pinned literal
text admitted by
nuif-svg-0. - User-mediated SVG import and SVG export through Affinity’s documented file interface.
- The source NUIF, exported bridge SVG, Affinity-produced SVG, canonical result, render artifacts and fidelity report are retained together.
Native .af, .afdesign, .afphoto, .afpub and template bytes are opaque
foreign artifacts. Paths, arbitrary transforms, CSS, paint servers, effects,
animation, scripts, external resources, editable Affinity-only objects,
responsive layout and component behavior are excluded.
Mapping boundary
NUIF-to-Affinity uses the existing nuif-svg-0 exporter. The SVG and its report
must pass cargo xtask gate-svg before a human opens it in Affinity. The user
records the exact Affinity and operating-system versions, imports the SVG and
exports a second SVG without unrelated edits.
Affinity-to-NUIF uses the existing bounded SVG importer. If the produced SVG
contains any construct outside nuif-svg-0, the import fails or reports that
construct as unsupported. The trial never rewrites the SVG to force acceptance.
Byte identity is not expected because Affinity may reserialize SVG; canonical
NUIF semantics and separately pinned render evidence are the comparison
surfaces.
Identity and resources
No stable Affinity object identifier or metadata-preservation contract is claimed. Correspondence is trial-local and derived from the two SVG mappings. Embedded or linked fonts and images are outside the first bridge because the current SVG profile excludes external resources. Native Affinity files may be retained as opaque provenance but are never decoded by NUIF tooling.
Required live evidence
- exact pre-Affinity NUIF and SVG bytes plus the exporter report;
- exact Affinity-produced SVG bytes and bounded importer report;
- named Affinity build, operating system, locale and unit settings;
- one unchanged import/export, one reorder, one edit and one unsupported-effect trial;
- visual comparison rendered from both sides with text and geometry metrics reported separately;
- a second-person review that every loss is represented in the report;
- repeated trials on every operating system advertised by the profile.
The profile remains external_runtime until these artifacts are checked in and
the trial runner prevents unsupported SVG from being promoted. A documented
Affinity document API or scripting runtime would require a new profile rather
than silently broadening this bridge.
Draft Canva Design Editing profile 0
Status: researched Apps SDK mapping specification; no reviewed Canva app or live-host conformance claim.
Profile identifier: nuif-canva-design-editing-0.
Primary evidence: nuif:research:canva-apps-and-connect-adoption and ADR 0012.
Host and scope
- A public Canva Apps SDK v2 app using generally available
@canva/designAPIs and the Design Editor intent. - One unlocked
current_pagesession whose page isabsoluteand has fixed dimensions. - Groups, rectangles, shapes and rich-text elements with ordered layering, position, size, rotation, transparency and the explicitly mapped solid-color and text properties.
- Image content only after a separate resource-upload and media-fill profile is admitted; profile 0 records existing image/video fills as unsupported.
Canva Docs, whiteboards and other unbounded pages, all-pages editing, tables, embeds, video, gradients, unavailable fonts, locked content, app elements, behaviors and preview APIs are excluded. All-pages editing may be reconsidered under a new profile after the documentation and package used for review agree on its stable status.
Transaction boundary
The app opens one session, rejects unsupported or locked input, creates a
complete normalized snapshot or mutation plan, validates every bound, asks for
user confirmation and calls sync once. A failed validation, expired session,
conflict or API error leaves the live design unchanged. A successful import is
one Canva undo action. The app does not replace the complete page with an
opaque app element or flattened screenshot.
Sessions are limited to one minute by the host. Profile 0 therefore caps one
page at 16,384 traversed elements, 4,096 rich-text code points per element,
1 MiB of normalized metadata and a 16 MiB NUIF input. Limit-plus-one inputs must
fail before sync. These are candidate limits pending live time and allocation
calibration.
Browser and package boundary
The app iframe may bundle nuif-wasm-api-0; Canva’s current CSP permits
packaged WebAssembly but forbids third-party scripts, nested frames and workers.
The module parses and validates NUIF locally. Only the Apps SDK reads or mutates
Canva objects. The build declares no remote code, and any optional backend is a
separate authenticated feature with explicit data disclosure.
Correspondence and fidelity
Stable Canva page identifiers and session object references are recorded in a
HostAdapterReport. No persistence of a custom NUIF entity identifier is
claimed until a generally available Canva metadata surface survives duplicate,
reorder, close/reopen and copy trials. Missing or duplicate portable identity
is repaired in the NUIF mapping and reported, never silently reused.
Every supported property has a correspondence entry. Every unsupported Canva element or property produces item-level fidelity. Images represented as rect fills, rich-text range conversion and layering by list order are mapped as Canva semantics rather than forced into a fictitious vendor-neutral object.
Connect API boundary
Connect API imports and exports are a separate server-side workflow, not this host profile. They require OAuth scopes and asynchronous jobs. The current import list includes Affinity files but not NUIF; current exports do not include NUIF. SVG or PDF bridges must therefore carry an explicit lossy fidelity report. Native NUIF support requires Canva to admit a NUIF media type and publish the associated semantic contract.
Required promotion evidence
- checked-in normalized current-page fixtures and canonical expected NUIF;
- exact repeated pure mapping in both declared directions;
- official SDK typecheck and deterministic single-file review bundle;
- CSP audit proving no remote code, workers or nested frames;
- locked, unsupported, expired-session, conflict and limit-plus-one failures before mutation;
- one-sync/one-undo and cancellation trials in a named Canva host/API version;
- public-review checklist, privacy disclosure and developer-verification readiness without credentials in the repository.
Marketplace submission and publication remain owner-authenticated operations. CI may produce the review bundle but does not submit or release it.
Retentive DTCG flat scalar-token profile zero
Status: executable integrated profile (nuif-dtcg-scalar-0). The library,
public CLI, blocking xtask gate and CI artifact path exercise the same
profile boundary.
Model projection
The profile is a bounded subset of the Design Tokens Format Module 2025.10:
- the root is a flat JSON object with one required
$extensionsmember and zero or more token members; - each token has exactly
$type,$valueand$extensionsmembers; - DTCG
boolean,stringandnumbervalues map to NUIF Boolean, String, Integer and finite Real property values; - token names obey the DTCG name grammar and become NUIF token names;
- root
org.nuifmetadata carries the profile and NUIF document identity; - token
org.nuifmetadata carries stable token identity and avalue_kinddiscriminator so DTCGnumberdoes not collapse NUIF Integer and Real; - unknown third-party members inside root and token
$extensionsobjects are retained byte-for-byte by synchronization.
The profile excludes groups, inherited type, $root, $extends, aliases,
JSON Pointer references, descriptions, deprecation, composite values and
token-local standard members beyond the declared three. These constructs are
not silently treated as scalar tokens. A full DTCG profile requires a token
model RFC for declared type, groups, aliases, descriptions, deprecation and
token-local opaque extensions.
Retentive laws
For a document inside the profile:
import_source(export_document(document).source).document == document.- Document identity and every token identity, name, declared type, value and NUIF value-kind discriminator have UTF-8 byte spans.
- Synchronization validates the retained source and all correspondence spans before applying any edit.
- Changed spans are replaced from the highest byte offset to the lowest.
- The synchronized source re-imports to the requested document exactly.
- Repeated synchronization from the same retained source and edited document produces identical source, edit records and reports.
- Document identity and token inventory changes return typed atomic errors.
Mapped strings, object keys and numbers use one canonical JSON spelling. This restriction makes stale-span comparison unambiguous while whitespace and unknown extension values remain retentive.
Resource and parser contract
Input is UTF-8 and limited to 1 MiB, 4,096 tokens and the serde_json default
recursion limit of 128. Duplicate root and token members fail parsing. Token
identities must be unique. Non-finite values are not valid JSON and do not
enter the model. The crate tests cover all four mapped NUIF scalar variants,
exact export/import, deterministic multi-property synchronization, root and
token extension retention, complete unchanged-byte locality, structural and
unsupported edits, stale source, aliases, duplicate members, excessive depth
and the source byte limit.
The command surface is:
nuif export <input.nuif> dtcg-scalar-0 <output.tokens.json> [report.json];nuif import dtcg-scalar-0 <input.tokens.json> <output.nuif> [report.json];nuif sync dtcg-scalar-0 <retained.tokens.json> <edited.nuif> <output.tokens.json> [report.json].
nuif-dtcg-scalar-0 is accepted as an explicit alias. cargo xtask gate-dtcg adds an eight-edit model trial, one-over token-count case and
public-CLI export/sync/import bridge. It writes target/dtcg-sync-report.json,
target/dtcg-sync-output.tokens.json, target/dtcg-sync-edited.nuif,
target/dtcg-sync-cli-report.json and
target/dtcg-sync-cli-output.tokens.json.
Draft Figma Plugin API profile 0
Status: pure normalized mapping and the compiled source-review shell are
executable as nuif-figma-plugin-snapshot-0; no live host conformance claim.
Profile identifier: nuif-figma-plugin-0.
Primary evidence:
nuif:research:figma,
nuif:research:figma-plugin-and-rest-api-as-automation-surface, and ADR 0008.
Host and scope
- Figma Design through Plugin API
1.0.0;editorType: ["figma"]. - One explicitly loaded
PageNodeper operation. - Import and export of frames, groups, rectangles, ellipses and literal text.
- Solid sRGB fills; ordered containment; freeform position; fixed width/height; frame auto-layout row/column, padding, gap and primary/counter alignment when exactly representable.
- Component/instance, variable, grid-layout, vector-network, effect, interaction and typography fields outside this list receive explicit fidelity entries and are not silently flattened.
The executable snapshot subset is narrower: nodes must be visible with node
opacity 1, auto layout must be packed with fixed dimensions, and text must
carry the exact pinned font identity. General visibility and node opacity are
not first-class fields in the current NUIF model. GRID auto layout, mixed text,
fill/hug sizing and SPACE_BETWEEN are explicit exclusions rather than guessed
lowerings. See SNAPSHOT-PROFILE.md.
The .fig encoding and undocumented multiplayer protocol are excluded.
Identity and correspondence
Each host node correspondence records SceneNode.id as host_object_id and
the affected property name when the entry is property-specific. The plug-in may
store these shared data entries:
| Namespace | Key | Value |
|---|---|---|
nuif | document_id | canonical NUIF document identifier |
nuif | entity_id | canonical NUIF entity identifier |
nuif | profile | nuif-figma-plugin-0 |
The bridge treats plug-in data as correspondence assistance, not authority. It scans the loaded scope before mutation. Missing identifiers are assigned; duplicate identifiers are replaced on every duplicate except the first stable host-tree occurrence. Every repair appears in the host report. No persistence claim is made for copy, paste or duplication until a live fixture proves it.
Import transaction
- The UI iframe reads a user-selected
.nuiffile under the NUIF encoded and semantic limits. - Pure mapping builds a host mutation plan and
HostAdapterReportwithout modifying the file. - The UI presents fidelity totals and all unsupported entries.
- On confirmation, the main thread creates/updates the declared scope. It does
not call
commitUndoduring the plan, so the host treats the run as one undo group when the plug-in closes. - Any exception stops the plan, triggers host undo when needed, and returns a failed report. Atomicity must be proven in the live-host gate.
Export transaction
- Export defaults to the current selection, or the current page when selection is empty; document-wide export is a separate user action.
- The plug-in loads only the required page nodes and records omitted pages.
- Pure mapping emits canonical NUIF plus a host report.
- The UI iframe downloads both files. Export does not mutate the host file.
Resource limits
The profile inherits NUIF profile-zero limits and additionally caps one run at one loaded page, 16,384 traversed nodes, 4,096 UTF-16 code units per text node, 100 kB per shared-data entry, and 16 MiB combined message payload between main thread and UI. Limit-plus-one inputs must fail before host mutation. These are candidate profile limits pending live Figma timing/allocation calibration.
Required fixtures
- covered one-page import/export and repeated-output determinism;
- reorder without identity drift;
- missing and duplicate shared-data identifiers;
- an invisible instance descendant and the traversal-mode report;
- unloaded-page omission versus explicit document-wide load;
- unsupported component, variable, vector, effect and interaction properties;
- unavailable font and mixed-style text;
- resource-limit and user-cancellation cases;
- undo returns the exact pre-import host tree.
The pure mapping, deterministic fixture, CLI bridge and no-network TypeScript
shell are implemented. The checked-in manifest is a template because Figma
assigns plug-in IDs; FIGMA_PLUGIN_ID=<assigned-id> npm run package produces a
reviewer’s local manifest. Publication as a Figma integration remains blocked
until a named live Figma product/version trial proves the required host
fixtures.
Figma plug-in snapshot profile 0
Profile identifier: nuif-figma-plugin-snapshot-0.
Status: executable pure mapping plus a compiled no-network review shell. No live Figma runtime, marketplace package or vendor interoperability claim is included.
The profile is the deterministic boundary between a thin Figma main-thread
shell and the NUIF engine. The shell normalizes public Plugin API objects into
the JSON snapshot defined by nuif-figma::PluginSnapshot; the Rust mapper
converts that snapshot to canonical NUIF. The reverse direction produces a
PluginMutationPlan tree for a shell to apply inside one user-initiated run.
Exact subset
- one selected
FRAMEas the NUIF surface root; - nested
FRAMEandGROUPcontainers; RECTANGLE,ELLIPSEandTEXTleaves;- ordered containment, finite relative position and finite fixed dimensions of
at least
0.01, matching the Figma resize contract; - one optional solid sRGB fill;
- freeform frames or packed horizontal/vertical auto layout with finite non-negative gap/padding and MIN/CENTER/MAX counter-axis alignment;
- visible nodes with node opacity
1; - literal text carrying the exact pinned Ahem Regular SHA-256 identity, positive font size and positive pixel line height;
- optional portable document/entity identifiers plus deterministic repair.
Figma GRID auto layout, wrapping, SPACE_BETWEEN, fill/hug sizing, constraints,
components and instances, variables, mixed text, strokes, effects, blend modes,
rotation, clipping, masks, interactions, images and vector networks are outside
this subset. The snapshot shell must list every active excluded property in
unsupported_properties. Hidden or partially transparent nodes produce an
unsupported appearance entry because the current NUIF model has no first-class
general visibility or node-opacity field. They are never treated as exact.
Identity
nuif_document_id and nuif_entity_id accept canonical 128-bit NUIF identity
strings. A valid unique value is retained exactly. A missing, malformed or
duplicate entity value is replaced by SHA-256 domain-separated derivation over
the profile, host document, page and object IDs. Collision retries add a
big-endian nonce. The same snapshot therefore produces the same repaired IDs;
the report classifies repaired identity as representable, not lossless.
Host object IDs must be non-empty and unique in one snapshot. The report binds
every mapped entity/property to its host object. Root frame page coordinates
are normalized to the surface origin; non-zero root coordinates are
representable rather than lossless.
Limits and failure
- 16 MiB encoded snapshot;
- 16,384 nodes;
- 64 containment levels;
- 4,096 UTF-16 code units per text node;
- 256 KiB combined normalized string payload.
Malformed JSON, duplicate host IDs, non-finite geometry, invalid colours,
dimensions below 0.01, negative spacing, non-default leaf layout, children on leaves and
limit-plus-one inputs fail before returning a document or plan. Export rejects
NUIF properties outside the subset with a property-attributed
HostAdapterReport.
Gate
cargo xtask gate-figma runs the release-mode mapper, CLI bridge, strict
TypeScript check, deterministic shell build and a mock Plugin API fixture
through the Rust importer. It requires repeated snapshot bytes, exact canonical
round trip, explicit loss, duplicate-ID and limit-plus-one rejection, and a
compiled manifest-template bundle with no network domains. The reports are
target/figma-snapshot-report.json and
target/figma-plugin-shell-report.json.
The shell is adapters/figma/plugin. Figma assigns its manifest ID, so CI does
not invent one. Live promotion still requires a Figma product/version record,
host mutation/undo/cancellation trials, page-load and identity persistence
evidence, and a human-confirmed import preview.
Web accessibility projection profile 0
Status: executable bounded research profile (nuif-web-accessibility-0). This
profile projects portable NUIF semantics into inert native HTML and ARIA, then
compares computed role, accessible name and supported state exposure across
pinned Chromium, Firefox and WebKit test engines. It is not an arbitrary ARIA
serializer, a behavior runtime or a claim about branded browsers and native
platform accessibility APIs.
Portable subset
The profile accepts at most 4,096 structurally valid entities and 8,192 relationships. Ten roles are admitted:
button,checkbox,radioandswitchfor the bounded interactive semantic surface;img,main,navigation,paragraph,regionandgroupfor bounded content and landmark semantics.
Native HTML is preferred where its implicit role matches: button, checkbox
and radio input, main, nav, named section and p. switch, img and
group use explicit roles because there is no equivalent element in this
profile. Required-name, prohibited-name and role/state combinations are checked
before output. switch requires an explicit Boolean checked state. A direct
accessible name and a labelled-by relationship cannot both supply the same
node’s name.
The Boolean state subset is deliberately role-specific:
- button:
disabled,expanded,pressed; - checkbox/radio:
checked,disabled,required; - switch:
checked,disabled; - non-widget roles: no state keys in profile 0.
Relationships map by stable entity identifier: labelled-by to
aria-labelledby, described-by to aria-describedby, controls to
aria-controls, owns to aria-owns and flow-to to aria-flowto.
Repeated relationship targets and unnamed labelled-by targets fail closed.
The source order of multiple relationship targets is retained because it can
affect accessible-name computation. owns additionally requires a directed,
acyclic, single-owner graph; multiple semantic parents and owned-tree cycles
are rejected before HTML is emitted.
Direct and referenced names are normalized to the whitespace form produced by the accessible-name algorithm before they become oracle expectations. A name that is empty after normalization fails closed rather than silently becoming an unnamed control.
Security and behavior boundary
Output contains no scripts, external URLs, event handlers or generated application behavior. Native controls retain their built-in focus and state exposure, but the profile does not invent activation results, navigation or business logic. A correct accessibility tree is not evidence of keyboard-flow, interaction or application behavior equivalence.
Unsupported roles, states, ambiguous names, malformed containment and
relationships outside the profile return typed errors without partial output.
The projection keeps data-nuif-id on every element so a foreign observation
can be attributed back to the stable NUIF entity.
Foreign oracle
cargo xtask gate-accessibility generates one eleven-node fixture covering
every admitted role and Boolean state, installs the
exact Playwright 1.62.1 browser set and compares the required role/name/state
subset plus full ARIA snapshots across its Chromium, Firefox and WebKit
engines. The report records package, engine, Node, operating-system and
architecture versions. Required-subset mismatches are classified as semantic
loss; non-required tree differences are retained separately as host-tree
differences.
The current macOS/arm64 trial reports identical bounded snapshots from Chromium 151.0.7922.34, Firefox 153.0 and WebKit 26.5. Hosted Linux evidence is produced by CI and must not be inferred merely from workflow configuration.
Artifacts:
target/accessibility-mapping-static-report.json;target/accessibility-mapping-report.json;target/accessibility-mapping-fixture.html;target/accessibility-mapping-expected.json.
Descriptions as direct semantic strings, numeric/value states, live regions, tables, trees, grids, composite-widget focus, keyboard interactions and native Apple/Microsoft/Linux/mobile mappings require separate model and profile work.
Web behavior projection profile 0
Status: executable bounded research profile (nuif-web-behavior-0). This is a
one-way host lowering from nuif-behavior-state-machine-0 plus
nuif-web-accessibility-0 to native HTML activation and observable DOM/ARIA
effects. It is not a JavaScript interchange format and does not import behavior
from HTML.
Admitted mapping
The complete source behavior program must first pass its own identifier, type, graph, capability and resource checks. The document must also pass the web accessibility projection. This web profile then admits:
activateonly on enabled nativebuttonelements or the button-backedswitchrole;visibility(Boolean)astarget.hidden = !value;- one non-empty
announcement(String)per transition through an unfocusedrole="status",aria-live="polite",aria-atomic="true"region; - ordered guards, sequential set/toggle/effect actions and run-to-completion state changes exactly as defined by the source profile.
Every enabled button/switch in the projected document is bound, including a control with no matching transition. This preserves the source profile’s observable unmatched-event no-op. Checkbox and radio activation are excluded because their native checked-state mutation has no corresponding action in the current behavior model. Disabled transition sources fail before output because native disabled controls do not produce the required activation. Visibility effects that can hide any admitted event source, including through an ancestor, also fail because a later abstract activation could no longer be produced by the native host.
At most one announcement may occur in a transition. Repeated effects of the same kind and target in one transition also fail. These rules prevent multiple abstract effects from being coalesced into a single host observation within one browser task.
Generated runtime and authority
The output contains one fixed interpreter template plus the validated behavior
program encoded as JSON data. JSON embedding escapes <, >, &, U+2028 and
U+2029, so behavior strings cannot terminate the script element. The runtime
uses no eval, Function, dynamic import, request API, timer, event-handler
attribute or authored code. A document-level Content Security Policy permits
the exact UTF-8 script body by its generated SHA-256 hash and otherwise denies
scripts, connections, images, fonts, forms and base-URL changes.
An HTTP Content-Security-Policy response header remains preferable when a host
serves the output; the generated meta policy makes the self-contained fixture
enforceable and testable. Hosts embedding the body into their own page must
merge policies themselves rather than assuming this document policy transfers.
Native commandfor was considered. Its built-ins address popovers and dialogs;
custom commands still require a script listener and do not represent the
profile’s state, guards, variables, visibility and announcement effects. A
small generated interpreter is therefore the narrower exact mapping.
Foreign browser oracle
cargo xtask gate-web-behavior generates the same two-state, five-event fixture
used by the independent trace gate, computes its reference Rust trace, and
drives separate pointer and keyboard sequences through exact Playwright 1.62.1
Chromium, Firefox and WebKit engines. The keyboard sequence alternates Enter
and Space. After every event it compares selected transition, target state,
retained visibility, live-region text and stable announcement target. It also
retains status/body ARIA snapshots, browser versions, Node/OS/architecture and
runtime errors.
The current macOS/arm64 trial passes Chromium 151.0.7922.34, Firefox 153.0 and WebKit 26.5 for all five events. Hosted Linux evidence is produced by CI and is not inferred from workflow configuration.
Artifacts:
target/web-behavior-static-report.json;target/web-behavior-report.json;target/web-behavior-fixture.html;target/web-behavior-expected.json.
The gate observes browser DOM state and the browser accessibility tree. It does not establish branded-browser behavior, screen-reader speech/timing, native platform UI, focus choreography, checkbox/radio semantics, navigation, animation, timers, networking, filesystem access or arbitrary host business logic. Those require separate profiles and oracles.
Retentive HTML/CSS profile 0
Status: executable bounded research profile (nuif-html-css-0). This profile proves a source-preserving synchronization mechanism; it is not a claim that arbitrary HTML/CSS or the complete NUIF v0 responsive-card fixture is representable.
Representable NUIF subset
- one document root and no relations, extensions or extension declarations;
- finite real-valued length tokens whose names contain only ASCII letters, digits,
.,_or-; - container and text entities only;
- containers use fixed pixel width/height, stack layout, row/column flow, finite gap and four padding edges, start/center/end/stretch alignment and one
token.spacingbinding; - text uses fill width, intrinsic height, literal content and pinned font name/hash/size/line-height; all other text-authored state is default;
- containment is represented by DOM nesting and identity by 32-digit
data-nuif-idvalues.
Every condition outside this list fails export or synchronization with a Fidelity::Unsupported item carrying a document/entity/token target and JSON pointer. The adapter never silently emits a fallback and calls it lossless.
Source contract
The HTML document declares data-nuif-profile="nuif-html-css-0" and data-nuif-document. Mapped entity elements declare data-nuif-id, data-nuif-kind and their profile fields. A single style[data-nuif-styles] block contains real CSS declarations for token custom properties, fixed sizing and stack layout.
Tree-sitter 0.26.10 validates the outer HTML with tree-sitter-html 0.23.2 and the injected stylesheet with tree-sitter-css 0.25.0. Import retains byte ranges for every editable scalar. Input is UTF-8 and capped at 1 MiB before parsing.
Unmarked comments, elements and CSS declarations are outside the semantic projection. Import ignores them and synchronization preserves them byte-for-byte. They are not promoted to NUIF entities and no semantic claim is made about them.
Laws and edit algorithm
For documents in the declared subset:
import(export(document)).document == documentexactly.- An unchanged document produces no source edits.
- A mapped edit replaces only its recorded byte span. Replacements are applied from the highest byte offset to the lowest, so earlier spans remain valid.
- Before replacement, each retained span must still equal the profile encoding of the imported value; otherwise synchronization returns
StaleSpanwithout partial output. - The synchronized source is re-imported and must equal the edited document exactly before it is returned.
- Structural changes, added/removed mapped properties and semantics outside the subset return
UnmappedChangeswith property-level fidelity.
Gate F changes one token value, four padding edges and escaped text. Its machine report proves exact re-import, repeat-identical edits/output and byte identity of every region outside those six spans while preserving inserted HTML/CSS comments and an unmapped element.
Automation
nuif export <input.nuif> html-css-0 <output.html> [report.json]nuif import html-css-0 <input.html> <output.nuif> [report.json]nuif sync html-css-0 <retained.html> <edited.nuif> <output.html> [report.json]cargo xtask gate-fwritestarget/html-sync-report.jsonandtarget/html-sync-output.html.
The full v0 responsive card still requires surface/component/instance/shape/unknown-kind, responsive-rule and opaque-extension mappings. Those remain explicit next-profile work rather than hidden metadata.
Retentive HTML/CSS v0 model profile
Status: executable research profile (nuif-html-css-v0). It carries the complete NUIF v0 responsive-card model through HTML/CSS and applies scalar semantic changes as byte-local source edits. It is not an arbitrary-HTML importer or a claim that browsers render every preserved NUIF kind.
Model projection
- document identity, relations, extension declarations and document extensions are quoted JSON attributes on the marked
htmlelement; - token identity is part of a CSS custom-property name, while safe token names and finite real or single-atom string values have retained source spans;
- every entity is an identity-bearing
div; DOM nesting is the canonical roots/children order; - kind, optional name, authored property values, semantics and opaque extensions are quoted JSON attributes;
- width/height intent, position, layout family/direction/gap/padding/alignment and sRGB fill are real CSS declarations;
- text content is escaped element text and its pinned font metadata is a quoted JSON attribute;
- width-conditioned direction/gap overrides have both a mapped JSON rule and a marked
@mediablock. Import requires the rendered block to equal the rule, preventing derived CSS from drifting silently.
The responsive mapping accepts min_width and/or max_width, direction and gap. Theme predicates and responsive width/height overrides are outside this profile. Token names use ASCII letters, digits, ., _ and -; string token values must be safe single CSS atoms. Input is UTF-8 and capped at 1 MiB.
All model fields used by v0-responsive-card round-trip exactly. Shape kind, path identity, component identity/reference, unknown-kind payloads and extension bytes survive, but target fidelity remains explicit:
- paths have no authored geometry in v0 and are not rendered by this adapter;
- instances retain their component reference but are not materialized into browser DOM;
- unknown kinds and opaque extensions are
preserved_unrenderable.
These target limitations coexist with lossless source correspondences. A field may be stored exactly while still lacking browser behavior.
Retentive laws
For documents inside the profile:
import_v0_source(export_v0_document(document).source).document == document.- Every editable scalar has a unique half-open UTF-8 byte span. Before synchronization, its retained bytes must equal the imported value’s profile encoding.
- Changed spans are replaced from the end of the source toward the beginning. No formatter or whole-file generator touches the retained source.
- The result is reparsed as HTML and CSS and must import to the requested edited document exactly.
- A repeated synchronization from the same retained source and edited document produces the same source and edit list.
- Root/entity insertion, removal or reorder; mapped-property set changes; stale values; unsupported profile values; inconsistent derived CSS; malformed syntax; and oversized input return typed errors without partial output.
Unmarked comments, elements, rules and declarations are retained byte-for-byte and remain outside the NUIF semantic projection. Such CSS can still affect a browser through the cascade; therefore exact NUIF round-trip does not imply browser-render equivalence after arbitrary unmapped CSS is inserted.
Automated evidence
cargo xtask gate-f-v0 runs two paths:
- the model trial exports the eight-entity responsive card, injects unmapped CSS/HTML, changes a token, four padding edges, escaped text and one responsive rule, then requires eight exact span edits, exact re-import, repeat determinism, byte identity everywhere else and exact opaque payload survival;
- the editor bridge generates the fixture through the CLI, edits name and width through the headless semantic editor, synchronizes through the CLI, imports through the CLI and requires canonical NUIF byte identity with the editor output.
Negative trials cover unsupported token values, structural edits, ordinary stale spans, derived-responsive CSS drift and the one-over source limit. The report also requires property-attributed fidelity for path, instance, unknown-kind and extension target limitations.
Artifacts:
target/html-sync-v0-report.jsonandtarget/html-sync-v0-output.html;target/html-sync-v0-editor-report.jsonandtarget/html-sync-v0-editor-output.html.
CLI:
nuif export <input.nuif> html-css-v0 <output.html> [report.json]nuif import html-css-v0 <input.html> <output.nuif> [report.json]nuif sync html-css-v0 <retained.html> <edited.nuif> <output.html> [report.json]
The smaller nuif-html-css-0 profile remains as an independently exercised mechanism proof with a deliberately narrow rejection boundary.
Retentive Penpot v3 package profile zero
Status: executable integrated profile (nuif-penpot-v3-0). The library,
public CLI, blocking xtask gate and retained CI artifacts exercise the same
profile boundary.
Foreign package boundary
The profile reads the current Penpot v3 ZIP-and-JSON representation with manifest version 1 and file data version 67. It maps exactly one file, one page, Penpot’s root frame, one board and that board’s direct rectangle, ellipse and text children. Manifest library relations must be absent. The legacy per-shape-member representation is required; the opt-in compact page representation is not accepted.
Penpot UUIDs map to NUIF document and entity identities. A page UUID is package
structure rather than a NUIF entity. The board maps to a positive fixed-size
surface. Board and leaf geometry maps finite positions and non-negative fixed
dimensions. An absent leaf fill remains absent; a present fill is one opaque
8-bit-exact sRGB colour. Text contains one literal run with positive finite
font size and line height. The pinned Ahem name and SHA-256 are retained in the
shape’s org-nuif plug-in data because the exported Penpot text object does not
otherwise carry NUIF’s content-addressed font identity.
Every mapped entity requires a non-empty name. Tokens, relations, extensions, portable semantics, responsive values, non-default layout, arbitrary authored values, nested mapped children, paths, groups, images, boolean/SVG-raw shapes, strokes, gradients, effects, constraints, grids, components, variants, libraries, interactions, multiple text runs and media are outside this profile. Their absence is a profile boundary, not a claim that Penpot lacks them.
Retentive package laws
For a document and package inside the profile:
import_package(export_document(document).bytes).document == document.- Repeated export produces identical ZIP bytes. Exported member timestamps are
fixed to the earliest ZIP date and file permissions are fixed. Native JSON
members below 4 KiB are stored without compression; larger members use
Deflate. Imported packages retain each member’s original method. The
manifest identifies the independently versioned adapter crate as
nuif-penpot/<crate-version>; it does not borrow the editor version. - An unchanged synchronization returns the original archive byte-for-byte, including central-directory representation.
- Mapped JSON scalars carry member-qualified UTF-8 byte spans. A change patches only those spans, from the highest offset to the lowest within each member.
- Member payloads without mapped edits remain byte-identical. Unknown members
are retained and reported as
preserved_unrenderable; unknown JSON fields are reported as unsupported while their bytes remain outside mapped spans. - A changed package is rebuilt, re-imported and required to equal the requested document exactly before any result is returned.
- Identity, containment, child order, kind, optional name/fill inventory or
text metadata-shape changes fail atomically as
UnmappedChanges.
ZIP container metadata may change when a mapped edit requires rebuilding the archive. The payload-locality law does not promise central-directory byte identity after an edit. The unchanged path has the stronger whole-archive law.
Resource and security contract
The complete package is limited to 16 MiB, 4,096 members, 4 MiB per expanded member and 32 MiB total expanded data. A member’s advertised expanded size may not exceed 1,000 times its compressed size. JSON is limited to depth 64 and 131,072 values.
Member paths must be ASCII, forward-slash-only and enclosed relative paths.
Duplicate names, directories, symbolic links and encrypted entries are
rejected. Only stored and Deflate compression methods are accepted. The adapter
uses zip 8.6.0 with default features disabled and only the Rust-backed Deflate
feature enabled. It reads every member into bounded memory and never extracts
an archive to the filesystem. This design also avoids the filesystem-extraction
surface described by CVE-2025-29787; the selected crate version is newer than
the advisory’s 2.3.0 patched boundary.
Executable evidence
The foreign fixture is produced by the official @penpot/library 1.1.0 npm
package and is committed under conformance/foreign/penpot/. The crate test
imports that package to the expected canonical model and proves no-op archive
identity. The profile runner additionally checks deterministic native export,
eight scalar edits, exact re-import, untouched member payloads, an opaque binary
member, an unknown JSON field, structural rejection, traversal rejection and
one-over package/member limits.
Run cargo xtask gate-penpot. The command also exercises the public CLI:
nuif export <input.nuif> penpot-v3-0 <output.penpot> [report.json];nuif import penpot-v3-0 <input.penpot> <output.nuif> [report.json];nuif sync penpot-v3-0 <retained.penpot> <edited.nuif> <output.penpot> [report.json].
nuif-penpot-v3-0 is accepted as an explicit alias. Evidence is written to
target/penpot-sync-report.json, target/penpot-sync-output.penpot,
target/penpot-sync-edited.nuif, target/penpot-sync-cli-report.json and
target/penpot-sync-cli-output.penpot.
Primary format sources are Penpot’s technical file-format
reference,
the official library source
and its binfile v3 implementation.
nuif-react-jsx-0 profile
Status: experimental executable source profile. It is not a React runtime, browser-layout or arbitrary-JSX equivalence claim.
Purpose
This profile maps a small, statically decidable React JSX subtree to NUIF and back while retaining source outside mapped scalar byte spans. Import never executes JavaScript. The profile exists for generated components and controlled developer tooling, not for interpreting an application.
The source must contain exactly one export default function with no
parameters and a body containing only a direct return of the marked JSX root.
Comments and unrelated module declarations outside that function are retained.
Mapped source
- lowercase intrinsic
<div>elements map to named NUIF containers; - lowercase intrinsic
<span>elements map to named NUIF literal text; - every mapped element has literal double-quoted
data-nuif-id,data-nuif-kindanddata-nuif-nameattributes; - the root additionally has exact
data-nuif-profileanddata-nuif-documentliterals; - container
styleis one object literal with numericwidth,height,gapand per-edge padding, fixedboxSizing: "border-box", fixeddisplay: "flex", literalflexDirectionand literalalignItems; - text
stylehas fixedwidth: "100%", the pinned literalfontFamily, numericfontSizeand a literal pixellineHeight; - text carries the exact font digest in
data-nuif-font-sha256and one raw JSX text run using the profile’s canonical entity escapes; - whitespace between mapped container children is formatting, not a NUIF node.
React documents numeric style values as receiving property-specific unit
handling, so this profile uses numbers only for properties React interprets as
CSS pixels. It uses a "<number>px" string for lineHeight, where a number
would instead be unitless. Style keys follow React’s camel-cased DOM property
vocabulary.
Retentive synchronization law
Import records the exact UTF-8 byte range of every mapped identity, name, layout scalar, font scalar and text run. Synchronization:
- proves the retained spans still contain the values imported from the original document;
- rejects entity insertion, deletion or child reordering;
- renders before and after documents through the same exporter;
- applies only changed mapped spans from the end of the source toward the beginning;
- reparses the result and requires exact canonical NUIF equality.
All bytes outside the returned edits remain byte-identical. A changed comment, unrelated import/export or other module-level user region is preserved. A caller-modified mapped span fails as stale rather than being overwritten.
Limits
| Resource | Limit |
|---|---|
| UTF-8 source | 1 MiB |
| JavaScript/JSX syntax nodes | 16,384 |
| Mapped JSX element depth | 128 |
The common NUIF model limits apply after import. Limits are checked before constructing the retained model, and every failure is atomic.
Rejected constructs
The profile rejects component tags, fragments, self-closing mapped elements,
spreads, event handlers, computed or extra style properties, variables,
member access, calls, templates, arrays, conditions, loops, hooks, state,
context, dangerouslySetInnerHTML, nested markup in text and any expression
other than the one profile-owned style object containing literal values. It
also rejects TypeScript/TSX syntax; a later TSX profile needs its own grammar,
toolchain and fixtures.
Unmarked JSX elsewhere in a module is outside the extracted document. The adapter preserves those bytes but makes no statement about their runtime relationship to the marked component.
Executable evidence
cargo xtask gate-react runs unit and release profile tests, 11 mapped scalar
edits, repeated synchronization, exact unchanged-byte-complement comparison,
typed stale/unsupported/structural failures and eleven hostile or excluded-source
trials. It writes:
target/react-sync-report.json;target/react-sync-output.jsx;target/react-sync-edited.nuif.json.
Tree-sitter JavaScript 0.25.0 supplies concrete JSX syntax and byte ranges. It is a syntax implementation, not a React renderer. The accepted syntax follows the React documentation for JSX, intrinsic DOM props and literal style objects; no foreign runtime comparison is claimed by this first profile.
nuif-svelte-static-0 profile
Status: experimental executable source profile. It is not a Svelte runtime, browser-layout or arbitrary-component equivalence claim.
Purpose
This profile maps one statically decidable .svelte component to NUIF and back
while retaining comments, whitespace and formatting outside mapped scalar byte
spans. Import never executes JavaScript. The profile is intended for generated
components and controlled developer tooling, not for interpreting an
application.
The component contains exactly one marked top-level regular element. Top-level whitespace and HTML comments are retained. Any other top-level markup, script, style or special node is rejected because it can render or execute alongside the marked root.
Mapped source
- regular
<div>elements map to named NUIF containers; - regular
<span>elements map to named NUIF literal text; - every mapped element has literal double-quoted
data-nuif-id,data-nuif-kindanddata-nuif-nameattributes; - the root additionally has exact
data-nuif-profileanddata-nuif-documentliterals; - container
styleis one literal declaration list with pixelwidth,height,gapand per-edge padding, plus fixedbox-sizing: border-box, fixeddisplay: flex, literalflex-directionand literalalign-items; - text
stylehas fixedwidth: 100%, the pinned literalfont-family, and pixelfont-sizeandline-height; - text carries the exact font digest in
data-nuif-font-sha256and one literal text run using the profile’s canonical HTML entity escapes; - whitespace and comments between mapped container children are formatting, not NUIF nodes.
Inline style is intentional. It makes scalar ownership unambiguous and avoids claiming CSS cascade, selector, specificity or Svelte scope-hash semantics. A component-CSS profile requires its own version and conformance suite.
Retentive synchronization law
Import records the exact UTF-8 byte range of every mapped identity, name, layout scalar, font scalar and text run. Synchronization:
- proves the retained spans still contain the values imported from the original document;
- rejects entity insertion, deletion or child reordering;
- renders before and after documents through the same exporter;
- compares all three correspondence inventories through the shared
nuif-adapterscalar planner; - applies only changed mapped spans from the end of the source;
- reparses the result and requires exact canonical NUIF equality.
All bytes outside the returned edits remain byte-identical. A caller-modified mapped span fails as stale rather than being overwritten. Every failure is atomic.
Limits
| Resource | Limit |
|---|---|
| UTF-8 source | 1 MiB |
| Svelte syntax nodes | 16,384 |
| Mapped element depth | 128 |
The common NUIF model limits apply after import.
Rejected constructs
The profile rejects components, special elements, self-closing mapped elements, scripts, module scripts, component CSS, preprocessors, expressions, blocks, snippets, render/raw tags, spreads, shorthand attributes, directives, events, bindings, actions, transitions, animations, class/style directives, extra properties, nested markup in text and noncanonical entity escapes.
Executable evidence
cargo xtask gate-svelte runs unit and release profile tests, 11 mapped scalar
edits, repeated synchronization, exact unchanged-byte-complement comparison,
typed stale/unsupported/structural failures, 13 hostile or excluded-source
trials and a public CLI export/import/sync bridge. It then installs the exact
lockfile with lifecycle scripts disabled and parses and compiles both direct and
CLI synchronized sources through official svelte/compiler 5.57.0 in modern
AST mode. Compiler warnings fail the gate.
The gate writes:
target/svelte-sync-report.json;target/svelte-sync-output.svelte;target/svelte-sync-edited.nuif.json;target/svelte-sync-cli-report.json;target/svelte-sync-cli-output.svelte;target/svelte-compiler-oracle-report.json.
Tree-sitter Svelte supplies concrete syntax and byte ranges. The official compiler is the separate semantic oracle. Neither is treated as a Svelte renderer, and no runtime pixel-equivalence claim is made.
Retentive SVG basic-shape profile zero
Status: executable integrated profile (nuif-svg-0). The library, public CLI,
blocking xtask gate and CI artifact path exercise the same profile boundary.
Model projection
- one positive, fixed-size surface maps to the root SVG element and its
viewBox; - freeform containers map to
gelements; - fixed-size, positioned rectangles map to
rectgeometry; - fixed-size, positioned ellipses retain NUIF bounding-box attributes and map
derived centre/radius geometry to
ellipseattributes; - fixed-size, positioned text maps to one literal SVG text node with the pinned font name, content hash, font size and line height;
- absent fill maps to
none; present fill is opaque sRGB with 8-bit-exact channels and maps to lowercase six-digit hexadecimal notation; - entity name, role and accessible name map to
data-nuif-name,roleandaria-label; data-nuif-document,data-nuif-id,data-nuif-kindanddata-nuif-profileretain canonical identities and the profile marker.
The profile excludes tokens, relations, extensions, responsive rules, layout families other than default freeform, property values, semantic states, paths, images, components, instances and unknown kinds. Paths, transforms, CSS, strokes, gradients, patterns, clipping, masks, filters, animation, scripts, external resources, text spans and per-character positioning require separate profiles.
Retentive laws
For a document inside the profile:
import_source(export_document(document).source).document == document.- Each mapped scalar has a UTF-8 byte span. Derived ellipse geometry uses multiple ordered correspondence records against the authored geometry.
- Synchronization compares every retained scalar with a canonical export of the imported document before applying any edit.
- Changed spans are replaced from the highest byte offset to the lowest.
- The synchronized source is re-imported and must equal the requested edited document exactly.
- Repeated synchronization from the same retained source and edited document produces identical source, edit records and reports.
- Containment, order, kind, optional mapped-property inventory and unsupported semantic changes return typed errors without partial output.
Unmarked elements, attributes, comments and processing instructions are outside the semantic projection and remain byte-identical during synchronization. They can affect SVG rendering. Exact NUIF round trips therefore do not imply visual equivalence after arbitrary unmarked SVG is inserted.
Resource and parser contract
Input is UTF-8 and limited to 1 MiB. roxmltree 0.21.1 parses at most 16,384
nodes with DTD parsing disabled and no external entity resolver. Mapped elements
must use the SVG namespace and be direct children of mapped surface/container
elements. Duplicate identities, inconsistent kind/tag pairs, non-canonical
numbers, non-canonical colors and inconsistent derived ellipse geometry fail
import.
The crate tests and cargo xtask gate-svg cover exact export/import,
deterministic seven-span synchronization, escaped text, unmarked-source
locality, stale spans, structural rejection, property-attributed unsupported
paint, inconsistent derived geometry, DTD rejection, the XML-node limit and
the source byte limit. The gate also performs export, edited synchronization
and exact canonical re-import through the public CLI.
The command surface is:
nuif export <input.nuif> svg-0 <output.svg> [report.json];nuif import svg-0 <input.svg> <output.nuif> [report.json];nuif sync svg-0 <retained.svg> <edited.nuif> <output.svg> [report.json].
nuif-svg-0 is accepted as an explicit alias. The gate writes
target/svg-sync-report.json, target/svg-sync-output.svg,
target/svg-sync-edited.nuif, target/svg-sync-cli-report.json and
target/svg-sync-cli-output.svg.
Research record index
This page is generated from 152 registered source records.
Status inventory
| Status | Records |
|---|---|
reviewed | 122 |
verified | 30 |
Records
Research corpus
research/ is a machine-readable evidence corpus, not a loose notes folder. AUDIT.md records the current accuracy/alignment review and the gated plan that turns claims into falsifiable work.
Evidence states
seed— discovered, not fully reviewed;reviewed— source identity and relevance checked, but not every material claim is necessarily verified at its locator;verified— material claims are checked against primary locators and implementation claims have reproducible evidence;supersededandrejected— retained with typed relations so history is not silently rewritten.
Confidence and status are independent. A high-confidence reviewed record is not described as verified.
Required properties
Each research record has a stable ID, source identity, retrieval date, tags, confidence, claim links, typed relations and repository links. Verified records additionally require Summary, Evidence, Mechanism, NUIF relevance and Open questions sections. Exact sections, pages, versions or commits support non-obvious source claims; synthesis is separated from source statements.
Layout
items/— one durable record per source or synthesized subject;index.yaml— architectural claims and topic-to-record map;questions.yaml— open and decided research questions;experiments/index.yaml— reproducible investigations and their executable artifacts;coverage.yaml— coverage status per architectural front;schema/— record schemas;AUDIT.mdandroadmap.md— current audit and gated research process.
Important relationships must be structural. tools/research/validate.sh checks record schema, identifiers, claims, relations, topics, questions, experiments, coverage links and artifact paths. Network source-health checks are periodic rather than part of offline conformance.
Research audit and corrected base plan
Audit date: 2026-08-29. Inventory synchronized: 2026-08-31. Current scope: the research index, 152 substantive source records plus the record template, questions, coverage map, experiments, whitepaper synthesis, accepted RFCs and ADRs, draft specification, conformance design and executable seams. The current record states are 122 reviewed, 30 verified and 1 seed template.
Outcome
The architectural thesis remains worth testing: stable identity, authored and resolved state, typed operations, explicit fidelity and opaque preservation form a coherent portability model. The corpus has unusually broad prior-art coverage and the newer records generally carry precise primary-source locators. It did not, however, justify calling the plan executable or the evidence verified. Before this audit, 97 records were reviewed, none was verified, every registered experiment was only planned, the Rust workspace had zero tests, and every substantive CLI command returned not_implemented.
The research base is solid enough to continue only under the gates below. It is not evidence that the full NUIF thesis is true, and it is not a standards-readiness claim.
Material findings
| Area | Finding | Disposition |
|---|---|---|
| Evidence state | reviewed and verified were effectively conflated in synthesis. Many early records summarize reputable sources but do not have an Evidence section with claim-level locators. | The corpus now states the distinction explicitly. Only source claims checked at their locator and backed by a regression may be verified. |
| Research graph | Topic aliases such as wgpu, harfbuzz, automerge, yjs, cbor, protobuf and kiwi did not resolve to record files. The validator ignored topic entries, questions and most experiment links. | Topic entries now resolve to actual records. Validator coverage is expanded; unresolved graph identifiers fail CI. |
| Canonical values | RFC 0005 collapsed integral reals into integers despite the logical model distinguishing the kinds, and incorrectly equated UTF-8 key order with CBOR encoded-key order. | RFC 0008 corrects both points. Codec regressions prove numeric-kind separation and the "z"/"aa" order counterexample. |
| Namespace grammar | NUIF_*, EXT_* and VENDOR_probe contradicted the lowercase identifier grammar in RFC 0005. | Extension lifecycle names are nuif.*, ext.* and collision-resistant lowercase vendor namespaces; the v0 probe is vendor.probe. |
| Accepted decisions versus code | RFCs 0006 and 0007 required anchors, unknown kinds and typed opaque payloads, while the code still used integer indices and byte vectors. | Stable anchors, typed explicit Unknown wrappers, opaque bytes, atomic apply, stale-base checks, replay and inverses are implemented. Automatic conversion of arbitrary future wire discriminants and namespace-registry authorization for SetUnknownPayload are not; RFC 0007 is therefore only partially implemented. |
| Oracle independence | The harness listed the reference implementation as oracle for several properties without separating self-consistency from independent correctness. | Reports identify oracle class. Codec, replay and inverse tests are metamorphic. Gate C now supplies Taffy and pinned Chrome foreign references for the declared CSS-compatible layout subset; adapters remain experimental until a foreign round trip is wired in. |
| Layout thresholds | A global < 0.1 px browser threshold and fixed visual thresholds were copied from prior systems without a NUIF calibration dataset. | Gate C now stores a bound per fixture: the measured Taffy/browser maximum rounded upward to 0.01 px and capped at 0.1 px. Exact foreign agreement retains a zero bound. These empirical bounds apply only to the pinned browser/platform report and are not normative across platforms. |
| CPU exactness | “CPU f32, tolerance 0 across operating systems” was asserted before a pinned math, font and raster pipeline existed. | Exactness is limited to the declared CPU profile 0. It pins color, coverage, composition, font, shaping, hard-line layout, outlines and grayscale masks; rectangle, ellipse and text hashes agree on the recorded macOS/aarch64, Linux/aarch64 and Linux/x86_64 matrix. Untested platforms and future visual operations are not covered. |
| Resource limits | Depth 1024 and one million nodes were listed without memory/time measurements. | RFC 0009 replaces them with measured profile-0 byte, syntax, semantic, diagnostic, allocation and time bounds. Orthogonal image and static-font profiles now publish measured ceilings; broader media, path and GPU budgets remain future-profile work. |
| Package/assets claim | The roadmap called package/assets complete although .nuif fixtures are bare canonical documents and profile 0 rejects images. | Phase 4 is split. RFC 0010 is proposed; package, image and font profiles require independent experiments before acceptance. |
| Capture versus inference | Static source synchronization, browser observation and screenshot reconstruction were described under one broad inference front. | RFC 0011/specification 14 define separate evidence classes, fidelity ceilings, typed-operation boundaries and planned capture/reconstruction/calibration gates. |
| Training proposal | Distillation and low-rank adaptation had no frozen evaluator, rights-cleared trace contract or untuned baseline. | Training is conditional Gate L work. Baseline, closed-loop, calibration and artifact/data governance precede any model adaptation. |
| Editor stack | ADR 0006 selects an unreleased Masonry revision with acknowledged API churn. The choice is plausible but not yet verified in this repository. | The stable boundary is the headless EditorDriver and accessibility action contract. Masonry is a replaceable shell client and cannot change document semantics. |
| Scope and adoption | The prior plan attempted model, layout, rendering, text, source synchronization, adapters, collaboration and a full editor before proving the hard round trip. | Work is gated by the v0 falsifier. No collaboration or broad GUI expansion precedes codec, responsive layout, opaque preservation and one minimal source patch. |
Evidence confidence rules
seed: discovered, not fully reviewed.reviewed: source identity and broad relevance checked; individual source-derived claims may still need locator verification.verified: every material source-derived claim has a primary locator, conflicts are recorded, and any implementation claim has a reproducible check or fixture.superseded: retained for history and linked to its replacement.rejected: evaluated and excluded, with the reason retained.
Confidence is not a substitute for status. A 0.99 reviewed record is not verified merely because its source is authoritative.
Gated base research plan
Gate A — evidence integrity
Exit only when the research validator resolves every record, claim, topic, question, experiment and artifact link; every accepted RFC has primary evidence; contradictions use contradicts/supersedes; and claims described as verified have locator-level evidence. Source-health checks are periodic and non-normative because network availability must not make conformance nondeterministic.
Gate B — canonical model, operations and encodings (complete)
Exit metrics:
- structural validation covers identity, reachability, parent uniqueness, cycles, relation endpoints, token references, version handling, finite numbers and extension declarations;
- seeded operation replay and inverse restoration run for at least 10,000 generated patches with the failing seed recorded;
- canonical text and CBOR reach byte fixpoints; integer/real kinds, negative real zero, map order and opaque bytes have positive and negative fixtures;
- hostile inputs are rejected under measured byte, depth, node and time budgets.
Gate C — responsive layout falsifier (complete)
Exit metrics:
- the v0 card resolves at 360, 768 and 1440 px with stable identity and declared responsive direction changes;
- a generated CSS-compatible subset is compared with a pinned browser and Taffy; every divergence is classified as schema loss, evaluator defect, target difference or implementation-defined behavior;
- tolerances are derived from the measured corpus and are stored per fixture, never as an unexplained global constant.
Gate D — visual and text profile (complete for profile 0)
Exit metrics:
- the CPU profile defines every supported operation by value and produces repeatable fixture bytes on the CI matrix;
- fonts, Unicode data, shaping options and raster parameters are content-addressed;
- text-layout divergence and raster divergence are reported separately;
- unsupported paints/effects create fidelity records and never disappear.
Gate E — editor/CLI parity (complete for profile 0)
Exit metrics:
- the v0 fixture can be constructed and edited through semantic editor actions without coordinate-based widget lookup;
- the editor operation log, direct API calls and CLI replay produce one canonical hash;
- snapshots include canonical document, context, layout, scene, raster and machine report;
- shell-specific screenshot failures cannot redefine model or renderer semantics.
Evidence: conformance/fixtures/v0-responsive-card/editor-authoring.jsonl starts from Document::empty, sets the document’s extension declarations and three tokens, and inserts all eight entities by author identity and semantic anchors. cargo xtask editor-trial requires exact bytes and the same canonical hash from direct fixture generation, editor state and replayed operation log. It also validates both the neighboring-edit output and the fully authored output, then writes target/editor-authoring-report.json plus a snapshot directory containing input.nuif, context, layout, scene, CPU PNG and a fidelity report. CI runs that one entry point and uploads both artifacts. Every entity is asserted present in the accessibility tree; no pointer coordinate is used.
Gate F — one real source synchronization path (complete; full-v0 follow-on complete)
Exit metrics:
- HTML/CSS is imported/exported for a declared representable subset;
- a text, token and padding edit changes only mapped source spans plus declared formatter effects;
- comments and unmapped source regions survive;
- every mismatch has an entity/property-level fidelity entry.
Evidence: nuif-html-css-0 uses pinned Tree-sitter HTML and CSS grammars, records 25 scalar correspondences and exactly re-imports its container/text/finite-token profile. cargo xtask gate-f changes one token value, four padding edges and escaped text, then asserts that exactly those six spans changed and every other source byte remained identical. Inserted HTML/CSS comments and an unmapped <aside> survive. A second synchronization produces identical source and edit records. Unsupported fill, stale-span and one-over-size trials return their named typed failures; unsupported changes include entity identity and JSON pointer. This independently retains the original narrow Gate F boundary.
The follow-on nuif-html-css-v0 profile carries the complete eight-entity responsive-card model through 181 source correspondences. cargo xtask gate-f-v0 changes a token, four padding edges, escaped text and one responsive rule through exactly eight spans; requires exact re-import, repeat-identical output, byte identity outside those spans and exact unknown-payload survival; and rejects unsupported tokens, structural edits, stale spans, inconsistent derived media CSS and one-over input. Its editor bridge then changes card name and width through semantic headless actions, applies exactly two source edits through the CLI, re-imports through the CLI and requires canonical NUIF byte identity with the editor output. Path and instance identities are losslessly stored but their missing browser geometry/materialization remains unsupported; unknown kinds and extensions are preserved_unrenderable. This completes nuif:experiment:v0-responsive-card under its declared model/source acceptance, not arbitrary HTML/CSS or browser visual equivalence.
Gate G — independent reproduction (complete for v0 profile 0)
Exit only when a second implementation, built from the specification and fixtures rather than reference-package calls, parses, writes, lays out and renders the v0 profile to its declared tolerances.
Evidence: implementations/python/nuif_profile0.py uses only the Python standard library and does not import, invoke or link a Rust/NUIF package. cargo xtask gate-g gives the implementations the same canonical fixture, generates reference observations at 360 × 640, 768 × 768 and 1,440 × 900, and requires the independent path to reproduce canonical text bytes, unknown opaque payload preservation after a neighbouring edit, all eight boxes, decoded RGBA and five fidelity records exactly. The report contains three matching RGBA SHA-256 values and zero layout delta in every context. Duplicate-key and deliberately corrupted layout/raster trials prove the negative path. This closes the mechanical Gate G metric; it is not external authorship, a general-purpose second implementation, neutral governance or standards publication.
Gate H — metadata-free collaboration checkpoints (complete for bounded register and existing-tree profiles)
Exit metrics:
- replica clocks, causal context and conflict candidates remain outside canonical NUIF documents;
- independently structured operation-set and replica-log materializers converge for every delivery of the same valid change set;
- concurrent semantic property conflicts remain explicit and property-attributed;
- incomplete causal history, identifier reuse, unsupported profile expansion and invalid materialization fail closed;
- existing-tree move/delete delivery preserves one-parent/acyclic structure and explicit structural conflicts;
- a pinned foreign engine convergently transports the exact structural operation set without being treated as the tree oracle.
Evidence: nuif-collab-registers-0 maps register-like NUIF semantic operations to causal multi-value registers. One implementation computes pairwise maximal changes from an operation set; the other incrementally maintains causal frontiers in per-replica logs. cargo xtask gate-h exhausts all 5,040 deliveries of a seven-change/three-replica history, compares both materializers and multiple merge orders, repeats duplicate delivery, requires two explicit property conflicts and proves canonical NUIF text contains no replica/context/conflict metadata.
nuif-collab-tree-0 separately handles existing-identity moves, reorders and trash deletion with unique Lamport order, cycle rejection and RGA-style stable sibling origins. The gate exhausts all 5,040 deliveries of a seven-replica conflict/stable-anchor fixture, checks two materializers, join/idempotence, every required structural conflict class and 4,096 moves across 4,097 entities. Pinned @automerge/automerge 3.4.1 reproduces the exact immutable operation set across merge orders and save/load. That foreign result proves transport, not independent tree semantics. Concurrent creation, causal-stability garbage collection, combined structural/property transactions and an externally authored tree materializer remain open.
Gate I — portable package and resources (container and narrow media segments active)
Exit only when:
- two independently implemented writers produce identical
nuif-package-0bytes from one normative document/resource fixture; - document, asset, resource and package identity changes obey RFC 0010 fixtures;
- package read/write reaches a byte fixpoint and no implicit resource fetch occurs;
- duplicate, traversal, symlink, directory, encryption, split, compression, missing/extra member, size and digest failures are atomic and typed;
- archive/member/descriptor/image/font boundary and one-over cases pass measured time/allocation limits;
- PNG interpretation and font policy/shaping inputs reproduce through independent implementations for their declared subsets.
Current evidence: stable AssetId/ResourceDigest semantics, deterministic
stored ZIP packages, exact manual/independent-writer bytes, package fixpoint,
separate document/resource/package identities, explicit resolver authority and
15 hostile/archive/one-over cases run through cargo xtask gate-i-package.
The CLI and editor write real packages and preserve embedded resources. The
nuif-png-rgba8-0 segment additionally agrees across png and zune-png on
12 filter/colour-marker fixtures. The separate
nuif-png-basic-rgba8-1 profile adds thirteen fixtures spanning every admitted
greyscale/indexed/RGB/greyscale-alpha/RGBA type and transparency form. Together
they retain exact encoded bytes, repeat resource-aware CPU rasterization and
reject 20 unsupported/hostile cases via cargo xtask gate-i-image. Gate I does
not yet pass: 16-bit/interlaced/colour-managed PNG, live host/GPU affine equivalence,
cross-platform image reproduction, and a cross-platform/external writer remain required. The separate
nuif-opentype-static-single-0 segment compares exact Ahem metrics, family,
tables and Unicode coverage between Skrifa and a pinned HarfBuzz metadata
capture, accepts four static TrueType fixtures, preserves the font through
package fixpoint, requires explicit license/review evidence and rejects 20
synthetic/real malformed or out-of-profile cases plus 10 policy cases through
cargo xtask gate-i-font.
Six additional trials distinguish package-level portable/private/linked/
substituted/unavailable outcomes. TTC, CFF/CFF2, variable/color/bitmap/WOFF2
acceptance, cluster-level fallback, arbitrary packaged-font shaping and
cross-platform font reproduction remain required. Six item-level trials now
separate requested, substituted and unavailable text/font identities through
layout and rendering. Four accepted-font
inspections and packaged validation now carry warmed 4 MiB allocated/2 MiB
retained regression ceilings. Package-to-session handoff shares an 8 MiB buffer
under a 1 MiB allocation ceiling, and 1,024 image instances retain one 1 MiB
surface under the 64 MiB preflighted scene total. Both media segments are separate
from CPU render profile 0 and do not establish general images or packaged-font
rendering.
Gate J — source-backed browser capture (local live segment automated)
Exit only when:
- a pinned browser/protocol/OS/context produces repeat-equivalent normalized DOM, layout, style, resource, font-use, accessibility and screenshot observations;
- downloaded source-resource bodies retain exact size/digest identity;
- multiple input viewports predict a held-out responsive context better than a one-screenshot/freeform baseline for the declared subset;
- cross-origin, local-font, canvas, video, worklet and behavior gaps are explicit;
- cookies, authorization, credentials, storage and secret canaries never enter exported evidence;
- captured scripts/resources remain inert in every package reader.
This gate creates a new runtime adapter; it does not enlarge the existing Tree-sitter source-synchronization profile by implication.
Current evidence: cargo xtask capture-baselines repeats fixed browser-provider
input through nuif-capture, requires identical normalized output/package
bytes, exact image-resource digest and body retention, absence of a query-token
canary from observations/proposals/packages, typed proposal application and
cyclic-parent rejection. cargo xtask gate-j-live then launches exact Chrome
for Testing 152.0.7977.64 through bounded loopback CDP, accepting four declared
contexts with at most three recorded fresh-profile attempts each. It records
browser/protocol/OS/viewport/locale/timezone/media/motion/settling/freeze
context, exactly retains the declared HTML/CSS/PNG/font/probe bodies, observes
actual custom-font and accessibility results, reproduces the repeated 360 px
capture/normalization/screenshot bytes, and excludes five query, cookie,
storage, authorization and header canaries after proving they were exercised.
Geometry fitted to 360/768 px beats copying the 360 px freeform geometry at the
held-out 900 px fixture. A distinct nuif-layout-inference-0 report now ranks
five candidate families on training data alone, retains every alternative and
its geometry observation provenance, labels the selection inferred, leaves
confidence uncalibrated and evaluates the untouched 900 px observation only
after selection. On this fixture the selected constraint records 0.0626
normalized held-out error versus 0.2918 for fixed freeform. This automates the
local live segment without establishing general accuracy or original authored
intent. Gate J remains
open for cross-OS/browser reproduction, opaque/cross-origin behavior,
matched-style/source correspondence, canvas/video bounded frames and licensed
real-page evidence.
Gate K — screenshot reconstruction and calibrated abstention (contract baseline active)
Exit only when:
- deterministic OCR/CV, one-shot, observation-assisted, hierarchical-crop, multi-context and corrective-loop baselines run through one frozen harness;
- every outcome is a validated document/transaction or explicit no-result;
- source-backed and screenshot-only cases remain separate evidence classes;
- reports include text, elements, tree, properties, geometry, resources, held-out contexts, provenance/fidelity, accessibility, visual diagnostics, confidence, latency, RAM/VRAM, iterations and cost;
- editable reconstruction rejects a flat screenshot cover as success;
- confidence is calibrated per decision type on disjoint data and review/abstain thresholds reproduce their declared risk/coverage;
- an independent evaluator reproduces the principal held-out result and one real editing task benefits from the reconstructed semantics.
No editor prerelease or visually selected demo can substitute for this gate.
Current evidence: the same automated report repeats strict fixed-PNG analysis,
round-trips observation bytes, distinguishes observed pixels from inference,
records four unavailable evidence categories, applies typed proposals, rejects
screenshot-derived flat-copy assets by default and exercises improved,
repeated-state, no-proposal, provider-call and memory-budget loop stops. A two-
point interpolation/selective-review fixture verifies the calibration API.
cargo xtask reconstruction-evaluation additionally validates a bounded typed
report containing every required per-example metric family, explicit
numerators/denominators, nullable unavailable cost measurements and separate
local-pixel/element failures. It rejects derived-rate drift, oversized edit
work and exact source-resource claims in screenshot-only suites. Its typed
three-example aggregate reports pooled rates, scored/unscored per-example
distributions and nearest-rank p50/p95 while rejecting mixed suites,
calibration/evaluator drift and mixed currencies. These are synthetic contract
fixtures: no OCR/model baseline, licensed real or leak-resistant held-out
accuracy corpus, predeclared perceptual thresholds/uncertainty method or
independent reproduction exists. The gate now executes the pinned LDR-FLIP
wrapper at 67 PPD, records its full evaluator parameters, separates the pooled
mean from exact/local semantic metrics and rejects implicit transparency. This
validates evaluator wiring, not perceptual accuracy. Gate K remains open.
cargo xtask reconstruction-provider-manifest separately makes provider
identity resolvable rather than decorative. Canonical manifest bytes bind
capabilities, execution modes, wire profiles and exact operational artifacts;
every observation bundle carries the manifests referenced by its observations
and proposals. Missing, duplicate, malformed or dangling entries fail before
mutation. Released/learned fixtures require external SPDX 3.0.1 or CycloneDX
1.7 inventory identity, and learned fixtures require a model card. The learned
fixture contains synthetic digests and the browser/screenshot providers are
development source-bundle identities, so this is not a released model,
inventory audit or accuracy result.
Gate L — conditional adaptation and distillation (blocked on Gate K)
This gate is skipped unless Gate K reveals repeatable learnable errors and a rights-cleared validated trace corpus exists. If opened, exit only when:
- the dataset has digest-pinned lineage, consent/rights/privacy/retention policy, leak-resistant splits and a datasheet;
- every base model, processor, task adapter and run has a digest-pinned manifest and model card;
- prompt/tool, retrieval, supervised, LoRA, QLoRA where architecture-compatible and sequence-distillation candidates are compared on identical frozen data, budgets and evaluator versions;
- the selected candidate improves predeclared quality or efficiency without an unacceptable validity, calibration, privacy, license or maintenance regression;
- rollback to the untuned baseline remains possible.
LoRA/QLoRA/distillation are methods tested inside the gate, not predetermined architecture or evidence that the gate should exist.
Thesis stop conditions
Stop or narrow the architecture if the v0 source patch routinely becomes whole-file regeneration, an ignorant implementation cannot preserve opaque bytes during neighboring edits, operation convergence requires collaboration metadata in canonical documents, tolerance tiers hide systematic semantic divergence, or a second implementation cannot reproduce the profile without reading reference code. Narrow the proposed resource/reconstruction path if independent package writers cannot agree, browser capture cannot exclude secrets reproducibly, correction loops improve pixels by deleting semantics, confidence cannot achieve useful risk/coverage, or tuned systems fail to beat the untuned tool-assisted baseline fairly.
Executable baseline after this audit
The repository now has a typed canonical model, structural validator, anchored atomic operations with stale-base rejection and replay/inversion, canonical text and deterministic CBOR codecs, responsive profile-0 layout, deterministic CPU rasterization, a seeded trial/ddmin/report library, validity-preserving subtree/scalar and choice-stream reducers, atomic regression-fixture emission, an executable conformance package, a multi-command CLI and a headless editor accessibility driver with complete mutation-log replay. The Gate B long run passes 10,000 generated patches (160,000 operations), checking replay, inverse and both encodings on every patch and sampling layout/raster checks every 100 patches. The hostile-input run measures byte, syntax-depth, semantic-cardinality, elapsed-time and allocator boundaries, records its platform, and rejects every one-over case. cargo xtask reduction-profile reduces the responsive fixture to the valid three-entity trigger path, records candidate counts and transformations, emits a canonical fixture and confirms non-overwrite behavior; failed seeded trials with report paths use this same mechanism with their recorded viewport and snapshot decision.
The active codec decision gate adds a separate four-scale release benchmark for size, encode, decode, canonicalize, allocation and decode-then-select behavior. Both implemented codecs must first pass semantic, canonical and opaque-data preservation through a neighboring edit. The first Apple M5 Pro run places CBOR near 41% of text size at 4,096 entities but shows the current typed CBOR decoder slower than canonical text. Protobuf and FlatBuffers remain outside the timing table because no complete NUIF mapping satisfies canonical and retentive editing requirements; Cap’n Proto is the next conditional candidate.
The unified performance gate also records portable release latency and allocation budgets, audits direction coverage from the per-profile adapter catalog and executes every Criterion path once. Its controlled-hardware suites cover core scaling, queries, both collaboration materializers, packages, resources, package-capability negotiation and all ten integrated adapter profiles. One-way accessibility and behavior projections are measured only as exports; the catalog no longer invents import or synchronization directions for them. Shared-runner Criterion timing remains smoke evidence, not a regression threshold.
Gate C pins Taffy 0.14.0 and Chrome for Testing 152.0.7977.64 and runs a deterministic three-way layout report over 27 cases, 81 comparisons and 1,536 box components. The v0 card agrees exactly across NUIF, Taffy and Chrome at 360, 768 and 1,440 px. Eight bounded-Grid cases exercise fixed/fr tracks, sparse row/column flow, explicit placement and spans in addition to generated stack/flex cases. All cases pass with zero classified, blocking or unexplained divergence; 26 fixtures have exact Taffy/browser agreement and one fractional Grid fixture uses its measured 0.02 px bound. The foreign oracles exposed both the earlier definite-size stretch defect and a Grid fill lowering defect, and both remain regression-covered. Gate C now claims the bounded explicit-Grid profile, not intrinsic, percentage, named, repeated, implicit, subgrid or masonry tracks.
Gate D is complete for the deliberately narrow CPU profile 0. It pins the 22,572-byte Ahem 1.50 font by SHA-256, HarfRust 0.13.3, Unicode 17.0.0, unhinted Skrifa 0.46.2 outlines and Zeno 0.3.3 grayscale masks. Eight ASCII/Unicode LTR/RTL runs match HarfBuzz 14.4.0 and five signed-26.6 paths match normalized hb-vector. Hard breaks, line-height placement, intrinsic shaped width, inline-start alignment and no automatic soft wrapping are executable lossless semantics. Encoded-sRGB solid rectangles, four-cubic ellipses and integer source-over composition are defined by value and have exact scene/raw-RGBA fixtures. PNG hashes are non-blocking encoder diagnostics. The text and paint reports reproduce on macOS/aarch64, Linux/aarch64 and Linux/x86_64. Paths, images, component instances and extension-defined visuals are not misrepresented as supported: their fidelity records retain document/entity identity and property pointers. Expanded render profiles remain future work.
Gate E is complete for the headless profile-0 editor instrument. Twelve semantic document/token/entity actions construct the full fixture from empty state. Direct generation, editor output and protocol replay converge to exact canonical text bytes and hash nuif-cbor-0:sha256:540363fe916a3a1926fecbcbd27fd0280666e3cbbb115e561d38f3b7f322a3d6. The 768×640 snapshot contains eight layout boxes, three supported render commands, five explicit fidelity entries, an exact CPU PNG and hashes for both RGBA and PNG bytes. cargo xtask editor-trial runs the old neighboring-edit/undo/redo trial and this complete-authoring trial, validates both outputs, emits the machine report and snapshot bundle, and is the CI entry point. The native research-preview shell is also exercised through AccessKit and deterministic CPU screenshots; it remains a client and cannot alter model, layout or renderer conformance. Expanded UI profiles and external reproduction remain open.
Gate F remains complete for nuif-html-css-0, a deliberately bounded two-entity source profile: 25 lossless correspondences and six local edits retain every unmapped byte. Its full-v0 follow-on is also complete for model preservation. nuif-html-css-v0 maps DOM containment, every responsive-card field, real size/layout/fill CSS, responsive media CSS and opaque metadata; its 181-correspondence trial and two-edit editor/CLI bridge both re-import exactly. Tree-sitter validates HTML and CSS under a 1 MiB bound, derived CSS drift fails closed, and target visual limitations are never described as lossless browser behavior. Arbitrary HTML/CSS, collaboration profiles and expanded path/instance rendering remain open.
The separate nuif-web-accessibility-0 projection makes the semantic web
boundary executable rather than hiding semantics in retained metadata. It
prefers native HTML for exact roles, admits ten roles with role-specific
Boolean states, maps five stable-ID relationships and rejects every unsupported
or ambiguous case atomically. cargo xtask gate-accessibility compares the
computed role, name and supported state of eleven entities through exact
Playwright 1.62.1 Chromium, Firefox and WebKit engines. The first macOS/arm64
run passes with identical full snapshots and records all engine/host versions.
This is web-engine evidence, not native platform API, assistive-technology,
keyboard or application-behavior conformance.
The separate nuif-behavior-state-machine-0 sidecar makes one behavior subset
executable without freezing it into the wire model. Stable semantic entity
activation, ordered guarded transitions, Boolean/string variables, sequential
actions and visibility/announcement effect records are statically bounded.
Missing required capabilities reject before execution; optional announcement
effects degrade only through an explicit traced no-op. The Rust reference and
independently written Node interpreter agree on every transition, state,
variable, effect and skipped operation for two capability runs over five
events. This is profile trace evidence, not browser DOM, native UI, animation,
network or arbitrary-script behavior evidence.
RFC 0012 now gives that sidecar one experimental wire transport without
freezing it into the semantic Document: canonical behavior CBOR is one
embedded, content-addressed source resource whose required capability and
descriptor are carried by the existing package manifest. The attachment gate
passes canonical/fixpoint, document-versus-package hash, disagreement,
duplicate, linked, malformed, rebinding and corruption probes. A separately
written Python standard-library reader checks exact ZIP bytes, ordering,
metadata, CRC and the behavior blob digest. Generic package decode remains
inert; a bounded generic SDK report distinguishes structural validity from full
host capability support and returns every missing requirement exactly. Explicit
attachment decode and runtime capability authorization remain separate steps.
This is not a second CBOR or behavior implementation.
The one-way nuif-web-behavior-0 adapter closes the bounded browser-DOM part
of that non-claim without widening the source profile. It admits enabled native
buttons and button-backed switches, maps visibility to hidden, maps one
announcement per transition to an unfocused polite status region and rejects
native-control or task-coalescing mismatches before output. Program data is
delimiter-escaped and interpreted only by one generated runtime whose exact
UTF-8 body is admitted by a SHA-256 CSP hash; resource and dynamic-code
authority stay denied. The five reference events pass event-by-event in exact
Playwright 1.62.1 Chromium, Firefox and WebKit engines on the recorded
macOS/arm64 run. This remains browser DOM/accessibility-tree evidence, not
screen-reader speech, focus, native UI or arbitrary-script compatibility.
Gate G is complete for the bounded v0 profile. The Python implementation independently validates and canonicalizes the full fixture, preserves the opaque vendor.probe payload across an unrelated edit, implements the profile-0 layout algorithm and rasterizes the fixture’s solid rectangles and pinned Ahem text. All 24 context/entity boxes and all three decoded RGBA buffers match exactly, with the fidelity list also byte-for-value equivalent. Its unsupported visual scope remains explicit, and an external implementation/reviewer is still required before a standards-readiness claim.
Gate H is complete for property registers and the bounded existing-tree structural profile. The property operation-set and replica-log materializers converge to hash nuif-cbor-0:sha256:29f24d0cb9613b7a6adaf1f57760031d12271c0eb06084e3807115ef869941ab across all 5,040 deliveries and tested merge orders. Concurrent values remain explicit, causal overwrites select only maximal values and the opaque entity stays exact. The structural operation-set and replica-log materializers separately converge over every delivery of move/reorder/delete/rescue conflicts while preserving one parent, acyclicity and stable sibling origins; a 4,096-change scale trial is bounded, and Automerge reproduces immutable operation transport. Checkpoints contain no collaboration metadata. Concurrent creation, causal garbage collection, combined property/structure transactions and a foreign tree materializer remain required before a general collaboration-profile claim.
RFCs 0010, 0011 and 0012 plus specifications 13 and 14 remain research-aligned proposals.
The executable baseline now includes the deterministic package writer,
asset/resource model, bounded provider-input browser/screenshot contracts and
the pinned local live-browser segment described above. It still has no general
image/font resource profile, cross-browser/OS capture corpus, reconstruction
accuracy corpus, independent reconstruction evaluator or trained artifact. The
editor version 0.1.0-alpha.3 identifies the developer application and must not
be cited as maturity evidence for those open proposals.
Continuous research roadmap
The operational plan and audit findings are in AUDIT.md. Work advances by evidence gates, not by document volume or implementation phase names.
Completed gates
Gate B: canonical model, operations and encodings. The executable baseline covers structural validation, anchored atomic operations, stale-base rejection, replay/inversion, text/CBOR fixpoints, negative numeric/canonical cases, opaque-byte cycles, a passing 10,000-patch seeded trial, and measured hostile-input byte, depth, node, allocation and time budgets. RFC 0009 and cargo xtask hostile-inputs close the final Gate B condition.
Gate C: responsive and bounded-Grid layout falsifier. cargo xtask gate-c
compares the v0 viewport matrix and 24 seeded stack/flex/Grid cases across the
independent NUIF evaluator, Taffy 0.14.0 and pinned Chrome for Testing
152.0.7977.64. Per-fixture measured bounds, raw boxes and classifications are
stored in target/layout-differential-report.json. Fixed/fr tracks, sparse
row/column flow, explicit placement and spans pass with no classified, blocking
or unexplained divergence; broader CSS Grid remains outside profile 0.
Gate D: bounded visual and text profile. cargo xtask gate-d runs separate text and paint reports. Profile 0 pins Ahem/HarfRust/Unicode/Skrifa/Zeno; defines hard lines without automatic soft wrapping; fixes rectangle, ellipse, encoded-sRGB and integer-composition behavior by value; reproduces scene/raw-RGBA hashes on macOS/aarch64, Linux/aarch64 and Linux/x86_64; reports PNG encoding separately; and keeps path/image/instance/extension semantics in property-attributed fidelity records.
Gate E: editor/CLI parity. Twelve semantic actions author the complete v0 fixture from an empty document. Direct generation, editor output and operation replay are byte-identical, while the archived snapshot carries canonical input, context, layout, scene, CPU raster and fidelity.
Gate F: bounded retentive HTML/CSS synchronization. The declared container/text/token subset reparses exactly; six mapped edits change only their scalar spans; comments and unmapped markup survive; unsupported properties remain typed and attributed.
Full-v0 source follow-on: nuif-html-css-v0 retains 181 model correspondences for the complete responsive card. Eight token/padding/text/responsive edits and the two-edit semantic editor/CLI bridge both re-import exactly, while path, instance and unknown target limitations remain explicit.
Web accessibility projection: nuif-web-accessibility-0 lowers a bounded
ten-role, Boolean-state and five-relationship subset to inert native HTML/ARIA.
The exact Playwright 1.62.1 Chromium, Firefox and WebKit engines expose all eleven
fixture entities with matching computed role/name/state and identical bounded
ARIA snapshots on the recorded macOS/arm64 run. Host versions and differences
remain separate from semantic loss; native APIs and interaction behavior are
not claimed.
Behavior portability sidecar: nuif-behavior-state-machine-0 runs one bounded
stable-identity program through independent Rust and Node interpreters. Full
and required-only capability traces agree over five events; required capability
absence rejects before execution and optional effects follow a recorded no-op.
nuif-behavior-package-resource-0 now binds the same canonical-CBOR program to
the delivered document through one inert content-addressed package resource and
an independently inspected deterministic ZIP. It remains outside the canonical
semantic Document and excludes timers, internal events, numeric computation
and host UI execution.
Web behavior projection: nuif-web-behavior-0 composes the bounded sidecar and
accessibility projection into enabled native-button activation, HTML hidden
visibility and one polite status announcement. A delimiter-safe generated
runtime is admitted by its exact CSP SHA-256 hash. Separate pointer and
Enter/Space keyboard sequences match the Rust reference transition/state/effect
sequence in pinned Playwright Chromium,
Firefox and WebKit; focus, control-state mutation, assistive-technology speech,
native UI and arbitrary authored scripts remain outside the profile.
Gate G: bounded mechanically independent reproduction. The standard-library-only Python implementation has no Rust/NUIF package dependency and exactly reproduces v0 canonical text, opaque preservation, 24 boxes, three decoded RGBA buffers and five fidelity records. External authorship and a general-purpose second implementation remain standards-publication work.
Gate H: bounded metadata-free collaboration checkpoint. Two algorithmically distinct in-repository materializers converge for every delivery of a conflict-bearing property-register history; conflicts remain explicit and canonical NUIF contains no replica state.
Publication infrastructure: docs/catalog.json selects canonical Markdown
documents without copying their bodies. The bounded xtask compiler validates
metadata and repository links, generates navigation and status indexes, builds
a searchable mdBook site and composes a 13-module working manuscript. The Pages
workflow retains pull-request artifacts and restricts deployment permission to
its deployment job. This infrastructure publishes evidence; it does not
promote evidence status.
Current falsifiers
The active codec decision gate measures canonical text and deterministic CBOR at 8, 64, 512 and 4,096 entities after exact semantic, canonical and opaque-edit preflight. It records native partial-load support separately from full decode followed by selection. Protobuf and FlatBuffers are not admitted because their documented default forms do not meet canonical and retentive-editing requirements. Cap’n Proto is the next candidate, conditional on a complete mapping, bounded cross-version edit trial and two canonical writers; no schema-codec timing claim exists yet.
nuif:experiment:v0-responsive-card, the bounded collaboration
property-register checkpoint and the existing-tree structural checkpoint are
complete under their declared acceptance. Structural move/reorder/delete/rescue
preserves one-parent/acyclic invariants and stable sibling origins across all
5,040 deliveries, while Automerge reproduces operation transport. The next
collaboration falsifier is concurrent entity creation, followed by causally
stable garbage collection, combined property/structure transactions and a
foreign materializer of the tree algorithm itself.
The package segment of nuif:experiment:portable-package-resources is active:
the manual writer agrees byte-for-byte with an independent ZIP writer, identity
relations and explicit resolution are exercised, and 15 hostile/one-over cases
produce target/package-resources-report.json. RFC 0010 remains proposed and
Gate I remains open. cargo xtask gate-i-image now provides a narrow
nuif-png-rgba8-0 cross-decoder, exact-resource and repeatable CPU-render
baseline. The separate nuif-png-basic-rgba8-1 profile covers the
non-interlaced colour/depth forms that normalize to RGBA8 without sample loss;
16-bit/interlaced/colour-managed PNG, live host/GPU affine equivalence and cross-platform
image reproduction remain excluded. cargo xtask gate-i-font
adds a deliberately narrow static TrueType external-oracle/package/policy baseline;
TTC, CFF, variable/color/bitmap/WOFF2 acceptance, cluster fallback and arbitrary
packaged-font shaping remain separate. Whole-text substituted/unavailable
bindings now have automated package, layout and rendering outcomes.
Package/session handoff now proves shared immutable bytes, and image scenes
deduplicate decoded surfaces under a preflighted 64 MiB total plus measured
allocation ceilings. Static-font inspection and packaged validation have their
own warmed allocation ceilings. Cross-platform writer reproduction remains an
independent requirement. Browser capture precedes
screenshot reconstruction because it provides stronger source-backed fixtures
and exposes which information is truly unavailable from pixels.
The automated capture/reconstruction contract baseline now produces
target/capture-reconstruction-report.json. It checks repeatable provider-input
normalization, exact browser resource retention, credential-query redaction,
honest screenshot omissions, typed proposal application, flat-copy rejection,
codec fixpoints, calibration interpolation and finite correction-loop stops.
cargo xtask gate-j-live separately drives Chrome for Testing 152.0.7977.64
through bounded loopback CDP. Four isolated runs retain the exact declared
response set, platform-font use, accessibility and PNG evidence; carry the
pinned runtime context into observations; reproduce the repeated 360 px
capture exactly; exclude five exercised secret canaries; and use 360/768 px
geometry to beat the one-view baseline at held-out 900 px. This closes the
local live-fixture segment. The same gate now emits a separate bounded layout
inference report: selection uses only 360/768 px training observations, retains
all row/column stack, Grid, constraint and freeform alternatives with raw
confidence and provenance, and evaluates the selected constraint only
afterward at 900 px. The observed 0.0626 versus 0.2918 normalized error is one
falsifiable fixture result, not calibrated confidence or general accuracy. The
typed confidence evaluator now has a deterministic smoke report over normal and
font-shifted holdouts, but this does not close the broader browser, screenshot,
closed-loop or calibration experiments: no cross-browser/OS capture corpus, opaque-frame
coverage, reconstruction accuracy corpus, independent evaluator or trained
artifact is claimed. Adaptation/distillation remains conditional on evidence
from that loop rather than a standing implementation commitment.
Queue
- Keep Gates B through H green with
cargo xtask alland the separate nightlycargo xtask fuzz-smoke; reduce fuzz failures before committing them as named fixtures and retain all machine reports as CI artifacts. - Implement the full Cap’n Proto candidate mapping only behind the codec admission preflight; compare it after canonical-writer, old-reader retention and hostile traversal tests pass. Keep the optimized typed CBOR decoder behind identical canonical-byte and hostile-input checks; investigate a streaming canonical validator only if profiling still justifies its added parser surface.
- Extend the executable existing-tree collaboration profile to concurrent creation, causal-stability garbage collection and combined property/structure transactions; obtain a foreign tree materializer rather than treating the completed Automerge transport oracle as one.
- Keep the implemented fixed/
fr, sparse-flow, explicit-placement Grid subset exact; intrinsic, percentage, named, repeated, implicit, subgrid and masonry tracks require a separately versioned schema and foreign-oracle matrix. - Keep the tested Masonry shell attached only through the editor driver boundary; extend the implemented one-transaction freeform move, eight-handle freeform resize, trailing managed resize, Shift-proportional corner gesture and resolved-axis Stack/Flex reorder only when a tested semantic operation exists. Cross-parent/tree drag and Grid/Constraint reorder remain separate design work.
- Treat soft wrapping, gradients, strokes, paths, images and instance materialization as a separately versioned expanded profile; do not weaken profile-0 exactness to add them.
- Keep
cargo xtask gate-i-packagegreen across CLI/editor/package changes and add a recorded cross-platform/external writer before accepting the wire profile. - Extend the narrow PNG and static TrueType baselines only through new declared fixtures, add cross-platform media reproduction, and complete the broader OpenType format/policy matrix with calibrated allocation/time budgets; do not expand profile 0 by fallback.
- Extend the passing local live-browser segment to cross-OS reproduction, opaque/cross-origin cases, matched-style/source correlation and licensed real pages before defining a portable browser-capture profile; keep WebDriver BiDi as the standards-track transport watch path.
- Freeze the reconstruction corpus and evaluator, then compare deterministic OCR/CV, one-shot, observation-assisted, hierarchical and corrective-loop baselines through the existing typed boundary.
- Train or distill only if the frozen evaluation demonstrates a learnable gap and rights-cleared validated traces exist.
- Maintain the credential-free Penpot package profile under its shared ZIP resource-limit, foreign-producer and unknown-member-retention gate; defer the compact representation until upstream stability and a second fixture.
- Run the bounded Figma profile in a named live host; run the Affinity SVG bridge as a retained user-mediated foreign-runtime trial; and implement the pure stable-API Canva current-page mapper before building its review shell. Retain host reports and never infer live behavior, marketplace approval or native NUIF support from API documentation.
- Keep
nuif-api::NuifDocumentas the single direct SDK façade and require semantic-API promotion, stable errors/ownership, sanitizer-backed native consumers and real platform packages before declaring C, Swift or Kotlin binding profiles. - Package the conformance kit for externally authored reproduction; do not treat the in-repository Python path or Rust adapters as external interoperability evidence.
- Keep standards-development work behind the implementer-draft and external-support gates in
docs/STANDARDS-ROADMAP.md.
Update policy
- Add evidence as a new record or source revision; use
supersedesandcontradictsrather than silently rewriting history. - Record source commit, tag or specification revision where available.
- Promote
reviewedtoverifiedonly after locator-level checks and executable evidence for implementation claims. - Every experiment declares seed/input, oracle class, acceptance criteria, artifacts and implementation path before it can become
active. - Every completed experiment stores a machine report and the exact engine/toolchain/profile identity.
- Claims become specification requirements only through RFC review and executable conformance fixtures.
Portable accessibility semantics, HTML lowering and foreign browser oracles
Document status:
verified. Canonical source.
Summary
Portable accessibility needs two distinct contracts: authored semantic intent in NUIF and target exposure through a host accessibility model. WAI-ARIA defines roles, states and properties; ARIA in HTML constrains how they may be used with native elements; Accessible Name and Description Computation defines the user-agent result; and Core-AAM/HTML-AAM map that result toward platform APIs. Copying arbitrary ARIA attributes into a document model would therefore be weaker than a bounded semantic profile with role-specific validity, deterministic lowering and observed browser results.
The best current test architecture is a small pure Rust projection plus a foreign three-engine oracle. The projection prefers native HTML semantics, uses explicit ARIA only where the profile has no exact element, retains stable entity IDs for attribution and rejects semantics it cannot represent. The oracle compares computed role, accessible name and supported state rather than checking emitted attributes alone. Full tree differences are kept separately from required-subset loss.
Evidence
- ARIA in HTML is a W3C Recommendation updated 11 August 2026. Its element
table defines implicit semantics and permitted role/state/property use and
explicitly discourages redundant explicit roles. This supports native
button, checkbox/radioinput,main,nav, namedsectionandplowering before explicit ARIA. Locator: https://www.w3.org/TR/2026/REC-html-aria-20260811/, retrieved 2026-08-31. - Accessible Name and Description Computation 1.2 defines flat computed names,
author/content/prohibited name sources and
aria-labelledbyprecedence. A direct name and label relationship can otherwise disagree, so the bounded profile rejects that ambiguity and requires named roles to have a resolvable name. Locator: https://www.w3.org/TR/accname-1.2/, retrieved 2026-08-31. - Core Accessibility API Mappings 1.2 explains that user agents expose web roles, values, Boolean states and relationships through differing platform APIs. It is a Candidate Recommendation Draft, not evidence that platform trees are byte-identical. NUIF must consequently separate semantic loss from host-tree difference. Locator: https://www.w3.org/TR/core-aam-1.2/, retrieved 2026-08-31.
- WebDriver defines Get Computed Role and Get Computed Label endpoints. Those operations confirm that a conformance test must ask the browser for computed semantics rather than infer success from source attributes. The current endpoints do not expose a complete portable accessibility property bag. Locator: https://w3c.github.io/webdriver/#get-computed-role and https://w3c.github.io/webdriver/#get-computed-label, retrieved 2026-08-31.
- Playwright 1.62.1 supplies a version-coupled Chromium, Firefox and WebKit set, role locators and ARIA snapshots. Its documentation says each Playwright release requires particular browser binaries and that its Firefox/WebKit builds are patched test engines rather than branded Firefox/Safari. This is appropriate as a repeatable foreign oracle, but the report must retain those non-claims. Locators: https://playwright.dev/docs/browsers and https://playwright.dev/docs/aria-snapshots, retrieved 2026-08-31.
Mechanism
nuif-web-accessibility-0 accepts at most 4,096 valid entities and 8,192
relationships. It admits ten roles with explicit name and Boolean-state rules,
maps five relationship kinds through stable entity IDREFs, retains target order
and rejects duplicate targets, unnamed labels, invalid containment, unknown
roles/states/relationships and competing direct/relationship names. Names are
whitespace-normalized before they become oracle expectations. The owned tree
must be acyclic and every owned target has at most one ARIA owner. Output is
inert HTML without scripts, external URLs or invented behavior.
cargo xtask gate-accessibility first generates the fixture, expected mapping
and static negative-test report through Rust. It then installs exact
Playwright 1.62.1 browser revisions and asks each engine to locate every NUIF
entity by its computed role, accessible name and supported state. Full body and
per-node ARIA snapshots, versions and mismatch categories are written to
target/accessibility-mapping-report.json. The gate is intentionally separate
from the main Rust loop because the three downloaded engines form a larger
foreign-runtime matrix, like sanitizer fuzzing and hosted platform jobs.
The first macOS/arm64 run passed all eleven nodes under Chromium 151.0.7922.34, Firefox 153.0 and WebKit 26.5. All three produced the same bounded ARIA snapshot covering all ten admitted roles. This verifies the named projection and oracle path on one host; CI is configured to produce independent Linux evidence.
NUIF relevance
NUIF should carry semantic intent independently of visual kind and lower it through profiles, not vendor-specific accessibility objects in the core. One authoritative role/name/state/relationship representation can feed HTML, AccessKit, SwiftUI, Android and other adapters, but each target must report its own unsupported or approximated surface. Browser agreement is valuable interoperability evidence, not a substitute for native assistive-technology or interaction testing.
The current wire model has only one direct accessible name and Boolean states. It cannot honestly claim direct descriptions, numeric values and levels, live-region details, table/grid metadata, composite focus management or behavior. Those additions need schema and operation design before widening the profile; adapters must not smuggle them through arbitrary strings and call the result portable.
Open questions
- Which direct description, numeric value/range, level, orientation and live-region fields belong in the portable baseline rather than extensions?
- Beyond profile 0’s acyclic single-owner rule, which portable relation model can reconcile semantic ownership with DOM containment without relying on target-specific tree repair?
- Which keyboard and focus traces are the minimum foreign behavior oracle for button, checkbox, radio and switch semantics?
- Which native API harnesses can compare macOS AX, Windows UIA, Linux AT-SPI, Android Accessibility and iOS UIAccessibility without reducing them to web role strings?
AccessKit as a cross-platform accessibility tree and semantic UI test surface
Document status:
reviewed. Canonical source.
Summary
AccessKit defines a Rust data schema for an accessibility tree (Node, Role, Action, TreeUpdate, ActionRequest) and platform adapters that expose that tree to Windows UI Automation, macOS NSAccessibility, Unix AT-SPI, Android and iOS. The provider (toolkit) pushes full and incremental TreeUpdates; the adapter retains the tree and pulls nothing, which the project states makes the design suitable for immediate-mode toolkits with stable node IDs. accesskit_consumer is the platform-independent tree store used by the adapters; it is also the basis of kittest, egui_kittest and masonry_testing, which turn the same tree into a test oracle: tests query nodes by role, label or value and dispatch Actions (Click, Focus, SetValue, ScrollIntoView) without synthesising pointer input.
NUIF interpretation: the AccessKit schema is a concrete, serialisable (serde, JSON Schema via schemars) semantic tree that NUIF can emit from resolved documents; it gives QA item 5 (inspect accessibility semantics) and QA item 3 (execute semantic actions) a shared representation. It is not a normative NUIF model: roles are Chromium-derived and platform-oriented, and the tree carries resolved geometry rather than authored intent.
Evidence
- Versions: accesskit 0.25.0 published 2026-08-29 (crates.io
max_version,updated_at), changelog entry “0.25.0 (2026-08-29)” with breaking change “Reuse property buffers when cloning nodes” and feature “Add html_id node property (#776)”; previous 0.24.1 (2026-06-12). accesskit_consumer 0.38.0 (2026-07-14; main manifest is 0.39.0 depending on accesskit 0.25.0). accesskit_winit 0.34.0 released 2026-08-29; accesskit_windows 0.34.0, accesskit_macos 0.26.3, accesskit_unix 0.22.1 (2026-07-14). Locator: crates.io API;accesskit/CHANGELOG.mdlines 1-20;accesskit/Cargo.toml,accesskit_consumer/Cargo.toml, main commit 42e53b0. - Schema description: “each node is either a single UI element or an element cluster”; “Each node has an integer ID, a role (e.g. button, label, or text input), and a variety of optional attributes”; “The schema is based largely on Chromium’s cross-platform accessibility abstraction”; canonical definition in Rust, other representations generated. Locator:
README.md“Data schema”. - Push model: the toolkit “initially pushes a complete accessibility tree, then it pushes incremental updates”; “only the platform adapter needs to retain a complete accessibility tree”; “suitable for immediate-mode GUI toolkits, as long as they can provide a stable ID for each UI element”. Locator:
README.md“Platform adapters”. - Adapters listed: Android, iOS (UIAccessibility), macOS (NSAccessibility), Unix (AT-SPI over D-Bus via zbus), Windows (UI Automation); planned: web. Language bindings: C (cbindgen), Python (PyO3). Locator:
README.md“The following platform adapters are currently available”, “Language bindings”. Roleenum has 182 variants (lines 62-271);Actionenum has 22 variants: Click, Focus, Blur, Collapse, Expand, CustomAction, Decrement, Increment, HideTooltip, ShowTooltip, ReplaceSelectedText, ScrollDown, ScrollLeft, ScrollRight, ScrollUp, ScrollIntoView, ScrollToPoint, SetScrollOffset, SetTextSelection, SetSequentialFocusNavigationStartingPoint, SetValue, ShowContextMenu. Locator:accesskit/src/lib.rslines 62-271, 289-320, main.Node(line 1107) is documented as “A single accessible object. A complete UI is represented as a tree of these.”; getters/setters act as properties (role(),set_role,add_action,supports_action). Locator:accesskit/src/lib.rslines 1095-1110, 1858-1871.TreeUpdate { nodes: Vec<(NodeId, Node)>, tree: Option<TreeInfo>, tree_id: TreeId, focus: NodeId }: nodes overwrite by ID; adding a child requires the updated parent; removal is expressed by omitting the child from the parent’schildren;focus“must be provided with every tree update”. Locator:accesskit/src/lib.rslines 3207-3256.ActionRequest { action, target_tree, target_node, data: Option<ActionData> }; traitsActivationHandler,ActionHandler,DeactivationHandler. Locator:accesskit/src/lib.rslines 3304-3380.- Cargo features:
serde,schemars(JSON Schema, withserde_json),pyo3,enumn; the only mandatory dependency isuuid. Locator:accesskit/Cargo.tomllines 16-30, main. - Consumer API:
Tree::new(initial_state: TreeUpdate, is_host_focused)panics unlessTreeUpdate::treeisSomeandtree_id == TreeId::ROOT;update_and_process_changes(update, &mut impl ChangeHandler);ChangeHandler { node_added, node_updated, focus_moved, node_removed };TreeState::{root, node_by_id, focus, active_dialog, toolkit_name, subtree_root};NodeRef::{role, label, description, value, parent, children, bounding_box, is_focused, is_hidden, toggled, supports_action, labelled_by, author_id, html_id}. Locator:accesskit_consumer/src/tree.rslines 67-708;node.rslines 89-985, main. - winit adapter:
Adapter::{with_event_loop_proxy, with_direct_handlers, with_mixed_handlers, process_event(window, &WindowEvent), update_if_active(|| TreeUpdate)}; platform adapters exposeupdate_if_activeso the tree is built only when assistive technology is active. Locator:adapters/winit/src/lib.rslines 127-264;adapters/{windows,macos,unix}/src/adapter.rs. - kittest 0.4.0 depends only on
accesskitandaccesskit_consumer; itsByfilter supports label, label-contains, role, value and predicate;NodeTwrapsaccesskit_consumer::Node. Locator: kittestCargo.tomllines 24-27;src/query.rslines 146-236;src/node.rslines 9-60. - egui_kittest
click_accesskit()sendsEvent::AccessKitActionRequest(ActionRequest { action: Action::Click, .. })and “can also click widgets that are not currently visible”;focus()sendsAction::Focus;scroll_to_me()sendsAction::ScrollIntoView. Locator: eguicrates/egui_kittest/src/node.rslines 101-160. - masonry_testing keeps
access_tree: accesskit_consumer::Tree, updates it inredraw(), and exposesprocess_access_event(ActionRequest)andaccessibility_click_on(WidgetId). Locator: xilemmasonry_testing/src/harness.rslines 146-160, 466, 576-608, 767. - egui’s workspace notes that kittest 0.4 pins accesskit_consumer 0.35 and blocks accesskit_winit upgrades. Locator: egui
Cargo.tomllines 73-75 at tag 0.36.1.
Mechanism
Data model and flow:
toolkit frame ──TreeUpdate{nodes, tree?, tree_id, focus}──▶ accesskit_consumer::Tree (retained)
│ ChangeHandler: node_added/updated/removed, focus_moved
▼
platform adapter (UIA / NSAccessibility / AT-SPI)
│
assistive technology ──ActionRequest{action, target_tree, target_node, data}──▶ ActionHandler (toolkit)
Test-surface instantiation (kittest / masonry_testing): the harness plays the role of the platform adapter. It owns the accesskit_consumer::Tree, applies each frame’s TreeUpdate, evaluates queries over NodeRef (role, label, value, labelled_by, bounding_box, supports_action), and injects ActionRequests into the toolkit’s event queue. Assertions read the next frame’s tree.
Invariants stated by the schema: node IDs are stable across updates; every update carries the current focus; a node’s children list is the only membership authority; roles are closed enumerations (182 roles, 22 actions in 0.25.0). Property presence is optional, so queries must treat None as “not exposed” rather than “false”.
Relation to WAI-ARIA (interpretation): AccessKit roles derive from Chromium’s ax::mojom::Role, which includes every ARIA role plus host-language and platform roles (window, document, list-marker, etc.); the Core Accessibility API Mappings that WAI-ARIA relies on are implemented inside the AccessKit adapters. A NUIF role vocabulary can therefore be lowered to AccessKit Role in the editor and to ARIA in a web adapter from one intent-level definition, subject to a mapping table that this record has not verified.
Serialization: with the serde feature a TreeUpdate is a plain data value; with schemars a JSON Schema can be generated, which makes the tree usable as a fixture format.
NUIF relevance
Borrow
TreeUpdateas the wire format for “inspect accessibility semantics” (QA item 5), because it is stable, serialisable, schema-describable and already consumed by two Rust test harnesses.ActionRequestas the pointer-free action channel for “execute semantic transactions” (QA item 3) in GUI wiring tests, becauseAction::Click,Focus,SetValue,SetTextSelection,ScrollIntoViewcover editor chrome interactions without coordinates.accesskit_consumer::TreeplusChangeHandleras the diff oracle for shell state across frames, becausenode_added/node_updated/node_removedcallbacks give a deterministic change log.
Adapt
- NUIF entity identity must be carried through the tree;
author_id(existing) andhtml_id(0.25.0) are the candidate properties forEntityIdor stable names, so queries by NUIF identity remain possible after lowering. - NUIF roles should be defined at intent level (spec 13) and lowered to AccessKit
Roleby the editor adapter; the record for nuif:research:accessibility-semantics already requires adapters to report semantic equivalence rather than claiming identical platform trees. - Version coupling must be isolated in the editor crate; the kittest/accesskit_consumer pin conflict in egui shows that harness, consumer and adapter versions drift independently.
Reject
- Making AccessKit
RoleorNodenormative NUIF state, because the schema is Chromium-derived, platform-oriented and includes resolved bounds and focus rather than authored intent. - Using platform adapters (UIA, AT-SPI) as the test surface, because they require a window and a live accessibility bus; the consumer tree already provides the same information headlessly.
Open questions
- Whether AccessKit issue #701 (public construction of
accesskit_consumer::NodeId) is closed in 0.39, removing theunsafehack in masonry_testing. - Whether the multi-tree (
TreeId, subtree) model introduced in recent releases can represent NUIF component instances as grafted subtrees. - Whether a published role-mapping table from AccessKit
Roleto ARIA roles exists; the mapping is implied by the Chromium lineage but not documented in the repository README. - Whether the planned web adapter will allow the same
TreeUpdateto drive DOM ARIA attributes for a browser build of the editor.
Adobe UXP host integration, permissions, mutation boundaries, and distribution
Document status:
reviewed. Canonical source.
Summary
Current NUIF disposition: ADR 0012 removes this host from the active adapter inventory and delivery queue in favor of Affinity interchange and Canva Apps SDK adoption. This record remains reviewed historical prior art; the UXP facts below are not an active implementation commitment.
UXP is a JavaScript/HTML plugin runtime whose manifest selects one Adobe host,
entry points and explicit permissions. The current InDesign manifest reference
lists Photoshop (PS), InDesign (ID) and XD (XD) host identifiers. A
plugin can request user-mediated filesystem access without requesting full disk
access, and can omit network permission entirely. InDesign supports UXP scripts
from version 18.0 and packaged plugins from 18.5. A plugin is distributed as a
host-specific .ccx package through direct distribution or the Creative Cloud
Marketplace; a Marketplace listing requires a Developer Distribution plugin
identifier and review.
Photoshop exposes a UXP document object model and a lower-level batchPlay
action-descriptor API. Adobe recommends the document object model first and
batchPlay for gaps. Every state-changing call must run within
core.executeAsModal, which provides exclusive mutation scope, cancellation,
progress and history suspension. The UXP XMP module can read and write
namespaced metadata, providing a place for NUIF identity/provenance when the
host document type preserves XMP.
NUIF should therefore ship one adapter profile and package per Adobe host, not one generic “Adobe” binary. The first public profile should target InDesign pages and simple page items because its document model is authored layout. A Photoshop profile must remain narrower and classify responsive layout, components and interactions as unsupported or preserved metadata. No retrieved primary source establishes an Illustrator UXP host identifier, so an Illustrator package must use a separately researched public SDK rather than assuming the InDesign/Photoshop UXP contract applies.
Evidence
- A UXP manifest defines one
host, entry points and permissions. The current host union in the InDesign reference isPS,IDorXD; incompatible plugins do not install or appear for that host.localFileSystemcan beplugin,requestorfullAccess, while network, process launch, webview and inter-plugin communication require separate declarations. Locator: Adobe InDesign UXP, Plugin manifest,HostDefinitionandPermissionsDefinition, retrieved 2026-08-30: https://developer.adobe.com/indesign/uxp/plugins/concepts/manifest/. - InDesign supports
.idjsUXP scripts from 18.0 and plugins from 18.5. Scripts have modal UI only; plugins may expose commands or persistent panels. Packaged plugins use.ccx. Locator: UXP Scripts and Plugins, comparison table, retrieved 2026-08-30: https://developer.adobe.com/indesign/uxp/introduction/next-steps/script-and-plugin/. - UXP file access is sandboxed by default. InDesign documents
localFileSystem: requestas the user-mediated choice and warns against asking forfullAccesswithout need. Locator: File operations, manifest permission and sandbox sections, retrieved 2026-08-30: https://developer.adobe.com/indesign/uxp/resources/recipes/file-operation/. - InDesign packages are created by UXP Developer Tool. A Marketplace package needs an identifier from Developer Distribution; packaged output should be installed and tested before publication. Locator: Packaging, retrieved 2026-08-30: https://developer.adobe.com/indesign/uxp/introduction/next-steps/distribution/packaging/.
- Adobe documents Marketplace and direct
.ccxdistribution as separate channels. Direct packages show trust warnings; Marketplace submission uses Developer Distribution. Locator: Distribution Options, retrieved 2026-08-30: https://developer.adobe.com/indesign/uxp/introduction/next-steps/distribution/distribution-options/. - Photoshop UXP exposes
require('photoshop').app, including active/open documents. State changes must execute throughcore.executeAsModal. Locator: Photoshop UXP, Photoshop API, overview and modal example, retrieved 2026-08-30: https://developer.adobe.com/photoshop/uxp/ps_reference/. - Adobe describes
batchPlayas the lower-level action-descriptor API and recommends the document object model before using it. Object IDs are preferred to indices because indices can change during the session. Locator: BatchPlay Details, overview and action references, retrieved 2026-08-30: https://developer.adobe.com/photoshop/uxp/ps_reference/media/batchplay/. executeAsModalgives one plugin exclusive mutation access, exposes cancellation/progress, and provides the history-state boundary for new code. Locator: Modal Execution in an Async World, retrieved 2026-08-30: https://developer.adobe.com/photoshop/uxp/ps_reference/media/executeasmodal/.- The UXP XMP module reads, modifies and serializes namespaced metadata and can
operate on host-provided packets or files. Locator:
require('uxp').xmp, retrieved 2026-08-30: https://developer.adobe.com/photoshop/uxp/2022/uxp-api/reference-js/modules/uxp/xmp/getting-started/xmp/. - Adobe’s Illustrator developer landing page describes HTML panels but the retrieved UXP host manifest does not list Illustrator. This is recorded as an evidence gap, not as proof that Adobe has no private or other Illustrator SDK. Locator: Illustrator developer landing page, retrieved 2026-08-30: https://developer.adobe.com/illustrator/.
Mechanism
An InDesign adapter panel requests a NUIF file with
localFileSystem: request, parses the canonical document under NUIF resource
limits, maps one declared page/page-item subset through the host DOM, and emits
a HostAdapterReport. The report records the InDesign version, UXP API
version, direction, profile, canonical hash, host-object correspondence and
per-property fidelity. Export performs the inverse mapping and writes the NUIF
document plus the same report contract.
A Photoshop adapter uses the DOM for covered document/layer operations and
isolated batchPlay descriptors only for covered gaps. All host mutations run
inside one cancellable executeAsModal call and one history state. Stable
NUIF identifiers are stored in a NUIF XMP namespace only when the target file
and workflow preserve XMP; otherwise the report marks identity as session-only
and synchronization cannot be claimed.
Pure UXP JavaScript is the default delivery because it is host portable at the
plugin level and avoids native architecture packaging. A hybrid/native plugin
is justified only after profiling proves that canonical parsing or conversion
cannot meet the bounded profile in JavaScript. Each production .ccx targets
one host and has a version stream independent from the NUIF editor.
NUIF relevance
Borrow explicit least-privilege permissions, host-version gates, modal mutation scope, user cancellation and host-specific package compatibility.
Adapt Photoshop history suspension and Figma undo grouping to NUIF’s transaction boundary. Every import/export produces a host report rather than claiming source-byte spans that an API host does not have.
Reject one generic Adobe adapter, unconditional fullAccess, unrestricted
network access, raw batchPlay as the primary model, and an Illustrator UXP
claim without a current primary contract.
Open questions
- Which InDesign object properties can retain a NUIF entity identifier through copy, package, IDML export and reopen?
- Which Photoshop file formats and save paths preserve a custom NUIF XMP namespace byte-for-byte?
- Which current public Illustrator SDK is appropriate for a bounded vector document profile, and what is its package/update contract?
- Does a pure JavaScript canonical NUIF parser meet the same hostile-input time and memory ceilings in each UXP host?
Forward affine image-paint coordinates and inverse sampling
Document status:
reviewed. Canonical source.
Summary
An image transform needs a declared coordinate direction, matrix layout, reference box, composition order and clipping rule. Six unnamed numbers are not interoperable semantics. NUIF uses the conventional forward 2D affine matrix layout and inverse-maps destination pixel centers during rasterization.
Evidence
- CSS Transforms Level 1 defines the current transformation matrix as the
mapping from local coordinates into the parent/viewport coordinate system and
represents
matrix(a,b,c,d,e,f)as[a c e; b d f; 0 0 1]. - The HTML Canvas transform API uses the same six-value layout. Rendering under a current transformation therefore has the same forward-coordinate reading, even though a rasterizer normally evaluates it through inverse sampling.
- Figma’s current plug-in
Transformis the top two rows of an affine 3×3 matrix, with identity[[1,0,0],[0,1,0]]; itsImagePaint.imageTransformcontrols crop positioning. This supports a direct adapter mapping for the matrix values, but does not by itself specify every NUIF fit/crop/sampling interaction. Sources: https://developers.figma.com/docs/plugins/api/Transform/ and https://developers.figma.com/docs/widgets/api/type-ImagePaint/.
Executable decision
After crop selection and fit calculation, (u,v) denotes the selected crop’s
unit square and (p,q) denotes normalized coordinates in the untransformed
fitted rectangle. The authored matrix is forward:
[p] [a c tx] [u]
[q] = [b d ty] [v]
[1] [0 0 1] [1]
The reference rasterizer clips to the entity rectangle, evaluates destination
pixel centers, applies the exact inverse matrix, rejects samples outside the
half-open crop unit square, then performs the declared nearest or fixed-weight
bilinear sample. Fit precedes this matrix. Crop selection follows inverse
mapping. The transform origin is (0,0); rotation around the center is encoded
by the caller in tx/ty.
The executable bound accepts finite components with absolute value at most
1,000,000, determinant magnitude at least 1e-12, and inverse components at
most 1,000,000. Singular or numerically unbounded authored transforms remain in
the document but lower to item-level unsupported fidelity. A manually supplied
invalid render command is rejected atomically.
Flip, clockwise rotation, translation, singular-matrix and repeatability
fixtures run through cargo xtask gate-i-image. This proves the reference CPU
semantics. It does not prove that a vendor host uses the same fit/crop
composition until live adapter trials compare named host versions.
NUIF relevance
The declared direction and inverse-sampling rule make image transforms portable core values instead of renderer-specific conventions. Adapters can classify a target mismatch explicitly, while the reference renderer and future foreign implementations share one falsifiable coordinate contract.
Affinity interchange surface and NUIF adoption path
Document status:
reviewed. Canonical source.
Summary
The all-new Affinity is a no-cost desktop application combining vector, photo and page-layout tools. That lowers the participation cost for live foreign-host trials and makes it a useful desktop adoption target. The reachable technical surface is much narrower than the product surface: official Affinity material documents SVG/PDF and other file interchange, but the reviewed material does not publish a stable document-object API, scripting SDK or schema for native Affinity files.
NUIF should therefore use Affinity first as a user-mediated interchange oracle,
not pretend that it has a native plug-in. The first profile composes the existing
bounded nuif-svg-0 adapter with a named Affinity import/export trial. Native
.af, .afdesign, .afphoto and .afpub bytes are opaque evidence. A public
Affinity API or native NUIF import would justify a new profile later.
Evidence
- Canva announced the all-new Affinity as one application combining photo editing, vector design and page layout and stated that it is free for everyone. The download uses an existing or newly created free Canva account. Locator: Canva Newsroom, Introducing the all-new Affinity: Professional design, now free for everyone, announcement and availability sections, 2025-10-29: https://www.canva.com/newsroom/news/all-new-affinity/.
- The official Affinity Designer 2 feature material documents a shared Affinity file family, PDF import, SVG import/export and export of slices, layers, pages and artboards to SVG/PDF and raster formats. This establishes file interchange, not the internals of the all-new native encoding. Locator: Affinity Designer 2, Key features, interoperability and file control/import/ export sections, retrieved 2026-08-31: https://affinity.help/designer2/en-US.lproj/pages/Introduction/keyFeatures.html.
- Canva’s current Connect API design-import overview lists
application/affinityand the.af,.afdesign,.afphotoand.afpubextensions as accepted input. It also lists PDF, AI and PSD. The endpoint accepting bytes does not publish the Affinity object model or return native Affinity structure. Locator: Canva Connect APIs, Design imports, supported file formats, retrieved 2026-08-31: https://www.canva.dev/docs/connect/api-reference/design-imports/. - A search of the official Affinity product/help sitemaps and current developer- oriented material on 2026-08-31 located no published scripting or document- object reference. A staff response on the official forum to a 2025 request for JavaScript API access said there was no additional release information. This is an evidence gap, not proof that no private or future API exists. Locator: official Affinity forum, 2025 Plugin Development with API Access using JavaScript, staff response, retrieved 2026-08-31: https://forum.affinity.serif.com/index.php?/topic/228053-2025-plugin-development-with-api-access-using-javascript/.
Options considered
Native Affinity parser
Rejected for the first profile. File extensions and Canva import support do not constitute a schema. Reverse engineering would create a release-by-release compatibility burden, uncertain preservation behavior and an unsupported trust boundary. Native bytes may be retained as content-addressed opaque evidence.
Desktop UI automation
Rejected as an adapter contract. Pointer/keyboard automation is fragile, platform-specific and cannot prove document semantics or atomic mutation. It may assist a recorded manual trial, but cannot establish an executable profile.
PDF bridge
Useful for render/reference comparison but not the first editable bridge. PDF can flatten structure, fonts and authoring intent; it cannot satisfy a semantic round-trip claim without a narrower PDF profile and property-level fidelity.
SVG bridge
Selected for the first experiment because both sides document the format and NUIF already has a strict executable SVG subset. The tradeoff is deliberately narrow coverage: paths, transforms, CSS, effects and external resources remain excluded until the NUIF SVG profile itself admits them.
Adoption path
- Publish a small versioned fixture kit containing canonical NUIF, bridge SVG, expected report and render reference for the existing SVG subset.
- Run user-mediated import/export trials in named Affinity and operating- system versions. Retain both SVGs, native file only as opaque provenance, renders, environment and property-level fidelity.
- Require a second reviewer and at least two operating systems before calling the bridge interoperable. Do not infer automation, identity persistence or undo behavior from file output.
- Present the fixture kit and capability matrix to the Affinity/Canva team as
an adoption proposal: native
.nuifimport/export, a documented scripting/ document API, or a published extension-preservation container would each unlock a stronger profile. - Version any future API-backed adapter separately. Do not broaden
nuif-affinity-svg-bridge-0in place.
NUIF relevance
Borrow the practical distinction between an authored desktop application and the file formats it can exchange, plus the value of a no-cost foreign runtime for repeatable human interoperability trials.
Adapt the existing SVG adapter as a checked-in bridge and carry native Affinity files as digest-pinned opaque provenance with explicit user and environment evidence.
Reject undocumented native-format parsing, pointer automation as a semantic oracle, and any claim that Canva’s ability to ingest an Affinity file exposes Affinity’s internal schema.
Open questions
- Which SVG constructs does the all-new Affinity preserve structurally across import/export on each desktop platform?
- Are IDs or names retained predictably enough for trial-local correspondence, and are unknown SVG namespaces preserved or discarded?
- Which exact font, text-range, unit, color-profile and page/artboard settings alter output?
- Will Affinity publish a stable scripting/document API or native format extension mechanism, and under what compatibility and distribution policy?
Alembic as a baked, time-sampled, non-procedural interchange cache
Document status:
reviewed. Canonical source.
Summary
Alembic is an open interchange framework from Sony Pictures Imageworks and ILM that “distills complex, animated scenes into a non-procedural, application-independent set of baked geometric results”. Its documentation states that it is “very specifically NOT concerned with storing the complex dependency graph of procedural tools”, that it “is not a dependency graph, nor a procedural data transformation tool”, and that it would not be used “to make lossless round trips out of and into the same computation context”. The data model is an archive containing a hierarchy of objects, each with compound, scalar and array properties whose values are stored as indexed samples related to time by a TimeSampling (uniform, cyclic or acyclic). Schemas (AbcGeom PolyMesh, Xform, Camera, Curves, Points, SubD, …) are conventions over this property model; ad hoc data lives in userProperties.
The Ogawa back end (Alembic 1.5.0, 2013) replaced HDF5 with a format optimized for multi-threaded reads and deduplicates array samples by a MurmurHash3 128-bit digest so repeated samples are written once. AbcCoreLayer adds read-time layering of multiple archives (sparse overrides), and AbcCoreFactory selects a back end on open. Alembic is therefore the canonical example of a resolved-only cache: it is what NUIF’s “resolved layer” would look like if the authored layer were discarded.
Evidence
- Purpose statement, “baked geometric results”, analogy to rendered images, and the sentence that Alembic “will not attempt to store a representation of the network of computations (rigs, basically)” — http://www.alembic.io/ (Introduction, retrieved 2026-08-29).
- “Alembic Is Not” list: not a dependency graph, not a replacement for native scene formats, not an asset manager, not a rigging storage solution; “Would Not Be Used” list includes transporting procedural rigs and “lossless round trips out of and into the same computation context” — http://www.alembic.io/ (sections “What is Alembic?”, retrieved 2026-08-29).
- Positioning as “the greatest common divisor between applications, the ‘periodic table of cg primitives’” — same page.
TimeSamplingTypesemantics: Uniform (start time plus fixed interval), Cyclic (N samples distributed over a cycle, e.g. shutter open/close), Acyclic (strictly increasing explicit times enabling bisection search for floor/ceiling/nearest) —lib/Alembic/AbcCoreAbstract/TimeSamplingType.hlines 48–138 (master, retrieved 2026-08-29).- Archive writer pools
TimeSamplingobjects; index 0 is reserved for identity uniform sampling; array compression level is a hint implementations may ignore —lib/Alembic/AbcCoreAbstract/ArchiveWriter.hlines 63–84. ArraySampleKeyholdsnumBytes, original and read POD, and aDigest;ArraySample::getKeycomputes it withMurmurHash3_x64_128—AbcCoreAbstract/ArraySampleKey.hlines 47–58;ArraySample.cpplines 73–126;Util/Digest.hstoresuint64_t words[2].- Ogawa
WrittenSampleMap: “A Written Sample ID is a receipt that contains information that refers to the exact location in an Ogawa file that a sample was written to” and is “used to ‘reuse’ an already written sample by linking it from the previous usage”;find(key)returns the prior receipt —lib/Alembic/AbcCoreOgawa/WrittenSampleMap.hlines 48–70. - Ogawa release notes (Alembic 1.5.0, 2013-07-22): 5–15% smaller files, ~4x single-thread and up to 25x multi-thread read improvement over HDF5, HDF5 kept for backward compatibility, “explicit hierarchical deduplication (OObject::addChildInstance)”, hierarchical hash keys (
IObject::getPropertiesHash,getChildrenHash),AbcCoreFactory::IFactory—NEWS.txtlines 1175–1200. - Library layering:
AbcCoreAbstract,AbcCoreOgawa,AbcCoreHDF5,AbcCoreLayer,AbcCoreFactory,Abc,AbcGeom,AbcMaterial,AbcCollection,Ogawa,Util— repository listinglib/Alembic/(retrieved 2026-08-29). AbcCoreLayer::OrImplcomposes an object from a vector of top-levelObjectReaderPtrs across archives (std::vector<AbcA::ObjectReaderPtr>& iTops) —lib/Alembic/AbcCoreLayer/OrImpl.hlines 50–62;ArImpl::getTopcollects each archive’s top object in list order —ArImpl.cpplines 138–156.- Layer merge rules:
CprImpl::inititerates compounds in order, honours property metadataprune == "1"(“since pruning is more destructive, it trumps replace”) andreplace == "1"(clears previously merged children), and merges compounds child-wise —lib/Alembic/AbcCoreLayer/CprImpl.cpplines 203–290. - HDF5 is optional (
-DUSE_HDF5=ON); dependencies are CMake 3.29+, C++11, Imath 3 —README.txtlines 1–60. - License BSD-3-Clause with Lucasfilm and Sony Pictures Imageworks copyright —
LICENSE.txtlines 1–12; latest release v1.8.12 published 2026-07-02 (GitHub releases API, retrieved 2026-08-29). - Recent releases are dominated by fuzzer-driven hardening fixes to Ogawa readers (buffer overruns on malicious dimensions, excessive allocation, infinite recursion) —
NEWS.txtlines 5–380. - USD documents
customproperties as equivalent to AlembicuserProperties— OpenUSDpxr/usd/usd/property.hlines 179–185 (cross-reference).
Mechanism
An Alembic archive is an immutable, write-once tree: one top object, child objects with headers (name, metadata), and per-object compound properties containing scalar or array properties. Every property value is a sample addressed by integer index; the property’s TimeSampling maps indices to times. Uniform and cyclic samplings are described by a start time and a period; acyclic sampling stores an explicit strictly increasing time list, and readers use bisection for floor, ceiling and nearest lookups. Static data is a property with one sample. Nothing in the format encodes how a sample was produced; interpolation, rig evaluation and simulation are all upstream. Consequently, there are no override semantics, no references between archives in the core model and no notion of an unresolved value.
Ogawa is a group/data tree with fixed-size headers designed for lock-free parallel reads. On write, each array sample is hashed (MurmurHash3 128-bit over bytes plus POD size); the WrittenSampleMap maps the key to a receipt with the file location of the previously written sample, so repeated samples (typical for static or partially animated properties) are written once and referenced thereafter. addChildInstance extends this to whole object subtrees. Hierarchical hashes on objects allow subgraph comparison across archives without decoding samples. Layering (AbcCoreLayer) is a read-time overlay: the factory opens several archives and presents a merged object hierarchy. Compound properties are merged child-wise in archive-list order; a property whose metadata carries replace = "1" discards previously merged data for that name, and prune = "1" removes the name entirely, with prune taking precedence over replace. This is the format’s only override mechanism, it operates on property names rather than semantic identities, and it is external to the archive.
Loss is by design: identity is a path in the object hierarchy, geometry is explicit vertex data, and the mapping back to authoring constructs (rig controls, procedural nodes, construction history) exists only in the producing application. The documentation states the intended usage boundary explicitly: hand-off between disciplines, not round trips into the originating computation context.
NUIF relevance
Borrow
- Use content-hash deduplication of resolved samples (Ogawa
WrittenSampleMap) for NUIF resolved caches keyed by evaluation context, so repeated layouts across breakpoints or states are stored once. - Adopt explicit sampling-domain descriptors (the
TimeSamplingpattern) for NUIF resolved state indexed by evaluation context (viewport, theme, state), with identity context reserved as index 0. - Reuse the “cache for hand-off” framing to define NUIF’s resolved-only export profile as a legitimate but declared lossy lowering for renderers and runtimes that do not need authored intent.
- Adopt fuzzer-driven hardening of binary readers as a conformance activity; Alembic’s release history shows parsers of baked data are the attack surface.
Adapt
- Alembic’s
userPropertiesare unschematized escape hatches; NUIF must namespace such data as extensions with used/required declarations rather than free-form properties. - Read-time layering across archives is a useful operational pattern but must be lifted into NUIF’s authored composition model with provenance rather than remaining an external merge.
- Hierarchical hashes for subgraph comparison map to NUIF canonical snapshot hashes, but NUIF hashes must exclude transport-only differences per spec/08.
Reject
- Resolved-only storage as the interchange truth: Alembic’s own documentation excludes lossless round trips, which is exactly the property the NUIF thesis requires (RFC 0003).
- Path-based identity: moving an object in an Alembic hierarchy changes its identity; NUIF identity is semantic and path-independent.
- Absence of an override or opinion model: NUIF resolved state must remain scoped to a context and never replace authored intent, whereas Alembic has no authored layer to protect.
Open questions
- Whether NUIF should specify a standalone “resolved cache” package profile (Alembic-like) with a mandatory back-reference to the authored document hash, or only allow resolved caches embedded in a full package.
- How much of Ogawa’s parallel-read layout is relevant to UI documents whose resolved data is small relative to 3D caches.
- Whether hierarchical content hashes should be normative for NUIF diff of resolved output across implementations.
Tree differencing with moves (Chawathe et al. 1996, GumTree 2014) and structural merge/diff tools (Mergiraf, difftastic)
Document status:
reviewed. Canonical source.
Summary
Chawathe, Rajaraman, Garcia-Molina and Widom (SIGMOD 1996) defined the tree change-detection problem as finding a minimum-cost edit script over insert, delete, update and subtree move, split it into finding a matching and then generating a conforming script, and gave an O(ne + e²) algorithm under domain assumptions. GumTree (ASE 2014) keeps the Chawathe script generator but replaces matching with a two-phase heuristic: a greedy top-down search for the largest isomorphic subtrees, then a bottom-up phase that matches containers by Dice similarity of already matched descendants and recovers further matches with an optimal tree-edit-distance algorithm on small subtrees; worst case O(n²). Mergiraf (2024-) applies GumTree classic matching to base/left/right trees, converts them to parent-child-successor triples, merges the triple sets, and emits conflict nodes or falls back to diff3 for the affected element; it treats designated “commutative parents” specially and identifies their children by signatures. Difftastic computes a diff as a lowest-cost path with Dijkstra’s algorithm over pairs of tree positions and does not detect moves. Across these systems the expensive and heuristic step is matching; every subsequent step (edit script, three-way merge) is defined relative to a matching. When entities carry stable identifiers, the matching is the identity map and the residual problems are ordering and move conflicts.
Evidence
- Chawathe et al.: DOI 10.1145/233269.233366, SIGMOD 1996 pp. 493-504 (SIGMOD Record 25(2), PDF retrieved 2026-08-29 from https://sigmodrecord.org/1996/06/24/change-detection-in-hierarchically-structured-information/). Edit operations
INS,DEL(leaf only; interior deletion requires moving descendants first),UPD,MOVof a subtree (§3.1); an edit script conforms to a partial matching if it does not insert or delete matched nodes (§3.2); cost model with acomparefunction in[0, 2]for updates (§3.3); five phases update, align, insert, move, delete in one breadth-first scan of the new tree plus a post-order delete pass (§4.1, §4.2); LCS-based child alignment yields the minimum number of moves (§4.2); Matching Criterion 1 (leaves: equal labels andcompare ≤ f,0 ≤ f < 1), Criterion 2 (internal nodes: fraction of common leaves> t,1/2 ≤ t < 1), Assumption 1 (acyclic labels), Assumption 2 (at most one close leaf), Theorem 5.1 (unique maximal matching is the best matching), algorithm FastMatch (§5); running timeO(ne + e²)withnleaves andethe weighted edit distance (§1). - GumTree problem statement: actions
update,add,delete,move(t, tp, i)moving a subtree; the shortest script with moves is NP-hard; the best add/delete/update algorithm (RTED) isO(n³). ASE 2014 PDF §2 (retrieved 2026-08-29 from https://www.labri.fr/perso/xblanc/data/papers/ASE14.pdf). - GumTree top-down phase (Algorithm 1): height-indexed priority lists, processing nodes of equal greatest height, isomorphism by hash then exact test, ambiguous candidates ranked by
dice(parent(t1), parent(t2), M), only nodes with height greater thanminHeight. §3.1. Dice:dice(t1, t2, M) = 2·|{t1' ∈ s(t1) | (t1', t2') ∈ M}| / (|s(t1)| + |s(t2)|). - GumTree bottom-up phase (Algorithm 2): a candidate
cfor unmatched internalt1requires equal labels,cunmatched, and matched descendants; the candidate with greatest Dice is matched ifdice > minDice; when the remaining subtrees are both smaller thanmaxSizean optimal algorithmopt(RTED) recovers descendant mappings for same-label nodes. §3.2. - GumTree script generation: “RTED does not handle moves,” so the script is produced with Chawathe et al.’s algorithm from the mappings. §2 and §3.3 (Complexity Analysis).
- GumTree complexity: worst case
O(n²),n = max(|T1|, |T2|), from the Cartesian products in both phases. §3.3. Replication settings:minHeight = 2,minDice = 0.5,maxSize = 100. §5.2.3. - GumTree implementation (main branch, retrieved 2026-08-29):
core/src/main/java/com/github/gumtreediff/matchers/heuristic/gt/AbstractSubtreeMatcher.javausesPriorityTreeQueue,HashBasedMapper,DEFAULT_MIN_PRIORITY = 1, optionsst_minprio,st_priocalc(defaultheight);GreedySubtreeMatcher.javaresolves ambiguous mappings by sorting on maximum subtree size thenFullMappingComparator;GreedyBottomUpMatcher.javausesDEFAULT_SIM_THRESHOLD = 0.5,DEFAULT_SIZE_THRESHOLD = 1000, optionsbu_minsim,bu_minsize, andZsMatcherfor last-chance matching;actions/ChawatheScriptGenerator.javawalks the destination tree breadth-first emittingInsert,Update,Move, aligns children with LCS, computes positions withfindPos, and emitsDeletein post-order. The code defaults (min priority 1, size threshold 1000) differ from the paper’s (minHeight2,maxSize100). - GumTree README cites hyperparameter optimisation (Martinez et al., IEEE TSE 2023, DOI 10.1109/TSE.2023.3315935) and a scalable variant (Falleri and Martinez, ICSE 2024, DOI 10.1145/3597503.3639148). https://github.com/GumTreeDiff/gumtree (retrieved 2026-08-29).
- Mergiraf architecture: tree-sitter parsing with multi-line leaves split into lines; “the GumTree classic algorithm” in top-down and bottom-up phases applied to base-left, base-right and left-right; class mapping with leader preference base, left, right; conversion to parent-child-successor triples
(p, c, s)with sentinels; quadruplets tagged by revision merged into a possibly inconsistent set with base triples removed when contradicted; commutative parents merged by applying right’s deletions and appending right’s additions; signature-based duplicate detection; delete/modify conflicts decided by a covering check that distinguishes moves; fallback to diff3 on the parent element’s source; output node kindsExactTree,Conflict,LineBasedMerge,MixedTree,CommutativeChildSeparator; fast mode cannot resolve “moving edited elements”. https://mergiraf.org/architecture.html (retrieved 2026-08-29). - Mergiraf conflict classes: commutative insertions (e.g. class members) are auto-resolved; order-dependent insertions (statements in a block, function arguments) are not; duplicate signatures under commutative parents are flagged; moved-and-edited code is replayed at the new location. https://mergiraf.org/conflicts.html (retrieved 2026-08-29). Repository: Rust, GPL-3.0, 24 releases. https://codeberg.org/mergiraf/mergiraf (retrieved 2026-08-29).
- Difftastic: a diff is “a route finding problem on a directed acyclic graph”; a vertex is a pair of positions in the two trees; edges marking a node novel cost more than matching; Dijkstra’s algorithm finds the lowest-cost route with vertices constructed lazily. https://difftastic.wilfred.me.uk/diffing.html (retrieved 2026-08-29). Tricky cases: no move detection; sliders; preference for matches at the same nesting depth. https://difftastic.wilfred.me.uk/tricky_cases.html (retrieved 2026-08-29). Repository: Rust, MIT, tree-sitter parsers, no merge or patch output. https://github.com/Wilfred/difftastic (retrieved 2026-08-29).
Mechanism
Problem decomposition (Chawathe et al. §1, §3; adopted by GumTree §2):
input: T1 (old), T2 (new)
step 1: matching M ⊆ nodes(T1) × nodes(T2), partial injective, label-preserving
step 2: edit script E conforming to M, minimising Σ cost(op)
ops: INS(x, parent, k, l, v) | DEL(x) | UPD(x, v) | MOV(x, parent, k)
Script generation from a matching (Chawathe §4.1; GumTree ChawatheScriptGenerator):
for y in BFS(T2):
if y unmatched: x := INS(new, partner(parent(y)), findPos(y)); M += (x, y)
else x := partner(y):
if value(x) ≠ value(y): UPD(x, value(y))
if partner(parent(y)) ≠ parent(x): MOV(x, partner(parent(y)), findPos(y))
alignChildren(x, y): keep LCS of matched children fixed, MOV the rest
for x in postorder(T1): if x unmatched: DEL(x)
GumTree matching:
top-down: process nodes by decreasing height; isomorphic subtrees (hash, then exact) are mapped wholesale;
ambiguous candidates ranked by Dice of their parents; stop below minHeight
bottom-up: for unmatched internal t1 in post-order: candidates c with label(c)=label(t1), c unmatched,
matched descendants; match argmax dice if dice > minDice;
if |t1|,|t2| < maxSize: run optimal edit distance (RTED / Zhang-Shasha) on the residue
Three-way structured merge (Mergiraf, PCS form):
PCS(T) = {(p, c, s) | s is the immediate successor of c under p} ∪ sentinels
merge = PCS(base)^tagged ∪ PCS(left) ∪ PCS(right) minus base triples contradicted by a side
inconsistency (two successors for one (p, c), two parents for one (c, s)) → Conflict node or diff3 fallback
commutative parent: children set = left ∪ (right additions) minus (right deletions); duplicates by signature → conflict
Identity observation (NUIF interpretation): with stable entity identifiers M = {(x, y) | id(x) = id(y)}, which removes both GumTree phases and the ambiguity they resolve heuristically. The residue is exactly the class Mergiraf cannot resolve automatically: order-dependent insertions and moved-and-edited elements, i.e. sibling-order and move conflicts.
NUIF relevance
Borrow
- The Chawathe edit-operation vocabulary (insert, delete, update, move-subtree) and the conforming-script discipline: NUIF operations in nuif-protocol already mirror it (
Insert,Remove,Rename/SetExtension,Move). - Mergiraf’s commutative-parent and signature concepts: NUIF relation sets and unordered property maps are commutative parents by construction, and entity IDs are exact signatures.
- Conflict nodes embedded in the merged tree (Mergiraf
Conflict) as the representation for spec/06’s typed conflicts.
Adapt
- GumTree’s matcher is still needed at the boundary: importing documents from formats without stable IDs (SVG, Figma exports, generated code) requires a matching step, and GumTree-style heuristics with tuned thresholds are the reference for that import path.
- The LCS-based child alignment (Chawathe §4.2) should be reused to convert two child sequences into a minimal move set when NUIF diffs snapshots rather than replaying operations.
- Mergiraf’s diff3 fallback on the enclosing element corresponds to NUIF’s textual fallback for human review in the canonical text form (
nuif-text-0).
Reject
- Treating move detection as a heuristic: NUIF records moves explicitly; inferred moves are only for import.
- Difftastic’s shortest-path diff as a merge primitive: it optimises display, ignores moves, and produces no patch.
Open questions
- Which alignment cost (Chawathe’s move-minimising LCS versus a Dice-weighted variant) produces the fewest spurious
Moveoperations when diffing NUIF snapshots that differ by reordering. - Whether Mergiraf’s covering algorithm for delete/modify conflicts has an identity-based analogue that distinguishes “moved out then deleted” from “edited while deleted”.
- How to calibrate GumTree thresholds for design-document trees (wide, shallow, many identical leaves) where the paper’s defaults were tuned on Java ASTs.
Automerge and Yjs CRDT architectures
Document status:
reviewed. Canonical source.
Summary
Automerge provides Rust-backed CRDT data structures, compact change encoding and sync protocols for local-first applications. Yjs uses a modified YATA-style sequence CRDT with state-vector-based differential synchronization.
Executable NUIF boundary
nuif-collab-registers-0 exercises operation-set convergence without depending on either library. It compares pairwise-maximal and incremental-frontier materializers across every delivery order, keeps conflicts explicit and strips collaboration metadata from the checkpoint document. This is a profile-mechanism test, not evidence that Automerge and Yjs interoperate or make identical choices.
The follow-on nuif-collab-tree-0 uses the proved tree-move design and
RGA-style stable sibling origins inside NUIF. Current Automerge Rust 0.11.0 and
JavaScript 3.4.1 documentation still exposes maps, RGA lists, element cursors,
change merging and synchronization but no tree-move operation. Its merge rules
explicitly note that reverse list insertion does not always preserve insertion
order. Gate H therefore pins JavaScript 3.4.1 only as a foreign transport
oracle: seven replicas write immutable NUIF change records under distinct map
keys, and forward, reverse, even/odd, duplicate and save/load merges must
recover the exact record set. NUIF, not Automerge, materializes the tree and
reports cycle/deletion conflicts. Sources: official merge rules
https://automerge.org/docs/reference/under-the-hood/merge-rules/ and Rust 0.11.0
API https://docs.rs/automerge/0.11.0/automerge/ (retrieved 2026-08-30).
NUIF relevance
CRDTs are suitable for a collaboration profile and operation history, but their implementation-specific metadata should not become mandatory content of every canonical NUIF document. Saved documents must remain portable to non-collaborative implementations.
Content-addressed behavior attachment without canonical-schema coupling
Document status:
verified. Canonical source.
Summary
The first behavior wire experiment should be one canonical CBOR resource in
the existing content-addressed package, not a new field in the canonical
Document and not a new ZIP member family. nuif-package-0 already binds each
resource’s exact bytes, size, media type and role into the same manifest as the
canonical document descriptor. Reusing that mechanism gives deterministic
delivery, digest verification, old-reader preservation and package-level
binding without freezing the semantic model.
The selected nuif-behavior-package-resource-0 profile admits exactly one
embedded source resource with provisional media type
application/nuif-behavior+cbor. Its bytes are canonical nuif-cbor-0 for a
validated nuif-behavior-state-machine-0 program. The package manifest also
declares that behavior profile as required. Generic package decoding verifies
and preserves the bytes but does not interpret or execute them; an explicit
behavior API checks cardinality, descriptor policy, canonical bytes and every
entity reference against the package document.
Evidence
- OCI content descriptors require a media type, digest and raw byte size, recommend embedding descriptors in other formats for secure content reference, and require consumers to verify size and digest before heavy processing. Its image layout stores content at a digest-derived blob path. This supports NUIF’s existing resource descriptor and cheap-verification order rather than a behavior-specific locator system. Locators: https://github.com/opencontainers/image-spec/blob/main/descriptor.md and https://github.com/opencontainers/image-spec/blob/main/image-layout.md, retrieved 2026-08-31.
- EPUB 3.3 requires publication resources to be declared in the package manifest and normally transported in one OCF ZIP container. EPUB Reading Systems 3.3 explicitly distinguishes applications that merely extract or validate a container from full reading systems, which may ignore rendering requirements. This is a useful precedent for verified package access not implying execution. Locators: https://www.w3.org/TR/epub-33/#sec-manifest-elem and https://www.w3.org/TR/epub-rs-33/#sec-ocf, retrieved 2026-08-31.
- KHR_interactivity keeps portable behavior in an extension graph rather than making it arbitrary script embedded in visual nodes. Its conformance assets are separately executed by engines that implement the extension. NUIF uses a smaller finite machine, but preserves the same separation between stored graph data and an implementing runtime. Locator: https://raw.githubusercontent.com/KhronosGroup/glTF/refs/heads/main/extensions/2.0/Khronos/KHR_interactivity/Specification.adoc, retrieved 2026-08-31.
- RFC 0010 already restricts package members to
mimetype,manifest.cbor,document.cborand digest-addressed blobs. Behavior fits the registered blob path and resource manifest, so adding a specialbehavior/member or a second package profile would duplicate identity and validation machinery.
Mechanism
Attachment is an explicit operation:
validate(program, package.document)
encode canonical CBOR
add embedded source resource
declare nuif-behavior-state-machine-0 required
encode deterministic package
Opening the package has two distinct levels:
NuifPackage::decode ZIP, manifest, size, digest and document validation
require_capabilities explicit complete package/host support negotiation
attached_behavior cardinality, role, canonical CBOR and entity binding
BehaviorRuntime::new caller-supplied effect-capability authorization
There is no automatic transition from one level to the next. In particular, resource presence never grants script, filesystem, network or host mutation authority.
This separation also constrains generic authoring. An editor that cannot
interpret a required capability cannot know whether an opaque resource binds
entity references, a document hash or other semantic preconditions. Preserving
that resource while changing the document would therefore manufacture an
unvalidated pairing. The reference editor uses structural read-only mode:
inspection and exact copying are allowed, but its shared driver and save
boundary reject semantic changes with the exact missing requirement set.
The same policy now lives below the editor in nuif-api::NuifDocument:
structural SDK and WASM loads reject mutation, history, evaluation and mode
conversion until complete-set authorization succeeds. The CLI deliberately
declares an empty support set, preserves unchanged packages and rejects its
evaluation and rewrite paths with a stable capability error. This prevents a
new wrapper from accidentally weakening the editor-only boundary.
The behavior digest identifies only the behavior bytes. It does not claim to be a standalone document-specific identity. The deterministic package manifest contains both the document descriptor and behavior descriptor, and the package hash binds that pair. Transplanting the same behavior bytes into a different package produces a different package hash and the explicit behavior loader still revalidates all entity references.
NUIF relevance
This closes the first delivery gap without collapsing NUIF’s layers. The semantic document remains readable by implementations that do not implement behavior, the package remains the authority for exact resource delivery, and a behavior/runtime adapter remains the authority for execution semantics. A future standards decision can therefore compare real multi-host evidence before deciding whether behavior belongs in a required semantic profile.
Executable evidence
cargo xtask gate-behavior-package generates a deterministic .nuif fixture,
checks canonical encoding, document/package hash separation, exact round trip,
capability/resource agreement, exact generic host-capability negotiation and
hostile mismatch cases, then invokes an
independent Python standard-library ZIP reader. The foreign reader checks the
exact archive hash, member ordering and metadata, CRC reads, media marker and
content-addressed behavior bytes. It intentionally does not decode CBOR; the
Rust side checks canonical CBOR and semantic references, so the report does not
misstate container agreement as a second behavior implementation. The gate also
runs the release CLI, records its exact structural copy and requires typed
render/mode-conversion rejection without output in
target/behavior-package-cli-report.json.
Rejected alternatives
- Add behavior to
Documentnow: would turn one successful trace experiment into a canonical-schema commitment before native and presentation adapters. - Add a dedicated ZIP member: would require a new path/profile while bypassing the existing resource descriptor, digest and limit machinery.
- Store JSON for browser convenience: would introduce a second canonical encoding and numeric/string rules; host adapters can generate escaped JSON from the validated in-memory program.
- Permit linked behavior in the portable profile: would make offline behavior depend on resolver authority and retrieval availability.
- Infer the attachment from any CBOR resource: would make discovery ambiguous; the exact provisional media type and required capability are both checked.
Open questions
- Whether enough independent native/presentation adapters will justify moving a future behavior profile into canonical semantics.
- Media-type registration and final naming remain standards-track work; the current identifier is explicitly provisional and must not be advertised as IANA-registered.
Bounded portable behavior through deterministic state-machine traces
Document status:
verified. Canonical source.
Summary
Portable behavior should begin with deterministic event traces, not arbitrary host scripts or an immediately universal visual-scripting graph. SCXML provides the mature semantics needed for the first layer: external events, ordered transition selection, guards, actions and run-to-completion execution. KHR_interactivity provides a current asset-format precedent for typed variables, explicit operations, bounded validation and no-op degradation of unsupported optional operations. Its much broader operation graph is still a Release Candidate and is intentionally Turing-complete, so copying it wholesale would expand NUIF’s safety and conformance surface before basic portability is proven.
The best first experiment is therefore a flat, bounded state-machine sidecar that references stable NUIF entities and emits abstract effects as data. The same authored program is executed by independent Rust and JavaScript runtimes; complete traces, not merely final pixels, are the conformance observation.
Evidence
- SCXML 1.0 is a W3C Recommendation. Its basic model selects transitions from an active state in response to events, permits conditions and resolves multiple matches by document order. Its interpreter principles require causality, deterministic behavior without external processors and run-to-completion before another external event is processed. This supports ordered guarded transitions and one-event-at-a-time traces. Locator: https://www.w3.org/TR/scxml/#Basic, and https://www.w3.org/TR/scxml/#AlgorithmforSCXMLInterpretation, retrieved 2026-08-31.
- The current KHR_interactivity specification identifies itself as Release Candidate. It describes directed acyclic behavior graphs, strictly typed value sockets, retained custom variables, explicit static/dynamic resource limits and unsupported extension operations demoted to no-ops. It also says arbitrary scripting is not a design goal and acknowledges that its complete execution model is Turing-complete, requiring runtime limits. NUIF profile 0 consequently adopts the type, capability and limit lessons while excluding internal loops, timers and general computation. Locator: https://raw.githubusercontent.com/KhronosGroup/glTF/refs/heads/main/extensions/2.0/Khronos/KHR_interactivity/Specification.adoc, sections Introduction, Concepts, Unsupported Operations and Limits, retrieved 2026-08-31.
- Khronos submitted KHR_interactivity for ratification on 16 July 2026. The announcement describes event/control/data/state nodes, custom variables, property writes and extension-defined capabilities; unavailable companion operations degrade to no-ops. Submission is not ratification, so NUIF must retain the Release Candidate status in its decision record. Locator: https://www.khronos.org/news/press/gltf-interactivity-extension-submitted-for-ratification, retrieved 2026-08-31.
- The Khronos interactivity test-asset repository currently reports 149 self-checking cases and 831 sub-tests and recommends both manual and automated engine execution. This supports generated fixtures and machine-readable traces rather than prose-only behavior claims. Locator: https://github.com/KhronosGroup/glTF-Test-Assets-Interactivity/blob/main/Tests/Interactivity/README.md, retrieved 2026-08-31.
Mechanism
nuif-behavior-state-machine-0 admits activate events only from stable NUIF
entities carrying an activatable semantic role. The active state’s transitions
are evaluated in authored order. The first exact event and equality-guard match
executes at most 64 sequential actions, then reaches its target state. There is
no internal event generation, recursion or asynchronous continuation, so every
accepted external event terminates under a statically known action budget.
The value surface is Boolean plus bounded string. State actions set or toggle
those values. Effects are abstract visibility(Boolean) and
announcement(String) records addressed to stable entities. A host declares
the effect capabilities it implements. Missing required capabilities reject
runtime construction; unavailable optional_noop effects are skipped and
recorded. This is an explicit fallback policy, not silent loss.
The Rust gate produces a five-event fixture and expected full/required-only traces. A separately written Node interpreter validates and executes the same program, then compares every selected transition, state, variable, effect and skipped optional operation. The initial trial passes both capability runs and the required-capability refusal probe on local Node 26.7.0; CI separately pins Node 24.20.0 and records its hosted result rather than inferring it from the workflow.
NUIF relevance
Behavior should be a modular layer over stable document identity. It should not make the canonical document an event log, grant scripts authority, or force every reader to implement timers, networking and host business logic. The sidecar experiment established semantics and test vectors before a wire decision. RFC 0012 now selects one canonical-CBOR, content-addressed package resource as the first experimental transport. This reuses the existing package manifest and hash without adding behavior to the canonical semantic document. Generic readers preserve inert bytes; explicit behavior loading revalidates the program against the package document before any separately authorized runtime can be constructed.
The abstract-effect boundary also keeps target fidelity honest. A web adapter
now maps the bounded subset to native activation, DOM visibility and one status
region under nuif-web-behavior-0; a presentation runtime may map it to scene
visibility; a device profile may reject announcements. Those adapters share a
trace contract but retain independent capability and host-observation reports.
Open questions
- What multi-host evidence would justify moving a later behavior profile from a package resource into the canonical semantic model?
- Which additional event kinds can be grounded in portable semantics rather than vendor input APIs?
- What numeric type and JSON encoding can preserve exact cross-language values without JavaScript precision loss?
- Which internal events, timers and animation triggers admit a static termination/budget rule strong enough for the next profile?
- What native UI adapter can observe the same effects without confusing host agreement with visual or assistive-technology equivalence?
Bidirectional evaluation with direct manipulation (evaluation update for a general-purpose functional language)
Document status:
reviewed. Canonical source.
Summary
The paper defines an evaluation update relation for LittleLeo, an ML-style lambda calculus with lists, records and dictionaries. Forward evaluation is the standard big-step judgement E ⊢ e ⇒ v; update is the judgement (E ⊢ e) ⇐ v′ ⇝ (E′ ⊢ e′), read as “when the output is changed to v′, the program E ⊢ e becomes E′ ⊢ e′”. Update rules retrace the evaluation derivation, replacing constants and closures at the leaves and propagating new bindings back through variables, let, application and conditionals. Conflicting bindings produced by different subderivations are reconciled by an environment merge; a conservative two-way merge yields a soundness theorem (re-evaluating the updated program produces v′), whereas the optimistic three-way merge used in the implementation abandons that guarantee in exchange for propagating a single edited use of a variable to all uses. List values are updated through a diff (Keep, Delete, Insert, Update) computed by dynamic programming, and the implementation propagates edit differences instead of whole values, which is reported as the decisive optimisation (70× average speed-up). Expert users may register custom lenses (apply/update pairs) that are invoked by applyLens, with access to the internal updateApp, diff and merge primitives. The update relation is nondeterministic; the Sketch-n-Sketch implementation enumerates solutions lazily and presents them as a menu with code and output previews. Across ten HTML-generating examples (about 1400 lines), 92 update calls produced 1.18 solutions on average and took 723 ms on average, close to the 833 ms average forward evaluation time. This record separates the paper’s claims from NUIF interpretation in the sections below.
Evidence
- Bibliographic data: Proc. ACM Program. Lang. 2, OOPSLA, Article 127 (November 2018), 28 pages, DOI 10.1145/3276497; arXiv:1809.04209v2, 18 October 2018. Source: arXiv preprint header and ACM reference format block, p. 127:1 (retrieved 2026-08-29).
- The PLDI 2016 system is characterised as having four limitations: only SVG-generating programs (A), only numeric values traced (B), no user customisation (C), and trace storage cost (D); the new approach addresses all four. Source: §1, p. 127:2-3.
- Syntax of LittleLeo (constants, closures, lists, records,
applyLens,updateApp,diff,merge,freeze): Figure 6, §3, p. 127:9. - Update judgement definition and the three rule families (replacement, primitive, propagation): §3.1, p. 127:9.
- Selected rules U-Const, U-Fun, U-Var, U-Let, U-App, U-If-True, U-Freeze: Figure 7, p. 127:10. U-Plus-1/U-Plus-2 (two valid updates for addition), U-Lt (operator flip), U-And: Figure 8, p. 127:11. List rules U-Cons, U-List with Diff operations Keep, Delete, Insert(v′), Update(v′): Figure 9, p. 127:13.
- Conservative two-way merge: Definition 3.1, p. 127:11; optimistic three-way merge: Definition 3.2, p. 127:12; Example 3.3 (
let x = 1 in [x, x]updated to[1, 2]yieldslet x = 2 in [x, x]only under three-way merge), p. 127:12; Example 3.4 (control-flow deviation under three-way merge), p. 127:12-13. - Theorem 3.5 (EvalUpdate): if
E ⊢ e ⇒ vthenE ⊢ e ⇐ v ⇝ E ⊢ e. Theorem 3.6 (Conservative UpdateEval): with two-way merge, ifE ⊢ e ⇐ v′ ⇝ E′ ⊢ e′thenE′ ⊢ e′ ⇒ v′. Proof sketch in supplementary appendices. Source: §3.1.5, p. 127:14. - Structural updates are permitted only in list literals (“pretty local updates”); cons expressions are never added or removed by the core rules “because of the amount of ambiguity they would introduce”. Source: §3.1.4, p. 127:13.
- The
mappattern cannot be repaired by the core algorithm; motivation for user-defined lenses. Source: §3.2, p. 127:14-15. Lens type{ apply: a -> b, update: {input: a, outputNew: b} -> {values: List a} }and rules E-Lens, U-Lens, E-Update-App, E-Diff, E-Merge: Figure 10 and §3.2.1, p. 127:15. Example lenses for MaybeOne map (Figure 11) and control-flow repairif_(Figure 12), pp. 127:16-18. - Implementation optimisations: continuation-passing style to avoid browser stack overflow; merging only bindings free in closure bodies to avoid exponential merge; propagation of edit differences instead of values, exposed to lenses through
outputOldanddiffsfields. Source: §4.1, pp. 127:18-19. - Whitespace-preserving abstract syntax for readable updated programs: §4.1, p. 127:19.
- Ambiguity presentation: candidate repairs are shown in a nested “Update Program” menu with previews of code and output; users freeze expressions (
Update.freeze) to remove undesired solutions. Source: §2.2 and Figure 3, pp. 127:5-6. - Performance table: 10 examples, 1469 LOC total, average Eval 833±400 ms, 92 update calls, average 1.18 solutions, average optimised update 723±900 ms, 70× speed-up over the version without edit differences; Node.js 6.9.5, Intel i7-6820HQ. Source: Figure 13 and §5.2, pp. 127:22-23.
- Round-trip laws are explicitly not required; many implemented lenses “violate even the basic laws”. Source: §6, “Round-Trip Laws”, pp. 127:23-24.
- Diff alignment is a single heuristic; nested differences are unsupported (example
[x,y,z]to[x, ["b",[],[y]], z]). Source: §6, “Alignment”, p. 127:24. - Follow-up: Mayer and Chugh, “A Bidirectional Krivine Evaluator”, Bx 2019 (CEUR-WS Vol. 2355, paper 5, pp. 56-60) restates the call-by-value system with Theorem 1 (structure preservation) and Theorem 2 (soundness) and gives call-by-name and Krivine-machine variants with Theorems 3-9. Retrieved from https://ceur-ws.org/Vol-2355/paper5.pdf on 2026-08-29.
- Implementation: Sketch-n-Sketch v0.7.1, more than 12,000 lines of Elm and JavaScript added. Source: §4, p. 127:18.
Mechanism
Judgements (§3.1):
Evaluation: E ⊢ e ⇒ v
Evaluation update: (E ⊢ e) ⇐ v′ ⇝ (E′ ⊢ e′)
Replacement axioms (Figure 7):
[U-Const] E ⊢ c ⇐ c′ ⇝ E ⊢ c′
[U-Fun] E ⊢ λp.e ⇐ (E′, λp.e′) ⇝ E′ ⊢ λp.e′
[U-Var] E = E1, x ↦ v, E2 ⟹ E ⊢ x ⇐ v′ ⇝ (E1, x ↦ v′, E2) ⊢ x
Propagation through binding forms (Figure 7): U-Let re-evaluates e1 to v1, pushes v2′ into e2 under E, x ↦ v1, obtains an updated binding v1′, pushes v1′ into e1, and merges the two resulting environments. U-App does the same through a closure: the new body and new closure environment are pushed back into the function expression, the new argument value into the argument expression, and the environments are merged. U-If-True pushes into the taken branch and assumes the guard is unchanged.
Environment merge (Definitions 3.1 and 3.2):
Two-way (conservative): (E1, x↦v1) ⊕ (E2, x↦v2) = (E′, x↦v)
v = v1 if v1 = v2; v = v1 if x ∉ fv(e2); v = v2 if x ∉ fv(e1); otherwise fail
Three-way (optimistic): x ↦ (v1 ⊕_v v2), base-case rule prefers v2 when v2 ≠ v, else v1
Correctness (§3.1.5):
Theorem 3.5 (EvalUpdate): E ⊢ e ⇒ v ⟹ E ⊢ e ⇐ v ⇝ E ⊢ e
Theorem 3.6 (Conservative UpdateEval): E ⊢ e ⇐ v′ ⇝ E′ ⊢ e′ (two-way merge) ⟹ E′ ⊢ e′ ⇒ v′
Theorem 3.5 is the analogue of the lens law GetPut (putting back the unchanged view leaves the source unchanged). Theorem 3.6 is the analogue of PutGet (re-evaluating the updated program reproduces the edited output) and holds only for the conservative merge. No PutPut-style law and no determinism or completeness result is claimed; the relation is a set of candidate repairs.
List diff (Figure 9): Diff(v, v′) produces a sequence over {Keep, Delete, Insert(v′), Update(v′)} by dynamic programming that prefers long contiguous preserved runs; U-List walks the literal and the diff in parallel, inserting exp(v′) (a literal expression synthesised from a value) for insertions. Dictionaries and records use analogous difference operations without insertion/deletion for records.
User-defined lenses (Figure 10): applyLens l e evaluates l.apply e; in the backward direction, l.update {input, outputOld, outputNew, diffs} returns {values = [...]} and each candidate argument is pushed back into e. updateApp exposes U-App, diff exposes Diff, and merge exposes three-way value merge to lens code.
Ambiguity handling: all solutions are enumerated lazily; the editor previews each; freeze e (U-Freeze) pins subterms to prune the solution space.
NUIF relevance
- Borrow: The judgement shape
(E ⊢ e) ⇐ v′ ⇝ (E′ ⊢ e′)together with Theorems 3.5/3.6 is a precise template for specifying an NUIF adapter’s design-to-source direction: an adapter’s put must satisfy an EvalUpdate law (unchanged resolved view yields an empty patch) and, where it claims lossless fidelity, an UpdateEval law (re-lowering the patched source reproduces the edited document). - Borrow: Propagating edit differences rather than whole values (§4.1, Optimisation 3) matches NUIF’s patch model; the reported 70× speed-up supports designing the protocol around operation deltas with base-snapshot identity instead of full-document diffs.
- Adapt: The explicit distinction between conservative (sound, may fail) and optimistic (always succeeds, may change unrelated output) merge should surface in NUIF as a fidelity class on the patch: a patch produced under an optimistic policy must be reported as
approximatedwith the affected uses listed, never aslossless. - Adapt: The freeze primitive corresponds to NUIF correspondence records marking source regions as non-editable from the design side; NUIF should expose the same pruning to adapters but store it in the correspondence map rather than in the source program.
- Adapt: Solution enumeration with previews is an editor concern; the NUIF protocol should instead return a ranked candidate list of patches with provenance so that any editor can implement the menu.
- Reject: The absence of round-trip laws for user lenses (§6) is acceptable for an interactive programming environment but not for a conformance-tested interchange specification; NUIF adapters must declare which laws they satisfy per fidelity class.
- Reject: Update through general recursion and higher-order code (U-App into closure environments) presupposes an interpreter for the target language; NUIF adapters for Svelte, React or SwiftUI cannot assume this and should restrict the source side to a syntactic correspondence fragment (literal values, static structure) as in nuif:research:tree-sitter.
Open questions
- Which subset of the U-rules can be realised without evaluating the target program, given only a syntax tree and correspondence records? The literal-replacement rules (U-Const, U-Var into let-bound literals, U-List on literals) appear feasible; U-App does not.
- Can Theorem 3.6 be checked mechanically per patch (re-lower and compare resolved geometry) as a conformance oracle for “minimal source patch after design edit” in nuif:experiment:v0-responsive-card?
- The Diff heuristic is fixed and alignment is not nested; NUIF entities carry stable identities, which removes most alignment ambiguity for the design side, but the source side still needs an alignment policy for lists without keys.
- Performance was measured on 37-534 line programs; scaling to component libraries with thousands of lines is not reported.
Blender DNA/SDNA self-describing files, RNA/operators, memfile undo and headless regression testing
Document status:
reviewed. Canonical source.
Summary
A .blend file is a sequence of typed blocks, each headed by a BHead carrying a block code, byte length, the writer’s old memory address, an index into the writer’s struct table and an element count. The struct table (SDNA) is generated at build time by makesdna from the DNA headers and is embedded in every file in a DNA1 block, so a reader compares the file’s SDNA with its own and reconstructs each struct member-by-member by name, dropping members that no longer exist and zero-initialising new ones; explicit versioning_*.cc code then applies semantic migrations gated on file version and on DNA_struct_member_exists. RNA is a reflection layer generated by makesrna from explicit rna_*.cc definitions; it adds metadata, callbacks and update hooks and drives the UI, animation, library overrides and the bpy Python API. Every user action is an operator with typed properties, registered under a bl_idname and callable as bpy.ops.<id>(); undo steps are pushed by the operator system, and global undo (memfile) is a chunked in-memory .blend write whose chunks are compared with the previous step and shared when identical. blender --background --python script.py executes the same operator and data API without a window, and the regression suite renders scenes headlessly and compares them with reference images using oiiotool thresholds tuned per category and platform.
Evidence
- DNA “describes the data structures stored in .blend files in a reproducible, forward- and backward-compatible manner”;
makesdnaparses headers undersource/blender/makesdna/into SDNA; “When saving a .blend file, Blender embeds this SDNA block alongside the actual binary data”; on load Blender “compares the file’s SDNA to the current binary’s SDNA and performs any necessary compatibility conversions”; DNA is sensitive to “structure padding, pointer size, and endianness”; structs prefixed with#\n#are excluded from SDNA. Blender developer docs, “DNA”, retrieved 2026-08-29. - Compatibility policy: backward compatibility expected for any previous version (conversion code may be removed two years after deprecation); forward compatibility “though with some loss of data”; when reading, “Unknown data is ignored. Missing data is initialized with default values.” and versioning code runs incrementally; unknown data “typically also cannot re-save”; partial re-save of unknown data exists only for ID properties and “should not be expected or relied on”; critical breakages only at major cycles, with the previous long-term support (LTS) release acting as converter. Blender developer docs, “Blend File Compatibility”, retrieved 2026-08-29.
enum eSDNA_StructCompare { SDNA_CMP_REMOVED = 0, SDNA_CMP_EQUAL = 1, SDNA_CMP_NOT_EQUAL = 2, SDNA_CMP_UNKNOWN = 3 }with comments: removed structs “will not be loaded by the current Blender”, equal structs load “with straight memory copy”, not-equal structs are “copied/converted field by field”;DNA_sdna_from_datadecodes an SDNA block;DNA_struct_get_compareflagsreturns one flag per old struct.source/blender/makesdna/DNA_genfile.h, main, retrieved 2026-08-29.- SDNA layout: magic
SDNA, thenNAME(count, strings),TYPE(count, strings),TLEN(shorts),STRC(count, then per struct<typenr><nr_of_elems>followed by<typenr><namenr>pairs); “Remember to read/write integer and short aligned!”;DNA_struct_reconstructmatches members byelem_streq, which compares names “excluding any array-size suffix”; members absent in the new struct are dropped, new members getRECONSTRUCT_STEP_INIT_ZERO;cast_primitive_typeconverts among the SDNA primitive types; 32/64-bit pointer casts warn that “pointers may lose uniqueness on truncation”.source/blender/makesdna/intern/dna_genfile.cc, main, retrieved 2026-08-29. - Block header:
struct BHead { int code; int SDNAnr; const void *old; int64_t len; int64_t nr; }witholddocumented as “the pointer that the memory had when it was written”, used “to remap memory blocks on load”; on-disk variantsBHead4(32-bitold),SmallBHead8(64-bitold, 32-bitlen/nr) andLargeBHead8(64-bitlen/nr); codes includeBLO_CODE_DNA1andBLO_CODE_ENDB.source/blender/blenloader_core/BLO_core_bhead.hh, main, retrieved 2026-08-29. - File header: “the first 12-17 bytes”, magic
BLENDER, file format version “Currently always 0 or 1”, pointer size 4 or 8, endianness, and writer version;LargeBHead8is the format-version-1 layout.scripts/modules/_blendfile_header.py, main, retrieved 2026-08-29. - Versioning code example:
if (!MAIN_VERSION_FILE_ATLEAST(bmain, 400, 9)) { ... }andif (!DNA_struct_member_exists(fd->filesdna, "bPoseChannel", "BoneColor", "color")) { ... }insideblo_do_versions_400(FileData *fd, Library *, Main *bmain)anddo_versions_after_linking_400.source/blender/blenloader/intern/versioning_400.cc, main, retrieved 2026-08-29. - RNA “provides a high-level description of Blender’s data structures and related functions, which is used to drive the UI, animation system, library overrides, and the Python API (bpy)”; definitions in
rna_*.ccprocessed bymakesrna; metadata includes ranges, enum items, get/set callbacks, “Update flags and functions to notify other systems”, override metadata (RNA_def_property_override_flag(prop, PROPOVERRIDE_IGNORE)); RNA “has since evolved into a more general-purpose runtime data definition system” with structures independent of DNA. Blender developer docs, “RNA”, retrieved 2026-08-29. - Operators: a class deriving from
bpy.types.Operatorwithbl_idname = "object.simple_operator"andexecute(self, context); registration viabpy.utils.register_class; invocationbpy.ops.object.simple_operator(); registering validates signatures (“expected Operator, SimpleOperator class ‘execute’ function to have 2 args”); add-on modules requireregister()/unregister(). Blender Python API, “API Overview” (api/current/info_overview.html), retrieved 2026-08-29. - Undo: “Undo is organized as an ‘undo stack’ storing a list of ‘undo steps’”; the stack is “fully relative”; steps are stateful or differential; “The main part of step creation (‘undo push’) is controlled by the operator management system”; layers
ed_undo.cc,undo_system.cc, and per-type implementations such asmemfile_undo.ccusingBLOread/write code. Blender developer docs, “Undo System”, retrieved 2026-08-29. - Memfile: chunks are compared with the reference step (
memcmp(compchunk->buf, buf, size) == 0setsis_identical);is_identical_futurehandles the redo direction;BLO_memfile_write_initbuilds a mapping from IDsession_uidto previous-step storage so reuse survives reordering ofMain.source/blender/blenloader/intern/undofile.cc, main, retrieved 2026-08-29. UndoTypecallbacksstep_encode/step_decode(is_finalflag),poll; memfile steps “must be read before loading other undo steps” when active.source/blender/blenkernel/BKE_undo_system.hh, main, retrieved 2026-08-29.- Command line:
-b, --background“Run in background (often used for UI-less rendering)”;-P, --python <filepath>“Run the given Python script file”;--python-expr;--python-exit-code <code>sets the exit code on uncaught Python exception “(only for scripts executed from the command line)”;--factory-startupskips the userstartup.blend;-f,-a,-o,-F,-E; “Arguments are executed in the order they are given” with worked examples of misordering. Blender Manual (latest), “Command Line Arguments”, retrieved 2026-08-29. - Render tests: “Each blend file generates a test in its category. It always renders frame number 1 to a PNG image”; results in
build/tests/report.html; references updated withBLENDER_TEST_UPDATE=1 ctest -R <name>; EEVEE reference is “the Nvidia OpenGL result”; threshold policy: strict on the reference platform, bumpfail_percentfirst, thenfail_thresholdwhen > 0.5% pixels fail, per platform if large; non-deterministic tests are blocklisted; tests are batched per Blender launch (WITH_TESTS_BATCHED). Blender developer docs, “Render Tests”, retrieved 2026-08-29. - Comparison implementation:
oiiotool ref out --fail <fail_threshold> --failpercent <fail_percent> --diff; defaultsfail_threshold = 0.016,fail_percent = 1;set_fail_threshold/set_fail_percent; onBLENDER_TEST_UPDATEthe new image overwrites the reference; HTML report with new/reference/diff images.tests/python/modules/render_report.py, main, retrieved 2026-08-29. - EEVEE tuning:
report.set_fail_percent(0.08),report.set_fail_threshold(4.0 / 255.0), tightened to0.049and2.0/255on NVIDIA OpenGL, loosened per category (e.g.transparency0.22 and10.0/255); Blender is invoked with--background --factory-startup --enable-autoexec --debug-memory --console-crash-handler --debug-exit-on-error, optional--gpu-backend <backend> --debug-gpu-backend-no-fallback, then<file> -E BLENDER_EEVEE -P <script> -o <out> -F PNG -f 1; AMD Vulkan blocklist is".*".tests/python/eevee_render_tests.py, main, retrieved 2026-08-29.
Mechanism
File model. A .blend is header || BHead+data ... || DNA1 || ENDB. Each BHead says which struct (index into the writer’s SDNA), how many, how many bytes, and the writer’s address of the block. Pointers inside data are the writer’s addresses; the reader builds an address map from old to the newly allocated block and patches pointers, which is why identity in a .blend is address-based rather than semantic. The SDNA is a schema: names (with array suffixes and pointer prefixes), type names and sizes, and struct member lists. The reader decodes the file SDNA, computes per-struct compare flags against its own SDNA, and for NOT_EQUAL structs builds reconstruction steps: per new member, find a same-named member of the old struct ignoring array size, and either memcpy, cast primitive, or zero-init; old members with no counterpart are dropped. Removed structs are skipped. This yields structural forward compatibility (an older reader ignores a new field) and backward compatibility (old files gain zeroed fields). Semantic migrations are then applied by version-gated code; the use of DNA_struct_member_exists on the file SDNA lets a migration run exactly when a file predates a field, independent of the version number bump discipline.
Preservation limits. Unknown structs and unknown members are discarded on load and therefore absent on re-save; the compatibility document states this and limits round-trip preservation to ID properties. Forward compatibility is defined as “open with some loss”, and critical breakages are scheduled at major version boundaries with the previous LTS as a converter.
Reflection and operations. RNA definitions are the single description used by the UI layout code, the animation system (which animates RNA paths), library overrides (per-property override flags) and Python. Operators are the command pattern: a registered type with an ID, typed properties, poll, invoke, execute, and flags such as REGISTER/UNDO; the window manager records executed operators with their property values (the redo panel re-executes them) and pushes an undo step after each undoable operator. Undo is a stack of relative steps; the global step type serialises Main into memory as a .blend-format MemFile of chunks and marks chunks identical to the previous step so that consecutive steps share memory; ID-level session_uid mapping keeps the chunk correspondence stable under reordering. Mode-specific step types (edit-mesh, sculpt, text) store their own state. Undo is therefore state-based (memento) at the file level, while the operator log provides re-executable commands.
Headless surface. The same binary runs with --background; arguments are executed in order, so --python after a file argument sees the loaded file. Scripts use bpy.ops (operators) and bpy.data (RNA data access) identically to the GUI; --python-exit-code maps uncaught exceptions to a process exit code for continuous integration (CI); --factory-startup removes user-state dependence. Regression tests are Python scripts run through this surface, batched, producing PNGs compared by oiiotool with an absolute per-pixel threshold and a maximum failing-pixel percentage, both tunable per test category and per GPU backend, with a reference-update mode and blocklists for non-deterministic cases.
NUIF relevance
Borrow
- Embedding the writer’s schema with the document so readers can reconstruct records by member name; NUIF’s CBOR/text profiles should carry a schema identifier and member names (or a schema registry hash) rather than positional layouts.
- Migration gated on “does the file’s schema contain member X” as well as on version numbers; NUIF
migrateshould be able to inspect declared schema features, not only a version integer. - Operator-as-command with typed properties,
pollpreconditions, registration-time validation and a re-executable log; this is the same shape as NUIF semantic operations and the QA replay requirement. - The headless invocation contract (
--background --factory-startup --python-exit-code) and threshold policy (absolute per-pixel error plus failing-pixel percentage, tuned on a reference platform, with blocklists and a reference-update mode) for NUIF render conformance.
Adapt
- Memfile undo shares identical chunks between steps; NUIF can use content-addressed canonical snapshots for checkpointing while keeping inverse operations as the primary undo representation (spec/06).
- Address-based block identity must become stable semantic IDs in NUIF; the
old-pointer remap is a useful reminder that identity derived from memory or byte offset is not portable (spec/02). - RNA’s override flags per property suggest that NUIF property definitions should declare override and merge policy in the schema.
Reject
- Dropping unknown structs and members on load; NUIF requires opaque preservation of unknown data and a fidelity report (RFC 0002, spec/07), which Blender explicitly does not promise.
- Coupling the on-disk layout to compiler padding, pointer size and endianness; NUIF encodings are logical, deterministic and platform-independent (spec/08).
- Undo semantics that vary by mode (global versus edit-mode step types); NUIF has one operation model for all editing contexts.
Open questions
- Whether recent work on ID-property preservation (the documented partial re-save of unknown data) has a stable specification; the compatibility document says not to rely on it.
- Whether the
session_uidmapping in memfile undo can be reused as a semantic identity for diffing two.blendstates, or is strictly session-local. - How
--python-exit-codeinteracts with batched test execution when one test in a batch fails.
Canonical serialization rules - JCS (RFC 8785), CBOR deterministic encoding (RFC 8949 s4.2, CDE, dCBOR) and lessons from XML C14N
Document status:
reviewed. Canonical source.
Summary
RFC 8949 §4.2.1 fixes four core requirements for deterministically encoded CBOR: shortest-form arguments for integers, lengths and tags; shortest floating-point form that preserves the value; no indefinite-length items; and map keys sorted by bytewise lexicographic order of their deterministic encodings. §4.2.2 leaves to the protocol the treatment of tags, big integers, negative zero, NaN payloads, subnormals and integral-valued floats. The CDE draft packages the core requirements as a profile (preferred serialisation, definite lengths, lexicographic map sorting) and keeps the data model intact (1.0 stays a float, -0.0 is encoded as 0xf98000). The dCBOR draft narrows further at the application level: integral floats within 64-bit range become integers, all zeros become 0x00, all NaNs become 0xf97e00, only false/true/null/floats are permitted as simple values, text must be NFC, and decoders must reject non-conforming input. RFC 8785 (JCS) canonicalises JSON text: no whitespace, fixed string escaping, numbers serialised per ECMAScript Number::toString (shortest round-trip, -0 becomes 0, NaN/Infinity are errors), properties sorted by UTF-16 code units, UTF-8 output, under the I-JSON constraint that numbers are IEEE 754 doubles. XML C14N 1.1 and the C14N 2.0 Note document why XML canonicalisation was hard: namespace and xml: attribute inheritance in document subsets, XPath node-set dependence, whitespace, QNames in content, and information lost (base URIs, notations, attribute types). The normative rules NUIF should adopt are listed under Mechanism.
Evidence
- RFC 8949 §4.1: preferred serialisation “always uses the shortest form of representing the argument”; floats use the shortest encoding that preserves the value; definite-length encoding is preferred when the length is known. https://www.rfc-editor.org/rfc/rfc8949.html#section-4.1 (retrieved 2026-08-29).
- RFC 8949 §4.2.1 Core Deterministic Encoding Requirements: integer arguments 0-23 in the initial byte, 24-255 in one byte, up to 65535 in two, up to 2^32-1 in four; floating-point in the shortest form that preserves the value (1.5 as binary16); “Indefinite-length items MUST NOT appear”; “keys in every map MUST be sorted in the bytewise lexicographic order of their deterministic encodings”. §4.2.1 (retrieved 2026-08-29).
- RFC 8949 §4.2.2: protocols must decide whether a tag must be present or absent; whether integers with absolute value at or above 2^64 use tags 2/3 and whether smaller values may also use them; negative zero may be disallowed; “the protocol needs to pick a single representation, typically 0xf97e00” for NaN; subnormals may be excluded; whether 1.0 is
0x01,0xf93c00,0xfa3f800000or0xfb3ff0000000000000. §4.2.2 (retrieved 2026-08-29). §4.2.3 defines the legacy length-first key order of RFC 7049. §5.6 lists three decoder behaviours for duplicate keys. - CDE: draft-ietf-cbor-cde-13, 2025-10-14, intended Best Current Practice, updates RFC 8949; CDE =
preferred-serialization+definite-length-only+lexicographic-map-sorting(§3); floats use the shortest of binary16/32/64 preserving the value, negative zero is encoded as0xf98000, integral floats remain floating-point (data model preserved), preferred serialisation applies to NaNs (§3.1.2); strictly increasing key order excludes duplicates (§3.3); application-level deterministic representation (ALDR) is separate, with dCBOR as the example (Appendix B). https://www.ietf.org/archive/id/draft-ietf-cbor-cde-13.html (retrieved 2026-08-29). - dCBOR: draft-mcnally-deterministic-cbor-18, 2026-08-10, authors McNally, Allen, Bormann, Lundblade; definite lengths only (§2.1); preferred serialisation validated by decoders (§2.2); ordered keys (§2.3); no duplicate keys (§2.4); numeric reduction: floats with zero fractional part within [-2^63, 2^64-1] become integers, all NaNs become
0xf97e00, 0, 0.0 and -0.0 become0x00, decoders reject non-reduced floats (§2.5); simple values limited tofalse,true,null, floats (§2.6); text strings must be NFC UTF-8 and decoders reject non-NFC (§2.7). https://datatracker.ietf.org/doc/draft-mcnally-deterministic-cbor/ (retrieved 2026-08-29). Draft -12 (2025-02-07) states the same numeric rules in §2.3 and rejects 65-bit negative integers (Table 4); tag 201 marks dCBOR content. https://www.ietf.org/archive/id/draft-mcnally-deterministic-cbor-12.html (retrieved 2026-08-29). - RFC 8785 (JCS), June 2020, Informational, Independent Submission, Rundgren, Jordan, Erdtman: input must be I-JSON (RFC 7493), “JSON number data MUST be expressible as IEEE 754 double-precision values” (§3.1); no whitespace between tokens (§3.2.1); strings escape U+0000-U+001F as
\uhhhhlowercase except\b \t \n \f \r, escape\and", emit other characters as-is, error on lone surrogates (§3.2.2.2); numbers “MUST be serialized according to Section 7.1.12.1 of [ECMA-262], including the ‘Note 2’ enhancement”, NaN and Infinity “MUST cause a compliant JCS implementation to terminate with an appropriate error” (§3.2.2.3); properties sorted by UTF-16 code units, shorter prefix first (§3.2.3); UTF-8 output (§3.2.4); Appendix B table shows both0000000000000000and8000000000000000serialise to0; larger precision “RECOMMENDED to represent such numbers as JSON strings”. https://www.rfc-editor.org/rfc/rfc8785.html and .txt (retrieved 2026-08-29). - XML C14N 1.1, W3C Recommendation 2008-05-02: UTF-8 without BOM, line breaks to
#xA, attribute value normalisation, entity references replaced, CDATA converted, empty elements expanded, superfluous namespace declarations removed and sorted, attributes sorted by namespace URI then local name; 1.1 changes the inheritance ofxml:base(URI joining) and excludesxml:idfrom inheritance for document subsets; information lost: base URIs, notations and unparsed entities, attribute types. https://www.w3.org/TR/xml-c14n11/ (retrieved 2026-08-29). - XML C14N 2.0, W3C Working Group Note 2013-04-11, not pursued to Recommendation; §1.4 motivations: performance (C14N 1.x depends on XPath node-sets; 2.0 is a tree walk), streaming, robustness (whitespace, QNames in attributes such as
xsi:type, optional prefix rewriting), portability of subdocuments, simplicity; §2.2 parametersIgnoreComments,TrimTextNodes,PrefixRewrite,QNameAware. https://www.w3.org/TR/xml-c14n2/ (retrieved 2026-08-29).
Mechanism
Layering used by the IETF documents (NUIF should mirror it):
Layer 0 well-formed CBOR (RFC 8949 §3)
Layer 1 preferred serialisation (RFC 8949 §4.1) shortest head; shortest float
Layer 2 CDE = core deterministic requirements (RFC 8949 §4.2.1; draft-ietf-cbor-cde)
+ definite lengths only + bytewise-lexicographic keys, no duplicates
Layer 3 application-level determinism (ALDR) (dCBOR draft; NUIF profile rules)
numeric reduction, NaN/zero canonical forms, simple-value and string constraints
Key ordering (RFC 8949 §4.2.1): compare the deterministic encodings of keys as byte strings; because the initial byte carries major type and argument size, shorter integers sort before longer ones and integers sort before strings.
Float rule as adopted by dCBOR §2.5:
canon(x):
if x is float and x is finite and frac(x) = 0 and -2^63 ≤ x ≤ 2^64-1: encode as integer (major 0/1)
elif x is NaN: emit f9 7e 00
elif x = ±0.0: emit 00
else: shortest of binary16/32/64 that round-trips exactly
decoders reject any float not in this form
JCS number rule (RFC 8785 §3.2.2.3): ECMAScript Number::toString produces the shortest decimal digit string that round-trips to the same double, with exponent notation thresholds fixed by ECMA-262; -0 prints as 0; this is the text-form counterpart of the binary shortest-form rule.
Normative rules NUIF must adopt for byte-stable canonical hashing (NUIF interpretation of the sources):
- Canonical hash is computed over the
nuif-cbor-0deterministic encoding, never over a text or compressed form (spec/08 “MUST exclude transport-only compression differences”). - CDE conformance: shortest heads, definite lengths, bytewise-lexicographic key order, duplicate keys rejected.
- Numeric model declared per property type: integer-typed values use major types 0/1 only; real-typed values use IEEE 754 binary64 semantics with dCBOR numeric reduction (integral values as integers, single NaN
0xf97e00, all zeros as0x00) or, if signed zero and NaN payloads carry meaning for a property, a documented exception; subnormals preserved (no information loss) but encoded in shortest form. - No big-integer tags unless a property type requires them; no other tags in the canonical body.
- Strings: valid UTF-8, no normalisation of user content (NFC normalisation changes content and is a data-model decision, see open questions), identifiers and keys restricted to a canonical repertoire.
- Sets and relation graphs serialised in a defined total order (by entity ID bytes), sequences in document order.
- Decoder is strict: any deviation from the canonical form is a rejection, not a re-canonicalisation, so that hash equality implies byte equality.
XML C14N lessons (source statements): canonical form must not depend on a query language over the document (C14N 2.0 §1.4.1), inherited context (namespaces, xml:base) must be made explicit or excluded (C14N 1.1 changes), QName-valued content must be known to the canonicaliser (C14N 2.0 §1.4.3), and some information is inevitably lost, so the canonical form must be defined as the reference form rather than a derived one.
NUIF relevance
Borrow
- RFC 8949 §4.2.1 and CDE as the base profile for
nuif-cbor-0, including bytewise-lexicographic key ordering and definite lengths. - dCBOR §2.5 numeric reduction (integral floats to integers, single NaN, single zero) and §2.6 simple-value restriction, because they remove the remaining encoder freedom that RFC 8949 §4.2.2 enumerates.
- JCS’s UTF-16 code-unit property ordering and shortest round-trip number printing for the
nuif-text-0canonical text form so text and binary forms agree on number identity. - The strict-decoder rule (dCBOR) so that a NUIF hash identifies exactly one byte sequence.
Adapt
- dCBOR’s NFC requirement (§2.7) must become a data-model rule in spec/02 rather than an encoder rule: text properties either store NFC by definition or store code points verbatim, and the choice affects round-tripping of imported documents.
- JCS’s -0 to 0 collapse and I-JSON double limit: NUIF property types that are integers wider than 2^53 must not be routed through the JSON/text form unquoted; the text form needs a typed integer syntax.
- C14N’s “context must be explicit” lesson maps to NUIF inheritance (tokens, styles, instance overrides): the canonical document must serialise authored values only, never resolved or inherited values, and resolved caches must be excluded from the hash.
Reject
- Length-first map key ordering (RFC 8949 §4.2.3), which exists only for RFC 7049 compatibility.
- Preserving NaN payloads, signalling NaNs or signed zero in canonical form; RFC 8949 §4.2.2 allows it but no NUIF property semantics require it.
- XML C14N-style parameterised canonicalisation (comments, whitespace trimming, prefix rewriting); a canonical form with parameters is not a single canonical form.
Open questions
- Whether a single NUIF numeric profile suffices, or whether layout properties (which may be authored as
1.0versus1) need a type-directed rule so that authored integers and reals hash differently only when the property type distinguishes them. - Handling of extension payloads (
SetExtension { payload: Vec<u8> }in nuif-protocol): opaque bytes are hashed verbatim, but if an extension is itself CBOR the canonical rules should apply recursively; whether to require nested canonical CBOR for registered extensions is undecided. - Whether CDE reaches RFC status with the -0.0 rule (
0xf98000) unchanged, which would conflict with dCBOR’s zero reduction for any NUIF profile that references CDE by name.
Canva Apps SDK and Connect API adoption surface
Document status:
reviewed. Canonical source.
Summary
Canva exposes two materially different integration surfaces. Apps SDK code runs inside the editor and the Design Editing API can read and edit supported page ingredients. Connect APIs are OAuth-protected server APIs for off-platform workflows such as import, export and return navigation. The Apps SDK is the primary semantic NUIF adoption path; Connect is a secondary workflow bridge and cannot currently import or export NUIF natively.
The first app profile should use only generally available current_page
Design Editing APIs on one fixed-dimension page, normalize groups/rects/shapes/
text through a bounded schema, validate the complete plan and call sync once.
Canva’s documented preview restriction, product-specific element model and app
review process make a broader “Canva adapter” claim unsound.
Evidence
- A Canva app is JavaScript embedded in a side-panel iframe. Canva injects API
packages including
@canva/design; the app does not gain direct access to an undocumented document file. Locator: Apps SDK, Integrating with Canva, basic app and API package sections, retrieved 2026-08-31: https://www.canva.dev/docs/apps/integrating-canva/. openDesignprovides page snapshots, helpers andsync. Sessions expire after one minute. Supported pages areabsolute; unsupported pages cannot be read or edited, and Canva Docs are explicitly incompatible. Fixed and unbounded absolute pages are distinguished and pages carry stablePageIdvalues. Locator: Apps SDK, Design Editing API, core concepts, sessions and pages, retrieved 2026-08-31: https://www.canva.dev/docs/apps/design-editing/.- The Design Editing API provides CRUD only for supported elements: embeds,
groups, rects, shapes and text. Images and videos are represented as rectangle
fills, text is exposed as rich-text ranges, and tables are unsupported.
Layering is list order rather than a
z-index. Locator: same document, elements, element types and layering sections. - Snapshot changes affect the live design only after
sync; Canva’s design guidelines recommend applying a logical change as one operation so the user can undo it as one action. Apps must present critical unsupported/failure states and must not unexpectedly replace or delete the whole design. Locator: Apps SDK, Design Editing API design guidelines, changing a design and error sections, retrieved 2026-08-31: https://www.canva.dev/docs/apps/design-guidelines/design-editing-api/. - The general Design Editing guide still labels
all_pagesas preview while the GA package changelog says multi-page editing was promoted to GA. This documentation inconsistency is why profile 0 remains on the unambiguous GAcurrent_pagecall. Locators: design editing guide above;@canva/designGA changelog, retrieved 2026-08-31: https://www.canva.dev/docs/apps/api/latest/design-changelog/. - Canva states that preview packages may change without a new version and apps using them cannot pass public app review. The same rule applies to preview Connect features. Locators: Apps SDK, Integrating with Canva, preview APIs; Connect API overview, preview APIs, retrieved 2026-08-31: https://www.canva.dev/docs/apps/integrating-canva/ and https://www.canva.dev/docs/connect/.
- Canva’s app iframe CSP allows
wasm-unsafe-evalbut blocks third-party JavaScript, frames and web workers. This permits a bundlednuif-wasmmodule but not remote executable code or worker-based assumptions. Locator: Apps SDK, Content Security Policy, allowed features and directives, retrieved 2026-08-31: https://www.canva.dev/docs/apps/content-security-policy/. - Connect design imports are asynchronous byte uploads requiring OAuth bearer
authorization and
design:content:write; the endpoint is rate-limited to 20 requests per minute per user. The supported list includes native Affinity extensions, PDF and other formats, but not NUIF. Locator: Connect APIs, Create design import job and Design imports, retrieved 2026-08-31: https://www.canva.dev/docs/connect/api-reference/design-imports/create-design-import-job/ and https://www.canva.dev/docs/connect/api-reference/design-imports/. - Connect exports are asynchronous and currently support JPG, PNG, GIF, PPTX,
MP4, PDF, CSV, HTML bundle and standalone HTML; completed download URLs expire
after 24 hours. The endpoint requires
design:content:readand has integration, document and user throttles in addition to the per-user request limit. Locator: Connect APIs, Create design export job, retrieved 2026-08-31: https://www.canva.dev/docs/connect/api-reference/exports/create-design-export-job/. - A public app is submitted as a source bundle through the Developer Portal, needs listing/testing material and Canva review, and can be released only after approval. Marketplace developers must provide identity and legal-entity information; team apps use an Enterprise-team review path. Released or rejected apps are changed by creating a new app version. Locators: Apps SDK, Submitting apps, App review process, Developer verification and App versioning, retrieved 2026-08-31: https://www.canva.dev/docs/apps/submitting-apps/, https://www.canva.dev/docs/apps/app-review-process/, https://www.canva.dev/docs/apps/developer-verification/ and https://www.canva.dev/docs/apps/versioning-apps/.
Options considered
Apps SDK Design Editing API
Selected as the primary path. It exposes typed semantic objects, a transaction- like sync boundary, user-visible undo and a public distribution channel. The first profile remains narrow enough to test without inventing Canva semantics.
Connect API native NUIF workflow
Not currently possible. NUIF is absent from the documented import/export format lists. Connect can route SVG/PDF with explicit loss or return users to a design, but cannot prove element-level NUIF round trips. Native NUIF support is an upstream adoption request, not an adapter feature that this repository can declare.
Render or app-element flattening
Rejected. Replacing a page with a screenshot or one opaque app element defeats editable semantics and conflicts with Canva’s design-editing guidance. A render may be diagnostic evidence only.
Preview API dependency
Rejected for the public profile. Preview behavior can change without versioning and blocks public review. Preview experiments may be kept in a separate branch and evidence class, never in a release bundle.
Adoption and release path
- Implement pure Canva snapshot and mutation-plan types with fixtures, bounds
and
HostAdapterReportoutput; do not call live APIs until that mapping is exact for the declared subset. - Build a no-network, single-file Apps SDK review shell that bundles
nuif-wasm, verifies its CSP and official TypeScript types, and uses only stable APIs. - Run named live-host trials for read, import, one-sync undo, cancellation, locked content, conflicts, session expiry and unsupported ingredients.
- Publish a source review bundle through GitHub release artifacts with digest, SBOM and fixture report. Submission remains a manual authenticated action; a Git tag must never publish the app automatically.
- After human developer verification and Canva approval, create versions in the Developer Portal for reviewed updates. Keep the app version independent from editor, WASM and format versions.
- Propose native NUIF MIME import/export to Canva only after the profile has public fixtures, independently reviewed fidelity and demonstrated user demand. The proposal must specify package safety, unknown-extension retention and profile negotiation rather than asking Canva to adopt the entire draft.
NUIF relevance
Borrow Canva’s explicit iframe, permission, session, sync, review and distribution boundaries. They are a useful model for a host adapter that makes mutation authority and user-visible undo explicit.
Adapt HostAdapterReport, canonical NUIF operations and the WASM binding to
the Apps SDK edge. Keep Connect API OAuth, rate limits, temporary URLs and
privacy policy outside the deterministic core.
Reject preview APIs in a public profile, opaque app-element flattening, remote executable code and any native NUIF interoperability claim until Canva publishes the corresponding MIME and semantic contract.
Open questions
- Which element IDs, metadata fields or app-owned data survive duplicate, reorder, close/reopen and cross-design copy under generally available APIs?
- Does the review production bundle execute
nuif-wasmidentically under the documented CSP on every supported Canva browser/desktop host? - Which rich-text, custom-path, image-fill crop and font semantics can be mapped exactly without undocumented assumptions?
- When the all-pages documentation and GA package surface agree, what page- ordering, session-expiry and atomicity rules are needed for a separate multi-page profile?
Cargo workspace layout, xtask automation, dependency/API linting, reproducible builds and test determinism for engine + apps + tests + fuzz
Document status:
reviewed. Canonical source.
Summary
Cargo workspaces share one Cargo.lock and one target directory across members and support inheritance of package metadata, dependencies and lints from the root manifest ([workspace.package], [workspace.dependencies], [workspace.lints], respected since Rust 1.74). Free-form automation is conventionally implemented as an xtask binary crate reachable through a .cargo/config.toml alias, which needs no tool beyond cargo. Feature-combination checks (cargo-hack), dependency policy (cargo-deny), public API compatibility (cargo-semver-checks), unused dependencies (cargo-udeps, nightly) and release packaging (cargo-dist) are separate installable subcommands. Reproducibility rests on a committed Cargo.lock with --locked, a rust-toolchain.toml pin, SOURCE_DATE_EPOCH for embedded timestamps and --remap-path-prefix for embedded paths. libtest runs tests in alphabetical order on a thread pool sized by available parallelism; --test-threads=1 (or RUST_TEST_THREADS) serialises tests that share state, and --shuffle remains unstable.
NUIF interpretation: the current repository already uses a virtual manifest with [workspace.lints] (unsafe_code = "forbid", clippy all/pedantic warn) and a pinned toolchain in CI; the missing pieces are a committed rust-toolchain.toml, --locked in CI, an xtask crate for fixture regeneration and differential runs, a fuzz/ member, conformance/ test crates, and a CI matrix covering feature powersets, wasm targets and dependency/API policy.
Evidence
[workspace]keys:resolver,members,exclude,default-members,package,dependencies,lints,metadata; a manifest with[workspace]and no[package]“is called a virtual manifest” and must setresolverexplicitly; “All packages share a commonCargo.lockfile which resides in the workspace root” and a common output directory. Locator: Cargo book “Workspaces”, retrieved 2026-08-29.- Lint inheritance:
[workspace.lints.rust] unsafe_code = "forbid"in the root and[lints] workspace = truein members; “MSRV: Respected as of 1.74”; lint entries acceptlevelandpriority(lower priority is overridden by higher). Locator: Cargo book “Workspaces” (“The lints table”) and “The Manifest Format” (“The [lints] section”). - Package and dependency inheritance:
[workspace.package]withversion.workspace = true;[workspace.dependencies]withregex = { workspace = true, features = ["unicode"] }andcc.workspace = truein build/dev dependencies. Locator: Cargo book “Workspaces” (“The package table”, “The dependencies table”). package.metadata“is completely ignored by Cargo and will not be warned about”, intended for external tools; cargo-fuzz’s generatedfuzz/Cargo.tomluses[package.metadata] cargo-fuzz = true. Locator: Cargo book “The Manifest Format”; cargo-fuzzsrc/templates.rslines 9-10.- Target conventions:
src/lib.rs,src/main.rs,src/bin/,examples/,tests/,benches/;harness = falserequires a usermain;testandbenchfields toggle default inclusion; “Each integration test results in a separate executable binary, andcargo testwill run them serially”. Locator: Cargo book “Cargo Targets”. cargo test: arguments after--go to the test binary (cargo test foo -- --test-threads 3);--jobsaffects the build only;--no-fail-fastruns all executables;--locked“Asserts that the exact same dependencies and versions are used as when the existingCargo.lockfile was originally generated”;--frozenequals--lockedplus--offline; each test’s working directory is the package root. Locator: Cargo book “cargo test”.- Cargo FAQ:
Cargo.lockgives “deterministic builds at different times and on different systems”; it aidsgit bisect, CI stability, MSRV verification and “snapshot testing error messages”; it “does not affect consumers of your package”;cargo installignores it unless--locked;cargo newtracks it in version control by default. Locator: Cargo book FAQ “Why have Cargo.lock in version control?”. - libtest CLI:
--test-threadsdefaults toavailable_parallelism, withRUST_TEST_THREADSas a deprecated alternative; default order is alphabetical;--shuffleand--shuffle-seedare unstable (-Z unstable-options, tracking issue #89583);--list,--exact,--skip,--ignored,--include-ignored,--formatare stable;--report-timeand JUnit output are unstable. Locator: rustc book “Tests” (src/doc/rustc/src/tests/index.md), lines 77-215. - rustup:
rust-toolchain.tomlwith[toolchain] channel = "...",components = [...],profile, optionaltargets; the nearest file up the directory tree applies;channelandpathare mutually exclusive. Locator: rustup book “Overrides” (“The toolchain file”). - xtask: “a polyfill for cargo workflows”; add an
xtaskbinary member and.cargo/config.tomlwith[alias] xtask = "run --package xtask --"; “It doesn’t require any other binaries besidescargoandrustc”; Cargo itself uses xtasks. Locator: matklad/cargo-xtaskREADME.md. - cargo-hack 0.6.45:
--each-feature,--feature-powerset(with--depth,--exclude-features/--skip,--group-features),--rust-version(check at the manifest MSRV),--version-range,--workspace. Locator:README.md“Usage”. - cargo-deny 0.20.2:
cargo deny initandcargo deny checkcovering licenses, bans, advisories and sources; GitHub Actioncargo-deny-action; README badge states MSRV 1.88.0. Locator:README.md. - cargo-semver-checks 0.50.0: analyses rustdoc JSON; stable toolchains supported, nightly “on a best-effort basis”;
--baseline-version <X.Y.Z>/--baseline-rev <REV>; GitHub Action available. Locator:README.md“FAQ”, lines 63-121. - cargo-udeps 0.1.61: “needs Rust nightly to actually run”; install with
--locked. Locator:README.md. - cargo-dist 0.32.0: builds tarballs and installers and “generates its own CI scripts” (
release.ymlon tag push). Locator:README.md. SOURCE_DATE_EPOCH“is a standardised environment variable that distributions can set centrally” giving seconds since the Unix epoch for the last source modification. Locator: reproducible-builds.org “SOURCE_DATE_EPOCH”.- rustc
--remap-path-prefixremaps source paths in output (debug info, panics). Locator: rustc book “Command-line Arguments”,--remap-path-prefix. - Existing repository state: virtual manifest with
resolver = "2", eight members undercrates/,[workspace.package](edition = "2024",license = "Apache-2.0 OR MIT",rust-version = "1.85"),[workspace.lints.rust] unsafe_code = "forbid",[workspace.lints.clippy] all = "warn",pedantic = "warn"; CI job pinsdtolnay/rust-toolchain@masterto 1.85.0 with rustfmt and clippy and runs fmt, check, test, clippy-D warningswithout--locked. Locator:Cargo.toml;.github/workflows/ci.yml.
Mechanism
Workspace layout (interpretation; every element cites a convention above):
Cargo.toml # virtual manifest; resolver = "2"; workspace.package/dependencies/lints
Cargo.lock # committed; CI uses --locked
rust-toolchain.toml # [toolchain] channel = "1.85.0", components = ["rustfmt","clippy"], targets = ["wasm32-unknown-unknown","wasm32-wasip1"]
.cargo/config.toml # [alias] xtask = "run --package xtask --"
deny.toml # cargo-deny policy (licenses allow-list: Apache-2.0, MIT, BSD-3-Clause, ...)
.config/nextest.toml # [profile.ci] retries = 0, junit.path = "junit.xml"
crates/nuif-*/ # engine crates (lib targets; [lints] workspace = true)
apps/editor/ # editor crate(s); depends on crates/* only through nuif-api
conformance/ # [[test]] harness = false suites; fixtures/ directory tree
fuzz/ # cargo fuzz init; [package.metadata] cargo-fuzz = true; fuzz_targets/*.rs; member or separate workspace
benches/ (per crate) # criterion/divan targets with harness = false
xtask/ # regeneration of generated fixtures, differential runs, report assembly
Feature and target policy: engine crates expose an optional arbitrary feature (fuzz derives) and serde feature; cargo hack check --workspace --feature-powerset --depth 2 --rust-version validates combinations at the declared MSRV. Editor crates are excluded from default-members so that engine-only commands stay fast.
CI matrix (interpretation): stable pinned toolchain on ubuntu/macos/windows for cargo fmt --check, cargo clippy --all-targets -D warnings, cargo nextest run --workspace --locked --profile ci; cargo hack powerset job; cargo deny check; cargo semver-checks --baseline-rev <last tag> on release branches; wasm32-unknown-unknown build plus wasm-pack test --headless --chrome; wasm32-wasip1 build plus wasmtime run; optional nightly job for cargo fuzz run <target> -- -max_total_time=60, cargo llvm-cov --branch, cargo udeps.
Determinism controls:
cargo nextest run --workspace --locked # one process per test; no shared static state across tests
cargo test -- --test-threads=1 # libtest fallback when tests share process-global state (fonts, env vars)
RUST_TEST_THREADS=1 # deprecated equivalent
SOURCE_DATE_EPOCH=$(git log -1 --format=%ct) # embedded build timestamps
RUSTFLAGS="--remap-path-prefix=$PWD=/src" # embedded paths
Invariants: Cargo.lock committed and enforced with --locked; toolchain pinned identically in rust-toolchain.toml and CI; libtest order is alphabetical, so tests must not depend on order; global mutable state (font databases, environment variables) must be process-local or serialised with --test-threads=1; nextest process isolation makes per-test global state safe but not shared fixtures on disk.
NUIF relevance
Borrow
rust-toolchain.tomlplus--lockedin every CI command, because the Cargo FAQ ties deterministic builds and snapshot stability to the lock file and the toolchain pin is currently only inci.yml.- The xtask alias for fixture regeneration and browser-differential runs, because it keeps automation in Rust with no extra binaries, consistent with ADR 0001.
- cargo-deny with an allow-list matching the workspace
Apache-2.0 OR MITpolicy, because the toolkit comparison surfaced Apache-2.0-only (Masonry) and GPL/commercial (Slint) candidates that a policy check would flag. - cargo-hack
--feature-powerset --rust-version, because optionalserde/arbitraryfeatures on engine crates must compile in every combination at 1.85.
Adapt
- The
fuzz/crate should be a workspace member only if the pinned stable toolchain cancargo checkit; otherwise use--fuzzing-workspace=trueand a nightly job, because cargo-fuzz requires nightly to run. - cargo-semver-checks applies once
nuif-apipublishes a versioned public API; until then run it oncrates/nuif-apiagainst the previous tag only. - nextest’s JUnit is the CI-facing report; the NUIF machine-readable report (QA item 10) must be produced by the harness itself and attached as an artifact.
Reject
- cargo-dist as a required component now, because the CLI is a prototype (
nuif versionprints 0.0.1) and release packaging is premature. - cargo-udeps in the required matrix, because it needs nightly.
- Relying on
--shufflefor order-independence checks, because it is unstable; use nextest process isolation and explicit--test-threads=1where state is shared.
Open questions
- Whether the MSRV pin remains 1.85.0; all candidate GUI toolkits require 1.88 or newer, and proptest’s main branch states 1.86.
- Whether editor crates live in the same workspace (shared lock, shared MSRV) or in a nested workspace with its own toolchain file, which the rustup proximity rule supports.
- Whether Windows and macOS runners are required for engine tests, or only for editor and snapshot tests where platform text and GPU differences matter.
Cassius, VizAssert and Troika - SMT formalisation of CSS layout and machine-checkable visual assertions
Document status:
reviewed. Canonical source.
Summary
Cassius (OOPSLA 2016) encodes a fragment of CSS 2.1 as a relation between an element tree, a rule set and a box tree, expressed in quantifier-free linear real arithmetic and solved with Z3. Every box coordinate is a real-valued constant, every layout rule of the standard becomes an equation or inequality over these constants, and the cascade is computed inside the solver. Because any field may be a hole, the same encoding verifies, debugs and synthesises stylesheets. The formalisation was validated against 2075 W3C CSS 2.1 conformance tests with Firefox 41.0.1 as the oracle, agreeing on all but six, all six traceable to Firefox’s fixed-point rounding. VizAssert (PLDI 2018) extends the fragment (line height, margin collapsing, full float semantics, positioned layout, media queries) through finitisation techniques, defines a visual logic of universally quantified assertions over boxes with linear arithmetic and ancestor navigation, and checks assertions for all renderings within bounded ranges of window size and font size; on 62 pages and 502 page-assertion pairs it found 64 true violations with 13 false positives and 11 timeouts. Troika (OOPSLA 2019) makes verification modular: a page is decomposed into components with rely/guarantee specifications, well-formedness of the decomposition is a pure-logic check, and component obligations are discharged by per-component tools, giving 13-1469× speed-ups over whole-page verification. NUIF interpretation follows in the relevance section.
Evidence
OOPSLA 2016, “Automated Reasoning for Web Page Layout” (DOI 10.1145/2983990.2984010, pp. 181-194; PDF from sandcat.cs.washington.edu retrieved 2026-08-29):
- Theory and solver: “theory of quantifier-free linear real arithmetic”, Z3; high-level specification written in SMT-LIB2 with quantifiers and grounded per problem. Source: §1, §3.2, §4.
- Layout is a relation on element tree E, rules R and boxes B; box types root, block, inline, line, text, opaque; each box has position, width, height and per-side border widths; any field except text width and height may be a hole (
?). Source: §3.2, Figure 2. - Cascade is computed declaratively:
e[p] = r[p]of the highest-scoring matching rule, else the default. Source: §3.2. - Block layout distils “36 pages of the CSS standard into just 790 lines”; naive grounding is O(|B|²), rewritten with auxiliary uninterpreted functions to at most one quantifier per rule, giving an encoding linear in |B|. Source: §3.2, §4.1.
- Supported fragment: CSS 2.1 cascade and box model, block and inline boxes, floats, line boxes, margin collapsing, text-align; selectors limited to tag, id and universal; font metrics, line breaking and hyphenation are unmodelled; tables are opaque boxes; four restrictions on float interactions. Source: §3.1, §3.4, Figure 7.
- Conformance: 2075 W3C CSS 2.1 conformance tests within the fragment, oracle Firefox 41.0.1, agreement on all but six, all due to Firefox rounding (1/60 px fixed point, pixel-rounded borders and text); full suite 138 minutes, under 3 s per test. Source: §5.1, Table 1.
- Rejection (mutation) testing: 20,750 mutants, 152 accepted (99.3% rejected); 126 acceptances due to unmodelled font metrics, 26 due to shrink-to-fit non-determinism in CSS 2.1. Source: §5.1.2.
- Case studies on Amazon, Baidu, Google, Wikipedia, Yahoo! (18-45 elements, 35-54 boxes): verification 2-12 s, debugging 1-5 s, synthesis of 25 holes in minutes; unsat cores of 1-5 rules and 1-6 properties. Source: §5.2, Tables 2 and 3.
- Float scalability: pages with 0-13 floats complete “within a few minutes”; without the float restrictions nothing finishes within an hour. Source: §5.3, Figure 11.
PLDI 2018, “Verifying That Web Pages Have Accessible Layout” (DOI 10.1145/3192366.3192407, printed on p. 1; pp. 1-14 of the conference PDF from homes.cs.washington.edu/~mernst, retrieved 2026-08-29):
- Thirteen formalised subsystems (styles, cascade, selectors, box types, layout mode, vertical, clearance, flow width, horizontal, height, floating layout, shrink-to-fit, line height, margin collapse), seven new relative to Cassius; selector matching and cascading moved outside the solver, enabling descendant, child and pseudo-class selectors and em/ex units. Source: §3, Figure 2, §5.
- Finitisation: line height via running baseline accumulators (§4.1); margin collapsing reduced to six reals and one boolean (§4.2); floats via exclusion zones with a register bound (|L|, |R| ≤ 5 “suffices for most web pages”, retry with more) and a per-run SMT proof that the float encoding satisfies the nine standard rules (§4.3).
- Visual logic grammar:
assertion ::= ∀ b1,... ∈ B : cond; conditions over real arithmetic (b.top,b.left, …, constants, multiplication by constants only), box navigation (.parent,.first-child,.next,.ancestor(cond)), box types (window, inline, line, text, block), selectors (b ∈ $(sel)), edge selection (b.left[margin|border|padding|content]), colours with gamma; universal quantification only, no recursion. Source: §3, Figure 3. - Semantics: an assertion is verified “for all rendering parameters in a user-chosen bounded set” or a counterexample is produced consisting of boxes plus concrete window size and font size. Source: §1, §2.
- Fourteen encoded guidelines (Table 1) include minimum text size, 200% resizing, line length ≤ 80 characters, screen-reader-only content off-screen, no horizontal scroll, heading hierarchy, no text overlap, line spacing, contrast, no text over background image, dropdowns hidden, aligned columns, visible link text, minimum button size.
- Conformance: 1006 W3C CSS 2.1 tests for §§8.3, 8.3.1, 9.5-9.5.2, 10.8, 10.8.1; VizAssert passes 915 versus Cassius 271; all 91 failures use unsupported features; five passing tests differ from Firefox where Firefox is documented as incorrect; comparison tolerance one-sixth of a pixel. Source: §5.3, Table 3, footnote 7.
- Evaluation: 62 of the 100 most recent Free Website Templates pages fit the subset; parameter ranges width 1024-1920, height 800-1080, font 16-32 px; 30-minute timeout; Z3 4.5.1. 502 page-assertion pairs: 64 true positives, 13 false positives, 11 timeouts (2.2%); false positives arise from glyph shapes (descenders). Verification times 10-1000 s (CDF), instances of 488k-1052k terms. Source: §5.1-5.2, Table 2, Figure 10.
- Excluded: vertical alignment, right-to-left text, SVG, tables, JavaScript. Source: §6.
OOPSLA 2019, “Modular Verification of Web Page Layout” (DOI 10.1145/3360577, Article 151, pp. 151:1-151:26, CC BY 4.0; PDF from homes.cs.washington.edu/~mernst, retrieved 2026-08-29):
- Component = subtree with holes and a symbolic computed style; a modular layout proof is a decomposition C plus specifications P_c such that
(∧_c P_c) ⇒ Q(Definition 4.3); well-formedness is “a matter of pure logic” (layout-agnostic) and checked by Z3 in 0.54 s; component specifications are layout-conscious, written as∧ R_j ⇒ ∧ A_i(rely/guarantee). Source: §4, §5. - Tools per component:
admit,random-test[n],model-check,whole-page,component-smt; random-test and admit are unsound;component-smtinherits VizAssert’s soundness by removing constraints only. Source: §5.3, §6.1. - Extensions to the logic:
collapsed-margin,non-negative-margins,starts-float-free,ends-float-free,no-floats-enter,float-flow-across. Source: §5.4. - Case study on a page “11× larger” than prior work: proofs of 36 lines; speed-ups of 13-1469× over VizAssert (Table 1); eight re-proved properties from prior work at 1.9-67× (Table 2); overall 2.6× serial, 4.3× with 8 threads, 13× with caching. Source: §7, Tables 1 and 2.
- Unsupported in component-smt:
transform,:before/:after, tables, flexbox, right-to-left. Source: §2.2, §7.
Repository: github.com/uwplse/cassius, Racket, MIT licence, requires Firefox with Geckodriver, Z3 ≥ 4.5, Racket ≥ 7.0 (README retrieved 2026-08-29). The project site cassius.uwplse.org did not resolve on 2026-08-29.
Mechanism
Encoding (Cassius §3-4):
Inputs: element tree E, rules R (with holes), boxes B (with holes)
Vars: for each box b: b.x, b.y, b.w, b.h, border widths ∈ ℝ
Cascade: e[p] = r[p] for the highest-specificity matching r, else default(p)
Layout: per box type, equations from CSS 2.1, e.g.
in-flow block: b.x = parent.content-left + b.margin-left
Query: ∃ holes . Layout(E, R, B) -- synthesis / debugging
¬∃ params . Layout(E, R, B) ∧ ¬P(B) -- verification
Theory: QF_LRA, Z3; grounding linear in |B| via uninterpreted helper functions
Visual logic (VizAssert §3):
assertion ::= ∀ b1 ... bn ∈ B : cond
cond ::= cond ∧ cond | ¬cond | cond ∨ cond | real ⋈ real | box = box
| box.type = type | box ∈ $(selector)
real ::= k | real + real | k × real | box.dir[edge] | color.channel
box ::= bi | root | null | box.parent | box.first-child | box.next | box.ancestor(cond)
Example: onscreen(b) := b.right ≥ root.left ∧ b.bottom ≥ root.top
∀ b ∈ B : for_screenreader(b) ⇒ ¬onscreen(b)
Checked over: width ∈ [1024,1920], height ∈ [800,1080], font ∈ [16,32]
Modular proof (Troika §4):
page p, decomposition C = {c1..cn}, specs Pc = (∧ Rj ⇒ ∧ Ai), goal Q
well-formed(C, P, Q) :⇔ (∧c Pc) ⇒ Q -- pure logic, no layout
each Pc discharged by admit | random-test | model-check | whole-page | component-smt
NUIF relevance
- Borrow: The visual logic’s assertion forms (no overlap, on-screen, containment, alignment via equal edges, minimum size, text-fits via line width, contrast) are a ready-made oracle vocabulary over resolved boxes; NUIF conformance fixtures can adopt this grammar and evaluate it concretely on each resolved snapshot without any solver, with symbolic checking as an optional stronger mode.
- Borrow: The validation methodology - a formal model checked against a W3C conformance suite with a browser oracle and an explicit tolerance (1/6 px), plus mutation-based rejection testing - is the template for validating NUIF’s CSS-family lowering and for nuif:experiment:layout-differential.
- Borrow: Rely/guarantee component specifications (Troika) map directly onto NUIF components with declared layout contracts; well-formedness as a pure-logic check is independent of any layout engine.
- Adapt: The quantified parameter ranges (window size, font size) correspond to NUIF evaluation contexts; NUIF should express ranges as context predicates and sample them concretely, since symbolic verification over ranges is available only for the CSS 2.1 fragment and costs 10-1000 s per assertion.
- Adapt: The fragment excludes flexbox, grid, tables, transforms and text shaping; NUIF’s flex and grid families therefore cannot be verified symbolically with this work, and the encoding effort (790 lines for block layout) indicates what a mechanised flex/grid semantics would require.
- Reject: A single browser (Firefox) as ground truth for the model; NUIF must record which browser and version served as oracle and treat browser disagreements as separate divergence classes.
- Reject: SMT verification as a conformance requirement; the timeouts (2.2%), false positives from glyph shapes (17% of counterexamples) and instance sizes (up to 10^6 terms) make it a research tool rather than a normative test harness.
Open questions
- Can the visual logic be evaluated concretely over NUIF resolved snapshots with identical semantics, so that the same assertion file serves both the concrete test oracle and a future symbolic checker?
- Which subset of the fourteen accessibility guidelines can be stated purely over NUIF resolved geometry and semantic annotations (spec/13) without CSS-specific box types?
- Is a mechanised semantics of CSS Flexbox §9 in QF_LRA feasible, given that flexible length resolution (§9.7) is an iterative freeze loop and intrinsic sizing is only partially specified (see nuif:research:css-flexbox-grid-algorithm-specs)?
- Troika’s component specifications were written by hand over hours; could NUIF derive component layout contracts automatically from authored layout intent (stack/flex constraints) instead?
Cassowary incremental linear constraint solving for user interfaces
Document status:
reviewed. Canonical source.
Summary
Cassowary is an incremental dual-simplex constraint solver designed for UI equalities, inequalities and preferences. It supports relationships that are awkward to express as a single parent-owned flow algorithm.
NUIF relevance
Constraint layout belongs as a distinct authored layout family rather than being forced into flex/grid. The core schema should model constraint identities, strengths and variables independently of one solver implementation.
CBOR integer/float identity and deterministic encoded-key ordering
Document status:
verified. Canonical source.
Summary
RFC 8949 defines integer and floating-point values as distinct members of the CBOR basic generic data model even when their mathematical values are equal. Its preferred serialization narrows the width of a floating-point encoding but does not change a floating-point data item into an integer. NUIF’s RFC 0005 instead reduced an integral real to a CBOR integer and relied on an external property schema to restore its type. That is not lossless for extension values, unknown properties, or generic tools and gives integer(1) and real(1.0) the same bytes despite the NUIF logical model distinguishing them.
RFC 8949 core deterministic ordering compares the complete deterministic encodings of map keys bytewise. For text keys, the encoded length byte participates in the comparison. UTF-8 lexical order therefore does not generally coincide with CBOR encoded-key order: UTF-8 sorts "aa" before "z", while their CBOR encodings 0x62 61 61 and 0x61 7a sort "z" first. RFC 0005’s claim of coincidence is false.
Evidence
- RFC 8949 §2 states that integer and floating-point values are distinct in the basic generic data model even when they have the same numeric value. Locator: heading “Data Models” and the paragraph beginning “Note that integer and floating-point values are distinct”.
- RFC 8949 §4.1 defines preferred serialization of a floating-point value as the shortest floating-point width that preserves its value. It does not authorize conversion to major type 0 or 1. Locator: heading “Preferred Serialization”, floating-point bullet.
- RFC 8949 §4.2.1 requires map keys to be sorted by bytewise lexicographic order of their deterministic encodings. Locator: heading “Core Deterministic Encoding Requirements”.
- RFC 8949 §4.2.3 gives the encoded examples
"z" = 0x617aand"aa" = 0x626161. The bytes provide the minimal counterexample for the ordering claim. Retrieved 2026-08-29. - Executable regressions:
nuif_codec::tests::encoded_key_order_is_not_utf8_orderandnuif_codec::tests::integer_and_integral_real_remain_distinct.
Mechanism
nuif-cbor-0 preserves the NUIF numeric kind: integers use CBOR major types 0/1; reals always use a CBOR floating-point data item at the shortest exact width. Because the logical model does not distinguish negative real zero, both real zeros use positive floating-point zero, while integer zero remains the integer 0x00. CBOR maps sort by complete encoded key bytes. nuif-text-0 independently sorts object keys by UTF-8 bytes because its hash is defined through the parsed CBOR form rather than its textual byte order.
NUIF relevance
The correction removes a schema-dependent decoding requirement from generic and extension values, prevents numeric type collisions in canonical hashes, and makes the text and CBOR ordering rules independently implementable. RFC 0008 supersedes only the conflicting rules of RFC 0005; its finite-number, strict-decoder, verbatim-string, opaque-byte and hash rules remain applicable.
Open questions
No open question remains for profile 0. A future profile that intentionally unifies integers and integral reals would define a different logical value model and a different profile identifier.
Chromium DevTools Protocol as a source-backed UI observation surface
Document status:
reviewed. Canonical source.
Summary
The Chrome DevTools Protocol (CDP) exposes complementary browser observations: DOM and layout snapshots, computed and matched CSS, platform-font usage, stylesheet text, response bodies, accessibility trees, screenshots and browser environment emulation. Together these provide much stronger evidence than a screenshot, but they still describe one browser execution under one context; they do not reveal arbitrary application intent or guarantee access to local font bytes.
NUIF should add a dedicated, pinned browser-capture adapter. It should not turn the current Tree-sitter source-synchronization adapter into a browser runtime. Static source retention and resolved browser capture are separate evidence channels that may be correlated through provenance.
Evidence
- CDP
DOMSnapshot.captureSnapshotreturns flattened documents including iframes, template contents, imported documents and flattened shadow trees, with requested computed styles. It can include DOM rectangles, inline text boxes, paint order and blended background/text colors. - CDP CSS exposes
getComputedStyleForNode,getMatchedStylesForNode,getStyleSheetText, media queries, pseudo-state forcing andgetPlatformFontsForNode. The last operation reports platform-font usage, not the original local font file bytes. - CDP Network exposes request/response events and
getResponseBody; downloaded image, stylesheet and web-font response bytes can therefore be captured while the request remains available to the protocol session. - CDP Accessibility exposes full or partial accessibility trees, adding browser-computed role/name/state evidence distinct from the DOM tree.
- CDP Page exposes screenshots and an MHTML snapshot. Emulation exposes device metrics, media features, locale, timezone and related execution context.
- CSS Font Loading Level 3 defines
document.fonts.readyas a readiness signal after font loading and layout operations complete; it does not make local font files distributable or directly retrievable.
Mechanism
A capture run pins browser build and records a CaptureContext: operating
system, viewport, device-pixel ratio, page scale, locale, timezone, color
scheme, reduced-motion preference, font environment, pseudo states, scroll
positions, navigation URL and a deterministic settling policy. The policy waits
for the load milestone, network quiescence bounded by a timeout,
document.fonts.ready, and an explicit animation-freeze point.
For each declared viewport and state the adapter collects:
- original HTML/CSS response bodies when available;
- DOMSnapshot nodes, layout, inline boxes, computed-style whitelist and paint order;
- matched rules and stylesheet text needed for source correspondence;
- downloaded resource bodies, final URLs, response media types and hashes;
- platform-font usage and font readiness;
- the accessibility tree;
- a reference screenshot and its exact capture parameters.
Canvas, WebGL, video and worklet output are observation boundaries: capture a bounded raster/frame and retain source/provenance where accessible, but do not pretend the pixels are an editable semantic reconstruction. Cookies, authorization headers, form secrets, storage and credentials are excluded from export. Captured scripts are inert source resources and are never executed by a NUIF reader.
NUIF relevance
Borrow CDP as the first source-backed Web observation port because its layout, style, resource and accessibility domains can be pinned to a browser build and replayed as fixture evidence.
Adapt observations into typed NUIF provenance and fidelity. Multiple viewports/states constrain responsive inference; authored source spans remain separate from resolved browser values.
Reject “DOM equals design intent,” capture under unspecified host settings, credential export, automatic external fetch during NUIF load, and lossless classification for canvas/video/script behavior based only on a frozen frame.
Open questions
- The first segment pins the complete Chrome for Testing build rather than a floating CDP schema and records its reported protocol version. CDP explicitly provides no tip-of-tree compatibility guarantee, so browser updates require a gate rerun and review.
- The first segment retains background colour plus actual font-use as its implemented style subset. Font family/size/line-height are requested in the snapshot but are not yet promoted to observations; matched-rule and stylesheet/source correspondence remain an explicit extension.
- How can cross-origin iframes and opaque responses report unavailable evidence without silently flattening them into screenshots?
- Which interaction states can be captured reproducibly without executing untrusted navigation actions?
Undo models - Command and Memento patterns, event sourcing, and undo in Blender, Photoshop and Figma
Document status:
reviewed. Canonical source.
Summary
Gamma et al. describe two complementary mechanisms: Command encapsulates a request as an object so that requests can be queued, logged and undone, with undo implemented by storing enough state in the command to reverse its effect and a history list for undo/redo; Memento captures an object’s internal state so it can be restored later without breaking encapsulation. Fowler’s event sourcing records every state change as an event, from which state can be rebuilt, queried at any past time, or replayed after correcting an event; reversal is either by an inverse event (possible only when the event carries enough information, “add $10” rather than “set to $110”) or by reverting to a snapshot and replaying. Editors combine these: Blender keeps a single undo stack of typed steps, stateful or differential, with global undo implemented as an in-memory .blend file written with the regular file-writing code; Photoshop keeps a bounded list of history states (snapshot-based, not saved with the document) with an optional non-linear mode; Figma’s multiplayer undo is defined so that undoing and redoing back to the present leaves the document unchanged, with undo rewriting the redo history relative to the current shared state. The invariants that transfer to NUIF are listed under Mechanism.
Evidence
- Command pattern: Gamma, Helm, Johnson, Vlissides, Design Patterns (1994), Command, pp. 233-242; intent is to encapsulate a request as an object, parameterise clients, queue or log requests, “and support undoable operations”; implementation notes cover storing state for reversal and a history list for undo and redo. Memento, pp. 283-291; intent is to capture and externalise internal state without violating encapsulation so the object can be restored later. (Book; page numbers from the table of contents; not retrieved online.)
- Event sourcing: definition “Capture all changes to an application state as a sequence of events”; capabilities Complete Rebuild, Temporal Query, Event Replay; reversal: “all the capabilities of reversing events can be done instead by reverting to a past snapshot and replaying the event stream”; the example contrasts “add $10 to Martin’s account” with “set Martin’s account to $110”; external-system updates and queries are the stated hazards of replay. Fowler, 2005-12-12, https://martinfowler.com/eaaDev/EventSourcing.html (retrieved 2026-08-29).
- Blender undo system: “Undo is organized as an ‘undo stack’ storing a list of ‘undo steps’”; steps are relative (must be loaded in sequence) or absolute; “Currently, Blender undo stack is fully relative”; steps are stateful (“stores the state of some data, and can be loaded regardless of the direction”) or differential (“only stores the difference to the previous step … either applied (redo) or unapplied (undo)”), and with differential steps undoing requires unapplying step n+1; only data is stored, not UI; one stack gathers global undo, edit-mode undo, sculpt/paint undo; skipped steps are hidden intermediates; undo push is driven by the operator system; layers are
ed_undo.cc,undo_system.cc(BKE), and per-type implementations such asmemfile_undo.ccandsculpt_undo.cc, where memfile undo “uses BKE_memfile_ functions from blender_undo.c, which in turns uses read/write .blend file code from BLO”. https://developer.blender.org/docs/features/core/undo/ (retrieved 2026-08-29; page marked WIP by its authors). Memfile improvement tracking: “Undo: support implicit-sharing in memfile undo step”, https://projects.blender.org/blender/blender/pulls/106903 (search result, not retrieved). - Photoshop history: Adobe help pages returned HTTP 403 and timeouts on retrieval; indexed text of https://helpx.adobe.com/photoshop/using/performance-preferences.html states Photoshop saves up to 1,000 history states with a default of 50, that “Allow Non-Linear History” permits editing from any state without deleting later ones, and that history states consume scratch memory (search snippets, 2026-08-29; treated as secondary evidence).
- Figma multiplayer undo: “if you undo a lot, copy something, and redo back to the present, the document should not change”; an undo “modifies redo history at the time of the undo, and likewise a redo operation modifies undo history at the time of the redo”, stated as necessary so users do not overwrite others’ later edits; changes are applied optimistically on the client, conflicting server updates are discarded while a client change is unacknowledged, “the server can define the order of events”, per-property last-writer-wins; parent link and fractional position are one property updated atomically. https://www.figma.com/blog/how-figmas-multiplayer-technology-works/ (retrieved 2026-08-29).
- Undo integrated with transformation: Sun et al. (TOCHI 1998) §7 integrate their GOT control algorithm with an undo/do/redo scheme, undoing later operations, applying the new one and redoing (https://www.cs.cityu.edu.hk/~jia/research/reduce98.pdf, retrieved 2026-08-29); Kleppmann et al. use the same undo-do-redo shape with a recorded prior parent (nuif:research:crdt-tree-move-operation, Fig. 4 lines 32-49).
Mechanism
Two undo representations:
inverse-operation undo (Command): history = [(op_i, inv_i)]; undo = apply(inv_k); redo = apply(op_k)
requires: inv_i computable at record time -> op must carry or capture prior state
memento / snapshot undo (Memento): history = [state_i]; undo = restore(state_{k-1})
requires: cheap snapshot (structural sharing, chunk dedup) ; no inverse needed
differential step (Blender): stores delta to previous step; undo unapplies step k+1, redo applies step k
Recording inverse operations for a replay log (NUIF interpretation, derived from Command, LogMove and Fowler’s “add $10” example):
record(op, doc):
pre := preconditions(op) -- e.g. exists(entity), parent(entity) = p_old, prop(entity,k) = v_old
inv := inverse(op, doc) -- Move{e,new} -> Move{e, p_old, order_old}
-- SetProperty{k,v} -> SetProperty{k, v_old} | UnsetProperty{k}
-- Remove{e} -> Insert{e, parent_old, order_old, payload_old}
-- Insert{e} -> Remove{e}
log += (op, pre, inv)
invariant: apply(inv, apply(op, doc)) = doc whenever pre holds in doc
Invariants that the sources support:
- Undo is a semantic inverse under preconditions: an inverse is only valid against the state the operation produced; if preconditions of the inverse fail (another actor changed the value), the editor must either transform, skip, or resolve relative to current state (Figma; Sun et al. §7).
- Transactions are atomic undo units: an undo step corresponds to one user-level operation, possibly comprising many primitive operations (Blender undo push per operator; RFC 6902 §5 atomicity).
- Redo stack invalidation: in linear history a new edit after undo discards the redo branch (Photoshop default; Blender relative stack); non-linear history retains it (Photoshop option); in multiplayer, undo rewrites redo entries against the current shared state so that undo-then-redo is the identity on the document (Figma).
- Snapshot undo is equivalent to inverse undo in capability (Fowler) but differs in cost: inverse logs are proportional to change size, snapshots to state size unless deduplicated (Blender memfile reuses
.blendwriting). - Replay must be side-effect free: events that trigger external effects cannot be replayed naively (Fowler), which constrains what a NUIF operation may do (pure document mutation).
- Undo history is not document state: Photoshop discards history on save; Blender stores steps in memory; spec/06 states undo “is not part of canonical document state”.
NUIF relevance
Borrow
- Record
(op, preconditions, inverse)triples per transaction in the profile log, so a NUIF patch can be inverted mechanically and replayed deterministically to the same canonical hash. - Blender’s stateful/differential step distinction: NUIF checkpoints (content-addressed snapshots) are stateful steps; patches between checkpoints are differential steps.
- Figma’s invariant that undo followed by redo returns the same document, as a test property for the collaboration profile.
Adapt
- Inverse computation must consult the base revision:
Removemust capture the removed subtree (or its content hash plus a retrievable object) so that the inverseInsertis complete; nuif-protocol’sRemove { entity }currently carries nothing to invert. - Multiplayer undo semantics belong in spec/10: the profile must define whether undo of a property change restores the user’s prior value, the current value’s predecessor, or a conflict object when another actor has since written the property.
- Transaction granularity: nuif-protocol
Transaction { id, operations }is the unit of undo; the log should also carry an actor and a base revision so that undo across replicas is well-defined.
Reject
- Snapshot-only undo for the canonical format: it is acceptable for editors but does not yield the serializable inverse operations that spec/06 requires.
- Storing undo history inside the canonical document.
Open questions
- Whether inverse operations should be stored explicitly or derived at replay time from the base snapshot; explicit storage doubles log size but makes inverse validity checkable without the base.
- The precise multiplayer undo rule for NUIF (Figma’s implementation details beyond the stated invariant are not public) and its interaction with CRDT reordering, where a later-timestamped undo may itself be reordered.
- Blender’s memfile chunk deduplication and implicit sharing details are only referenced through issue trackers; a primary description of the chunk comparison algorithm was not retrieved.
Community Specification license and repository governance
Document status:
verified. Canonical source.
Summary
Community Specification 1.0 provides repository-based legal and governance terms for collaborative specification development. Its contributor agreement, scope, notices and license files define participation, patent coverage and source-code licensing. The process recommends separate repositories for a specification and its reference source code where practical.
Evidence
- The contributor license agreement binds participants to the legal and
governance terms of the working group. Locator:
getting-started.md, lines 199–200, retrieved 2026-08-30. Scope.mddefines the working group’s subject matter and bounds the patent licensing obligations. Locator:getting-started.md, lines 201 and 209, retrieved 2026-08-30.Notices.mdrecords contacts, patent exclusions, implementers and withdrawn participants.License.mdidentifies the specification license and the separate license for source or sample code. Locator:getting-started.md, lines 203–213, retrieved 2026-08-30.- The best-practice section recommends a contributor-agreement check, use of
the specification license for specifications rather than code, careful scope
definition and separate specification and code repositories. Locator:
getting-started.md, lines 235–245, retrieved 2026-08-30.
Mechanism
Each contributor accepts the common agreement before a contribution is merged. The declared scope bounds patent commitments. Notices record exclusions and implementer assertions. The specification license grants rights applicable to independent implementations, while an Open Source Initiative-approved license continues to govern implementation code.
NUIF relevance
Borrow the explicit scope, notices and contributor-agreement model before a multi-party specification is published as an implementable draft.
Adapt the separate-repository recommendation only when the specification has independent contributors. The current monorepository keeps experiments, fixtures and draft modules reviewable at the same revision.
Reject applying Community Specification terms retroactively without legal review and contributor consent. The current code licenses do not establish the specification-wide patent commitments described by this process.
Open questions
- The entity that would administer contributor agreements and notices has not been selected.
- The patent scope requires legal review after the implementable draft boundary is stable.
Confidence calibration, risk coverage and explicit abstention
Document status:
reviewed. Canonical source.
Summary
Inference confidence is useful only when it predicts observed correctness under a declared condition. Guo et al. show that modern neural-network confidence can be poorly calibrated and evaluate post-hoc calibration, including temperature scaling. SelectiveNet studies an explicit reject option and risk/coverage trade-off. Together they support calibrated confidence and abstention instead of forcing every screenshot region into a confident semantic claim.
Evidence
- Guo et al., ICML 2017, defines calibration as predicted probability matching empirical correctness likelihood, documents miscalibration in studied modern networks, and reports temperature scaling as a strong simple baseline on its classification datasets.
- Geifman and El-Yaniv, SelectiveNet, ICML 2019, https://proceedings.mlr.press/v97/geifman19a.html, trains prediction and rejection jointly and evaluates risk as coverage changes.
- Both studies concern classification/regression. Applying their procedures to structured UI reconstruction requires task-specific correctness events and cannot reuse their thresholds directly.
Mechanism
NUIF confidence is attached to individual observations and inferred decisions, not only a whole document. Calibration sets map raw scores to empirical success for events such as correct text, region class, parent, layout family, resource match or operation acceptance. A policy can abstain, retain alternatives or request review when expected risk exceeds a profile limit.
NUIF relevance
Borrow reliability diagrams, expected calibration error as one diagnostic, proper scoring where applicable and risk/coverage curves.
Adapt correctness to multiple structured outcomes. A candidate can be visually close but structurally wrong, so confidence is typed by decision and evaluated on frozen, shifted and out-of-distribution subsets.
Reject raw model likelihood as portable confidence, one global threshold for every property and forced guesses where evidence is absent.
Open questions
- Which structured correctness events have enough validation examples for stable calibration?
- How should confidence compose when a parent/layout/resource decision depends on several uncertain observations?
- What risk/coverage target is acceptable for automatic application versus a suggestion shown for human review?
Content-addressed Merkle DAGs for immutable resources and snapshots
Document status:
reviewed. Canonical source.
Summary
Merkle DAGs assign immutable nodes identifiers derived from their contents and referenced children. This provides verifiable immutable snapshots and deduplication but changes identity whenever content changes.
NUIF relevance
Use content hashes for immutable assets, packages and canonical snapshots, but not for editable semantic entity identity. Stable entity IDs and content-addressed snapshot/resource IDs solve different problems and must remain separate.
A highly-available move operation for replicated trees (Kleppmann, Mulligan, Gomes, Beresford)
Document status:
reviewed. Canonical source.
Summary
The paper defines an operation-based CRDT for trees whose only operation is Move t p m c (timestamp, new parent, metadata, child). Node creation is a first move; deletion is a move under a designated trash node. Each replica keeps the tree as a set of (parent, meta, child) triples plus a log of applied moves in descending timestamp order, each log entry recording the child’s previous parent. A remote operation with timestamp t is applied by undoing every logged operation with timestamp greater than t, applying the new operation, and redoing the undone operations. Applying an operation whose child is an ancestor of its destination is a no-op. The result is a state identical to sequential application in timestamp order, so any permutation of the same operation set converges. Convergence, acyclicity and unique-parent invariants are mechanised in Isabelle/HOL, and Scala code is extracted from the proofs. Automerge does not yet ship this algorithm; Da and Kleppmann (PaPoC 2024) adapt it to the Automerge operation set and report that Automerge currently models a move as delete-plus-reinsert. Sibling order is delegated to a list CRDT identifier stored in the metadata field, the alternative being fractional indices as used by Figma.
NUIF interpretation is separated from source statements in the relevance section.
Evidence
- Bibliographic data: IEEE TPDS vol. 33, no. 7, pp. 1711-1724, DOI 10.1109/TPDS.2021.3118603; open-access PDF at https://martin.kleppmann.com/papers/move-op.pdf; code and proofs at https://github.com/trvedata/move-op (author page https://martin.kleppmann.com/2021/10/07/crdt-tree-move-operation.html, retrieved 2026-08-29).
- Motivation: concurrent moves of the same directory produced duplication in Dropbox (Fig. 1a) and one of the two intended outcomes in Google Drive (Fig. 1c/d); concurrent reciprocal moves (A under B, B under A) can create a cycle (Fig. 2). PDF §2.1-2.2, pp. 2-3.
- Algorithm definition: Fig. 4 (59 lines of Isabelle/HOL) defines
state = log_op list × (n × m × n) set(line 14),get_parent(lines 16-20), the inductiveancestorrelation (lines 22-24),do_op(lines 26-30) with the guardif ancestor tree c newp ∨ c = newp then tree(line 29),undo_op(lines 32-35),redo_op(lines 37-40),apply_op(lines 42-49),apply_opsasfoldl(lines 51-52), and theunique_parentandacyclicpredicates (lines 54-59). PDF p. 6. - Undo-do-redo: “it first undoes the effect of any operations with a timestamp greater than t, then performs the new operation, and finally re-applies the undone operations.” PDF §3.4, p. 7. The log is kept in descending timestamp order;
LogMoveadds anoldp :: (n × m) optionfield (PDF §3.2, p. 7). - Conflict semantics: concurrent moves of one node are resolved by the greater timestamp; an operation that would close a cycle is ignored because
do_opchecks against the tree produced by all lower-timestamped operations; ignored operations must remain in the log because later lower-timestamped operations can change their safety. PDF §3.5, pp. 7-8. - Creation/deletion: creation is the first move of a fresh node; deletion moves to a trash node; children of deleted nodes are retained so a concurrent move can bring them back. Node creation may bypass undo-redo under three stated assumptions, proved safe in Isabelle; deletion cannot. PDF §3.6, p. 8.
- Log truncation and garbage collection use causal stability: operations with timestamp at or below the causally stable threshold can be dropped; trashed subtrees can be discarded once the trashing operation is stable. PDF §3.7, p. 8.
- Sibling ordering: “This can be implemented by maintaining an additional list CRDT for each branch node, e.g. using RGA [14] or Logoot [15]”; the list element ID is placed in the metadata field, and reordering is a move with unchanged parent and a new ID. PDF §3.7, pp. 8-9.
- Theorems (all machine-checked):
apply_ops_unique_parent,apply_ops_acyclic, andapply_ops_commutes(assumesset ops1 = set ops2and distinct timestamps, showsapply_ops ops1 = apply_ops ops2); strong eventual consistency is obtained through the framework of Gomes et al. PDF §4.1-4.2, p. 9. - Proof size: 59 lines of definitions plus 2,495 lines of proof (203 unique parent, 443 acyclicity, 450 commutation/convergence, 327 SEC, 743 executable refinement, 779 creation optimisation); checking takes about 3 minutes. PDF §5.3, p. 11. Repository layout:
proof/Move.thy,proof/Move_Acyclic.thy,proof/Move_SEC.thy,proof/Move_Code.thy,evaluation/Scala, MIT licence (GitHub README, retrieved 2026-08-29). - Complexity: worst case
O(nd)per applied operation,nbeing the number of logged operations to undo and redo anddthe tree depth. PDF §5.1, p. 10. Local operations need no undo/redo because their Lamport timestamp exceeds all logged ones (median 1-2 µs hand-written, about 50 µs generated code); remote saturation at 5,700 ops/s hand-written versus 600 ops/s generated, with about 200 undos/redos per remote operation at peak. PDF §5.1, p. 11. - Comparison to state machine replication: leader ordering reaches 22,000 ops/s but requires a 145-176 ms round trip per operation and no offline editing. PDF §5.2, pp. 11-12.
- Automerge status: Automerge merge-rules documentation lists no move operation and orders concurrent inserts at one position by an arbitrary but deterministic choice (https://automerge.org/docs/reference/under-the-hood/merge-rules/, retrieved 2026-08-29). The Automerge JSON CRDT states “Our approach for handling insertions is based on the RGA algorithm” (Kleppmann and Beresford, IEEE TPDS 2017, DOI 10.1109/TPDS.2017.2697382, §4, INSERT1/INSERT2 rules; arXiv 1608.03960 PDF retrieved 2026-08-29).
- Automerge extension: Da and Kleppmann, “Extending JSON CRDTs with Move Operations”, PaPoC 2024 (arXiv 2311.14007; Cambridge repository PDF retrieved 2026-08-29): “Currently, Automerge handles moves by deletion and reinsertion” (§1); operations are applied in ascending operation-ID order with a
treemap (child to parent, deletion as parentnull) and awinnersmap (greatest move ID per element) (§3.1, Algorithm 1); optimisations are batch updating and lifecycle tracking (§3.3); a Go prototype exists and integration into the Rust implementation is planned (§4). Kleppmann’s 2024-01-04 review states the algorithm “is not yet fully implemented within Automerge” (https://martin.kleppmann.com/2024/01/04/year-in-review.html, retrieved 2026-08-29). - List move: naive delete-and-reinsert duplicates an element under concurrent moves; the fix treats the element’s position as a register over list-CRDT positions. Kleppmann, “Moving Elements in List CRDTs”, PaPoC 2020, DOI 10.1145/3380787.3393677, §2 (PDF retrieved 2026-08-29).
- Fractional indexing: “An object’s position in its parent’s array of children is represented as a fraction between 0 and 1 exclusive”; parent link and position are stored as a single property so they update atomically; the server rejects parent updates that would cause a cycle. Figma engineering blog (https://www.figma.com/blog/how-figmas-multiplayer-technology-works/, retrieved 2026-08-29).
Mechanism
Types (Fig. 4):
Move t p m c -- timestamp, new parent, metadata, child
LogMove t oldp p m c -- oldp : (parent × meta) option, recorded at apply time
state = LogMove list × (parent × meta × child) set
Core functions (Fig. 4, lines 22-49, transcribed):
ancestor tree a c ⇔ (a, _, c) ∈ tree ∨ ∃p. (p, _, c) ∈ tree ∧ ancestor tree a p
do_op (Move t newp m c, tree) =
(LogMove t (get_parent tree c) newp m c,
if ancestor tree c newp ∨ c = newp then tree
else {(p', m', c') ∈ tree | c' ≠ c} ∪ {(newp, m, c)})
undo_op (LogMove t None newp m c, tree) = {(p', m', c') ∈ tree | c' ≠ c}
undo_op (LogMove t (Some(op,om)) newp m c, tree) = {(p', m', c') ∈ tree | c' ≠ c} ∪ {(op, om, c)}
redo_op (LogMove t p m c) (ops, tree) =
let (op2, tree2) = do_op (Move t p m c, tree) in (op2 # ops, tree2)
apply_op op1 ([], tree1) = let (op2, tree2) = do_op (op1, tree1) in ([op2], tree2)
apply_op op1 (logop # ops, tree1) =
if move_time op1 < log_time logop
then redo_op logop (apply_op op1 (ops, undo_op (logop, tree1)))
else let (op2, tree2) = do_op (op1, tree1) in (op2 # logop # ops, tree2)
apply_ops ops = foldl (λs o. apply_op o s) ([], {}) ops
Invariants proved for every reachable state: every child has at most one (parent, meta) pair (unique_parent); no node is its own ancestor (acyclic); apply_ops is invariant under permutation of an operation set with distinct timestamps (apply_ops_commutes). Timestamps must form a total order with unique values (Lamport timestamps suffice). Preconditions are not user-visible: an unsafe move is silently ignored rather than rejected, and its log entry preserves the information needed to re-evaluate it.
Sibling order is external to the proof: metadata m carries a list-CRDT identifier (RGA or Logoot), so reordering is Move t sameparent newid c. Fractional indexing (Figma) is the other established choice; it requires an authority or a tie-break rule for equal fractions and periodic renormalisation, which the Figma post does not detail.
Cost model: local operations are O(d) for the ancestor check; remote operations are O(nd) where n grows with the number of in-flight concurrent operations; the log can be truncated at the causally stable timestamp.
NUIF relevance
Borrow
- The single-operation model (create as first move, delete as move to trash) because it yields a small proof surface and identical conflict handling for all structural edits.
- The
LogMoveshape (operation plus recorded prior parent and metadata) as the canonical way to record inverse information in a replay log, matching spec/06’s “inverse semantic operations or transaction history”. - The acyclicity precondition expressed as an ancestor check evaluated against the state produced by all lower-ordered operations, and the rule that ignored operations stay in the log.
- Causal stability as the criterion for truncating profile-level logs and discarding tombstoned subtrees.
Adapt
- At retrieval, NUIF’s
Operation::Move { entity, new_parent, new_index }used a non-commutative integer index. RFC 0006 subsequently replaced it withAnchor::Start/Anchor::After(id); collaboration-specific list identifiers remain outside canonical documents. - Silent ignoring of cycle-inducing moves is correct for convergence but must be surfaced as a typed conflict object in the collaboration profile (spec/10) rather than lost, because NUIF requires semantic conflicts to remain explicit.
- The timestamp total order belongs to the collaboration profile; canonical NUIF documents must not carry Lamport timestamps or trash subtrees, so checkpoint materialisation must strip them (spec/10 “materialize a canonical NUIF snapshot without collaboration metadata”).
Reject
- Making undo-do-redo the semantics of the canonical patch format: a NUIF patch is applied against a declared base revision in order, and operations with failed preconditions are reported, not reordered by timestamp.
- Unbounded operation logs in the document: log retention is a profile concern.
Open questions
- Which order key should the collaboration profile mandate for sibling order: RGA-style identifiers (require tombstones) or fractional indices (require renormalisation and an equal-key tie-break)?
- How should a cycle-rejected move be represented to a user: as a conflict object with both intended parents, or as an automatic loss with an audit entry?
- Da and Kleppmann’s lifecycle tracking changes validity as later operations arrive; whether an equivalent incremental validity check can be defined for NUIF patches with preconditions is untested.
- The paper leaves the interaction between move and concurrent property edits inside a moved subtree to the enclosing data model; NUIF property operations need their own commutation argument.
CSS Flexbox §9, Grid §12 and Box Sizing 3/4 - algorithm structure, phases and known implementation divergences
Document status:
reviewed. Canonical source.
Summary
CSS Flexible Box Layout Level 1 (Candidate Recommendation Draft, 14 October 2025) specifies flex layout as a sixteen-step algorithm across line length determination, main size determination, cross size determination and alignment, with §9.7 “Resolving Flexible Lengths” as an iterative loop that distributes free space proportionally to flex factors, clamps by min/max, and freezes violating items until none remain. CSS Grid Level 2 (CRD, 26 March 2025) specifies grid sizing as track sizing run for columns, then rows, then re-run once per axis if min-content contributions changed; track sizing itself has five phases (initialise, resolve intrinsic, maximise, expand flexible, stretch auto), with a detailed sub-algorithm for distributing extra space across spanned tracks. CSS Box Sizing Level 3 (Working Draft, 17 December 2021) defines min-content, max-content, fit-content and stretch-fit sizes and the treatment of cyclic percentages; Level 4 (WD, 20 May 2021) defines aspect-ratio, ratio-dependent and ratio-determining axes, automatic content-based minimums and min/max size transfer. The specifications state that implementations may use any algorithm producing the same results, but several regions are explicitly incomplete or known to diverge from shipping engines: the flex container intrinsic main size algorithm is labelled “not Web-compatible” with a placeholder for a replacement (issue #8884, open); flex intrinsic cross sizing for multi-line column containers is acknowledged as approximate; grid intrinsic sizing “may be updated”; percentage tracks under indefinite sizes changed after all engines had shipped the older behaviour (#1921); engines differ on whether the grid track sizing algorithm runs once or twice for intrinsic container sizes (#2303); the automatic minimum size of flex items with aspect-ratio produced Chrome/Firefox differences (#6794); and Box Sizing 3 leaves float sizes and several intrinsic sizes to CSS 2 “and/or existing implementations”. NUIF interpretation follows.
Evidence
CSS Flexible Box Layout Module Level 1, W3C CRD 14 October 2025 (https://www.w3.org/TR/css-flexbox-1/, retrieved 2026-08-29):
- Algorithm preamble: algorithms are “written to optimize readability”; “Implementations may use whatever actual algorithms they wish, but must produce the same results.” Source: §9 introduction.
- Structure: §9.1 Initial Setup, §9.2 Line Length Determination, §9.3 Main Size Determination, §9.4 Cross Size Determination, §9.5 Main-Axis Alignment, §9.6 Cross-Axis Alignment, §9.7 Resolving Flexible Lengths, §9.8 Definite and Indefinite Sizes, §9.9 Intrinsic Sizes (§9.9.1 container main sizes with §9.9.1.1 Ideal Algorithm, §9.9.1.2 Web-compatible Intrinsic Sizing Algorithm, §9.9.1.3 Multi-line Min-content Algorithm; §9.9.2 container cross sizes; §9.9.3 item contributions).
- Step 3 flex base size cases A-E (definite flex basis; aspect ratio with definite cross size; min/max-content constraint on container; infinite available main size with parallel inline axis; otherwise size into available space treating content as max-content); min/max are ignored while computing flex base size; hypothetical main size = flex base size clamped by used min/max. Source: §9.2, step 3.
- Automatic minimum size (
min-width: auto) for non-scroll-container items: larger of content size suggestion and transferred size suggestion, capped by the specified size suggestion (replaced elements: smaller of content and transferred); specified and transferred suggestions are “otherwise undefined” when their preconditions fail. Source: §4.5. - §9.7 step structure: (1) determine used flex factor (grow if sum of hypothetical outer main sizes is less than inner main size, else shrink), target main size initialised to flex base size; (2) freeze inflexible items (flex factor 0, or base size already beyond hypothetical in the flexing direction); (3) initial free space; (4) loop: (a) exit if all frozen; (b) remaining free space, scaled by the sum of unfrozen flex factors when that sum is below 1; (c) distribute proportionally (grow: by flex grow factor; shrink: by scaled flex shrink factor = shrink factor × inner flex base size); (d) clamp to min/max, recording min and max violations; (e) total violation decides which items to freeze (zero: all; positive: min violators; negative: max violators); (5) used main size = target main size. A note states that at least one item freezes per iteration, guaranteeing termination.
- §9.8: definite main size of container implies definite post-flexing item main sizes; a definite flex basis makes the item’s main size definite; a note says “definite” sizes in flex layout can require performing layout so that percentages inside items resolve.
- §9.9.1.1 note: the ideal algorithm “is not Web-compatible” and implementers and the working group “are investigating to what extent” engines can approach it. §9.9.1.2 contains only “Outline Web-compatible algorithm here, once we have one. [Issue #8884]”.
- §9.9.2 notes for multi-line column containers: min-content “effectively assumes a single flex line”; the max-content approach is “not a perfect fit in some cases” and a fully correct computation is described as prohibitively expensive.
- Changes since the 2018 CR include: identification of §9.9.1 as ideal and not web-compatible (#8884); fixed flexing rules in §9.9.1 to avoid division by zero (#7189); reformed cross-size intrinsic sizing for column-wrap containers (#6777); aspect-ratio interaction with the automatic minimum via the transferred size suggestion (#6069, #6794); main size definite whenever flex basis is definite (#4311). Source: “Changes” section.
CSS Grid Layout Module Level 2, W3C CRD 26 March 2025 (https://www.w3.org/TR/css-grid-2/, retrieved 2026-08-29):
- §12 outer steps: placement, container size per §5.2 (note: cyclic percentages in track sizes treated as
auto), grid sizing algorithm (percentages resolved against the resulting container size), item layout with definite grid areas. - §12.1 Grid Sizing Algorithm: (1) size columns; (2) size rows using column sizes; (3) if any item’s min-content contribution changed because of row sizes, re-run column sizing once; (4) likewise re-run row sizing once; (5) align tracks. A note lists the cases that trigger the re-run: column-wrap flex containers, orthogonal flows, multicol, aspect-ratio items.
- §5.2: max-content (min-content) size of a grid container is the sum of track sizes including gutters when sized under the corresponding constraint.
- §12.3 five phases: Initialize Track Sizes (§12.4), Resolve Intrinsic Track Sizes (§12.5), Maximize Tracks (§12.6), Expand Flexible Tracks (§12.7 with §12.7.1 Find the Size of an fr), Expand Stretched auto Tracks (§12.8).
- §12.5 order: baseline shims; span-1 items into intrinsic non-flexible tracks (minimums then maximums, with limited contributions capped by fixed max sizing functions); spanning items by increasing span, not crossing flexible tracks; items crossing flexible tracks all at once (flex-factor-proportional, with the sum-below-1 rule); infinite growth limits set to base size. §12.5.1 distributes extra space with per-track “planned increase” to avoid order dependence, freezes at limits, then distributes “beyond limits” into tracks with intrinsic maximums; “infinitely growable” tracks are those whose growth limit became finite in the intrinsic-maximums step.
- §12.5 closing note: “There is no single way to satisfy intrinsic sizing constraints” and the algorithm “may be updated in the future to take into account more advanced heuristics”.
- §12.7.1: hypothetical fr size = leftover space ÷ (sum of flex factors floored at 1); restart treating any track whose factor × fr size is below its base size as inflexible.
- §7.2.1: percentage track sizes are relative to the container’s inner size; if the container size depends on its tracks, the percentage “must be treated as auto” for intrinsic sizing and then resolved against the resulting size for layout.
CSS Box Sizing Module Level 3, W3C WD 17 December 2021 (https://www.w3.org/TR/css-sizing-3/, retrieved 2026-08-29):
- Definitions (§2.1): stretch-fit size (available space minus margins, border, padding, floored at zero; “Undefined if the available space is indefinite”); max-content size (size under infinite available space); min-content size (smallest size without avoidable overflow; formally the size under a min-content constraint); fit-content size =
clamp(min-content, stretch-fit, max-content)when available space is definite, min-content under a min-content constraint, otherwise max-content.fit-content(x)=min(max-content, max(min-content, x))(§3.2). - Intrinsic size contribution (§2.2): the outer size a box contributes, auto margins treated as zero.
- §5.2.1 cyclic percentages: non-replaced boxes treat cyclic percentages in
width,max-width,height,max-heightas the initial value for contributions; replaced boxes resolve them against zero for the min-content contribution; minimum sizes, margins, padding and gutters resolve against zero; the note states “These rules specify the previously-undefined behavior” of CSS 2. - Under-specification: “This specification does not define how to determine the sizes of floats” (§5.1); intrinsic sizes of some boxes are deferred to CSS 2 “and/or existing implementations” (§5.1, §5.2); the UA “may enforce a minimum” on form-control intrinsic sizes and “may additionally floor the min-content contribution” for UI reasons (§5.1, §5.2.1).
stretchandfit-contentkeywords are deferred to Level 4.
CSS Box Sizing Module Level 4, W3C WD 20 May 2021 (https://www.w3.org/TR/css-sizing-4/, retrieved 2026-08-29):
- §4.1
aspect-ratio: auto || <ratio>;autouses the natural ratio of replaced elements,<ratio>uses the box selected bybox-sizing,auto && <ratio>uses the content box; degenerate ratios behave asauto. - §4.2 ratio-dependent axis (preferred size depends on the ratio, definite only if inputs are definite) and ratio-determining axis; an inline issue block records that the sizing text may move.
- §4.3 automatic content-based minimum size in the ratio-dependent axis: min-content size capped by the maximum size, for non-replaced, non-scroll-container boxes.
- §4.4 min/max transfer: the definite minimum is transferred first and “capped by any definite preferred or maximum size in the destination axis”; the maximum is then transferred and floored by definite preferred or minimum sizes and by the transferred minimum; definite sizes are never affected.
- Status: exploratory Working Draft; the document instructs implementers to use Level 3 as reference; one section carries the marker “This section might not be written correctly” (issue 6071).
csswg-drafts issues (github.com/w3c/csswg-drafts, retrieved 2026-08-29):
- #8884 (opened 2023-05-30, davidsgrogan): the §9.9.1 intrinsic main size algorithm for single-line row flexboxes is not web-compatible; the spec yields 300 px where existing content requires 100 px; labelled Agenda+ and Needs Edits; open.
- #7189 (2022-03-31, tabatkins): the intrinsic main size algorithm floors flex factors below 1 per item, creating discontinuities; closed, accepted by CSSWG resolution.
- #1147 (2017-03-30, tabatkins): implementations do not match the intrinsic main size algorithm, producing 200 px where the equivalent grid yields 150 px; closed, rejected as invalid.
- #6794 (2021-11-04, davidsgrogan): a flex item with
aspect-ratiorenders at 50 px in Chrome (per spec) and 100 px in Firefox (matching block layout); proposal to use min-intrinsic instead of min-content for the content size suggestion; closed, accepted by CSSWG resolution. - #2303 (2018-02-12, mrego): Firefox runs the grid track sizing algorithm twice for min-content and max-content container sizes as specified; Blink, WebKit and Edge run it once with zero available space; closed.
- #1921 (2017-10-30, mrego): the specification changed percentage rows to resolve against the intrinsic container size, but all implementations had shipped the older “treated as auto” behaviour; closed, accepted by CSSWG resolution.
- #5566 (2020-10-01, mrego): proposal to resolve percentage row tracks as
autoand gutters as0under indefinite height; Chromium and WebKit follow the spec for both, Firefox follows the proposal for tracks; closed, rejected as wontfix by CSSWG resolution. - web-platform-tests/interop #139 (2022-09-21):
css/css-sizing/aspect-ratio/flex-aspect-ratio-004.html“fails in the same identical way across the 3 engines”; open test-change proposal.
Mechanism
Flex layout phases (Flexbox §9):
1 setup: generate flex items
2 available main/cross space
3 per item: flex base size (cases A-E) → hypothetical main size = clamp(base, min, max)
4 container main size
5 collect items into lines (single-line, or by outer hypothetical main size)
6 resolve flexible lengths (§9.7) → used main sizes
7 hypothetical cross size per item (layout with used main size)
8 line cross sizes (baseline groups, largest outer hypothetical cross size)
9 align-content: stretch lines
10 visibility: collapse handling (strut, re-run)
11 used cross size (stretch → line cross size clamped; else hypothetical)
12 main-axis: auto margins, justify-content
13 cross-axis auto margins
14 align-self
15 container cross size
16 align-content
Flexible length resolution (§9.7):
factor = (Σ outer hypothetical main < inner main) ? grow : shrink
target_i = flex_base_i ; frozen_i = (factor_i == 0) or already beyond hypothetical
free_0 = inner_main − Σ (frozen ? outer target : outer flex base)
loop:
if all frozen: break
free = inner_main − Σ ...; if Σ unfrozen factors < 1: free = min(|free|, |free_0 × Σfactors|)
grow: target_i = base_i + free × grow_i / Σ grow_unfrozen
shrink: scaled_i = shrink_i × inner_base_i ; target_i = base_i − |free| × scaled_i / Σ scaled_unfrozen
clamp targets to [min_i, max_i], content box ≥ 0 ; violation_i = clamped − unclamped
total = Σ violation_i ; freeze all if 0, min-violators if > 0, max-violators if < 0
used_main_i = target_i
Grid sizing (Grid §12):
size_columns(); size_rows(cols)
if any min-content contribution changed: size_columns() once more
if any min-content contribution changed: size_rows() once more
track_sizing(axis):
initialize (base size, growth limit)
resolve intrinsic (span 1 → increasing spans → flexible-crossing items; planned increases)
maximize (distribute free space to base sizes up to growth limits)
expand flexible (fr size = leftover / max(1, Σ flex); restart on under-sized tracks)
stretch auto tracks
Intrinsic sizes (Sizing 3 §2, §3.2; Sizing 4 §4):
fit-content = max(min-content, min(max-content, stretch-fit)) (definite available space)
fit-content(x) = min(max-content, max(min-content, x))
stretch-fit = available − margins − border − padding, floored at 0 (undefined if indefinite)
aspect-ratio transfer: min first (capped by dest preferred/max), then max (floored by dest preferred/min and transferred min)
NUIF relevance
- Borrow: The phase structure of Flexbox §9 and Grid §12 gives NUIF a stable vocabulary for diagnostics from a CSS-family evaluator (flex base size, hypothetical main size, line cross size, base size, growth limit, fr size); NUIF resolved-layout diagnostics can name the phase in which a value was fixed.
- Borrow: Box Sizing 3’s definitions of min-content, max-content, fit-content and stretch-fit are the definitions NUIF’s shared sizing primitives should reference normatively, including the cyclic-percentage rules.
- Adapt: The re-run-once rule of Grid §12.1 and the “performing layout to make sizes definite” note of Flexbox §9.8 mean that a NUIF evaluator contract must allow multi-pass layout; the resolved snapshot should record whether a second pass occurred so differential tests can attribute divergences.
- Adapt: NUIF’s
flexfamily must state which intrinsic main size algorithm it uses formin-content/max-contentcontainers, since the specification currently has none that is web-compatible (§9.9.1.2, #8884); the practical choice is to follow the reference browser’s behaviour and label itrepresentablerather thanlosslessrelative to the specification. - Adapt: Percentage tracks under indefinite size (#1921, #5566) and one-pass versus two-pass intrinsic grid sizing (#2303) are documented engine divergences; NUIF’s differential experiment should include fixtures for each and classify results as “target semantic difference”, not evaluator bugs.
- Reject: Treating the CRD text as a complete executable specification; the explicit “implementations may use whatever actual algorithms they wish” clause plus the open placeholder sections mean conformance can only be defined against test fixtures plus a named reference implementation.
- Reject: Adopting Box Sizing Level 4
aspect-ratiotext as normative for NUIF’s aspect-ratio primitive while the draft carries “might not be written correctly” markers; NUIF should specify its own transfer semantics and cite Level 4 as the intended alignment target.
Open questions
- Which web-compatible intrinsic main size algorithm will replace §9.9.1.2, and does Taffy’s current behaviour (Chrome-derived) match the eventual text?
- How many of Taffy’s 17 excluded fixtures and 26 grids without track-list assertions correspond to the issues listed here versus Taffy-specific limits?
- Does any engine implement the Grid §12.1 double re-run exactly, or do all engines approximate it (as #2303 suggests for intrinsic sizes)?
- Should NUIF expose
stretchandfit-contentas keywords now (Sizing 4) even though Level 3 defers them?
CSS formatting tree, Flexbox and Grid algorithms
Document status:
reviewed. Canonical source.
Summary
CSS separates the source element tree from an intermediary formatting box tree and then applies family-specific layout algorithms. CSS Display defines box-tree generation; Flexbox and Grid define normative algorithms whose results implementations must reproduce even if their internal algorithms differ.
NUIF relevance
This strongly supports separating semantic containment from formatting/layout structure. NUIF can borrow exact CSS-compatible semantics where selected, but should not claim arbitrary HTML/CSS equivalence when anonymous boxes, generated content, cascade, writing modes or other web-specific fixups are absent.
Delta debugging and test-case reduction (ddmin, HDD, C-Reduce, Hypothesis choice-sequence shrinking, proptest shrinking)
Document status:
reviewed. Canonical source.
Summary
Delta debugging (Zeller and Hildebrandt 2002) defines 1-minimality and the ddmin algorithm, which reduces a failing input by testing subsets and complements at increasing granularity; the worst case is |c|² + 3|c| tests, the best case 2·log₂|c|. Hierarchical delta debugging (Misherghi and Su 2006) applies ddmin level by level over a parse tree so that every candidate is syntactically valid and needs orders of magnitude fewer tests. C-Reduce (Regehr et al. 2012) generalises to a fixpoint over many pluggable transformation passes with an external interestingness test and validity checking. Hypothesis (MacIver and Donaldson 2020) reduces the sequence of random choices consumed by the generator instead of the generated value, so every reduced candidate is generatable by construction; shortlex order and adaptive passes drive it. proptest shrinks through ValueTree::simplify/complicate binary search, with collections deleting elements before shrinking them; proptest-state-machine deletes transitions while re-checking preconditions.
For NUIF the failing artefact is a seed plus an operation sequence over a tree document. The algorithm below combines ddmin over the operation list, HDD over the base document, and choice-sequence reduction so that the minimised fixture remains a valid document and a valid patch.
Evidence
- A test case c is 1-minimal if removing any single change makes the failure disappear; determining a local minimum requires 2^|c| tests in general. Zeller and Hildebrandt, IEEE TSE 28(2), 2002, DOI 10.1109/32.988498, §III-A Definitions 8–10 (PDF https://www.st.cs.uni-saarland.de/papers/tse2002/tse2002.pdf, retrieved 2026-08-29).
- ddmin(c) = ddmin2(c, 2) with three rules: reduce to subset (continue with n = 2), reduce to complement (continue with max(n − 1, 2)), increase granularity (n = min(|c|, 2n)); Proposition 11: the result is 1-minimal; Proposition 12: worst case |c|² + 3|c| tests; Proposition 13: best case 2·log₂|c|. Same paper, §III-B Fig. 5 and Propositions 11–13.
- GCC case: 755 characters reduced to 77 after 731 further tests in 34 seconds; the failure-inducing option −ffast-math was isolated among 31 options in 7 tests. Same paper, §IV-A.
- Mozilla case: 95 user actions reduced to 3 after 82 runs; 896 lines of HTML reduced by a hierarchical approach (lines, then characters) to
<SELECT>after 57 line-level runs. Same paper, Abstract and §IV-B. - The isolating variant dd works on the pair (passing, failing) and needs only log₂|c| tests without unresolved outcomes; isolating the GCC difference took 59 tests where minimising took 731. Same paper, §V and §VI.
- HDD applies ddmin to each level of the tree from coarsest to finest, prunes irrelevant nodes, and “All the generated input configurations are syntactically valid”; it may not produce 1-minimal results, HDD* iterates to a 1-tree-minimal fixpoint in O(n³) worst case; finding a global minimum is NP-complete. Misherghi and Su, ICSE 2006, DOI 10.1145/1134285.1134307, Abstract, §3.2 Algorithm 1, §3.4 (PDF https://web.cs.ucdavis.edu/~su/publications/icse06-hdd.pdf, retrieved 2026-08-29).
- HDD numbers: bug.c ddmin 680 tests/53 tokens vs HDD 86/51; boom7.c 3727/102 vs 144/57; XSL case ddmin-line 1092 tests to 92 lines vs HDD 124 tests to 8 lines. Same paper, §4.1 Table 1, §4.2 Table 2.
- C-Reduce: a generic fixpoint over modular transformations, each an iterator with new/transform/advance, parameterised by a test that decides whether a variant is successful; outputs average more than 25 times smaller than Berkeley delta; validity is checked with Frama-C or KCC because unguarded reduction introduces undefined behaviour. Regehr, Chen, Cuoq, Eide, Ellison, Yang, PLDI 2012, DOI 10.1145/2254064.2254104, Abstract, §5, §6.3 Listing 2, §7 Table 1 (preprint https://users.cs.utah.edu/~regehr/papers/pldi12-preprint.pdf, retrieved 2026-08-29).
- Regehr’s guidance: an interesting variant seeds further reduction, an uninteresting one is a dead end; order interestingness checks fastest first and run biggest-win passes first; “If these criteria contain any kind of loophole, C-Reduce is likely to find it.” https://blog.regehr.org/archives/1678, retrieved 2026-08-29.
- Hypothesis reduces “the sequence of random choices made during generation”, “ensuring that any reduced test case is one that could in principle have been generated”; generators are viewed as parsers of choice sequences; a too-short sequence is a parse error; order is shortlex. MacIver and Donaldson, ECOOP 2020, DOI 10.4230/LIPIcs.ECOOP.2020.13, Abstract, §2.1–2.2, §3.2 (PDF https://drops.dagstuhl.de/opus/volltexte/2020/13170/pdf/LIPIcs-ECOOP-2020-13.pdf, retrieved 2026-08-29).
- Hypothesis 5.15.1 used 15 passes: six deleting contiguous regions, region-to-subregion, region-to-zeroed sequence, four lexicographic, three combined, one float-specific; the list generator emits a “more” bit before each element so element deletion is a contiguous deletion. Same paper, §3.1, §3.3 Fig. 7.
- Evaluation: on Csmith programs Hypothesis reduced to 812 bytes (floor 410) versus C-Reduce 120 and Picire 345, using 762 SUT invocations versus 3968 (C-Reduce) and 3139 (Picire). Same paper, §4.1.2 Fig. 8 and Fig. 12.
- Hypothesis source (
hypothesis/src/hypothesis/internal/conjecture/shrinker.py, master at 49a797b, retrieved 2026-08-29):sort_keyimplements shortlex over choice indices (lines 73–91); theShrinkerdocstring requires that progress be deterministic, that passes not iterate to a fixed point internally, and recommends adaptive passes that turn O(m) successful calls into O(log m); the pass list includesnode_program("X"*k)deletions for k = 5..1,reorder_spans,minimize_duplicated_choices,minimize_individual_choices,redistribute_numeric_pairs,lower_integers_together(lines 343–359);fixate_shrink_passesloops until no pass improves, with 20 consecutive failures per pass and length-reducing passes sorted first (lines 865–958). - Haskell QuickCheck 2.18:
shrink :: a -> [a]lists immediate shrinks; candidates are tried in list order, so aggressive steps go first;genericShrinktries subterms then recursive shrinks;shrinkListshrinks lists given an element shrinker. https://hackage-content.haskell.org/package/QuickCheck-2.18.0.0/docs/Test-QuickCheck.html, retrieved 2026-08-29. - proptest 1.11.0:
ValueTree::simplifymoves current to a halfway point between low and high,complicatepartially undoes the last simplification;prop_mapshrinks in terms of the source value;prop_flat_mapshrinks both the input and the derived values;prop_filtercan largely prevent shrinking;VecValueTreeusesShrink::DeleteElementthenShrink::ShrinkElement(“delete elements from the list until we can do so no further, then to shrink each remaining element”). https://docs.rs/proptest/latest/proptest/strategy/trait.ValueTree.html, trait.Strategy.html, and https://docs.rs/proptest/latest/src/proptest/collection.rs.html, retrieved 2026-08-29. - Perses guarantees that each reduction step considers only smaller, syntactically valid variants by reducing over a grammar in a normal form with quantifiers; results are 2% and 45% of the size of DD and HDD outputs on 20 C programs. Sun, Li, Zhang, Zhang, Su, ICSE 2018, DOI 10.1145/3180155.3180236, Abstract, §3, §4.3 (PDF https://web.cs.ucdavis.edu/~su/publications/perses.pdf, retrieved 2026-08-29).
Mechanism
Definitions: an interestingness test interesting(x) ∈ {FAIL, PASS, UNRESOLVED}; a validity oracle valid(x) run before the system under test; a result is 1-minimal when no single element can be removed while preserving FAIL.
test(x) = memo( valid(x) ? interesting(x) : UNRESOLVED ) # C-Reduce §5, §6.3
ddmin(c, n = 2): # TSE 2002 Fig. 5
chunks = split(c, n)
if ∃i: test(chunks[i]) == FAIL: return ddmin(chunks[i], 2)
if ∃i: test(c − chunks[i]) == FAIL: return ddmin(c − chunks[i], max(n − 1, 2))
if n < |c|: return ddmin(c, min(|c|, 2n))
return c # 1-minimal
hdd(tree): # HDD Alg. 1
level = 0
while nodes(tree, level) ≠ ∅:
keep = ddmin(nodes(tree, level)) # test(prune(tree, level, keep))
tree = prune(tree, level, keep) # manipulator keeps required children
level += 1
return tree
repeat hdd until no node removed # HDD*
reduce_choices(seq): # ECOOP 2020, shrinker.py
order = shortlex(seq) # length first, then choice indices
passes = [zero_spans, delete_k_consecutive(5..1), subregion, reorder_spans,
minimize_duplicates, minimize_individual (binary search), redistribute]
until no pass improves:
for p in passes: run p until 20 consecutive non-improvements
sort passes: length-reducing first
return generator.parse(seq) # valid by construction
NUIF reducer over (seed, base document, operation sequence), synthesis with attribution:
- Level A, operations: ddmin over the transaction list with
test= replay on the base document followed by the failing comparison; validity = every operation’s preconditions hold on the intermediate document (TSE 2002; eqc_statem precondition rule via nuif:research:property-based-testing-state-machines). - Level B, document: HDD over the base document tree by depth, pruning subtrees not referenced by remaining operations; the tree manipulator must keep component definitions referenced by instances, token definitions referenced by bindings, and parents of moved entities (HDD §3.2; Perses grammar validity).
- Level C, values: choice-sequence reduction of the generator input so that names, sizes, extension payloads and viewport contexts shrink toward the simplest generatable values (ECOOP 2020; Hypothesis passes); for hand-written fixtures, per-value
ValueTreebinary search instead (proptest). - Isolation: when a passing base revision exists, run dd on the pair (passing, failing) to isolate the failure-inducing operation subset in log₂|c| tests before minimising (TSE 2002 §V).
- Output: a fixture consisting of the reduced canonical document, the reduced patch, the seed, and the interestingness predicate identifier, as required by item 9 of
apps/editor/QA.md.
Invariants: every candidate passed to the system under test is a valid document and a valid patch; the reducer never edits the document directly when a generator exists; progress is deterministic for a fixed seed; memoised results keep the test count within the ddmin bounds.
NUIF relevance
Borrow
- ddmin as the operation-list reducer and dd as the isolator; both are small, proven 1-minimal and have known bounds (TSE 2002 Propositions 11–13, 16–18).
- Hypothesis’s generator-as-parser principle: reducing the seed-derived choice sequence guarantees that minimised documents are generatable and valid, which is the property the task requires (ECOOP 2020 §2.2, §3.2).
- C-Reduce’s separation of a domain-independent fixpoint driver from pluggable passes, and its rule that validity is checked before the system under test (PLDI 2012 §5–6).
- proptest-state-machine’s delete-then-shrink order for transition lists and its precondition re-check on every deletion (proptest
VecValueTree; proptest-state-machineShrink).
Adapt
- HDD’s tree levels map to NUIF entity depth, but pruning must respect graph references (instances to components, token bindings, relations) that a parse tree does not have; the tree manipulator becomes a document-aware pruner that also removes dangling references.
- The “more” bit trick from the Hypothesis list generator should be used in the NUIF operation generator so that deleting an operation is a contiguous choice-sequence deletion.
- Interestingness for tolerant oracles (image or box differences) must be stable under reduction; the predicate should compare against the same declared tolerance and record the maximum delta.
Reject
- Character- or line-based reduction of canonical text (Berkeley delta style); it produces invalid documents and is orders of magnitude slower than structural reduction (PLDI 2012 §3.2; HDD Table 2).
- Reduction without a validity oracle; Csmith and C-Reduce show that unguarded reducers converge to invalid inputs (PLDI 2011 §3.7; Regehr blog).
Implemented decision
NUIF stores the reduced canonical document and semantic operations as the
durable regression fixture, together with the seed, predicate identifier,
content hashes and every accepted transformation. The choice sequence remains
reproduction evidence for generator/fuzzer failures but is not the only durable
form because generators evolve. The document reducer iterates subtree deletion
at progressively finer granularity and then runs explicit graph-collection,
extension and known-scalar passes; full structural validation precedes every
interestingness call. Unknown-kind opaque payload bytes are held fixed because
only their owner can define meaningful byte-level simplification, though a
whole irrelevant unknown entity or namespace can be removed. The bounded NUIF
profile does not need general cubic HDD*: the subtree pass returns only after no
remaining individual subtree can be removed, and all later passes are strictly
simplifying and content-hash memoized. cargo xtask reduction-profile records
the resulting three-entity ancestor path and emitted fixture in CI.
Direct dependency and implementation-subsystem alternatives audit
Document status:
verified. Canonical source.
Summary
Cargo metadata reports 35 distinct direct external crates across the workspace. Each is now registered with a role, a current decision, at least one considered alternative and repository evidence. The executable audit fails when a direct crate is added without ownership or when a stale registration remains. Cargo Deny separately gates advisories, duplicate-version bans, licences and sources; the two checks answer different questions.
The critical result is not a wholesale dependency replacement. NUIF’s unusual
requirements—canonical bytes, exact source correspondence, independent layout
and raster oracles, bounded hostile inputs and a semantically queryable native
editor—make several generally faster or broader libraries worse fits at the
actual boundary. Four version lines warranted immediate compatibility trials:
json5 0.4 to 1.3, sha2 0.10 to 0.11, font-test-data 0.7 to 0.9 and
Tree-sitter 0.26.10 to 0.26.13. The complete tests accepted the JSON5, font-data
and Tree-sitter updates without changing canonical fixtures, hostile-input
classification, pinned font hashes or adapter source spans. SHA-2 0.11 removed
the digest output’s hexadecimal formatting implementation; NUIF retains 0.10
because 0.11 provides no required fix or measured benefit that justifies
duplicating a hex adapter across the report-producing crates.
Evidence
cargo metadata --locked --format-version 1identifies workspace members, their direct dependency declarations and the exact resolved graph. The Cargo reference defines the JSON output as stable when consumers ignore unknown fields. Locator: Cargocargo metadatadocumentation, retrieved 2026-08-30: https://doc.rust-lang.org/cargo/commands/cargo-metadata.html.cargo searchon 2026-08-30 reports current stable lines for the registered crates.io dependencies. NUIF is already on the current stable line for Ciborium, Criterion, Harfrust, PNG, RFD, roxmltree, serde, serde-bytes, serde-json, Skrifa, stats_alloc, Taffy, thiserror, tracing, the HTML and CSS grammars, and Zeno. The version-trial candidates are listed in the summary. The three Masonry packages are full-SHA Git dependencies and are therefore evaluated as one forked toolkit boundary rather than by crates.io maximum version.cargo deny checkpasses after the reviewed Xilem and UI Events fork pins and the retirement of directttf-parserfor RUSTSEC-2026-0192.ttf-parser,rustybuzz, metal-rs and rust-block are absent from the active graph. The complete check is a CI and release gate with no advisory exception.- The version trial ran all workspace unit and documentation tests, the release hostile-input allocation profile, text and render goldens, all eight executable adapter profiles, and workspace Clippy with warnings denied on rustc 1.98.0. The three accepted updates passed; SHA-2 0.11 failed at compile time before runtime evidence and was reverted (2026-08-30).
Mechanism
The register maps each direct crate to the subsystem boundary it serves. The comparison is made at that boundary, so an alternative is accepted only when it preserves the same observable contract and improves a measured workload.
Subsystem comparisons
Serialization and hashing
- Serde explicitly separates data structures from format implementations through its 29-type data model. This is useful here because the model derives typed traversal while NUIF retains control of canonical JSON5 and CBOR output. Serde is not a parser and therefore does not weaken the format-specific resource checks. Locators: https://serde.rs/data-model.html and https://serde.rs/data-format.html, retrieved 2026-08-30.
- Ciborium remains preferable to Minicbor for profile zero because the current bounded decoder needs serde’s generic logical-value distinctions before its own deterministic-order validation. A schema-specific Minicbor profile may be worthwhile only after a benchmark demonstrates an end-to-end gain without changing unknown-value preservation.
json51.0 replaced its Pest grammar with a handwritten parser and 1.1–1.3 added wide integer and UTF-16 surrogate-pair support. This is a semantic parser change, so a major-version update is acceptable only with the complete codec and hostile-input suites. Locators: release notes and comparison, retrieved 2026-08-30: https://github.com/callum-oakley/json5-rs/releases and https://github.com/callum-oakley/json5-rs/compare/0.4.1…1.3.1.- SHA-256 is retained over BLAKE3 because NUIF hashes are exchanged with browser, release and independent Python tooling, not used as a high-throughput internal hash table. RustCrypto 0.11 changes to Digest 0.11, edition 2024 and newtype hash implementations; its MSRV 1.85 is below NUIF’s. Locator: https://github.com/RustCrypto/hashes/blob/master/sha2/CHANGELOG.md, retrieved 2026-08-30.
Source adapters
- Tree-sitter supplies concrete nodes, byte offsets, edit descriptions and
incremental reparsing.
html5everis the right browser-grade oracle for full WHATWG error correction, but it mutates a caller-supplied tree through callbacks and does not provide a concrete source tree. Normalizing and reserializing a DOM is incompatible with the byte-complement preservation postcondition of the declared adapter. Locators: https://tree-sitter.github.io/tree-sitter/using-parsers/ and https://github.com/servo/html5ever, retrieved 2026-08-30. - The official Tree-sitter JavaScript grammar includes JSX in the same concrete syntax tree and therefore extends the existing retained-byte contract to the static React profile. SWC and Oxc are stronger choices for semantic JavaScript transforms, but their normalized AST boundary does not improve a profile that explicitly refuses evaluation and patches exact source ranges. Locator: https://github.com/tree-sitter/tree-sitter-javascript, retrieved 2026-08-30.
- Svelte uses the same split boundary:
tree-sitter-svelte-next0.1.1 supplies concrete byte spans for the bounded static adapter, while exact officialsvelte/compileris a test-only foreign parser/compiler oracle. The unofficial Rustsvelte-compileris rejected because its broader compiler graph and documented manual recovery debt do not improve retentive scalar patching. Locator:nuif:research:svelte-source-adapter-surface, retrieved 2026-08-30. - wasm-bindgen 0.2.127 is the current browser/Node ABI generator and already
resolves transitively in the native editor graph. Making it direct only in
nuif-wasmadds no second resolved version. WIT is the stronger long-term language-neutral component interface, but browsers and Figma-style iframe hosts consume JavaScript modules today. The binding therefore passes only canonical documents, diagnostics and patches as bytes instead of generating a parallel JavaScript model. Locators: https://wasm-bindgen.github.io/wasm-bindgen/reference/deployment.html and https://component-model.bytecodealliance.org/, retrieved 2026-08-30.
Process and agent adapters
rmcp3.1.4 is the official Rust SDK for the breaking MCP 2026-07-28 stateless lifecycle. It is preferable to a handwritten JSON-RPC loop because request metadata, discovery, result types and schemas changed together; it is preferable to a TypeScript or Python sidecar because NUIF can call the same Rust core without a second private RPC. NUIF enables only the server, macro and stdio features, pins the exact release, and covers the wire with an independent subprocess harness. Locator: official SDK README and roadmap, retrieved 2026-08-30: https://github.com/modelcontextprotocol/rust-sdk and https://github.com/modelcontextprotocol/rust-sdk/blob/main/ROADMAP.md.- Tokio 1.53.1 is already the official SDK’s executor. NUIF makes it direct
only in
nuif-mcp, enables a current-thread runtime and standard I/O, and keeps every async concern outside the deterministic core. Async-std or Smol would add runtime interoperation; a blocking loop would reproduce official lifecycle behavior without a semantic benefit. - The live Chromium capture port uses the repository’s existing exact Chrome
for Testing lock through raw CDP rather than adding Playwright’s separate
browser-release/download lifecycle. A tiny synchronous Tungstenite 0.29.0 client is
limited to the browser’s loopback
ws://endpoint and caps messages/frames at 32 MiB. The current 0.30.0 server-side validation change does not apply to this client-only boundary, while its dependency refresh introduced four parallel Digest-family version lines, so the smaller prior line is pinned and watched. Base64 decoding occurs only after CDP and response ceilings; Tempfile supplies one automatically cleaned, credential-empty browser profile per run. Playwright remains the stronger future cross-engine runner, and the W3C WebDriver BiDi Working Draft remains the portability watch path, but neither currently replaces the Chromium-only DOMSnapshot and platform-font evidence used by this segment. Locators: https://playwright.dev/docs/browsers, https://www.w3.org/TR/webdriver-bidi/ and https://github.com/snapview/tungstenite-rs, retrieved 2026-08-31. - roxmltree represents XML as a read-only tree and exposes original byte positions. Quick XML is an almost-zero-copy pull parser and a better candidate for very large streaming documents, while usvg is a better renderer-facing normalized SVG model. The current bounded SVG profile needs a small complete tree plus exact attribute/text ranges; neither alternative improves that contract. Locators: https://docs.rs/roxmltree/0.21.1/roxmltree/ and https://github.com/tafia/quick-xml, retrieved 2026-08-30.
Layout, text and rendering
- The NUIF layout kernel remains implementation-owned, with Taffy and browser engines as independent differential oracles. Substituting Taffy into the reference would erase one of the two implementations being compared; Yoga has the same self-oracle problem and adds a foreign-function boundary.
- Harfrust and Skrifa remain the shaping/outline stack, and the separate narrow
package-font profile now uses Skrifa behind NUIF-owned sfnt/checksum/OS/2
checks. A committed HarfBuzz 14.4.0 metadata capture replaces the former
in-process Skrifa oracle. Fontations describes
read-fontsas a no-allocation, no-copy parser suitable for shaping, forbids unsafe code in Skrifa and subjects the stack to OSS-Fuzz. The directttf-parserdependency was removed after RUSTSEC-2026-0192 reported no patched version. Locators: https://github.com/googlefonts/fontations and https://rustsec.org/advisories/RUSTSEC-2026-0192.html, retrieved 2026-08-30. - The
pngcrate owns the production RGBA8 decoding path; test-onlyzune-pngsupplies independent exact-pixel evidence with unsafe paths disabled and integrity checks enabled. Replacing production decoding with the oracle would erase that differential boundary. - Zeno stays the small deterministic glyph-mask rasterizer. Tiny-skia or resvg would add broader path and SVG behavior that is outside the current reference commands, while Vello remains the interactive renderer and its CPU path is a non-normative visual-harness oracle.
Editor and verification
- Masonry remains the only compared editor toolkit that combines a retained
widget tree, AccessKit semantics and a same-tree CPU visual harness. Its
alpha churn is contained behind full-SHA
refpath/xilemandrefpath/ui-eventspins. The fork changes dependencies and safe API call sites; it contains no new foreign-interface implementation. - RFD 0.17.2 offers synchronous and asynchronous native dialogs on Windows,
macOS, Linux/BSD and asynchronous web dialogs. NUIF uses only the synchronous
desktop path and keeps parsing, fidelity reporting and filesystem writes in
editor-owned code. Per-platform APIs or Linux
ashpdwould increase platform code without changing the user-visible contract. Locator: https://docs.rs/rfd/0.17.2/rfd/, retrieved 2026-08-30. - Criterion is retained for controlled same-machine statistical comparisons; the release smoke profile records portable latency and allocation ceilings. Divan is a simpler runner and iai-callgrind gives stable Linux instruction counts, but neither replaces both current roles.
- thiserror produces standard typed errors, while
anyhowwould erase the error categories asserted by conformance tests. Tracing provides structured events and spans and is already the toolkit’s diagnostics vocabulary. Locators: https://docs.rs/thiserror/2/thiserror/ and https://docs.rs/tracing/0.1.44/tracing/, retrieved 2026-08-30.
NUIF relevance
This audit turns dependency choice into checked repository state. It also keeps implementation libraries separate from the independent engines and formats used as conformance oracles.
Decision boundary
Retain focused libraries whose data model directly matches a tested NUIF boundary. Performance claims require a profile workload, not a microbenchmark from another project.
Fork only the native toolkit chain, at full commits, while the reviewed wgpu and dependency-feature fixes are absent from the selected upstream revision.
Watch RFD because native-dialog behavior is platform-owned and must be covered by release-platform smoke tests. It is not allowed to own document I/O.
Reject substituting an oracle into the implementation it verifies, whole source regeneration in a retentive adapter, or a broad framework solely because it advertises more format coverage.
Open questions
- Whether future JSON5 grammar expansion needs an explicit accepted-source corpus in addition to the existing canonical, malformed, non-finite, byte and depth cases.
- Whether collaboration convergence should gain a portable allocation ceiling; adapter import/export/synchronization now runs inside the allocation-aware smoke profile after its Criterion fixtures were calibrated.
- Whether a future Masonry release incorporates the refpath wgpu, resvg/usvg and UI Events feature corrections, allowing both fork pins to be removed.
- Whether a future WebAssembly Component Model browser path is mature enough to
replace JavaScript glue. The current 0.2.127 CLI build also reports
future-incompatible
buf_reduxandmultipartin its optional packaging/test tool graph; neither crate is linked into NUIF or the emitted module. The pinned compiler remains an isolated build tool until upstream removes them or verified prebuilt-tool acquisition is adopted.
Shared and vendor-specific layout conventions across design editors
Document status:
reviewed. Canonical source.
Summary
Six editors were compared from their vendor documentation: Figma UI3, Penpot, Sketch (Mac), Adobe XD (maintenance mode since 2023), Framer and Canva. All six place structural navigation (pages, layers, assets) in a left panel, the canvas in the centre and selection-dependent properties in a right panel or inspector; all six expose creation tools in a toolbar. The toolbar position is the main variable: Sketch, Penpot and Canva use the top; Adobe XD uses a left vertical strip; Framer and Figma UI3 float it at the bottom. Property panels are sectioned by concern (position/size, layout, appearance, fill, stroke, effects, export) in every editor that documents sections. The shared composition is therefore a genre convention, and the bottom floating toolbar is a recent Figma choice (shared with Framer) rather than a universal one.
Legal framing, recorded briefly and not as advice: the US Copyright Office states that ideas, methods, systems and “format” or “layout” are not copyrightable, while names and logos may be protected by trademark; the First Circuit held a menu command hierarchy to be an uncopyrightable “method of operation” (Lotus v. Borland), and the Ninth Circuit held that Apple “cannot get patent-like protection for the idea of a graphical user interface” (Apple v. Microsoft). A test editor that reproduces spatial arrangement, section taxonomy and shortcuts, but no icons, logos, names or copied artwork, stays within that documented boundary.
Evidence
Retrieval date for all locators: 2026-08-29. Adobe and Canva pages could not be retrieved in full (timeouts / bot protection); those rows rely on search excerpts of the named vendor pages and are marked accordingly.
- Figma UI3: five regions (navigation bar, left sidebar, canvas, right sidebar, toolbar); toolbar at the bottom; Design/Prototype tabs on the right; sections Position, Auto layout, Layout, Appearance, Fill, Stroke, Effects, Export. https://help.figma.com/hc/en-us/articles/15297425105303-Explore-design-files; https://www.figma.com/blog/behind-our-redesign-ui3/ (“a slim new toolbar at the bottom of the canvas”); https://www.figma.com/blog/our-approach-to-designing-ui3/ (“Toolbars will float at the bottom of all Figma products”).
- Penpot: toolbar at the top; Pages and Layers left; Design/Prototype/Inspect right; design groups size and position, layout/constraints, opacity and blend, fill, stroke, shadow, blur, text, export, interactions. https://help.penpot.app/user-guide/first-steps/the-interface/; https://help.penpot.app/user-guide/designing/layers/.
- Sketch: Toolbar top; Layer List left; Canvas centre; Inspector right (“design properties for any layers you’ve selected”); Minimap bottom right; Cmd . toggles the interface. https://www.sketch.com/docs/designing/the-interface/; https://www.sketch.com/docs/designing/the-interface/the-toolbar/ (toolbar “at the top of the Mac app window”, contextual).
- Adobe XD (search excerpts of vendor pages; full page not retrieved): workspace elements include Design/Prototype/Share modes, Property Inspector, Pasteboard, Artboard, Plugins, Layers, Libraries, Toolbar; Layers panel via Cmd Y / Ctrl Y or a toolbar icon; the toolbar is described as a vertical strip on the left in vendor material (unverified in retrieved text). https://helpx.adobe.com/xd/help/workspace-basics.html; https://helpx.adobe.com/xd/help/layers.html.
- Adobe XD status: “Adobe XD continues to be in maintenance mode” and Adobe is “not investing in ongoing development or shipping new features” (search excerpt of the vendor support page; page not retrieved). https://helpx.adobe.com/support/xd.html.
- Framer: “The canvas controls are located in the bottom toolbar” with selection, pan, comment tools and a zoom menu “on the right side”; Space + drag pans; Cmd/Ctrl + and − zoom. https://www.framer.com/help/articles/how-to-use-the-canvas/. The Actions menu was “placed it in the Toolbar” in May 2023. https://www.framer.com/updates/may-update-2023. Left layers panel and right properties panel are stated in Framer Academy excerpts (lesson bodies not retrieved). https://www.framer.com/academy/lessons/framer-fundamentals-framer-interface.
- Canva (search excerpts; pages blocked by bot protection): a left side panel for elements, text, uploads and apps; a contextual toolbar above the design; layers reached via Position on the toolbar or a Layers tab in the side panel. https://www.canva.com/help/finding-and-arranging-layers/; https://www.canva.com/help/glow-up-variantb/.
- Copyright Office Circular 33, “Ideas, Methods, and Systems”: copyright excludes “any idea, procedure, process, system, method of operation, concept, principle, or discovery”; “Layout and Design”: the Office “will not accept a claim to copyright in ‘format’ or ‘layout’”; “Names, Titles, Short Phrases”: names and slogans are uncopyrightable but “may be protectable under federal or state trademark laws”. https://www.copyright.gov/circs/circ33.pdf — pages 1–3.
- Lotus Development Corp. v. Borland International, 49 F.3d 807 (1st Cir. 1995): the menu command hierarchy is “an uncopyrightable ‘method of operation’” under 17 U.S.C. § 102(b); “methods of operation” are “the means by which a user operates something”. Full text mirrored at https://www.bitlaw.com/source/cases/copyright/Lotus.html — Part II.D.
- Apple Computer, Inc. v. Microsoft Corp., 35 F.3d 1435 (9th Cir. 1994): “Apple cannot get patent-like protection for the idea of a graphical user interface, or the idea of a desktop metaphor”; GUIs are dissected because “copyright protection extends only to protectable elements of expression”. Full text at https://law.resource.org/pub/us/case/reporter/F3/035/35.F3d.1435.93-16883.93-16869.93-16867.html.
- Unverified: Adobe XD toolbar orientation and exact Property Inspector sectioning; Canva panel names beyond the excerpts; Framer properties-panel section names; whether Sketch’s toolbar can be hidden independently of the whole interface.
Mechanism
Comparison table (positions as documented; “excerpt” marks rows built from vendor search excerpts only):
| Editor | Left panel | Canvas | Right panel | Toolbar position | Property sectioning | Status |
|---|---|---|---|---|---|---|
| Figma UI3 | File tab: pages + layers; Assets; Tools | infinite, rulers, zoom menu top-right | Design / Prototype tabs | bottom, floating, collapsible | Position, Auto layout, Layout, Appearance, Fill, Stroke, Effects, Selection colors, Export | verified |
| Penpot | Pages, Layers (Alt L); Assets (Alt I) | infinite viewport, rulers | Design / Prototype / Inspect; palettes | top, horizontal | size/position, layout, constraints, layer, fill, stroke, shadow, blur, text, export, interactions | verified |
| Sketch (Mac) | Layer List (pages, frames, layers) | infinite canvas, minimap | Inspector | top (macOS toolbar), contextual | selection-dependent inspector | verified |
| Adobe XD | Layers (Cmd Y), Libraries, Plugins | pasteboard with artboards | Property Inspector | left, vertical (excerpt) | dimensions, appearance, design specs tab | excerpt; maintenance mode |
| Framer | Layers (sections, frames) | infinite canvas | Properties | bottom toolbar with zoom menu | size, content, design (excerpt) | partially verified |
| Canva | side panel (elements, text, uploads, apps); Layers tab | page-based canvas | contextual toolbar above design | top (contextual) | element-type dependent | excerpt |
Shared conventions: left panel holds document structure; canvas occupies the centre; right panel holds selection-dependent properties, sectioned by geometry, layout, appearance, paint, effects and export; creation tools live in a toolbar; hierarchical layer trees with visibility and lock; zoom control near the canvas edge.
Figma-specific or minority conventions: a floating, collapsible toolbar at the bottom of the canvas (shared only with Framer among the six); a mode switch (Design / Dev Mode) inside that toolbar; an actions/command palette in the toolbar (shared with Framer); optional property labels; minimize-UI behaviour that re-expands the right panel on selection.
Legal boundary as documented (three sentences): layouts, methods of operation and menu/command structures are outside copyright per Circular 33, Lotus and Apple; names, logos, icons and other original artwork remain protectable by trademark or copyright; the test editor therefore reproduces arrangement, taxonomy and bindings while using its own icon set, names and visual assets. This is a summary of public sources, not legal advice.
NUIF relevance
Borrow
- The shared left-structure / centre-canvas / right-properties composition, because every compared editor uses it and it is documented as a genre convention.
- Concern-based property sectioning (geometry, layout, appearance, fill, stroke, effects, export) as the inspector taxonomy, mirrored in NUIF’s property groups.
- Figma UI3’s bottom floating toolbar and minimize-UI behaviour, since the test editor is specified to replicate the UI3 interaction model and the pattern is a method of operation rather than protectable expression.
Adapt
- Author an original icon set, palette and naming; do not reproduce Figma’s icons, logo, product names or Help Center artwork (
nuif:research:naming). - Where editors disagree (toolbar position, ellipse/board shortcuts), document the chosen convention in the editor’s own reference rather than presenting it as a Figma feature.
- Treat Adobe XD as a historical data point only, given its maintenance-mode status.
Reject
- Editor-specific product features that are not layout conventions: Figma Dev Mode, FigJam, AI actions, community marketplace, version history and branching UI, multiplayer cursors, comments, Canva’s template and media browsers, Framer’s CMS and publishing controls, Sketch’s Components view and prototyping player beyond test needs. Reason: they are product scope, not conventions, and are outside the test editor’s testing/import/export remit.
Open questions
- Should the comparison be extended with Lunacy, Affinity Designer or Pencil-class editors to test whether the bottom-toolbar pattern is spreading?
- Are there jurisdictions outside the US where UI layout or “look and feel” receives stronger protection that would affect the test editor’s distribution?
- Which Adobe XD and Canva documentation pages can be retrieved through an alternative channel to upgrade their rows from excerpt to verified?
Design2Code real-world screenshot-to-code benchmark
Document status:
reviewed. Canonical source.
Summary
Design2Code evaluates screenshot-to-frontend-code generation on 484 manually curated real webpages. Its fine-grained analysis reports that contemporary multimodal models especially miss visible elements and produce incorrect layouts. The benchmark is useful evidence that screenshot reconstruction is not solved by a single high-level similarity score or a one-shot prompt.
It is not a NUIF conformance corpus: its output is HTML/CSS, its exact source and licensing conditions need to be reviewed before fixture reuse, and matching one viewport cannot establish authored structure, responsiveness or behavior.
Evidence
- ACL Anthology paper, abstract and §2: 484 diverse real-world webpages are manually curated as test cases for screenshot-conditioned code generation.
- §3 defines automatic evaluation over rendered output and complements it with human evaluation to validate system ranking.
- The paper’s fine-grained results identify element recall and layout generation as major failure categories even when aggregate visual similarity improves.
- The official repository adds an 80-example hard subset and publishes the benchmark/evaluation implementation: https://github.com/NoviScl/Design2Code.
Mechanism
The task provides one webpage screenshot to a multimodal model, executes the generated frontend, renders the result and compares it with the target using aggregate and element-level measures. Human evaluations provide a check on metric ranking. This is an end-to-end code-generation evaluation, not recovery of the original source program.
NUIF relevance
Borrow a held-out real-page benchmark, element-level recall, layout breakdowns and human validation of metric ranking.
Adapt output validation to typed NUIF operations and a deterministic renderer. Add document validity, tree structure, text, geometry, resources, responsive held-out viewports, accessibility, provenance and confidence.
Reject one screenshot/one viewport as proof of exact reconstruction, aggregate screenshot similarity as the sole reward, and source-code similarity as a semantic NUIF oracle.
Open questions
- Which Design2Code assets can be redistributed as NUIF test fixtures under documented terms rather than only evaluated in place?
- How strongly do its metric rankings correlate with editable structure and held-out responsive behavior?
- Which failures remain after deterministic OCR, region proposals and a render-difference correction loop are supplied to the same model?
Deterministic CBOR profiles (CDE, draft-ietf-cbor-serialization, dCBOR) and numeric canonicalization for a binary and a text profile with one hash
Document status:
reviewed. Canonical source.
Summary
As of 2026-08-29 no deterministic-CBOR profile beyond RFC 8949 §4.2 has reached RFC status. draft-ietf-cbor-cde (CDE) passed Working Group Last Call on 2025-03-06, was moved to “Parked WG Document” on 2025-10-19 after the CBOR Working Group found no consensus to continue with the document, and expired on 2026-04-16 at revision -13. The working group adopted draft-lundblade-cbor-serialization as draft-ietf-cbor-serialization on 2025-11-19; revision -08 (2026-07-29, Standards Track) entered Working Group Last Call on 2026-07-30. It defines “preferred-plus serialization” (shortest-form arguments for every major type, definite lengths only, a single NaN 0xf97e00, no leading zeros in big numbers, mandatory subnormal support in the shortest float width) and “deterministic serialization” (preferred-plus with map entries sorted bytewise-lexicographically by the deterministic encoding of their keys). It keeps the data model: integral floats stay floats, 0.0 is 0xf90000, and negative zero is not mentioned. draft-mcnally-deterministic-cbor (dCBOR) remains an individual submission at revision -18 (2026-08-10); its title changed from “A Deterministic CBOR Application Profile” (-13, intended Experimental) to “dCBOR: Deterministic CBOR” (-14, intended Standards Track) and its abstract now describes “a set of narrowing rules”. dCBOR §2.5 reduces integral floats in [-2^63, 2^64-1] to integers, all zeros to 0x00 and all NaNs to 0xf97e00; §2.6 permits only false, true, null and floats among simple values; §2.7 requires NFC text; every decoder rule is MUST reject. RFC 8949 erratum 8589 (verified 2025-10-01) adds the sign bit to NaN map-key equivalence. RFC 8785 (JCS) prints numbers with the ECMAScript Number::toString algorithm (shortest round-trip digits, exponent notation at magnitude ≥ 10^21 or < 10^-6, -0 printed as 0, NaN and Infinity are errors). Rust’s Display for f64 (core::num::flt2dec, shortest mode since PR #24612, 2015-05-09) produces the same digit string but a different layout: it prints -0 for negative zero and never uses exponent notation; {:e} (LowerExp) yields the shortest digits in exponent form. Among Rust CBOR crates at MSRV 1.85, only dcbor 0.25.2 implements and enforces a complete deterministic rule set (numeric reduction, NaN reduction, NFC, key order, duplicate rejection); ciborium 0.2.2 narrows floats and integers to the shortest bit-preserving width but does not sort keys or check input; minicbor 2.3.0 and cbor4ii 1.2.2 write fixed-width floats and preserve iteration order; serde_cbor is unmaintained (RUSTSEC-2021-0127). None of CDE, draft-ietf-cbor-serialization, dCBOR or DAG-CBOR constrains the content of a byte string, so opaque extension payloads survive a strict decoder unchanged.
Evidence
Status of the IETF documents
- CDE datatracker history: WG -00 adopted 2023-11-27 from draft-bormann-cbor-cde; Working Group Last Call initiated 2025-03-06 with intended status Best Current Practice; -13 published 2025-10-13; “IETF WG state changed to Parked WG Document from In WG Last Call” 2025-10-19; document expired 2026-04-16; no RFC number. https://datatracker.ietf.org/doc/draft-ietf-cbor-cde/history/ (retrieved 2026-08-29).
- Interim 2025-10-15 minutes (interim-2025-cbor-18): Lundblade “I oppose publication of -cde in its current form. We should publish a document about serialization in general, not about determinism.”; Hoffman “So far no consensus on continuing with CDE document, but consensus for CDE topic.”; Bormann on chat: consensus that the definite-length-only constraint can be addressed in the same document as determinism. https://datatracker.ietf.org/doc/minutes-interim-2025-cbor-18-202510151400/ (retrieved 2026-08-29).
- Interim 2025-10-01 minutes (interim-2025-cbor-17): Lundblade “Decoder checking has to be optional. Normative behavior can’t depend on it.” and “Let’s keep determinism clean about determinism, and not extend it to defense about malicious input.”; Bormann “The encoder has the situation under control and shouldn’t need to check.” https://datatracker.ietf.org/doc/minutes-interim-2025-cbor-17-202510011400/ (retrieved 2026-08-29).
- IETF 124 minutes (2025-11-07): Lundblade “My document is std track.” with “No update to 8949, align closely, eg. new serialization only differs in NaN handling.”; Bormann: “We now have 3 choices in the wild” (well-known, legacy canonical, common deterministic); Leiba: technical erratum on NaN verified. https://datatracker.ietf.org/doc/minutes-124-cbor-202511071430/ (retrieved 2026-08-29).
- draft-ietf-cbor-serialization history: WG -00 approved 2025-11-19 (replaces draft-lundblade-cbor-serialization), shepherd Paul E. Hoffman; -08 2026-07-29; “IETF WG state changed to In WG Last Call from WG Document” 2026-07-30. https://datatracker.ietf.org/doc/draft-ietf-cbor-serialization/history/ (retrieved 2026-08-29).
- IETF 125 minutes (2026-03-16) discuss only draft-ietf-cbor-serialization (normative language, test vectors, “2k test vectors” from the hackathon); IETF 126 minutes (2026-07-23): the draft “is close to ready for Working Group Last Call”, open items are byte-string handling inconsistencies across bundle protocol, C509 and COSE, “the usual discussion of nontrivial NaNs”, and bignums. https://datatracker.ietf.org/doc/minutes-125-cbor-202603160830/ and https://datatracker.ietf.org/doc/minutes-126-cbor/ (retrieved 2026-08-29).
- dCBOR datatracker: revision -18 dated 2026-08-10, “Active Internet-Draft (individual)”, no stream, IESG state “I-D Exists”, no replaced-by entry, no RFC number; the document header reads “Intended status: Standards Track”, “Expires: 11 February 2027”, authors McNally, Allen, Bormann, Lundblade. https://datatracker.ietf.org/doc/draft-mcnally-deterministic-cbor/ and https://www.ietf.org/archive/id/draft-mcnally-deterministic-cbor-18.txt (retrieved 2026-08-29).
- dCBOR title history: -11 (2024-08-07) and -13 (2025-08-10) are titled “dCBOR: A Deterministic CBOR Application Profile” with intended status Experimental; -14 (2025-11-01) is titled “dCBOR: Deterministic CBOR” with intended status Standards Track. https://datatracker.ietf.org/doc/draft-mcnally-deterministic-cbor/11/, /13/, /14/ (retrieved 2026-08-29).
- dCBOR -18 references: normative [RFC8949], [RFC8610], [IEEE754], [UNICODE-NORM]; informative [cbor-deterministic] (the CDE draft), [cbor-dcbor], [BCRustDCBOR], [BCSwiftDCBOR], [BCTypescriptDCBOR], [GordianEnvelope]; draft-ietf-cbor-serialization is not referenced. -18 §9 (retrieved 2026-08-29).
Normative rules
- RFC 8949 §4.2.1: shortest integer arguments, shortest float that preserves the value, no indefinite-length items, map keys sorted bytewise-lexicographically by deterministic encoding; §4.2.2 leaves tags, big integers, negative zero, NaN, subnormals and integral floats to the protocol. https://www.rfc-editor.org/rfc/rfc8949.html#section-4.2 (retrieved 2026-08-29).
- RFC 8949 erratum 8589, Verified 2025-10-01, §5.6.1: NaN values are equivalent as map keys “if they have the same significand after zero-extending both significands at the right to 64 bits, and if they both have the same sign bit.” https://errata.rfc-editor.org/rfc8949 (retrieved 2026-08-29).
- draft-ietf-cbor-serialization-08 §4.1 (preferred-plus): shortest-form argument for all major types; definite-length encoding only for strings, arrays and maps; floats “MUST be encoded in the shortest of double, single or half-precision that preserves precision”; “Subnormal numbers MUST be supported in this shortest-length encoding”; “For example, 0.0 can always be reduced to half-precision so it MUST be encoded as 0xf90000”; “Encoders MUST NOT output any NaN other than the half-precision NaN 0xf9 0x7e 0x00”; a value representable in major type 0 or 1 “MUST be encoded with major type 0 or 1, never as a big number”; no leading zeros in big numbers. §5.1 (deterministic): “If a map is encoded, the items in it MUST be sorted in the bytewise lexicographic order of their deterministic encodings of the map keys.” §1.3: “This document defines new serializations rather than updating those in [STD94]”. §8: the CDDL control
.serial“applies recursively through nested arrays and maps, but does not extend into byte strings”. Appendix H: checking decoders are permitted, not required. Appendix I.1: wrapping CBOR in a byte string isolates encoding errors of the wrapped data. Negative zero, integral-float reduction and duplicate keys are not addressed in the retrieved text. https://www.ietf.org/archive/id/draft-ietf-cbor-serialization-08.html and .txt (retrieved 2026-08-29). - CDE -13 §3.1.2: shortest head that preserves the value; “an encoder that is asked by an application to represent a negative floating point zero (-0.0) will generate 0xf98000”; “there is no attempt to mix integers and floating point numbers”; typical applications encode the quiet non-negative NaN as
0xf97e00. §3.3: each key MUST be lexicographically strictly greater than the preceding key (which excludes duplicates). Appendix B: Application-level Deterministic Representation (ALDR) rules are “a concept that is separate from CDE itself”; “An early example of a separate document is the dCBOR specification”, which “specifies the use of CDE together with some application-level rules, i.e., an ALDR ruleset”; ALDR rules “do not ‘fork’ CBOR”. Appendix C.3.2: CDE-checking decoders “MUST check the input for keeping the preferred-serialization and definite-length-only encoding constraints” and “MUST NOT present to the application a decoded data item that fails one of these checks”; generic decoders are not required to check. §4: CDDL controls.cdeand.cdeseqrequire the byte-string content to be CDE, for exampleleaf = #6.24(bytes .cde any). https://www.ietf.org/archive/id/draft-ietf-cbor-cde-13.html and the “disentangle” editor’s copy https://cbor-wg.github.io/draft-ietf-cbor-cde/disentangle/draft-ietf-cbor-cde.html (retrieved 2026-08-29). - dCBOR -18: §2.1 definite lengths, decoders MUST reject indefinite-length items; §2.2 encoders MUST emit only preferred serialization, decoders MUST validate and reject; §2.3 bytewise-lexicographic key order, decoders MUST validate; §2.4 decoders MUST reject duplicate keys; §2.5 numeric reduction: encoders “MUST check whether floating point values to be encoded have the numerically equal value in DCBOR_INT = [-2^63, 2^64-1]” and convert them to that integer, “the three representations of a zero number in CBOR (0, 0.0, -0.0 in diagnostic notation) are all reduced to the basic integer 0”, encoders “MUST reduce all encoded NaN values to the quiet NaN value having the half-width CBOR representation 0xf97e00”, decoders “MUST reject any encoded floating point values that are not encoded according to the above rules”; §2.6 only
false(0xf4),true(0xf5),null(0xf6) and floats are valid simple values, decoders MUST reject others; §2.7 encoders “MUST only emit text strings that are in NFC”, decoders “MUST reject any encoded text strings that are not in NFC”; §3 tag 201 declares enclosed dCBOR “at the data model level and the encoded data item level”; §4 implementation status lists Swift, Rust and TypeScript (Blockchain Commons, BSD-2-Clause-Patent) and Ruby (Bormann, Apache-2.0, exclusion checking not implemented); §7.1 test vectors include the smallest half, single and double subnormals; §7.2 Table 4 invalid encodings include12.0asf94a00(“Can be reduced to 12”),-2^63-1as3b8000000000000000and-2^64as3bffffffffffffffff(“65-bit negative integer value”). No rule constrains byte-string content. https://www.ietf.org/archive/id/draft-mcnally-deterministic-cbor-18.txt (retrieved 2026-08-29). - RFC 8949 §3.4.5.1 (tag 24): a contained byte string “is valid if it encodes a well-formed CBOR data item”; §3.1 major type 2 carries an arbitrary byte sequence whose length is the argument. https://www.rfc-editor.org/rfc/rfc8949.html#section-3.4.5.1 (retrieved 2026-08-29).
- RFC 8785 §1: hashing and signing “need the data to be expressed in an invariant format”; §3.2.2.3: numbers “MUST be serialized according to Section 7.1.12.1 of [ECMA-262], including the ‘Note 2’ enhancement”, NaN and Infinity “MUST cause a compliant JCS implementation to terminate with an appropriate error”; Appendix B:
0000000000000000and8000000000000000both serialise as0,0000000000000001as5e-324,4340000000000000as9007199254740992,7fefffffffffffffas1.7976931348623157e+308. https://www.rfc-editor.org/rfc/rfc8785.html (retrieved 2026-08-29). - ECMAScript
Number::toStringlayout as documented by MDN: “Scientific notation is used if radix is 10 and the number’s magnitude (ignoring sign) is greater than or equal to 10^21 or less than 10^-6”; “Both 0 and -0 have ‘0’ as their string representation”; the algorithm “uses the least number of significant figures necessary to distinguish the output from adjacent number values”. https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Number/toString (retrieved 2026-08-29). The ECMA-262 §6.1.6.1.20 algorithm text itself was not retrieved in this session (the multipage fetch returned only the table of contents); the RFC 8785 citation of ECMA-262 §7.1.12.1 with Note 2 is the normative locator.
Rust float formatting
- rust-lang/rust PR #24612 “New floating-to-decimal formatting routine” (lifthrasiir, merged 2015-05-09) introduced
core::num::flt2decwith Grisu3 and a Dragon4 fallback; “all specifiers default to the shortest representation”. https://github.com/rust-lang/rust/pull/24612 (retrieved 2026-08-29). library/core/src/num/flt2dec/mod.rsat tag 1.85.0, lines 13–17: the shortest mode output is “correctly rounded when parsed back”, “shortest such one, i.e., there is no representation with less than n digits that is correctly rounded”, and “closest to the original value”; lines 18–32 name this the shortest mode. https://raw.githubusercontent.com/rust-lang/rust/1.85.0/library/core/src/num/flt2dec/mod.rs (retrieved 2026-08-29).library/core/src/fmt/float.rsat tag 1.85.0:float_to_decimal_display(lines 77–88) callsfloat_to_decimal_common_shortestwithSign::Minuswhen no precision is given; the Debug implementation switches to exponent form when(abs != 0.0 && abs < 1e-4) || abs >= 1e+16(lines 15–18); thefloating!macro (line 226) implements Display, Debug, LowerExp and UpperExp for f32 and f64. https://raw.githubusercontent.com/rust-lang/rust/1.85.0/library/core/src/fmt/float.rs (retrieved 2026-08-29).- Empirical run of a 30-line probe compiled with
rustc 1.85.0 (4d91de4e4 2025-02-17), edition 2024, on this repository’s pinned toolchain (2026-08-29):
| bits (binary64) | {} Display | {:?} Debug | {:e} LowerExp | JCS Appendix B / ECMAScript |
|---|---|---|---|---|
0000000000000000 | 0 | 0.0 | 0e0 | 0 |
8000000000000000 | -0 | -0.0 | -0e0 | 0 |
3ff0000000000000 | 1 | 1.0 | 1e0 | 1 |
3fb999999999999a | 0.1 | 0.1 | 1e-1 | 0.1 |
444b1ae4d6e2ef50 (1e21) | 1000000000000000000000 | 1e21 | 1e21 | 1e+21 |
4415af1d78b58c40 (1e20) | 100000000000000000000 | 1e20 | 1e20 | 100000000000000000000 |
3e7ad7f29abcaf48 (1e-7) | 0.0000001 | 1e-7 | 1e-7 | 1e-7 |
3eb0c6f7a0b5ed8d (1e-6) | 0.000001 | 1e-6 | 1e-6 | 0.000001 |
0000000000000001 | 0.000…005 (326 characters) | 5e-324 | 5e-324 | 5e-324 |
7fefffffffffffff | 309 digits | 1.7976931348623157e308 | 1.7976931348623157e308 | 1.7976931348623157e+308 |
4340000000000000 | 9007199254740992 | 9007199254740992.0 | 9.007199254740992e15 | 9007199254740992 |
Every Display string parsed back to the identical bit pattern. f32 values widened to f64 print under the f64 shortest rule: 0.1f32 prints as 0.10000000149011612 after widening, and as 0.1 when formatted as f32. f64::NAN, INFINITY and NEG_INFINITY print as NaN, inf, -inf.
Rust crates at MSRV 1.85
| Crate | Version (date) | Licence | MSRV or edition | Deterministic rules | Unknown tags | Decoding model |
|---|---|---|---|---|---|---|
| ciborium | 0.2.2 (2024-01-24) | Apache-2.0 | crates.io metadata 1.58; main-branch Cargo.toml rust-version = "1.85", edition 2021 | integers shortest (ciborium-ll/src/hdr.rs lines 85–91); floats narrowed to f16 or f32 when the widened value is bit-identical (lines 104–118); map keys written in caller order (ciborium/src/ser/mod.rs lines 269–274); no canonical check on input; docs: “liberal in what we accept” | Value::Tag(u64, Box<Value>) retains any tag | whole Value tree or serde; Value::Map is Vec<(Value, Value)> preserving wire order |
| minicbor | 2.3.0 (2026-07-23) | BlueOak-1.0.0 | MSRV unspecified; edition 2024 (requires 1.85 or later) | Encoder::f16, f32, f64 write the requested width; no sorting or canonical check documented | derive ignores unknown fields; manual Decoder sees every tag | non-allocating Decoder with position, set_position, probe, skip, tokens |
| cbor4ii | 1.2.2 (2025-11-30) | MIT | unspecified | src/core/enc.rs lines 323–339 write f32 and f64 at full width; maps in iteration order (lines 316–322); datetime, bignum and bigfloat not implemented | not documented | serde and core API |
| dcbor | 0.25.2 (2026-03-16) | BSD-2-Clause-Patent | MSRV unspecified; edition 2024; no_std feature; deps half ^2.4.1 (half 2.7.1 declares MSRV 1.81), unicode-normalization ^0.1.22, chrono ^0.4.28; no serde | README enforces shortest integers and floats, key order, definite lengths, duplicate rejection, numeric reduction to [-2^63, 2^64-1], single NaN, simple-value restriction, NFC; CBOR::try_from_data rejects data that “violates dCBOR encoding rules” or has trailing content; Error variants NonCanonicalNumeric, NonCanonicalString, MisorderedMapKey, DuplicateMapKey, UnusedData, InvalidSimpleValue, UnsupportedHeaderValue, Underrun | CBORCase::Tagged(Tag, CBOR) holds any tag | whole reference-counted tree |
| cbor-edn | 0.0.10 (2026-03-23) | MIT OR Apache-2.0 | MSRV 1.76 | diagnostic-notation converter, not a canonical codec | n/a | n/a |
| serde_cbor | 0.11.2 (2021-08-15) | MIT/Apache-2.0 | n/a | RUSTSEC-2021-0127 (2021-11-30): unmaintained, repository archived; suggested replacements ciborium and minicbor | n/a | n/a |
Sources: https://crates.io/api/v1/crates/{ciborium,minicbor,dcbor,cbor4ii,cbor-edn,serde_cbor}; https://raw.githubusercontent.com/enarx/ciborium/main/ciborium/Cargo.toml; https://raw.githubusercontent.com/enarx/ciborium/main/ciborium-ll/src/hdr.rs; https://raw.githubusercontent.com/enarx/ciborium/main/ciborium/src/ser/mod.rs; https://docs.rs/ciborium/latest/ciborium/; https://raw.githubusercontent.com/twittner/minicbor/develop/minicbor/Cargo.toml; https://docs.rs/minicbor/latest/minicbor/encode/struct.Encoder.html; https://docs.rs/minicbor/latest/minicbor/decode/struct.Decoder.html; https://docs.rs/minicbor-derive/latest/minicbor_derive/; https://raw.githubusercontent.com/quininer/cbor4ii/master/src/core/enc.rs; https://raw.githubusercontent.com/BlockchainCommons/bc-dcbor-rust/master/Cargo.toml; https://raw.githubusercontent.com/BlockchainCommons/bc-dcbor-rust/master/README.md; https://docs.rs/dcbor/latest/dcbor/struct.CBOR.html; https://docs.rs/dcbor/latest/dcbor/enum.Error.html; https://docs.rs/dcbor/latest/dcbor/enum.CBORCase.html; https://raw.githubusercontent.com/starkat99/half-rs/main/Cargo.toml; https://rustsec.org/advisories/RUSTSEC-2021-0127.html (all retrieved 2026-08-29).
Numeric values in authored interface documents
- nuif-core:
SizeIntent::Fixed(f64)(crates/nuif-core/src/lib.rsline 54),EntityId(pub u128)(line 6),Extensions(pub BTreeMap<String, Vec<u8>>)(line 60); nof32and no integer-typed property exists in the current model. spec/03 lists “number” as a parameter class; spec/04 lists percentage sizing; spec/05 requires affine transforms and colour values with a declared colour space; no timestamp property is specified (grep overspec/andrfcs/, 2026-08-29). - DTCG Format Module 2025.10: colour
componentsare numbers in the range 0–1 ([0, 0.4, 0.8]),dimension.valueis “a numeric value (integer or floating-point)”,number“MUST be a JSON number value”,duration.valueis integer or floating-point; no precision constraint. https://www.w3.org/community/reports/design-tokens/CG-FINAL-format-20251028/ (retrieved 2026-08-29). - glTF 2.0 JSON encoding: integer-typed properties “MAY be stored as decimals with a zero fractional part or by using exponent notation” and “MUST NOT contain any non-zero fractional value”; floating-point values
NaN,+Infinity,-Infinity“MUST NOT be present”; non-integer numbers “SHOULD be written in a way that preserves original values” across a round trip. https://raw.githubusercontent.com/KhronosGroup/glTF/main/specification/2.0/Specification.adoc, section “JSON Encoding” (retrieved 2026-08-29; the GLB chunk section was beyond the retrievable length and is not cited). - OTIO test utilities compare JSON with trailing-decimal-zero normalisation (existing record nuif:research:opentimelineio, evidence line for
test_utils.pylines 15–31).
Hash precedents
- IPLD CID:
<cidv1> ::= <CIDv1-multicodec><content-type-multicodec><content-multihash>; a CID is “a tuple of (content-type, content-address)”, so the same data under two codecs has two CIDs. https://github.com/multiformats/cid (retrieved 2026-08-29). - Automerge binary format: “A change hash is the 32-byte SHA256 hash of the concatenation of the chunk type (0x01) chunk length and chunk contents fields of a change represented as a Change Chunk”; “Implementations must generate the shortest possible uLEB encodings, and should reject documents with overly long encodings.” https://automerge.org/automerge-binary-format-spec/ (retrieved 2026-08-29).
- OpenUSD:
.usdcis “losslessly, bidirectionally convertible to the .usda text format” andusdcat --usdFormat usda|usdcconverts between them; no content hash is defined (existing record nuif:research:openusd-composition-and-crate, evidence lines for the glossary and toolset; https://openusd.org/release/toolset.html retrieved 2026-08-29).
Mechanism
Rule matrix (source statements; “unspecified” means the retrieved text contains no rule):
| Rule | RFC 8949 §4.2.1 | CDE -13 | draft-ietf-cbor-serialization-08 deterministic | dCBOR -18 | DAG-CBOR |
|---|---|---|---|---|---|
| integer arguments | shortest | shortest | shortest, big numbers only above 64-bit range | shortest; 65-bit negatives rejected | shortest, signed 64-bit range |
| float width | shortest preserving value | shortest preserving value | shortest of double, single, half | shortest after reduction | always binary64 |
| integral float | protocol decides | stays float | stays float | integer if in [-2^63, 2^64-1] | stays float |
| -0.0 | protocol decides | 0xf98000 | unspecified | 0x00 | should not appear; encode as 0x0000000000000000 |
| NaN | protocol decides | typically 0xf97e00 | only 0xf97e00 | only 0xf97e00 | rejected |
| Infinity | allowed | allowed | allowed | allowed | rejected |
| subnormals | protocol decides | shortest form | must be supported in shortest form | test vectors include them | binary64 |
| map order | bytewise lexicographic of encoded keys | same, strictly increasing | same | same | length-first then bytewise (RFC 7049 order) |
| duplicate keys | disallowed by sorting | excluded by strict order | unspecified | decoders MUST reject | rejected |
| text normalisation | none | none | none | NFC, decoders MUST reject | none |
| simple values | any | any | any | false, true, null, floats | false, true, null, floats |
| decoder strictness | not required | checking decoders MUST reject; generic decoders exempt | checking decoders optional | MUST reject every deviation | should reject; decoders may relax by default |
| byte-string content | opaque | opaque unless .cde control applied | opaque; .serial stops at byte strings | opaque | opaque |
Digit generation and layout. Both JCS (via ECMAScript) and Rust Display emit the shortest decimal digit string that parses back to the same binary64 value, and both pick the closest candidate; the two algorithms differ only in layout: negative zero (0 in JCS, -0 in Rust) and exponent thresholds (JCS switches to d.ddde±x outside 10^-6 ≤ |v| < 10^21; Rust Display never switches, Rust Debug switches outside 10^-4 ≤ |v| < 10^16). A text profile can therefore specify a layout over the shortest digits without ECMAScript semantics:
digits, exp := shortest round-trip decimal of v as produced by `{:e}` (d[.ddd]e[-]x)
n := exp + 1 // position of the decimal point relative to the digits
if 0 < n <= 21: integer part = digits padded with zeros to n places, fraction = remaining digits
if -6 < n <= 0: "0." + (-n zeros) + digits
otherwise: d[.ddd] + "e" + sign + |n-1|
The {:e} output on rustc 1.85.0 for the probe values (1e21, 5e-324, 1.7976931348623157e308, 9.007199254740992e15) contains the digit strings JCS Appendix B expects; the layout step adds the explicit + and the threshold switch.
Typed reduction. dCBOR’s integral-float reduction is lossless only when the reader knows the type of the slot. For a property typed real the wire item 0x01 decodes to 1.0, and for a property typed integer the encoder never produces a float; the reduction is then a wire-level normalisation. In an untyped slot (a generic Value, or an extension payload interpreted by a foreign decoder) 1 and 1.0 collapse, which is exactly the problem OTIO’s trailing-zero normalisation and glTF’s “integer stored as 1.0” rule work around at the JSON layer.
Negative zero. Authored input rarely contains -0.0; it results from arithmetic (negation of zero, products of a negative factor with zero as in a mirroring transform, rounding of small negative results). IEEE 754 equality treats it as equal to +0.0; NUIF geometry has no property whose meaning depends on the sign of zero. Reducing it removes a source of hash instability between an authored 0 and a computed -0.0.
Width. The same real number stored as f32 and as f64 yields the same shortest binary width under every CBOR profile (the widened f32 value is exactly representable in binary32), but not the same shortest decimal string (0.1 versus 0.10000000149011612). A logical model with a single real type (binary64) keeps binary and text canonicalisation consistent.
Hash definition. Every precedent hashes one designated byte sequence (JCS text, DAG-CBOR block, Automerge change chunk) and does not claim identity across encodings; IPLD makes the codec part of the identifier. The workable definition of “the same hash from text and binary” is therefore: the hash is computed over the nuif-cbor-0 bytes, and nuif-text-0 is defined as a lossless surface syntax over the same value set, so that hash(text) is by definition hash(cbor(parse(text))). This requires the value sets to coincide: binary64 reals, integers within the CBOR 64-bit range, one NaN, one zero, NFC or verbatim text chosen once.
Strictness and opaque payloads. A strict decoder (dCBOR, CDE-checking, DAG-CBOR) rejects non-canonical structure but never inspects byte-string content, so an extension payload carried as a byte string survives byte-for-byte whatever its internal format. Tag 24 is unsuitable for opaque payloads because RFC 8949 §3.4.5.1 requires the content to be well-formed CBOR. If a registered extension promises canonical CBOR content, the promise can be expressed with the .cde-style CDDL control and checked by the extension’s own decoder, not by the container decoder.
NUIF relevance
Borrow
- draft-ietf-cbor-serialization-08 §4.1 and §5.1 as the structural base of
nuif-cbor-0(shortest arguments, definite lengths, shortest float width with subnormals, single NaN0xf97e00, bytewise-lexicographic key order, no big-number tags inside the 64-bit range), because it is the only deterministic-CBOR text in Working Group Last Call and it is Standards Track. - dCBOR §2.4, §2.5 and §2.6 (duplicate rejection, integral-float reduction within [-2^63, 2^64-1], zero reduction to
0x00, NaN reduction, simple-value restriction) as the application-level rules, stated in the NUIF specification by value rather than by reference, because dCBOR is an individual draft whose text may change. - dCBOR’s decoder rule set (MUST reject every deviation) for canonical hash inputs, so that hash equality implies byte equality.
- The
dcborcrate (0.25.2, BSD-2-Clause-Patent, edition 2024,no_stdcapable) as the initialnuif-cbor-0implementation, behind the existingnuif-codectraits, because no other crate at MSRV 1.85 checks canonical form on input. - Shortest round-trip digits from
core::fmt{:e}plus a fixed layout fornuif-text-0, matching JCS digit strings without ECMAScript. - Byte strings for opaque extension payloads; no tag 24; no content check by the container decoder.
Adapt
- The logical model needs exactly two numeric types,
integer(i64/u64 within the CBOR range) andreal(binary64);f32colour components and percentages are stored as binary64 in the model and narrowed on the wire by the shortest-width rule. The empirical width difference in text output is the reason. - dCBOR’s NFC rule becomes a data-model rule on text properties (spec/02), decided once for both profiles; whichever choice is made, the text profile and the binary profile apply it identically or the hashes diverge.
- The
-0and exponent-layout differences between RustDisplayand JCS meannuif-text-0MUST NOT be specified as “print with Rust Display” and MUST NOT be specified as “print with ECMAScript”; it is specified by the digit-plus-layout rule above. - Because negative zero reduces to
0in the canonical form,SizeIntent::Fixed(-0.0)andFixed(0.0)hash equally;PartialEqonf64already treats them as equal, so the in-memory model and the hash agree.
Reject
- CDE -13 as a normative reference: parked and expired, and its
-0.0rule (0xf98000) contradicts the zero reduction chosen here. - DAG-CBOR’s length-first key order and always-binary64 floats: they are RFC 7049 legacy choices that no current IETF text recommends.
- Accept-and-recanonicalize decoding for hash inputs: it makes one hash identify several byte sequences and depends on decoder-specific behaviour.
serde_cbor(unmaintained), andciborium,minicbororcbor4iias canonical encoders without an additional canonicalisation layer (none sorts keys or checks input; two write fixed-width floats).
Open questions
- Whether draft-ietf-cbor-serialization will add a negative-zero rule or a duplicate-key rule before publication; the NUIF profile states both explicitly so that the outcome does not change
nuif-cbor-0. - Whether the
dcborcrate’s 0.x API and its tracking of dCBOR revisions are stable enough for a reference implementation, or whethernuif-codecneeds an independent canonical encoder overciborium-llwith a conformance test against the dCBOR §7 test vectors. - Whether Rust’s shortest-digit tie-breaking (closest value,
flt2declines 13–17) and ECMA-262 Note 2 (closest value, even digit on ties) ever differ for binary64; no counterexample was found in this session and the question is unverified. - Whether text properties store NFC by definition (dCBOR §2.7) or code points verbatim; the choice changes round-trip fidelity of imported documents and must be made in spec/02 before either profile is frozen.
- Whether registered extensions that carry CBOR inside their payload are required to be canonical (
.cde-style control in their CDDL) so that extension payloads remain diffable, or whether only opaque preservation is promised.
Deterministic simulation testing (FoundationDB, TigerBeetle VOPR, Antithesis)
Document status:
reviewed. Canonical source.
Summary
Deterministic simulation testing (DST) runs an entire system inside one single-threaded process in which time, scheduling, network, disk and randomness are simulated from one seeded pseudo-random number generator. A failure is reproduced by rerunning the same build with the same seed. FoundationDB introduced the practice with the Flow actor language and the Sim2 simulator; TigerBeetle’s VOPR adds swarm-randomised fault distributions, hash-chained state checkers and a liveness mode; Antithesis moves determinism into a hypervisor so unmodified binaries can be simulated. Swarm testing (Groce et al., ISSTA 2012) supplies the evidence that randomising which features each run enables improves defect discovery.
NUIF is not a distributed system, but its trial-and-error loop has the same nondeterminism sources: operation ordering, floating-point layout, font and image loading, adapter I/O and renderer scheduling. The DST recipe transfers as a design constraint on the headless engine: every source of nondeterminism sits behind an injectable interface and every run is replayable from (build, seed, fixture).
Evidence
- FoundationDB simulation is “a deterministic simulation of an entire FoundationDB cluster within a single-threaded process” and determinism “allows perfect repeatability of a simulated run”. https://apple.github.io/foundationdb/testing.html, §Simulation (mirrors
documentation/sphinx/source/testing.rst), retrieved 2026-08-29. - Simulated runs have roughly a 10:1 real-to-simulated time ratio and the project runs tens of thousands of simulations nightly. Same page, §Simulation.
- The simulated failure model includes network, machine and datacenter failures, reboots, degraded performance and “swizzle-clogging” (stopping connections in random sequence, then unclogging). Same page, §Simulation.
- Flow is an actor-based extension of C++ whose output feeds “our simulation tool, which conducts deterministic simulations of the entire system”. https://apple.github.io/foundationdb/flow.html, retrieved 2026-08-29. On the
mainbranchflow/README.mdnow describes cooperative scheduling over standard C++ coroutines. - The simulator is
class Sim2 final : public ISimulator, public INetworkConnections;runLoop()pops aTaskQueue<PromiseTask>and advances virtual time withdeterministicRandom()->random01();delay()schedules timers on virtual time and buggifies extra delay with probability 0.25.fdbrpc/sim2.cpp,mainbranch (lines ~1064–1075 and ~1379), retrieved 2026-08-29; therelease-7.1filefdbrpc/sim2.actor.cppnotes that time is modified only from the main thread. - Network and disk are simulated by
SimClogging,Sim2ConnandSimpleFile, with latency, disconnects and open delays drawn fromdeterministicRandom(). Same file, lines ~277–402 and ~654. - BUGGIFY sections activate with probability 0.25 and fire with probability 0.25;
buggify()returns true only if buggify is enabled for the file/line anddeterministicRandom()->random01() < probability.flow/include/flow/Buggify.h,mainbranch, lines 52–53, 92–101, retrieved 2026-08-29. fdbserveraccepts-r simulation,-f TESTFILE,-s SEED(“Random seed.”),-b [on,off](buggify, default off),-fi [on,off]and-R/--restarting.fdbserver/fdbserver.cpp,mainbranch, usage text lines ~609–630, retrieved 2026-08-29. The wiki page “How to reproduce a restart test failure” showsfdbserver -r simulation -f <test> --seed 523887594 --buggify on.- The 2014 Strange Loop abstract states that disks, network links and machines are “replaced in testing with software” so that “the exact same series of events can be replayed”. https://www.thestrangeloop.com/2014/testing-distributed-systems-w-slash-deterministic-simulation.html, retrieved 2026-08-29. Talk video https://www.youtube.com/watch?v=4fFDFbi3toc (no transcript retrievable); secondary notes at https://alex-ii.github.io/notes/2018/04/29/distributed_systems_with_deterministic_simulation.html record the interface swap
INetwork -> SimNetwork,IAsyncFile -> SimFileand the single-thread requirement. - Wilson (Antithesis blog, 2024-02-13) states that a “fully-deterministic event-based network simulation” was written before the database, run as a single-threaded process with one RNG and rerun “with the same random seed”. https://antithesis.com/blog/is_something_bugging_you/, retrieved 2026-08-29.
- Antithesis distinguishes FoundationDB-style DST, where “all nondeterministic components are pluggable”, from running unmodified software inside a deterministic hypervisor; the controlled sources are clocks, thread interleaving and system randomness. https://antithesis.com/docs/resources/deterministic_simulation_testing/, retrieved 2026-08-29.
- The Antithesis hypervisor runs each instance on one physical core, virtualises time and routes I/O through a VMCALL channel; reproducibility enables time-travel debugging. https://antithesis.com/blog/deterministic_hypervisor/ (2024-03-20), retrieved 2026-08-29.
- A “Sometimes” assertion asserts that a state is reached in at least one run; a never-hit sometimes assertion indicates an unreachable state or weak testing. https://antithesis.com/docs/best_practices/sometimes_assertions/, retrieved 2026-08-29.
- TIGER_STYLE requires an average of at least two assertions per function, pair assertions on different code paths, and assertions of both positive and negative space; it states that assertions “downgrade catastrophic correctness bugs into liveness bugs” and are “a force multiplier for discovering bugs by fuzzing”.
docs/TIGER_STYLE.md,mainbranch, §Safety (lines ~105–150), retrieved 2026-08-29. The phrase “assertions as oracles” does not appear in the document. - VOPR uses a random seed to tune fault-injection parameters; “the seed and Git commit hash can be used to replay back the exact simulation”; storage checkers verify data files byte-for-byte across caught-up replicas.
docs/internals/vopr.md,mainbranch, lines ~9–39 and §Assertions and Checkers, retrieved 2026-08-29. - Replay command:
./zig/zig build vopr -- 123“produces a fully deterministic, reproducible outcome”.docs/internals/HACKING.md,mainbranch, §Simulation (lines ~48–60), retrieved 2026-08-29. src/vopr.zig(main, 1805 lines): default seedstd.crypto.random.int(u64),var prng = stdx.PRNG.from_seed(seed),options_swarm(&prng)randomises replica/client counts, packet loss, partition mode, storage fault probabilities and crash probabilities; failure message “you can reproduce this failure with seed={}”; safety mode thentransition_to_liveness_mode(core)withfatal(.liveness, "no state convergence: ...")on timeout. Lines ~83–84, 127–157, 263–265, 349–350, 374ff, 802–808, 888, retrieved 2026-08-29.- Testing doubles:
src/testing/packet_simulator.zig(delay, loss, replay, partition modesnone,uniform_size,uniform_partition,isolate_single, clogging);src/testing/storage.zig(“In-memory storage, with simulated faults and latency”, read/write fault and misdirect probabilities,ClusterFaultAtlasguaranteeing one valid copy);src/testing/time.zig(TimeSimwith tick-based monotonic and drifting realtime clocks);src/testing/cluster/state_checker.zig(hash-chain assertions such asassert(header_b.?.parent == checksum_a)).mainbranch, retrieved 2026-08-29. - Liveness mode: pick a core quorum, heal its partitions, freeze non-core faults, require convergence within a timeout. https://tigerbeetle.com/blog/2023-07-06-simulation-testing-for-liveness/, retrieved 2026-08-29. VOPR’s default mode swarm-randomises the fault distributions themselves. https://tigerbeetle.com/blog/2025-11-28-tale-of-four-fuzzers/ and https://tigerbeetle.com/blog/2025-04-23-swarm-testing-data-structures/, retrieved 2026-08-29.
- Swarm testing: a “swarm” of random configurations, “each of which omits some features”, found 42% more distinct compiler crashes in a week (104 vs 73 for the default Csmith configuration); features can suppress interesting behaviour and compete for space in a test. Groce, Zhang, Eide, Chen, Regehr, ISSTA 2012, DOI 10.1145/2338965.2336763, Abstract and §1 (https://users.cs.utah.edu/~regehr/papers/swarm12.pdf, retrieved 2026-08-29).
- Rust equivalents: turmoil runs multiple hosts “within a single thread” with a seeded RNG and injects latency, drops, partitions and torn writes (https://github.com/tokio-rs/turmoil README,
main); madsim requires “All I/O-related interfaces must be mocked”, providesRuntime::with_seed,MADSIM_TEST_SEEDandMADSIM_TEST_CHECK_DETERMINISM(https://github.com/madsim-rs/madsim README, docs.rs 0.2.34). Retrieved 2026-08-29.
Mechanism
Recipe, with attribution:
- Single-threaded scheduler over virtual time. All concurrency is cooperative; a task queue ordered by virtual timestamp is drained in one thread (FoundationDB
Sim2::runLoop; TigerBeetleTimeSim.tick(); turmoil). - One seeded PRNG. Every random choice, including simulated latency, fault firing and workload generation, is drawn from a generator initialised from the CLI seed (
deterministicRandom()with-s SEED;stdx.PRNG.from_seed(seed)). - Nondeterminism behind injectable interfaces. Network, disk, clock and randomness are traits with a production and a simulated implementation (
ISimulator,INetworkConnections,IAsyncFile;packet_simulator.zig,storage.zig,time.zig; madsim mocks). Antithesis relocates this boundary to the hypervisor. - Fault injection at two levels: environment faults (partition, loss, crash, misdirected write, clock drift) and in-code probabilistic hooks (
BUGGIFY, 0.25 × 0.25). - Replay by
(commit, seed). The failure report prints the seed; the same binary and seed reproduce the run (fdbserver -r simulation -s,zig build vopr -- <seed>). - Oracles are invariants, not expected outputs: dense assertions (TIGER_STYLE), state checkers with hash chaining, byte-identical storage across replicas, convergence within a liveness timeout, and reachability (“sometimes”) assertions.
- Swarm-randomised configurations: each seed also selects which features and fault classes are enabled and their probabilities (Groce 2012; VOPR
options_swarm). - Volume: many short simulated runs per night, with time compression relative to wall-clock.
run(seed, build):
prng = Prng::from_seed(seed)
config = swarm_config(&prng) # which features/faults are on, and their rates
env = SimEnv { clock: VirtualClock, io: SimIo(prng, config), rng: prng }
sys = System::new(&env) # all I/O through env traits
model = ReferenceModel::new()
while env.clock.now() < config.ticks_max:
env.step() # drain one virtual-time task; may fire faults
if let Some(op) = workload.next(&prng, &model):
sys.apply(op); model.apply(op)
check_invariants(&sys, &model) # assert, never log-and-continue
assert_convergence(&sys, &model) # liveness phase
report { seed, commit, config, coverage, sometimes_hits }
Invariants: no wall-clock, thread or OS entropy reaches the system under test; any two runs with equal (build, seed) produce identical traces; every failure is emitted with the seed needed to reproduce it.
NUIF relevance
Borrow
- Make
(implementation version, capability profile, fixture, seed)the replay key of every conformance run, matching the report fields already required inconformance/PLAN.md(FoundationDB-s SEED; VOPR seed plus commit hash). - Put every nondeterminism source of the headless engine behind traits with simulated implementations: font and image loading, adapter file I/O, renderer scheduling, timestamps in provenance records (FoundationDB interface swap; madsim mocking rule).
- Adopt assertion density and pair assertions in
nuif-core,nuif-protocolandnuif-layoutso that invariants (stable IDs, containment, acyclic references,Extensionsunchanged by unrelated operations) fail inside the loop rather than in later comparison (TIGER_STYLE §Safety). - Swarm-randomise the operation mix, layout families and adapter set per seed instead of fixing one generator distribution (Groce 2012; VOPR
options_swarm). - Add reachability (“sometimes”) assertions for rare paths such as move-into-instance, extension preservation through an unaware intermediate, and lossy adapter fallbacks (Antithesis).
Adapt
- NUIF has no network or clock to virtualise; the analogue of environment faults is adapter loss (unsupported feature, approximated value), corrupted or truncated inputs, and resource-limit hits from
spec/11-security.md. Fault injection should target those. - The liveness phase becomes a convergence phase: after fault injection stops, canonical hashes across the round-trip path must converge, and operation replay from the same base must yield the same hash (
conformance/fixtures/v0-responsive-card/README.md). - Time compression is irrelevant; the equivalent budget is operations per second through the CLI/API contract of
spec/12-cli-api-and-automation.md.
Reject
- A deterministic hypervisor is unnecessary: NUIF controls its own process and can achieve determinism at the interface level.
- BUGGIFY-style probabilistic hooks inside production code paths conflict with a library that must be embeddable; fault hooks belong in the simulated trait implementations only.
Open questions
- Which floating-point paths in layout and rasterisation are deterministic across CPU architectures, and must the seed key include target triple and font rasteriser version?
- Should browser-based differential oracles be excluded from seeded runs, given that a browser cannot be made deterministic from NUIF’s side?
- How are seeds and swarm configurations recorded in the report so that a coverage-guided scheduler can prioritise seeds without breaking replayability?
Source-built developer installation and operating-system trust boundaries
Document status:
verified. Canonical source.
Summary
NUIF Editor is a reference, conformance and research tool rather than a consumer desktop product. Its primary installation path therefore builds a reviewed revision locally with the checked-in Cargo lock file and installs it inside the current user’s account. Tagged GitHub archives remain durable CI evidence, reproducibility inputs and an expert download path; they are not the default trust mechanism for running the editor.
Local compilation does not create a universal security-policy exemption. Apple Gatekeeper evaluates software downloaded from outside the App Store, and Microsoft Smart App Control can require trusted signatures for all executable files. The installer must not disable Gatekeeper, System Integrity Protection, Defender, SmartScreen or Smart App Control. A managed device whose policy rejects the local build requires an organization-approved signing identity or device policy. That identity may be internal and does not require marketplace publication.
The source installer resolves a named alpha channel through the published,
attested release manifest; pins its tag and source revision; checks out that
revision; builds with Cargo.lock; and records the resulting binary digest,
toolchain and source identity. Installation is user-scoped, explicit updates
retain one rollback version, and uninstall removes only directories carrying a
NUIF-owned marker.
Evidence
- Cargo builds installable binaries from a Git repository and accepts an exact
tag or revision, a selected binary and
--lockedto use the checked-in lock file. Locator: Cargo Book,cargo install, “Install Options” and “Dealing with the Lockfile”, retrieved 2026-08-30: https://doc.rust-lang.org/cargo/commands/cargo-install.html. - GitHub verifies binary attestations with
gh attestation verify; repository, signer workflow, source ref and source digest can be constrained by the verifier. Locator: GitHub Docs, Using artifact attestations to establish provenance for builds, binary verification, retrieved 2026-08-30: https://docs.github.com/en/actions/how-tos/secure-your-work/use-artifact-attestations/use-artifact-attestations. - Gatekeeper verifies downloaded applications from outside the App Store for an identified developer, notarization and alteration, and requests first-run approval. Locator: Apple Platform Security, Gatekeeper and runtime protection in macOS, retrieved 2026-08-30: https://support.apple.com/guide/security/gatekeeper-and-runtime-protection-sec5599b66df/web.
- Microsoft documents publisher and file-hash reputation, states that an unsigned version starts without transferable publisher reputation, and notes that Smart App Control signature checks apply to all executable files. Locator: Microsoft Learn, SmartScreen reputation for Windows app developers, retrieved 2026-08-30: https://learn.microsoft.com/windows/apps/package-and-deploy/smartscreen-reputation.
- Homebrew taps distribute external formulae through Git repositories and support source builds; its Cargo formula guidance separates dependency fetch from offline installation. Locator: Homebrew Documentation, How to Create and Maintain a Tap and Formula Cookbook, retrieved 2026-08-30: https://docs.brew.sh/How-to-Create-and-Maintain-a-Tap and https://docs.brew.sh/Formula-Cookbook.
- Scoop buckets are Git repositories of JSON manifests. Their archive URL and SHA-256 fields provide convenient user-scoped installation but still execute the downloaded Windows artifact, so Scoop does not remove SmartScreen’s publisher boundary. Locator: Scoop Wiki, Buckets and App Manifests, retrieved 2026-08-30: https://github.com/ScoopInstaller/Scoop/wiki/Buckets and https://github.com/ScoopInstaller/Scoop/wiki/App-Manifests.
- Nix flakes name source inputs and lock their resolved references. They are a
useful optional reproducible environment for macOS and Linux, but are not a
native Windows installation path. Locator: Nix Reference Manual,
nix flake, retrieved 2026-08-30: https://releases.nixos.org/nix/nix-2.25.5/manual/command-ref/new-cli/nix3-flake.html.
Mechanism
The local installer has two trust modes. source installs the current clean
checkout and records its revision. alpha first resolves the newest published
NUIF prerelease, downloads release-manifest.json, verifies its GitHub
attestation against the NUIF release workflow and tag, and then requires the
checked-out source revision to equal the attested revision. Both modes build
with the repository lock file.
Each installation lives in an immutable version directory identified by the application version, source revision and installed binary digest. A small state document selects the active and previous installations. Platform integration points reference only the active version: a user Applications bundle on macOS, a per-user program and Start-menu shortcut on Windows, and XDG executable, desktop and icon entries on Linux. Changing the active version is separate from building it, allowing rollback without a rebuild.
macOS applies a local ad-hoc signature after copying the locally built bundle and verifies that signature. This provides a structurally valid local code signature, not a Developer ID or notarization claim. Windows never modifies a certificate store or security policy. Linux never writes outside user-owned XDG paths. A marker and receipt constrain doctor, rollback and uninstall to the directories created by the installer.
NUIF relevance
Adopt a source-built, user-scoped developer channel as the normal way to run the reference editor. It matches the project’s research role and makes the reviewed source, lock file and toolchain part of the install receipt.
Retain GitHub packages, checksums, the SBOM and attestations as independent build evidence and recovery material. Their existence does not imply an operating-system publisher identity.
Offer later a source-building Homebrew tap and a Nix flake. A Scoop bucket can be a convenience for explicitly opted-in Windows users, but cannot be described as a trust bypass.
Reject silent self-updates, mutable branch installation, piping an uninspected network script into a shell, automatic trust-store changes, and instructions that disable platform security controls.
Open questions
- Whether an organization wants to publish and trust an internal macOS or Windows signing identity for managed development machines.
- Whether sufficient demand exists to maintain a Homebrew source formula, a Nix flake and a Scoop convenience bucket in addition to the built-in source lifecycle.
- Whether the explicit alpha update resolver should later support stable and nightly channels after those channels have separate publication policies.
Differential testing (McKeeman, Csmith) and browser-referenced layout oracles (Taffy and Yoga gentest, R2Z2, X-PERT, WPT)
Document status:
reviewed. Canonical source.
Summary
Differential testing feeds one input to several comparable implementations and treats disagreement, crashes or hangs as bug candidates (McKeeman 1998). Csmith (Yang et al., PLDI 2011) is the canonical generator for this oracle: every generated program has a single defined meaning, the observable is a checksum, and voting across compilers identifies the minority; 325 compiler bugs were reported. Layout engines use the same oracle with a browser as the reference implementation: Taffy’s gentest drives headless Chrome through WebDriver, reads getBoundingClientRect, writes XML fixtures and compares with a 0.1 px tolerance; Yoga’s gentest uses Selenium and emits exact-equality C++, Java and TypeScript tests. Browser-to-browser work (Mesbah and Prasad 2011, X-PERT 2013, R2Z2 2022) supplies divergence classifications and filters for benign differences. WPT reftests define the fuzzy-match syntax for rendered images.
For NUIF the browser is an alternative implementation for the CSS-compatible subset of the flex, grid and stack families; the NUIF evaluator (initially Taffy) is the system under test; canonical hashes and metamorphic relations give self-consistency oracles where no browser semantics exist.
Evidence
- Definition: “If a single test is fed to several comparable programs … and one program gives a different result, a bug may have been exposed”; differential testing trades “many computer cycles instead of human effort”. McKeeman, “Differential Testing for Software”, Digital Technical Journal 10(1), 1998, pp. 100–107, Abstract and p. 101 (PDF https://www.cs.tufts.edu/comp/150FP/archive/bill-mckeeman/DifferentailTesting.pdf, retrieved 2026-08-29).
- Test quality levels for C: ASCII characters, tokens, syntactically correct, type-correct, statically conforming, dynamically conforming, model-conforming; results become interesting from level 4. Same paper, pp. 102–103.
- Outcome classification: results are filed as crash, loop, abend (some but not all terminate abnormally) and diff (all complete, outputs differ); tests where a comparison compiler crashes are discarded. Same paper, p. 105.
- Reduction applies 23 heuristic transformations to a fixpoint, often requiring more than 10,000 compilations. Same paper, p. 105.
- Csmith: randomised differential testing “has the advantage that no oracle for test results is needed”; with three or more implementations “a tester can use voting to heuristically determine which implementations are wrong”. Yang, Chen, Eide, Regehr, PLDI 2011, DOI 10.1145/1993498.1993532, §2.1 and Fig. 2 (preprint https://users.cs.utah.edu/~regehr/papers/pldi11-preprint.pdf, retrieved 2026-08-29).
- Design goal: every program “must be well formed and have a single meaning according to the C standard”; the observable is a checksum of non-pointer globals; C99’s 191 undefined and 52 unspecified behaviours are avoided structurally or by checks; implementation-defined behaviour is allowed, so comparison is valid within an equivalence class of compilers only. Same paper, §2.2, §2.6.
- Bug classes: compile-time crash, wrong-code (wrong result, crash, wrong termination), and “silent wrong-code error” without any warning. 325 bugs reported to 11 teams (79 GCC, 202 LLVM); no interesting split vote was ever observed. Same paper, §2.6, §3.1–3.2.
- Delta-debugging variants for C “introduce undefined behavior” and produce small but useless programs, so validity checkers are needed during reduction. Same paper, §3.7.
- Taffy gentest (
main, retrieved 2026-08-29):scripts/gentest/Cargo.tomldepends onfantoccini = "0.22.0"(WebDriver) and a localgetchromecrate;scripts/gentest/src/main.rslaunches Chrome with--headless --no-sandbox --disable-gpu, loads eachtest_fixtures/**/*.htmlviafile://, callsclient.execute("return getTestData()"), and writes XML totests/xml/. CONTRIBUTING.md: layouts are tested “by validating that layouts written in this crate perform the same as in Chrome”;just gentestdownloads matching Chrome for Testing and ChromeDriver builds; fixtures starting withxare disabled. Note: CONTRIBUTING still mentionstests/generated, but the current emitter writes XML. - Taffy fixture conventions: root element
id="test-root";scripts/gentest/test_base_style.cssembeds Ahem as a data URI, sets#test-root { font-family: ahem; line-height: 1; font-size: 10px; },box-sizing: border-box, fixed 15 px scrollbars;test_helper.jsreadsgetBoundingClientRect()relative to the parent, offers unrounded and “smartRounded” (Math.round(right) - Math.round(left)) values controlled bydata-test-rounding, and emits four trees (border-box/content-box × ltr/rtl). Files retrieved 2026-08-29. - Taffy comparison:
tests/xml.rsimplementsPartialEqfor output nodes with(expected - actual).abs() < 0.1on x, y, width, height, scroll dimensions and grid tracks; theuse-roundingattribute togglesenable_rounding/disable_rounding;tests/xml/flex/holds about 2,656 files such asabsolute_layout_width_height_start_top__border_box_ltr.xmlwith<viewport width="max-content" height="max-content"/>. Retrieved 2026-08-29. - Yoga gentest (
main, retrieved 2026-08-29):gentest/gentest-driver.tsusesselenium-webdriverwith--force-device-scale-factor=1 --window-position=0,0 --hide-scrollbars(ChromePool.tsadds--headless), loadstest-template.html(Ahem via@font-face,font: 10px/1 Ahem, every elementdisplay: flex; flex-direction: column; align-items: stretch), and reads results from console lines prefixedgentest-log:.src/buildLayoutTree.tsusesgetBoundingClientRect()rounded asMath.round(right) - Math.round(left). Emitters writetests/generated/*.cpp(ASSERT_FLOAT_EQ),java/tests/generated/**/*.java(assertEquals(..., 0.0f)) andjavascript/tests/generated/*.test.ts(toBe): exact equality.gentest/gentest.jsandgentest/README.mdno longer exist. - Playwright:
locator.boundingBox()returns{x, y, width, height}relative to the main-frame viewport or null if not visible;page.evaluate()returns the function result including-0,NaNand infinities;browser.newContext()setsdeviceScaleFactor(default 1),viewport(default 1280×720),reducedMotion,colorScheme,locale,timezoneId; screenshots wait fordocument.fonts.readyunlessPW_TEST_SCREENSHOT_NO_FONTS_READYis set (packages/playwright-core/src/server/screenshotter.ts,main). https://playwright.dev/docs/api/class-locator, class-page, class-browser, retrieved 2026-08-29. - Playwright
toHaveScreenshot:threshold0.2 (YIQ perceived colour difference),maxDiffPixels,maxDiffPixelRatio,animations: "disabled",caret: "hide",scale: "css",mask; comparison uses pixelmatch. https://playwright.dev/docs/api/class-pageassertions#page-assertions-to-have-screenshot-1 and https://playwright.dev/docs/test-snapshots, retrieved 2026-08-29. - R2Z2: cross-version differential fuzzing of Chrome with a Domato-derived grammar; screenshots compared by 4,096-bit pHash Hamming distance with threshold 140; bisection finds the culprit commit; an interoperability oracle treats Firefox agreement as correctness (bug only when old Chrome equals Firefox and new Chrome differs); a non-feature-update oracle excludes commits that add WPT tests; stage analysis compares DOM, style, layout (“same size and location” per node) and paint records. 22,629 candidates yielded 13 confirmed regressions, 11 new. Song et al., ICSE 2022, DOI 10.1145/3510003.3510044, §4–§6 (PDF https://lifeasageek.github.io/papers/suhwan-r2z2.pdf, retrieved 2026-08-29).
- Mesbah and Prasad: cross-browser oracle at trace level (state-graph isomorphism) then screen level (DOM diff via XMLUnit ignoring case, whitespace, attribute order, text values, plus configurable ignore patterns); screen-level false positives ranged from 12% to 37%. ICSE 2011, §4.3, §5, §6 (PDF https://www.cs.columbia.edu/~junfeng/12fa-e6121/papers/browser-compat.pdf, retrieved 2026-08-29).
- X-PERT classifies cross-browser issues as structure, content (text, visual) and behaviour; structure is compared with an alignment graph of contains and sibling relations (left-align, above, leftOf) because users notice relative position rather than absolute size; visual content uses χ² colour histograms on leaf elements; 98 true issues at 76% precision. Roy Choudhary, Prasad, Orso, ICSE 2013, §IV, §VI, §VII (PDF http://shauvik.com/public/pubs/roychoudhary13icse_cr.pdf, retrieved 2026-08-29).
- WPT reftests pass only if test and reference render “pixel-for-pixel identically within a 800x600 window”; fuzzy syntax
<meta name=fuzzy content="maxDifference=15;totalPixels=300">, ranges10-15;200-300inclusive, per-reference prefixoption1-ref.html:...; screenshots are taken after load, web fonts and pending paints. https://web-platform-tests.org/writing-tests/reftests.html, retrieved 2026-08-29.
Mechanism
differential_run(fixture, context, impls, policy):
outcomes = { impl: run(impl, fixture, context) for impl in impls } # boxes or image + status
if any(o.status in {crash, timeout}): return classify_abnormal(outcomes)
ref = policy.reference or majority(outcomes) # Csmith voting
for impl, o in outcomes:
d = compare(o, ref, policy.tolerance) # per-box abs diff or perceptual
record(impl, d.kind, d.max_delta, d.entities)
return divergences filtered by policy.known_gaps # feature-gap filter
Oracle classes (attributed):
- Reference implementation: a browser is ground truth for CSS semantics (Taffy, Yoga gentest; X-PERT uses one browser as reference).
- Alternative implementation, N-version: several engines with no designated truth (McKeeman; Mesbah and Prasad; R2Z2 change detector across versions).
- Majority vote: with three or more implementations the minority is suspect (Csmith §2.1).
- Interoperability consensus: agreement of independent engines is taken as correct; a change that breaks agreement is a regression (R2Z2 §4.3.1).
- Self-consistency: the same engine must agree with itself across equivalent encodings (Taffy’s border-box/content-box × ltr/rtl variants; WPT reftests; metamorphic relations in nuif:research:metamorphic-testing-graphics).
Tolerance policy (attributed):
- Boxes: absolute difference below 0.1 px after optional rounding (Taffy
tests/xml.rs); exact afterMath.round(Yoga). Sub-pixel rounding must be part of the fixture contract (data-test-rounding,use-rounding). - Images: per-channel
maxDifferenceandtotalPixelsbudgets (WPT), YIQ or OKLab threshold plusmaxDiffPixels(Playwright, pixelmatch), pHash distance for coarse triage (R2Z2). - Structure: relative alignment relations instead of absolute coordinates when the target is a different engine with its own rounding (X-PERT).
Divergence classification (attributed):
- Abnormal: crash, hang, abend in some implementations (McKeeman; Csmith compile-time crash and timeouts).
- Structural mismatch: tree or alignment-graph differences (Mesbah; X-PERT; R2Z2 DOM stage).
- Numeric within tolerance: accepted and recorded with the observed maximum delta.
- Numeric beyond tolerance: bug candidate, reduced before reporting (McKeeman p. 105; Csmith §3.7).
- Feature gap: the reference implements semantics the system under test does not claim, or vice versa; filtered by known-gap lists (R2Z2 non-feature oracle; Csmith equivalence classes; Mesbah ignore patterns; Taffy
x-prefixed fixtures). - Silent wrong output: plausible boxes with no diagnostic; the analogue of Csmith’s silent wrong-code error and the class NUIF’s fidelity reports must make impossible.
NUIF relevance
Borrow
- The Taffy gentest pipeline (WebDriver-driven Chrome, Ahem,
#test-root, four box-model/direction variants, XML fixtures, 0.1 px tolerance) as the template for NUIF’s browser-referencedlayoutfixtures; Playwright can replace fantoccini with the samegetBoundingClientRectextraction anddocument.fonts.readywait. - Csmith’s rule that generated inputs must have a single defined meaning: the NUIF document generator must avoid authored constructs whose lowering to CSS is
approximatedorunsupported, or the comparison is not sound (Csmith §2.2;spec/04-layout.mdfidelity records). - McKeeman’s outcome buckets (crash, loop, abend, diff) and Csmith’s silent-wrong-output class as the top level of the divergence taxonomy in the machine-readable report.
- WPT fuzzy syntax as the declared per-fixture tolerance format for render comparisons.
Adapt
- The browser is a reference only for CSS-compatible families; for
freeform,constraintandcustomfamilies the oracle must be self-consistency or a second NUIF implementation, so the report must record the oracle class per fixture. - Round-trip differential testing compares NUIF → HTML/CSS export → browser boxes against NUIF resolved boxes, which tests the adapter and the evaluator jointly; disagreement must be attributed to a pipeline stage as R2Z2 does (DOM, style, layout, paint).
- Rounding: NUIF should compare unrounded boxes with an epsilon and treat device-pixel snapping as a separate, declared step rather than adopting Yoga’s exact-after-rounding policy.
Reject
- Majority voting across engines is not applicable while only one NUIF evaluator exists; voting becomes relevant once a second independent implementation exists (roadmap governance item).
- Yoga’s zero-tolerance assertions, because NUIF resolved geometry is f64 and browsers expose rounded layout.
Open questions
- Which Chrome version should be pinned as reference, and how are reference-side regressions distinguished from NUIF regressions without a second browser (R2Z2’s interoperability oracle suggests adding Firefox or WebKit)?
- How should the feature-gap filter be derived automatically from the
FidelityReportof the export so that onlylosslessorrepresentableentities are compared? - Is the 0.1 px tolerance adequate for percentage and
fit-contentsizing at 1440 px viewports, or should tolerance scale with box size?
Design Tokens Format Module 2025.10
Document status:
reviewed. Canonical source.
Summary
The Design Tokens Format Module 2025.10 defines JSON interchange for typed
design tokens, hierarchical groups, aliases, group extension and
vendor-specific metadata. A token is identified structurally by an object with
$value; its name is the containing object key.
Evidence
- §4 assigns
application/design-tokens+jsonand recommends.tokensor.tokens.json. A conforming file remains JSON. https://www.w3.org/community/reports/design-tokens/CG-FINAL-format-20251028/#file-format (retrieved 2026-08-29). - §5.1 requires a name and
$value. Names are case-sensitive, cannot begin with$, and cannot contain{,}or.because those characters are reserved by the reference syntax. §5.2 requires an unambiguous type and prohibits type inference from the value. https://www.w3.org/community/reports/design-tokens/CG-FINAL-format-20251028/#name-and-value (retrieved 2026-08-29). - §5.2.3 and §6.3.2 require processors to preserve unknown
$extensionsdata. Reverse-domain keys are recommended to reduce collisions. https://www.w3.org/community/reports/design-tokens/CG-FINAL-format-20251028/#extensions-0 (retrieved 2026-08-29). - §6 defines hierarchical groups, inherited
$type,$root,$extends, empty groups and cycle/error handling. §7 defines curly-brace and JSON Pointer references, chained resolution and circular-reference rejection. https://www.w3.org/community/reports/design-tokens/CG-FINAL-format-20251028/#groups and https://www.w3.org/community/reports/design-tokens/CG-FINAL-format-20251028/#aliases-references (retrieved 2026-08-29). - §8 defines primitive and composite types. A conforming processor must validate the value against the resolved type rather than preserving an untyped JSON value as if it were interoperable. https://www.w3.org/community/reports/design-tokens/CG-FINAL-format-20251028/#types (retrieved 2026-08-29).
NUIF relevance
Borrow the media type, name grammar, explicit type system, alias resolution, group model, cycle rejection and unknown-extension preservation rules.
Adapt DTCG paths to stable NUIF token identity by storing an EntityId in a
reverse-domain $extensions entry for generated files. A first bounded profile
can map boolean, string and finite number tokens exactly. It must retain the
declared DTCG type and distinguish an alias from its resolved value.
Reject full DTCG conformance with the current Token { id, name, value }
model. The model lacks declared type, description, deprecation, group identity,
group extension, alias syntax and token-local opaque extensions. These fields
require a token-model RFC or an adapter-owned retentive package before a full
round trip can be claimed.
egui, eframe, egui-wgpu and the egui_kittest headless harness
Document status:
reviewed. Canonical source.
Summary
egui is an immediate-mode GUI library for Rust; the application closure re-declares the whole UI every frame and receives a FullOutput containing shapes, platform output and an AccessKit TreeUpdate. eframe is the official native/web application framework and egui-wgpu is the wgpu render integration. egui_kittest is a headless test harness that runs the egui Context without a window, feeds the per-frame AccessKit update into kittest (an accessibility-tree query library built on accesskit_consumer), lets tests locate nodes by label, role or value, synthesises pointer/keyboard events or dispatches AccessKit actions, and optionally rasterises the frame with egui-wgpu on a CPU adapter for image snapshot comparison. The harness is used in egui’s own CI with git LFS snapshot storage, but the toolkit’s minimum supported Rust version (1.95 at tag 0.36.1) exceeds the NUIF pin (1.85.0), and the text stack and docking layers are outside egui core.
NUIF interpretation: egui_kittest is the closest existing implementation of the “AccessKit tree as semantic test surface” pattern. Its run() convergence loop, kittest.toml thresholds, UPDATE_SNAPSHOTS regeneration protocol and CPU-adapter preference are directly reusable design elements regardless of which toolkit NUIF adopts.
Evidence
- egui 0.36.1 was released 2026-08-07; 0.36.0 on 2026-08-05; 0.35.0 (2026-06-25) introduced the inspection protocol and
egui_mcp. Locator:CHANGELOG.mdlines 17, 21, 77-97, main branch, retrieved 2026-08-29. - The tagged workspace manifest sets
rust-version = "1.95",wgpu = "30.0",kittest = "0.4.0",accesskit = "0.24.1",accesskit_consumer = "0.35.0"with a comment that kittest 0.4 pins accesskit_consumer 0.35 and blocks upgrades. Locator:Cargo.tomllines 27, 73-75, 103, 160 at tag0.36.1. - Crates and versions on crates.io (2026-08-29): egui 0.36.1, egui_kittest 0.36.1, eframe 0.36.1, egui-wgpu 0.36.1, epaint 0.36.1, egui_extras 0.36.1, kittest 0.4.0 (2026-03-24), egui_dock 0.21.1 (2026-08-06), egui_tiles 0.17.1 (2026-08-18). Locator: crates.io API
/api/v1/crates/<name>. - egui describes itself as “a simple, fast, and highly portable immediate mode GUI library for Rust” that “runs on the web, natively”, with “Accessibility via AccessKit” and epaint as “A simple 2D graphics API for custom painting”. Locator:
README.mdlines 22, 98, 130. - egui_kittest features:
wgpu(pulls egui-wgpu, pollster, image, wgpu with metal/dx12/vulkan/gles; comment “Enable DX12 because it always comes with a software rasterizer.”),snapshot(dify, image/png),eframe. Locator:crates/egui_kittest/Cargo.tomllines 20-44, tag 0.36.1. - Harness surface:
Harness::new_ui,new_ui_state,new_eframe,builder(),step,run,try_run,run_ok,run_steps,try_run_realtime,fit_contents,set_size,set_pixels_per_point,input_mut,output,kittest_state,event,key_down/up/press,key_combination,hover_at,drag_at,drop_at,mask,render,root,spawn_eframe_app. Locator:crates/egui_kittest/src/lib.rslines 188-905, tag 0.36.1 (identicalpub fnset on main). - Per-step data flow:
ctx.run_ui(input, |ui| app.run(...)), thenself.kittest.update(output.platform_output.accesskit_update.take().expect("AccessKit was disabled")). Locator:src/lib.rslines 259-275. run()loopsstep()untilrepaint_delay != Duration::ZERO(no immediate repaint requested) and returns the step count; exceedingmax_stepsyieldsExceededMaxStepsErrorcarryingrepaint_causes. Locator:src/lib.rslines 327-375.- Node interaction:
click()synthesisesPointerMovedandPointerButtonpress/release at the node rect centre;click_accesskit()dispatchesaccesskit::ActionRequest { action: Action::Click }and “can also click widgets that are not currently visible”;focus()usesAction::Focus;scroll_to_me()usesAction::ScrollIntoView;type_text,value,is_focusedexist. Locator:crates/egui_kittest/src/node.rslines 56-200, main. - kittest queries:
Queryabletrait generatesget_by_label,get_by_label_contains,get_by_role,get_by_role_and_label,get_by_value,get_by(predicate)plusquery_*,get_all_*,query_all_*variants over aByfilter; label-by nodes are excluded from label matches. Locator: kittestsrc/query.rslines 146-236, main commit bd19226 (2026-08-03). - kittest depends only on
accesskit = "0.24.1"andaccesskit_consumer = "0.38.0"(main); its README states it is “inspired by Testing Library” and framework-agnostic. Locator: kittestCargo.tomllines 24-27,README.md. - Snapshot protocol:
Harness::snapshot(name),try_snapshot,snapshot_options,SnapshotOptions { threshold, max_failed_pixels, output_path }with per-OSOsThreshold,SnapshotResultsaggregation,UPDATE_SNAPSHOTS=true(update failing only) orforce;.new.png,.diff.png,.old.pngside files. Locator:src/snapshot.rslines 13-950; README “Snapshot testing”. kittest.tomldefaults:output_path = "tests/snapshots",threshold = 0.6(weighted squared YIQ distance per pixel),max_failed_pixels = 0, optional[windows]/[macos]/[linux]overrides. Locator: README “Configuration”.- wgpu test renderer:
default_wgpu_setup()callsWgpuSetupCreateNew::without_display_handle(), removesBackends::BROWSER_WEBGPU, and sorts adapters soDeviceType::Cpuranks first;WAIT_TIMEOUTis 10 s and the comment names lavapipe. Locator:crates/egui_kittest/src/wgpu.rslines 10-58. - README enumerates cross-machine image differences (MSAA sample placement, texture filtering, WGSL floating-point evaluation, derivative variants) and recommends disabling MSAA and avoiding NaN/Inf. Locator: README “What to do when CI / another computer produces a different image?”.
- README guidance: “prefer regular Rust tests or
instasnapshot tests over image comparison tests”; images should be checked in via git LFS at low resolution. Locator: README “Guidelines for writing snapshot tests”. - egui CI checks out with
lfs: trueand uploads**/tests/snapshotsas artifacts. Locator:.github/workflows/rust.ymllines 18, 228, 247, main. PainterAPI:new(ctx, layer_id, clip_rect),with_clip_rect,set_opacity,add(Shape) -> ShapeIdx,set(idx, shape),extend,rect_filled,rect_stroke,line,circle,image,text,layout,layout_no_wrap,round_to_pixel_center. Locator:crates/egui/src/painter.rslines 47-503, tag 0.36.1.- egui_dock: tabs, moving tabs between nodes, dragging tabs into new windows, programmatic layout manipulation; badge targets egui 0.36. egui_tiles: horizontal/vertical/grid layouts, tabs, drag-and-drop docking,
unsafeforbidden. Locator: respectiveREADME.mdfiles, main.
Mechanism
Frame loop and tree extraction:
#![allow(unused)]
fn main() {
// egui_kittest/src/lib.rs (0.36.1), simplified
fn step_impl(&mut self, sizing_pass: bool) {
self.input.predicted_dt = self.step_dt;
let mut output = self.ctx.run_ui(self.input.take(), |ui| {
self.response = self.app.run(ui, &mut self.state, sizing_pass);
});
self.kittest.update(output.platform_output.accesskit_update.take()
.expect("AccessKit was disabled")); // accesskit::TreeUpdate -> accesskit_consumer::Tree
self.renderer.handle_delta(&mut output.textures_delta);
self.output = output;
self.handle_viewport_commands(); // InnerSize, Screenshot
}
}
Invariants: AccessKit must be enabled on the Context; every frame produces a complete TreeUpdate; a Node handle borrowed from the harness is invalidated by the next step(), so tests re-query after run(). Events queued on nodes are drained one per frame by step().
Typical test:
#![allow(unused)]
fn main() {
let mut harness = Harness::new_ui(|ui| { ui.checkbox(&mut checked, "Check me!"); });
let cb = harness.get_by_label("Check me!");
assert_eq!(cb.accesskit_node().toggled(), Some(Toggled::False));
cb.click(); // or cb.click_accesskit() for Action::Click
harness.run(); // converge until no repaint requested
harness.fit_contents();
harness.snapshot("readme_example"); // needs features wgpu + snapshot
}
Rendering path for snapshots: Harness::render() clones FullOutput, optionally paints a cursor triangle, and calls WgpuTestRenderer::render(&ctx, &output) -> RgbaImage; the renderer is created without a display handle on the first CPU adapter found (lavapipe, WARP via DX12, or a GPU fallback). Comparison uses dify with the YIQ per-pixel threshold and an absolute failed-pixel budget.
Custom canvas painting: an editor canvas is an egui::Painter obtained from ui.painter() or Painter::new; shapes are epaint::Shape values (paths, rects, meshes, text galleys). Text layout is epaint’s own glyph atlas; custom fonts are installed via Context::set_fonts (README line 252). Complex script shaping and bidirectional layout are not provided by epaint (NUIF interpretation from the API surface; not verified against an egui issue in this retrieval).
NUIF relevance
Borrow
- The harness contract: run-to-convergence with a bounded step count and reported repaint causes (
ExceededMaxStepsError), because NUIF QA item 6 requires deterministic snapshots and a bounded loop makes non-convergence a test failure rather than a hang. - The dual action path (
click()synthesises pointer input;click_accesskit()dispatchesAction::Clickto a possibly invisible node), because QA.md mandates testing “without synthetic mouse input” while GUI wiring tests still need pointer simulation. - The
kittest.tomlper-OS threshold table,UPDATE_SNAPSHOTS=true|forcesemantics and.new/.diff/.oldside files, because they encode a regeneration protocol that already survived CI use at egui scale. - CPU-adapter-first wgpu selection without a display handle, because ADR 0003 retains a CPU/reference path for conformance and this is a concrete implementation of the same policy.
Adapt
- Node queries should target NUIF entity identity (
EntityId, name, role) exposed through AccessKitauthor_id/html_idor a NUIF-owned semantic tree, because kittest’s label/role queries alone cannot express QA item 2 (identity/type/name/relationship queries). - Snapshot storage must be moved from image files toward structured
RenderScenesnapshots (insta) with image comparison only for the renderer conformance suite, because the README itself ranks image tests as slow and brittle. - egui_dock or egui_tiles can host panels, but panel layout must remain ephemeral shell state (ARCHITECTURE.md), so docking state must not be serialised into NUIF documents.
Reject
- Adopting egui 0.36 under the current toolchain pin, because
rust-version = "1.95"contradictsrust-version = "1.85"inCargo.tomlandtoolchain: 1.85.0in.github/workflows/ci.yml; either the pin moves or the toolkit is excluded. - Using epaint text as the text oracle for NUIF text diagnostics (QA item 5), because NUIF’s text semantics target HarfBuzz-compatible shaping (whitepaper section 06) and epaint does not expose a shaping pipeline.
- Depending on the kittest/accesskit_consumer version lock (egui
Cargo.tomllines 74-75) in the engine crates, because the engine must stay free of GUI-toolkit version coupling.
Open questions
- Whether the NUIF project will raise its MSRV to track egui (1.95) and kittest (1.95 on main), or freeze on an older egui release line compatible with 1.85.
- Whether the egui inspection protocol (0.35) and
egui_mcpcan serve as the “local automation endpoint” required by ARCHITECTURE.md, or whether that would make MCP the canonical contract, which ARCHITECTURE.md forbids. - Whether wgpu CPU adapters (lavapipe, WARP) yield bit-identical output across CI hosts for NUIF’s render conformance tolerances, or whether a
vello_cpu/tiny-skia reference rasteriser is still required. - Whether epaint’s shape model can carry NUIF
RenderScenecommands losslessly (gradients, clipping, images at explicit scale) or whether a Vello scene should be composited into the egui frame as a texture.
Deterministic CBOR, Protobuf unknown fields and Kiwi schema evolution
Document status:
reviewed. Canonical source.
Summary
RFC 8949 defines deterministic CBOR encoding profiles suitable for hashing and reproducible binary forms. Protobuf demonstrates mature field-number evolution and binary unknown-field preservation but warns that JSON conversion loses unknown fields. Kiwi demonstrates schema-bundled forward decoding and compact tree serialization.
NUIF relevance
Separate logical schema from encoding. Use a canonical human-readable form for review/spec fixtures plus deterministic CBOR as the first binary/wire encoding. Unknown extensions must be represented explicitly rather than relying solely on codec-specific unknown field behavior.
The executable comparison and the current Protobuf, FlatBuffers and Cap’n Proto
admission decision are maintained in
schema-codec-admission-and-benchmarking.md rather than inferred from vendor
microbenchmarks.
EPUB 3.3 OCF package and resource-manifest discipline
Document status:
reviewed. Canonical source.
Summary
EPUB 3.3 separates an abstract publication container from its physical OCF ZIP
representation. Publication resources are declared in a package manifest and
are normally carried inside the container; remote resources are a deliberate
exception. OCF narrows ZIP to an interoperable, inspectable subset and reserves
an uncompressed first mimetype member for early format identification.
NUIF can borrow the package discipline without borrowing EPUB’s publication model. The important precedent is that a portable document does not depend on an unconstrained filesystem or arbitrary ZIP behavior: it has a manifest, well-defined members, restricted compression and explicit external resources.
Evidence
- EPUB 3.3 §3.3 requires publication resources to be listed in the package document manifest and normally bundled in the container. §3.6 defines remote resources as a distinct resource-location case.
- §4.2 defines one rooted abstract filesystem, reserves
mimetype, and keeps container configuration separate from publication resources. - §4.3.2 prohibits split or spanned archives and ZIP encryption, permits only stored and Deflate entries, and requires UTF-8 file names.
- §4.3.3 requires
mimetypeto be the first member, stored and unencrypted, with no extra field or surrounding whitespace. The value identifies the package before processing the rest of the archive. - §4.4 treats embedded-font handling as an explicit package concern rather than assuming that every font may be redistributed.
Mechanism
An OCF reader identifies the archive from a fixed first member, validates the ZIP profile, resolves one package document, then obtains the declared resource set from its manifest. Physical member paths locate bytes; the package document defines their semantic role. Those two responsibilities are not conflated.
For NUIF this suggests a similarly narrow ZIP envelope:
mimetype fixed first stored member
manifest.cbor descriptors and semantic roots
document.cbor canonical semantic document
blobs/sha256/<hex-digest> immutable resource bytes
The path is a package locator, not the resource identity. The manifest binds a media type, byte length and digest to every resource before a decoder consumes it.
NUIF relevance
Borrow the abstract/physical container separation, first-member media-type identification, mandatory manifest, rooted paths, UTF-8 names and small ZIP feature subset.
Adapt member identity to content digests and separate the semantic document hash from the complete package hash. Required portable resources are embedded; linked resources remain explicit, digest-pinned and unavailable without an opt-in resolver.
Reject publication reading order, EPUB-specific metadata, font obfuscation, container XML and any assumption that a remote URL is stable resource identity.
Open questions
- Should the first NUIF package profile permit Deflate at all, or use stored blobs initially so package hashing and resource limits remain simplest?
- Which fixed ZIP metadata fields are required for byte-reproducible packages across independent writers?
- Does streaming import require the manifest before
document.cbor, or is the fixed ordering only an authoring recommendation aftermimetype?
Figma engineering on multiplayer synchronization, fractional indexing and the custom renderer
Document status:
reviewed. Canonical source.
Summary
Figma’s engineering posts describe the document as a two-level map Map<ObjectID, Map<Property, Value>> synchronized over a WebSocket to a per-document server process. Concurrency control is per-property last-writer-wins ordered by server arrival; the server is authoritative and rejects parent updates that would form a cycle; clients apply local edits optimistically and discard incoming values that conflict with unacknowledged local writes. Child order is a fractional index stored together with the parent link so both change atomically; indices are arbitrary-precision base-95 strings. Undo is per-user and rewrites redo history so that undo-copy-redo leaves the document unchanged. The editor is C++ compiled to WebAssembly with a custom tile-based WebGL renderer (and, since 2025, a WebGPU path with dynamic fallback), chosen over DOM, SVG and Canvas 2D for consistency and retained-mode performance. Performance is guarded by headless per-pull-request benchmarks with a 20 percent regression margin.
Evidence
- Document structure: a single root, page objects beneath it, and per page “a hierarchy of objects”; conceptual model
Map<ObjectID, Map<Property, Value>>. E. Wallace, “How Figma’s multiplayer technology works”, 2019-10-16, retrieved 2026-08-29. - Conflict rule: servers “keep track of the latest value that any client has sent for a given property on a given object”; concurrent edits to the same property yield “the last value that was sent to the server”; text edits are whole-value (“either AB or BC but never ABC”). Same post.
- Operational transformation (OT) rejected as “very complicated and hard to implement correctly” for the requirements; design goal was to be “no more complex than necessary”. Same post.
- Optimistic client rule: unacknowledged local changes are the best prediction, so clients “discard incoming changes from the server that conflict with unacknowledged property changes”. Same post.
- Cycles: servers “reject parent property updates that would cause a cycle”; a client may transiently hold both an unacknowledged reparent and a conflicting server reparent; Figma temporarily removes the affected objects from the tree “until the server rejects the client’s change”. Same post.
- Ordering: position “is represented as a fraction between 0 and 1 exclusive”; insertion sets the average of neighbours; “The parent link and this position must both be stored as a single property so they update atomically”. Same post.
- Undo: the principle “If you undo a lot, copy something, and redo back to the present … the document should not change”; “an undo operation modifies redo history at the time of the undo, and likewise a redo operation modifies undo history”. Same post; the principle is first stated in E. Wallace, “Multiplayer Editing in Figma”, 2016-09-28, retrieved 2026-08-29.
- Offline: arbitrary offline editing; on reconnect the client “downloads a fresh copy of the document, reapplies any offline edits on top” and resumes over a new WebSocket. 2019 post.
- Architecture: “a separate process for each multiplayer document”; multiplayer server ported to Rust for lower latency and memory; “Serialization time is now over 10x faster”; problems listed with lifetimes, error stack traces, immature compression libraries and futures. E. Wallace, “Rust in Production at Figma”, 2018-05-02, retrieved 2026-08-29.
- Fractional indexing detail: indices as strings with “averaging … done using string manipulation”; leading
0.omitted and full printable ASCII used (“base 95 instead of base 10”); the server assigns a unique position to the second of two identical inserts; index length grows with repeated edits; concurrent inserts may interleave. E. Wallace, “Realtime Editing of Ordered Sequences”, 2017-03-06, retrieved 2026-08-29. - Renderer rationale: HTML/SVG “often much slower than the 2D canvas API due to DOM access”; Canvas 2D “is an immediate mode API instead of a retained mode API so all geometry has to be re-uploaded … every frame”; text layout “inconsistent between browsers”; missing features such as angular gradients; result is “a highly-optimized tile-based engine” in WebGL with masking, blurring, dithered gradients, blend modes, nested layer opacity; “a browser inside a browser” with own DOM, compositor and text layout; C++ via emscripten with compact 32-bit floats. E. Wallace, “Building a professional design tool on the web”, 2015-12-07, retrieved 2026-08-29.
- WebAssembly: load time “improved by more than 3x … regardless of document size”; wasm “parses around 20x faster than asm.js”. E. Wallace, “WebAssembly cut Figma’s load time by 3x”, 2017-06-08, retrieved 2026-08-29.
- Renderer restructuring yielded up to 3x faster load, zoom and drag; tracked metrics are average frame time and maximum frame time. J. Wong, “Figma, faster”, 2018-08-13, retrieved 2026-08-29.
- WebGPU: shipped in Chromium in 2023; enables compute shaders (planned blur optimisation) and avoids WebGL’s “bug-prone global state”; “a dynamic fallback system” starts on WebGPU and falls back to WebGL on asynchronous test failure or mid-session failure. A. Ringlein, L. Anderson, “Figma Rendering: Powered by WebGPU”, 2025-09-18, retrieved 2026-08-29.
- Performance continuous integration (CI): benchmarks run “in GPU-enabled virtual machines, in a headless Chromium process on every code change in every pull request” with “20% pass margin”; scenarios include local edits and “a stream of simulated multiplayer changes” (e.g. 100 simulated editors); a hardware lab handles precise cases; CPU profiles are captured per run; rendering prioritises local edits over remote changes. S. Kim, L. Woods, “Keeping Figma Fast”, 2023-08-29, retrieved 2026-08-29.
Mechanism
Synchronization. Each object is a set of registers keyed by property name. A client edit produces (objectID, property, value) messages applied locally at once and buffered as unacknowledged. The server serializes all messages per document, applies last-writer-wins per register, persists, and broadcasts. On receipt, a client applies a server value unless it holds an unacknowledged write to the same register, in which case the server value is dropped because the local write will arrive later in server order. Tree structure is a register too: parent and position are one composite value, so a reparent is a single register write and cannot be split by concurrent edits. Acyclicity is a server-side precondition on parent writes; the client’s transient inconsistency is contained by detaching the involved subtree until the rejection arrives. Reconnection is state-based: reload, then replay the local unacknowledged log; there is no operation log merge beyond that.
Ordering. position is a string over a 95-symbol alphabet interpreted as a fraction in (0, 1). Insertion between a and b chooses the shortest string strictly between them (by averaging with string arithmetic); the server perturbs duplicates. This makes reorder a single-register write and avoids index shifts, at the cost of unbounded string growth and interleaving under concurrent insertion at the same gap.
Undo. Undo is per user and operates on the user’s own history; undoing a property write re-writes the previous value as a new write (so the server sees it as a normal last-writer-wins update), and the redo stack is rewritten at undo time to reflect the state that the undo produced. The invariant is idempotence of undo-then-redo sequences with respect to the document.
Rendering. The document is retained in WebAssembly memory with compact numeric types. The renderer rasterizes into tiles on the GPU, so partial invalidation redraws only affected tiles and pan/zoom reuse cached tiles; text is laid out by an in-house engine to avoid cross-browser divergence. WebGPU adds compute passes; a runtime feature probe selects the backend and can downgrade mid-session. Performance regressions are caught by scripted scenarios in headless Chromium under GPU virtual machines.
NUIF relevance
Borrow
- Property-level registers with server-ordered last-writer-wins as the default conflict policy for scalar authored properties in the collaboration profile, with the explicit consequence that concurrent edits to one value are not merged (spec/10).
- Parent and order stored as one atomic value; NUIF
move/reorder entityshould be one operation carrying both target parent and fractional position. - Fractional indexing in a printable alphabet for sibling order in the operation log, with server-side deduplication of identical positions.
- Acyclicity as a precondition enforced at the authority, and a defined client-side containment strategy for transiently invalid states.
- Per-pull-request headless GPU benchmark scenarios with a fixed regression margin as a model for
nuif-renderand layout performance gates in conformance.
Adapt
- NUIF undo is defined as inverse semantic operations rather than value rewrites; Figma’s “undo rewrites redo history” invariant should be stated as a testable property of the operation log rather than a client implementation detail.
- Offline reconciliation by reload-and-replay is acceptable only if replayed operations are the same typed operations as live ones and preconditions (spec/06) surface conflicts instead of silently overwriting.
- Tile-based GPU rendering is a renderer strategy, not a document property; NUIF’s resolved layer must remain renderer-independent.
Reject
- Treating text content as a single register; NUIF text runs need sequence semantics or explicit conflict objects for concurrent edits.
- Undocumented wire protocol as an integration boundary; NUIF keeps the collaboration profile normative and materializable to canonical snapshots (ADR 0005).
- Renderer-specific text layout as the canonical result; NUIF resolved text diagnostics must be attributable to a declared shaping context, not to one engine’s behaviour.
Open questions
- How Figma handles register writes to deleted objects and whether tombstones are retained; not covered by the retrieved posts.
- Whether the fractional index ever gets rebalanced (reindexing all siblings) and how that interacts with concurrent edits.
- Whether the WebGPU path changes the tile strategy or only the blur/compute stages; the 2025 post gives no rendering detail.
Figma Plugin API and REST API as evidence for a programmable, testable editor surface
Document status:
reviewed. Canonical source.
Summary
Figma exposes two programmatic surfaces. The Plugin API runs inside an open editor session: a sandboxed JavaScript main thread manipulates the document through a global figma object (node creation, selection, properties, events, export, undo grouping, per-node plugin data), while an optional iframe hosts UI and browser APIs. The REST API runs outside the editor without a user present; it is “largely read-only” for design content, returning the node tree as JSON, rendering nodes to images, and, on Enterprise plans, reading and writing variables. Figma’s documentation states that plugins cannot run in the background, that a user must initiate them, and that only one plugin runs at a time; no headless plugin execution is offered. A newer MCP server (remote or desktop) allows agents to read design context and create native content, but only through catalogued clients.
NUIF interpretation: Figma proves that a design editor’s entire semantic surface can be exercised without pointer input, which is the premise of apps/editor/QA.md. Its execution model also shows the gap NUIF must close: the programmable surface is bound to a running GUI session, so headless conformance testing is impossible against Figma itself.
Evidence
Retrieval date for all locators: 2026-08-30.
figma.createFrame(): FrameNode— “similar to using the F shortcut followed by a click”; the frame defaults to 100×100 with a white background and is parented tofigma.currentPage. https://developers.figma.com/docs/plugins/api/properties/figma-createframe/.- The global object exposes
currentPage: PageNode(settable),root: DocumentNode,editorType(‘figma’ | ‘figjam’ | ‘dev’ | ‘slides’ | ‘buzz’),mode(‘default’ | ‘textreview’ | ‘inspect’ | ‘codegen’ | ‘linkpreview’ | ‘auth’),create*constructors (Frame, Rectangle, Ellipse, Polygon, Star, Text, Component, Page, Section),getNodeByIdAsync,loadAllPagesAsync,on/off/once,commitUndo,triggerUndo,notify,closePlugin,viewport,ui,clientStorage,variables,teamLibrary,skipInvisibleInstanceChildren. https://developers.figma.com/docs/plugins/api/figma/. PageNode.selection: ReadonlyArray<SceneNode>; “Each page stores its own selection separately”; order unspecified;selectedTextRange;loadAsync()required under dynamic page loading. https://developers.figma.com/docs/plugins/api/PageNode/. Whether assignment toselectionis permitted was not confirmed in the retrieved text (unverified; the older URL /properties/figma-currentpage/ returns 404).setPluginData(key: string, value: string): void— entry (pluginId, key, value) limited to 100 kB; private to the plugin ID; privacy is “for stability, not security”; empty string deletes the key. https://developers.figma.com/docs/plugins/api/properties/nodes-setplugindata/.- Private plug-in data becomes inaccessible if the plug-in ID changes. Shared plug-in data is namespaced, readable by every plug-in and also limited to 100 kB per entry. Locators: https://developers.figma.com/docs/plugins/api/properties/nodes-setplugindata/ and https://developers.figma.com/docs/plugins/api/properties/nodes-setsharedplugindata/.
- New plug-ins must declare
documentAccess: "dynamic-page". The manifest can constrain network requests withnetworkAccess.allowedDomains;["none"]declares no network. Locator: Plugin Manifest, retrieved 2026-08-30: https://developers.figma.com/docs/plugins/manifest/. - The manifest
idis assigned by Figma, normally through Create new Plugin; Figma can also allocate one during publication. A repository can therefore compile a credential-free manifest template, but an importable development manifest requires the reviewer’s assigned ID. The field is specified only as a string, and Figma’s official sample repository includes descriptive sample IDs, so tooling must not impose a numeric-only grammar. Locators, retrieved 2026-08-31: same manifest page and https://github.com/figma/plugin-samples/blob/main/post-message/manifest.json. - Figma recommends loading pages only as needed. Document-wide traversal under
dynamic loading requires explicit page loads, and several
DocumentNodesearches requireloadAllPagesAsync(). Locators: Accessing the Document and Migrating Plugins to Dynamically Load Pages, retrieved 2026-08-30: https://developers.figma.com/docs/plugins/accessing-document/ and https://developers.figma.com/docs/plugins/migrating-to-dynamic-loading/. setSharedPluginData(namespace: string, key: string, value: string): void— readable by all plugins; namespace at least 3 alphanumeric characters; 100 kB limit;getSharedPluginDataKeysenumerates a namespace. https://developers.figma.com/docs/plugins/api/properties/nodes-setsharedplugindata/.- REST
GET /v1/files/:keyqueryplugin_dataaccepts “Comma separated list of plugin IDs and/or the string shared” and addspluginDataandsharedPluginDatato nodes in the response; other parametersversion,ids,depth,geometry=paths,branch_data; response includesdocument,components,componentSets,styles,schemaVersion,version. Tier 1, scopefile_content:read. https://developers.figma.com/docs/rest-api/file-endpoints/. - REST
GET /v1/files/:key/nodes(ids, version, depth, geometry, plugin_data);GET /v1/images/:keyrenders nodes withscale0.01–4,formatjpg/png/svg/pdf,svg_outline_text,svg_include_id,svg_include_node_id,svg_simplify_stroke,contents_only,use_absolute_bounds,version;GET /v1/files/:key/imagesreturns image-fill URLs expiring within 14 days. Same page. - The REST API is “Largely read-only” except comments, comment reactions, variables and dev resources; it operates where “a user does not need to be present”; the Plugin API requires that “A user has a particular Figma design or FigJam file open” and can “only read and edit the current file that a user has open”. https://developers.figma.com/compare-apis/.
- Plugin execution: the main thread runs in an ES2020+ sandbox without the DOM
or the full browser API; UI runs in an iframe created by
figma.showUI()and communicates with the main thread by messages. A plug-in must close when its work ends. Locator: How Plugins Run, retrieved 2026-08-30: https://developers.figma.com/docs/plugins/how-plugins-run/. - Current documentation now exposes a Figma Fetch API in the sandbox, governed by the manifest domain allow-list; UI iframes continue to provide browser APIs. A credential-free NUIF bridge does not require network access. Locator: Making Network Requests, retrieved 2026-08-30: https://developers.figma.com/docs/plugins/making-network-requests/.
- By default, one undo reverses all actions performed by a plug-in run.
figma.commitUndo()partitions later actions into another undo segment. Locator:commitUndo, retrieved 2026-08-30: https://developers.figma.com/docs/plugins/api/properties/figma-commitundo/. - “It’s not possible to build plugins that run in the background”; users run one plugin at a time; actions are “initiated by the user”. https://developers.figma.com/docs/plugins/ — introduction.
- Dev Mode plugins (
editorType: ["dev"], capabilitiesinspect/codegen) are read-only; “setter methods in the Plugin API do not work in Dev Mode” except metadata such aspluginDataandrelaunchData; pages are always dynamically loaded. https://developers.figma.com/docs/plugins/working-in-dev-mode/. figma.onevents: selectionchange, currentpagechange, documentchange, close, run, drop, timer events, stylechange, textreview.documentchangerequires"documentAccess": "dynamic-page"in the manifest and a priorfigma.loadAllPagesAsync(); Figma “will not call the ‘documentchange’ callback synchronously and will instead batch the updates”. https://developers.figma.com/docs/plugins/api/properties/figma-on/.exportAsyncoverloads:(settings?: ExportSettings): Promise<Uint8Array>for PNG/JPG/PDF/SVG bytes;(ExportSettingsSVGString): Promise<string>;(ExportSettingsREST): Promise<Object>returningJSON_REST_V1, the REST-compatible node JSON; MP4/GIF/WEBM for animated top-level frames; default PNG at 1x. https://developers.figma.com/docs/plugins/api/properties/nodes-exportasync/.figma.skipInvisibleInstanceChildren: boolean— defaulttruein Dev Mode,falsein Figma and FigJam; when enabled,children,findAll,findOne,findAllWithCriteriaskip invisible instance descendants andgetNodeByIdAsyncreturns null for them;findAll/findOnebecome “up to several times faster” andfindAllWithCriteria“up to hundreds of times faster in large documents”. https://developers.figma.com/docs/plugins/api/properties/figma-skipinvisibleinstancechildren/.resize(width, height)requires both dimensions to be at least0.01except that a line has height zero. The bounded frame/shape/text profile does not include lines, so0.01is the uniform import/export minimum. Locator:ResizeMixin, retrieved 2026-08-31: https://developers.figma.com/docs/plugins/api/ResizeMixin/.layoutModecurrently admitsNONE,HORIZONTAL,VERTICALandGRID. Changing it can move children and resize the frame; the documented padding, spacing and axis-alignment properties apply to HORIZONTAL or VERTICAL modes. A bounded mapper therefore cannot toggle layout merely to inspect it, and GRID needs a separate profile. Locator:layoutMode, retrieved 2026-08-31: https://developers.figma.com/docs/plugins/api/properties/nodes-layoutmode/.TextNodeexposescharacters,fontName,fontSizeandlineHeight, each of the style properties may befigma.mixed, and writes that affect rendered text require the font to be loaded. The API exposes a family/style identity, not the immutable font-byte SHA-256 required by NUIF. Exact text export must therefore carry previously verified NUIF font metadata or report the font as unsupported. Locators:TextNodeand Working with Text, retrieved 2026-08-31: https://developers.figma.com/docs/plugins/api/TextNode/ and https://developers.figma.com/docs/plugins/working-with-text/.- Variables REST:
GET .../variables/localandGET .../variables/published(scopefile_variables:read),POST .../variables(scopefile_variables:write, edit permission, 4 MB body, up to 5,000 variables per collection, 40 modes per collection); all require an Enterprise organisation. https://developers.figma.com/docs/rest-api/variables-endpoints/. - REST authentication is by personal access token or OAuth2; base URL
https://api.figma.com. https://developers.figma.com/docs/rest-api/. - MCP server: remote (Figma-hosted) or desktop-app server; agents can read variables, components, layout and design context, generate code, and “create and modify native Figma content directly”; only clients in the Figma MCP Catalog can connect. https://developers.figma.com/docs/figma-mcp-server/.
- After initial approval, plug-in updates publish immediately to every user and users cannot select an older version. Rollback requires republishing earlier code as a new update. Locator: plug-in introduction, Versioning, retrieved 2026-08-30: https://developers.figma.com/docs/plugins/.
- Headless execution: no Figma developer page retrieved offers headless or CLI plugin execution. A community feature request confirms plugins cannot auto-run headlessly (secondary source). https://forum.figma.com/suggest-a-feature-11/are-headless-auto-start-figma-plugins-possible-39156.
- Unverified: whether
pluginDatasurvives copy/paste and duplication of nodes; the setPluginData page does not state it. - Unverified: whether
figma.currentPage.selectionis assignable (commonly used in plugin code, but not confirmed in the retrieved text). - Unverified: MCP server plan tiers and write-capability details beyond the quoted summary.
Mechanism
Call surface relevant to a programmable editor, grouped by the QA capabilities in apps/editor/QA.md:
- Create/open:
figma.root,figma.currentPage,figma.createFrame()and siblings,figma.createPage(),loadAllPagesAsync(); RESTGET /v1/files/:keyfor out-of-editor read access. - Query:
node.findAll,findOne,findAllWithCriteria,getNodeByIdAsync,skipInvisibleInstanceChildrenas a traversal filter; RESTids,depth,geometry=paths. - Transact: property setters on nodes,
commitUndo()to group actions into an undo step,triggerUndo();figma.on('documentchange')as a batched change feed. - Selection as state:
PageNode.selection,selectionchangeevent. - Opaque data:
setPluginData(private, keyed by plugin ID) andsetSharedPluginData(namespaced, public), each ≤100 kB per entry, surfaced by REST viaplugin_data=<id>|shared. - Render:
exportAsync(PNG, JPG, SVG, PDF, SVG string, JSON_REST_V1, video); RESTGET /v1/images/:key. - Tokens:
figma.variablesin-editor; REST variables endpoints (Enterprise). - Execution boundary: plugin main thread inside an open file, initiated by a user; REST outside the editor, read-mostly; MCP server as an agent bridge with catalogued clients.
- Delivery boundary: manifest API version and plug-in identifier are host contracts. Updates are global, so a NUIF bridge needs independent semantic versioning, fixture gates and an explicit rollback release.
- Pure mapping boundary:
nuif-figma-plugin-snapshot-0now maps a normalized one-frame subset in both directions and runs through the CLI. It requires visible, fully opaque, fixed-size nodes, packed row/column auto layout and exact pinned-font metadata. This is executable mapping evidence, not proof of page loading, object creation, undo, messaging or persistence in Figma. - Static shell boundary: the pinned official typings, strict local sources,
inline iframe UI and
allowedDomains: ["none"]manifest template compile deterministically. A mock public-API object crosses the TypeScript normalizer and Rust importer incargo xtask gate-figma. The manifest ID remains reviewer-assigned and all runtime host claims remainnot_run.
NUIF relevance
Borrow
- The principle that every inspector control has an API-level property and event, so that tests drive the document through operations rather than pointer input (
nuif:claim:semantic-automation). - Batched, asynchronous change notification (
documentchange) as the model for the editor’s event log and replay capture inrfcs/0004-headless-qa-contract.md. - Undo grouping via an explicit commit (
commitUndo) as the pattern for NUIF transactions with inverse logs. - A traversal flag that skips invisible instance descendants as a documented performance lever for query evaluation in large documents.
- Opaque, size-bounded, namespaced per-node string stores that survive file save and external export, as precedent for extension preservation (
rfcs/0002-extension-preservation.md).
Adapt
- Replace the plugin-ID-keyed private store with NUIF’s dialect/extension namespaces so preserved data is portable rather than tied to a vendor plugin identity (
spec/07-extensions-and-dialects.md). - Make the export-to-JSON path (
JSON_REST_V1) the canonical serialisation rather than a secondary export, because NUIF’s canonical document is the neutral format itself. - Provide the same operation surface in-process and over a local endpoint so the editor is not required to be open (contrast with Figma’s editor-bound plugin runtime).
- Use dynamic page loading, a no-network manifest by default and one undo group
per confirmed import. Store portable identity in a shared
nuifnamespace, while treating host node IDs and duplicate detection as the authoritative correspondence evidence. - Normalize host objects into a bounded, serializable snapshot before invoking the core. This lets credential-free CI test mapping, identity repair and loss reports while leaving the thin host shell and live certification separate.
Reject
- Editor-bound plugin execution with no headless mode; single-plugin-at-a-time and user-initiated constraints; Enterprise-gated variables API; MCP access restricted to a client catalogue; comments and dev-resources write endpoints; FigJam/Slides/Buzz editor types. Reason: the NUIF test editor must be scriptable headlessly and without plan or client gating.
Open questions
- Does Figma document plugin-data persistence across copy, paste, duplicate and component instantiation? Needed to compare with NUIF’s preservation guarantees.
- Is there an official statement on determinism of
exportAsyncoutput (identical bytes for identical documents)? Relevant to snapshot comparison. - Can the MCP server’s write path be characterised as a semantic operation API, and does it expose undo grouping or change events?
Figma Design tools, keyboard shortcuts and canvas interactions relevant to a test editor
Document status:
reviewed. Canonical source.
Summary
Figma publishes its shortcut inventory primarily through an in-app panel (Ctrl Shift ?) rather than a single reference article; the Help Center article on keyboard use documents keyboard navigation, the keyboard box-selection tool and toolbar focus, while individual feature articles document the tool and command bindings. This record consolidates the bindings that a NUIF test editor needs, each tied to the article that states it. Bindings that no retrieved primary source states (zoom to 100%, redo, Move tool V, Slice tool S, big-nudge key) are listed but marked unverified. Modifier mapping is Cmd/Option on macOS and Ctrl/Alt on Windows throughout the Help Center.
NUIF interpretation: the bindings define the pointer-and-keyboard layer that apps/editor/QA.md reserves for shell testing. Every binding below must have a semantic-operation equivalent so that the headless QA contract can execute the same action without synthetic input.
Evidence
Retrieval date for all locators: 2026-08-29. “Snippet” marks claims verified only through the Help Center search excerpt of the named article rather than the full article body.
- Shortcut panel: Ctrl Shift ? on both platforms, also via Help and resources or the actions menu; the panel “appears along the bottom of your screen” with category tabs and a Layout tab for keyboard layout. https://help.figma.com/hc/en-us/articles/360040328653-Use-Figma-products-with-a-keyboard — “View keyboard shortcuts”.
- Keyboard-only canvas: arrow keys pan when nothing is selected, Shift + arrows pan faster; Cmd/Ctrl + or − zooms; F6 (Mac) / Ctrl F6 (Windows) focuses the toolbar; keyboard box selection is Option Space / Ctrl Space; objects needing multiple clicks (lines, vector paths) cannot be inserted by keyboard. Same article.
- Hand tool H; Space held temporarily activates Hand; Cmd/Ctrl + scroll zooms. https://help.figma.com/hc/en-us/articles/30925881896727-FD4B-Navigate-Figma-Design-files; https://help.figma.com/hc/en-us/articles/360041064174-Access-design-tools-from-the-toolbar.
- Scale tool K. https://help.figma.com/hc/en-us/articles/360040451453-Resize-layers-with-the-Scale-tool.
- Frame tool F or A; frame selection Option Cmd G / Ctrl Alt G. https://help.figma.com/hc/en-us/articles/360041539473-Frames-in-Figma-Design.
- Section tool Shift S (snippet). https://help.figma.com/hc/en-us/articles/9771500257687-Organize-your-canvas-with-sections.
- Rectangle R, Line L, Arrow Shift L, Ellipse O; Polygon and Star have no listed shortcut; Shift constrains proportion, Option/Alt draws from centre. https://help.figma.com/hc/en-us/articles/360040450133-Shape-tools.
- Pen P; Escape leaves a path open and deselects. https://help.figma.com/hc/en-us/articles/360040450213-Vector-networks.
- Pencil Shift P, in the Creation tools menu. https://help.figma.com/hc/en-us/articles/4402723791511-Sketch-on-the-canvas-with-the-pencil-tool.
- Vector edit mode: select vector layers and press Enter. https://help.figma.com/hc/en-us/articles/360039957634-Edit-vector-layers.
- Text tool T; Dev Mode toggle Shift D. https://help.figma.com/hc/en-us/articles/360041064174-Access-design-tools-from-the-toolbar.
- Comment mode C. https://help.figma.com/hc/en-us/articles/360039825314-Guide-to-comments-in-Figma.
- Actions menu Cmd K / Ctrl K. https://help.figma.com/hc/en-us/articles/23570416033943-Use-the-actions-menu-in-Figma-Design.
- Add auto layout Shift A; remove Option Shift A / Alt Shift A. https://help.figma.com/hc/en-us/articles/5731482952599-Toggle-on-auto-layout-in-designs.
- Group Cmd G / Ctrl G; ungroup Shift Cmd G or Cmd Delete / Shift Ctrl G or Ctrl Backspace; frame Cmd Option G / Ctrl Alt G; double-click selects a layer inside a group. https://help.figma.com/hc/en-us/articles/360039832054-The-difference-between-frames-and-groups.
- Create component Option Cmd K / Ctrl Alt K. https://help.figma.com/hc/en-us/articles/360038663154-Create-components-to-reuse-in-designs.
- Detach instance Option Cmd B / Ctrl Alt B. https://help.figma.com/hc/en-us/articles/360038665754-Detach-an-instance-from-the-component.
- Duplicate Cmd D / Ctrl D; Option/Alt + drag duplicates; copy Cmd C, paste Cmd V; paste to replace; copy as PNG. https://help.figma.com/hc/en-us/articles/4409078832791-Copy-and-paste-objects.
- Option/Alt drag on an instance creates another instance; the click must be released before the modifier. https://help.figma.com/hc/en-us/articles/360039150173-Create-and-insert-component-instances — “Drag to copy”.
- Selection: click; Shift click adds/removes; marquee on empty canvas; Cmd/Ctrl drag marquee selects nested layers; Shift drag removes; Cmd/Ctrl click deep-selects; Enter “Select Child”, Shift Enter “Select Parent”, Tab / Shift Tab next/previous sibling; Cmd/Ctrl A select all; Cmd/Ctrl Shift A select inverse; Option Cmd A / Ctrl Alt A select matching layers; Esc deselects; right-click “Select layer” submenu. https://help.figma.com/hc/en-us/articles/360040449873-Select-layers-and-objects.
- Nudge: small nudge 1, big nudge 10 “in resolution-independent points”, set under Preferences > Nudge amount. https://help.figma.com/hc/en-us/articles/4404575206295-Set-small-and-big-nudge-values. Arrow keys apply the small nudge; the Shift + arrow binding for the big nudge is stated in the position article only by reference to “big nudge” and is marked unverified as a key binding. https://help.figma.com/hc/en-us/articles/360039956914-Adjust-alignment-rotation-position-and-dimensions.
- Align: Option/Alt + A (left), D (right), W (top), S (bottom), H (horizontal centres), V (vertical centres); flip Shift H / Shift V; Shift while rotating snaps to 15°. Same position article.
- Snapping: “Snap to objects” aligns centres and outer points; “Snap to pixel grid”; “Snap to geometry” in vector edit mode; settings under Preferences and the actions menu; hold Control to disable temporarily (snippet). Same position article.
- Measurement: select a layer, hold Option/Alt and hover another layer to show a red measurement line and distances between bounds (snippet). https://help.figma.com/hc/en-us/articles/360039956974-Measure-distances-between-layers.
- Zoom: Shift + / Shift − zoom in/out; Shift 1 zoom to fit; Shift 2 zoom to selection; pixel grid Cmd ’ / Ctrl ’; snap to pixel grid Cmd Shift ’ / Ctrl Shift ’; pixel preview Ctrl P / Ctrl Alt P; layout guides Ctrl G (Mac) / Ctrl Shift 4 (Windows); multiplayer cursors Option Cmd \ / Ctrl Alt . https://help.figma.com/hc/en-us/articles/360041065034-Adjust-your-zoom-and-view-options.
- Layout guides toggle Shift G. https://help.figma.com/hc/en-us/articles/360040450513-Create-layout-guides-with-rows-columns-and-grids. Conflicts with the zoom article; both recorded.
- Rulers Shift R. https://help.figma.com/hc/en-us/articles/360040449713-Add-guides-to-the-canvas-or-frames.
- Minimize UI Cmd Shift \ / Ctrl Shift ; hide UI Cmd \ / Ctrl . https://help.figma.com/hc/en-us/articles/41414918021271-Hide-or-minimize-the-UI.
- Export all configured selections Shift Cmd E / Shift Ctrl E. https://help.figma.com/hc/en-us/articles/360040028114-Export-from-Figma-Design.
- Undo Cmd Z / Ctrl Z (Help Center search excerpt only; the originating article was not identified, so the binding is snippet-level).
- Keyboard layouts (US, UK, German and others) can be selected so that shortcuts map to the physical keyboard; article located but not retrieved. https://help.figma.com/hc/en-us/articles/5665442977431-Select-keyboard-layout.
- Unverified: Move tool V (search excerpt only, article not identified).
- Unverified: Slice tool S (the slice article was located, https://help.figma.com/hc/en-us/articles/360040028394-Using-the-Slice-Tool, but the key was not visible in retrieved text).
- Unverified: zoom to 100% Shift 0 (documented for FigJam at https://help.figma.com/hc/en-us/articles/1500004414582-Pan-and-zoom-in-FigJam, not found for Figma Design).
- Unverified: redo Cmd Shift Z / Ctrl Shift Z and Ctrl Y; no Help Center locator found.
- Unverified: Shift + arrow as the big-nudge binding (see above); Cmd/Ctrl + / − listed as zoom in the keyboard article while Shift + / − listed in the zoom article; both are recorded.
- Unverified: place image Shift Cmd K / Shift Ctrl K.
Mechanism
Consolidated binding table (Mac / Windows). V = verified from the article cited above; S = snippet-level; U = unverified.
| Action | macOS | Windows | Status |
|---|---|---|---|
| Move/select tool | V | V | U |
| Hand tool / temporary pan | H / hold Space | H / hold Space | V |
| Scale tool | K | K | V |
| Frame tool | F or A | F or A | V |
| Section tool | Shift S | Shift S | S |
| Slice tool | S | S | U |
| Rectangle / Ellipse / Line / Arrow | R / O / L / Shift L | same | V |
| Polygon / Star | none listed | none listed | V (absence) |
| Pen / Pencil | P / Shift P | P / Shift P | V |
| Text | T | T | V |
| Comment | C | C | V |
| Actions menu | Cmd K | Ctrl K | V |
| Dev Mode toggle | Shift D | Shift D | V |
| Add / remove auto layout | Shift A / Option Shift A | Shift A / Alt Shift A | V |
| Group / Ungroup | Cmd G / Shift Cmd G | Ctrl G / Shift Ctrl G | V |
| Frame selection | Option Cmd G | Ctrl Alt G | V |
| Create component | Option Cmd K | Ctrl Alt K | V |
| Detach instance | Option Cmd B | Ctrl Alt B | V |
| Duplicate / duplicate by drag | Cmd D / Option drag | Ctrl D / Alt drag | V |
| Copy / Paste | Cmd C / Cmd V | Ctrl C / Ctrl V | V |
| Undo | Cmd Z | Ctrl Z | S |
| Redo | Cmd Shift Z | Ctrl Shift Z, Ctrl Y | U |
| Select all / inverse / matching | Cmd A / Cmd Shift A / Option Cmd A | Ctrl A / Ctrl Shift A / Ctrl Alt A | V |
| Select child / parent | Enter / Shift Enter | same | V |
| Next / previous sibling | Tab / Shift Tab | same | V |
| Deep select | Cmd click | Ctrl click | V |
| Marquee (nested / subtract) | drag (Cmd drag / Shift drag) | drag (Ctrl drag / Shift drag) | V |
| Enter vector edit mode | Enter | Enter | V |
| Nudge small / big | Arrow (1) / Shift Arrow (10) | same | V values, U binding |
| Align L/R/T/B/HC/VC | Option A/D/W/S/H/V | Alt A/D/W/S/H/V | V |
| Flip H / V | Shift H / Shift V | same | V |
| Zoom in / out | Shift + / Shift −, Cmd + / − | Shift + / −, Ctrl + / − | V |
| Zoom to fit / selection / 100% | Shift 1 / Shift 2 / Shift 0 | same | V / V / U |
| Rulers / pixel grid / layout guides | Shift R / Cmd ’ / Shift G (or Ctrl G) | Shift R / Ctrl ’ / Shift G (or Ctrl Shift 4) | V (conflict noted) |
| Minimize UI / hide UI | Cmd Shift \ / Cmd \ | Ctrl Shift \ / Ctrl \ | V |
| Export | Shift Cmd E | Shift Ctrl E | V |
| Shortcut panel | Ctrl Shift ? | Ctrl Shift ? | V |
| Keyboard box selection | Option Space | Ctrl Space | V |
| Focus toolbar | F6 | Ctrl F6 | V |
| Measure to hovered layer | hold Option | hold Alt | S |
| Temporarily disable snapping | hold Control | hold Control | S |
Canvas interaction model as documented: marquee selection from empty canvas; additive and subtractive selection with Shift; hierarchical traversal by Enter/Shift Enter/Tab; deep select with the platform command key; duplication by modifier drag; smart-guide snapping to objects and pixel grid, suspended while Control is held; measurement overlay on modifier hover; pan by Space drag; zoom by command-key scroll.
NUIF relevance
Borrow
- The full binding table as the default keymap of the test editor, because the bindings are widely shared with Penpot and other editors and are not protected expression (see the synthesis record).
- The hierarchical selection grammar (Enter, Shift Enter, Tab, deep select) as the canvas counterpart of NUIF’s relationship queries in
spec/12-cli-api-and-automation.md. - Small/big nudge as configurable resolution-independent values, matching NUIF’s authored coordinates.
Adapt
- Map each binding to a named semantic operation in
spec/06-operations-and-patches.mdso that the QA client can replay the same operation log without pointer input (rfcs/0004-headless-qa-contract.md). - Resolve the two documented conflicts (layout-guides toggle; zoom modifier) by choosing one binding and documenting it in the editor’s own shortcut panel.
- Provide a keyboard-layout switch only if the test matrix needs non-US layouts; otherwise fix a US layout to keep tests deterministic.
Reject
- Comment mode (C), Dev Mode toggle (Shift D), multiplayer cursor toggle, actions-menu AI entries, keyboard box-selection tool, Figma Draw tools and the Scale-tool K binding if scale is not a first-class NUIF operation. Reason: outside the testing/import/export scope or dependent on services the test editor will not have.
Open questions
- Which primary source, if any, publishes the complete Figma Design shortcut list outside the in-app panel? The in-app panel could be transcribed under controlled conditions and recorded as an experiment.
- Should NUIF’s editor treat Shift + arrow as “big nudge” or as “pan faster when nothing is selected”, given that Figma overloads the key by selection state?
- How should conflicting bindings from different Help Center articles be reconciled in a conformance keymap fixture?
Figma UI3 design editor layout, panels and Design-tab sections
Document status:
reviewed. Canonical source.
Summary
Figma’s third interface generation (“UI3”) was announced at Config 2024 and became the only interface on 30 April 2025 (Figma blog, “Making the move to UI3”). The Help Center describes a design file as five regions: navigation bar, left sidebar, canvas, right sidebar and toolbar. The toolbar is a single slim strip at the bottom of the canvas; the left sidebar carries file, pages, layers and assets; the right sidebar carries Design and Prototype tabs whose sections are Position, Auto layout, Layout, Appearance, Fill, Stroke, Effects, Selection colors and Export, with Typography, Component/Properties and Instance sections appearing contextually. Panels are fixed but resizable; the UI can be minimized (sidebars collapse, the right sidebar reappears on selection) or hidden entirely. Panel pixel widths, typographic metrics and the exact rendering order of some sections are not stated in any primary source retrieved and are marked unverified below.
NUIF interpretation: the layout is a stable, documented target for a test editor that reproduces spatial arrangement and interaction grammar without brand assets. The section list doubles as a checklist of property groups whose semantic operations the editor must expose to automation.
Evidence
Each bullet is one claim, followed by its locator. Retrieval dates: 2026-08-29 through 2026-08-31.
- A design file has five regions, lettered A–E: navigation bar, left sidebar, canvas, right sidebar, toolbar. The toolbar “contains various creation tools, the quick actions menu, and switcher to switch between file modes”. https://help.figma.com/hc/en-us/articles/15297425105303-Explore-design-files — region legend.
- The right sidebar “contains actions like sharing and exporting”; viewers see Comment and Properties tabs, editors see Design and Prototype tabs. Same article, region D.
- Canvas panning: hold Space and drag; zoom via keyboard or trackpad. Same article, region C.
- The course article names the same four working areas (toolbar, left sidebar/navigation panel, right sidebar/properties panel, canvas); Hand tool is H; zoom is Cmd/Ctrl + scroll. https://help.figma.com/hc/en-us/articles/30925881896727-FD4B-Navigate-Figma-Design-files — “Get to know the interface”.
- Left sidebar content changes with the navigation-bar tab: File (Pages panel, Layers panel), Assets, Tools (plugins, widgets), Agents, and a Variables view. https://help.figma.com/hc/en-us/articles/360039831974-View-layers-and-assets-in-the-left-sidebar — tab list.
- Left sidebar width is user-adjustable by dragging its right edge; no pixel value is given. Same article, “adjust the width of the left sidebar”.
- The file name sits at the top of the File tab; an “Edit file menu” opens next to it; a Find/Replace tool searches the file. Same article.
- Minimize UI: Cmd Shift \ (Mac) / Ctrl Shift \ (Windows). “the navigation bar and left sidebar remains minimized, while the right sidebar expands” when an object is selected and “minimizes again” on deselect. https://help.figma.com/hc/en-us/articles/41414918021271-Hide-or-minimize-the-UI — “Minimize the UI”.
- Hide UI: Cmd \ / Ctrl \ conceals “the navigation bar, left and right sidebars, and the toolbar”. Same article, “Hide the UI”. Note: the blog post below states “Shift " for minimize; the Help Center article is treated as authoritative and the discrepancy is recorded as unverified.
- UI3 headline changes: “Bottom toolbar”; “Flexible panels and modals” (resizable panels, horizontal scrolling); “Logical layout controls” (width, height, resizing, direction, alignment, spacing grouped in one section); Dev Mode toggle in the toolbar; “Actions menu for everything”; “Optional property labels” toggled from the dropdown next to the zoom percentage; UI2 retired 30 April 2025. https://www.figma.com/blog/making-the-move-to-ui3-a-guide-to-figmas-next-chapter/ — section headings as quoted.
- Redesign rationale and visual changes: “a slim new toolbar at the bottom of the canvas”; panels “resizable” and “collapsible”; component controls (variants, instances) given “top billing above attributes like color and size”; layout options “merged into a single panel”; inputs gained backgrounds, dropdowns gained borders, corners rounded; 200 redrawn icons. https://www.figma.com/blog/behind-our-redesign-ui3/ — published 2024-06-26.
- Design-process account: “Toolbars will float at the bottom of all Figma products”; the navigation panel “linearly lists the file name, branch name, and project name, followed by pages and layers”; after beta feedback “panels are fixed, but still resizable”; constraints “expanded by default”; auto layout controls “always show pixel values and resize mode”. https://www.figma.com/blog/our-approach-to-designing-ui3/ — published 2024-10-10.
- Toolbar groups in order: Move tools (Move, Hand, Scale); Region tools (Frame, Section, Slice); Shape tools (Rectangle default; Rectangle, Line, Arrow, Ellipse, Polygon, Star, Image/video); Creation tools (Pen, Pencil); Text; Comment tools (Comment, Annotation, Measurement); Actions menu (AI tools, asset search, plugins, widgets, commands); Dev Mode toggle (Shift D); a Figma Draw button. https://help.figma.com/hc/en-us/articles/360041064174-Access-design-tools-from-the-toolbar — section headings. The article does not state the toolbar’s screen position; position is taken from the blog posts and the file-overview article above.
- Actions menu shortcut: Cmd K (Mac) / Ctrl K (Windows). https://help.figma.com/hc/en-us/articles/23570416033943-Use-the-actions-menu-in-Figma-Design.
- Right sidebar tabs and property groups: Design and Prototype for editors; Comment and Properties for viewers; listed property groups include alignment/rotation/position, frame size, corner radius, constraints, layout guides, component properties, instance, auto layout, blend modes, text, fill, stroke, effects, export settings. With nothing selected the tab shows styles, local variables, canvas background colour and page export. A dropdown “next to the 100% zoom percentage” exposes “Property labels”. https://help.figma.com/hc/en-us/articles/360039832014-Design-prototype-and-explore-layer-properties-in-the-right-sidebar.
- Zoom percentage is shown “in the top-right corner”; clicking it opens the Zoom/view options menu (zoom in/out, zoom to fit, pixel grid, snap to pixel grid, pixel preview, layout guides, multiplayer cursors). https://help.figma.com/hc/en-us/articles/360041065034-Adjust-your-zoom-and-view-options.
- Position section: alignment row (align left/right/top/bottom/horizontal centres/vertical centres, Option/Alt + A/D/W/S/H/V), X/Y measured from the top-left of the layer bounds, rotation field “at the top of the Design panel”, flips via Shift H / Shift V, W/H fields with aspect-ratio lock. https://help.figma.com/hc/en-us/articles/360039956914-Adjust-alignment-rotation-position-and-dimensions.
- Selected layers can be resized by dragging their canvas bounding box or by editing W/H; Shift temporarily preserves aspect ratio when it is unlocked, while Control temporarily suspends an existing aspect-ratio lock. The NUIF alpha editor implements transient Shift-proportional corner resizing but does not claim a persisted aspect-ratio constraint. https://help.figma.com/hc/en-us/articles/360039956914-Adjust-alignment-rotation-position-and-dimensions — “Resize layers” and “Lock aspect ratio”, retrieved 2026-08-31.
- Constraints are opened “from the Position section of the right sidebar”; options Left/Right/Left and right/Center/Scale and Top/Bottom/Top and bottom/Center/Scale; not available for layers outside a frame or inside an auto-layout frame. https://help.figma.com/hc/en-us/articles/360039957734-Apply-constraints-to-define-how-layers-resize.
- Auto layout section controls: flow (vertical, horizontal with wrap, grid), gap (numeric or auto spacing), padding (uniform or per side), alignment, resizing (hug contents, fill container, fixed), min/max width and height. Shortcut Shift A. https://help.figma.com/hc/en-us/articles/360040451373-Guide-to-auto-layout.
- Horizontal and vertical auto-layout children can be reordered by selecting and dragging them; the ordering axis follows the active flow, and instance children cannot be reordered. NUIF borrows the same visible-axis rule for same-parent Stack/Flex children but does not infer Grid or cross-parent semantics. https://help.figma.com/hc/en-us/articles/31289464393751-Use-the-horizontal-and-vertical-flows-in-auto-layout — “Vertical and horizontal flows” and “Arrange or reorder objects”, retrieved 2026-08-31.
- The controls sit under a right-panel section labelled “Auto layout”; removal via “Remove auto layout” or Option Shift A / Alt Shift A. https://help.figma.com/hc/en-us/articles/5731482952599-Toggle-on-auto-layout-in-designs.
- Frame properties: Frame tool F or A; frame presets list “in right sidebar” while the tool is active (Phone, Tablet, Desktop, Presentation, Watch, Paper, Social Media, Figma Community, Archive); “Clip content” hides children beyond the bounds; Layout guides; frame selection Option Cmd G / Ctrl Alt G. https://help.figma.com/hc/en-us/articles/360041539473-Frames-in-Figma-Design.
- Layout guides live in a “Layout guide” section; types uniform grid, column, row; visibility toggle Shift G. https://help.figma.com/hc/en-us/articles/360040450513-Create-layout-guides-with-rows-columns-and-grids. The zoom article above lists Ctrl G (Mac) / Ctrl Shift 4 (Windows) for the same toggle; the conflict is recorded as unverified.
- Text resizing (auto width, auto height, fixed) is in the Layout section; the “Typography” section holds text styles, font family, weight/style, size, line height, letter spacing, horizontal and vertical alignment, and a “Type settings” panel (text case, decoration, truncation, paragraph spacing, wrap). https://help.figma.com/hc/en-us/articles/360039956634-Explore-text-properties.
- Appearance section hosts the layer blend mode (“Apply blend mode in the Appearance section”); modes: Pass through, Normal, Darken, Multiply, Plus darker, Color burn, Lighten, Screen, Plus lighter, Color dodge, Overlay, Soft light, Hard light, Difference, Exclusion, Hue, Saturation, Color, Luminosity. https://help.figma.com/hc/en-us/articles/360040667874-Apply-blend-modes-to-layers-fills-and-effects.
- Corner radius: an “Independent corners” control opens a per-corner Corner radius panel; corner smoothing applies to the whole shape. https://help.figma.com/hc/en-us/articles/360050986854-Adjust-corner-radius-and-smoothing (search-snippet level; article body not retrieved).
- Stroke section controls in order: stroke fill, opacity, weight (px), position (inside/outside/center), individual per-side strokes, then advanced stroke settings: style (dashed, dotted, brush, dynamic), dash and gap, cap, join (miter, bevel, rounded), miter angle. https://help.figma.com/hc/en-us/articles/360049283914-Apply-and-adjust-stroke-properties.
- Effects section types: Glass, Drop shadow, Inner shadow, Layer blur, Background blur, Noise, Texture. Shadows expose X, Y, blur, spread, colour, and “Show drop shadows through transparent layers”; blurs are uniform or progressive; only one layer blur or background blur per layer. https://help.figma.com/hc/en-us/articles/360041488473-Apply-effects-to-layers.
- Selection colors section appears only “when your selection contains objects with mixed fills”; lists solid colours and gradients on fills and strokes grouped by variable, style and plain fill; a target icon selects all layers using a colour. https://help.figma.com/hc/en-us/articles/360042553434-View-and-adjust-colors-in-a-mixed-selection.
- Export section is “toward the bottom of the right sidebar” in Design mode with edit access, and under the Properties tab with view access; “Export” of all configured selections is Shift Cmd E / Shift Ctrl E. https://help.figma.com/hc/en-us/articles/360040028114-Export-from-Figma-Design.
- Export settings: formats PNG, JPG, SVG, PDF (PDF 1.7); scale presets 0.5x, 0.75x, 1x, 1.5x, 2x, 3x, 4x plus custom width/height via
w/h; SVG and PDF export only at 1x; optional suffix; options include ignore overlapping layers, include bounding box (text), include “id” attribute, outline text, simplify stroke, image quality, image resampling, colour profile. https://help.figma.com/hc/en-us/articles/13402894554519-Export-formats-and-settings. - Main component selection: a “Properties” section for creating Boolean, Instance swap, Text, Variant and Slot properties. https://help.figma.com/hc/en-us/articles/5579474826519-Explore-component-properties. The “Create component” button is in the right sidebar “next to the selection’s name” (Option Cmd K / Ctrl Alt K). https://help.figma.com/hc/en-us/articles/360038663154-Create-components-to-reuse-in-designs.
- Instance selection: the instance menu in the properties panel offers “Detach instance” (Option Cmd B / Ctrl Alt B). https://help.figma.com/hc/en-us/articles/360038665754-Detach-an-instance-from-the-component.
- Vector selection: Enter enters vector edit mode; a secondary toolbar offers Variable width, Shape builder, Cut, Bend, Eraser, Lasso and Paint. https://help.figma.com/hc/en-us/articles/360039957634-Edit-vector-layers.
- Scale tool (K) replaces the Design tab contents with a scale multiplier, W/H fields and an anchor selector. https://help.figma.com/hc/en-us/articles/360040451453-Resize-layers-with-the-Scale-tool.
- Rulers appear along the top and left edges of the canvas; toggle Shift R; guides are dragged from rulers. https://help.figma.com/hc/en-us/articles/360040449713-Add-guides-to-the-canvas-or-frames.
- Unverified: exact pixel widths of the left and right sidebars; default sidebar widths; minimum widths. No primary source states them.
- Unverified: UI3 typography (font family, sizes) and control heights. The blog posts describe icon and input styling only.
- Unverified: the exact on-screen order of Design-tab sections; the order given in Mechanism is a reconstruction from the sources above (Position before Auto layout, Layout and Appearance from the blog descriptions; Export last from the export article).
- Unverified: Share button, Present/play control and collaborator avatars in the top-right area. The file-overview article states only that the right sidebar “contains actions like sharing and exporting”.
- Unverified: page background colour control location (right sidebar with nothing selected per the right-sidebar article; the section name is not given).
- Unverified: the Help Center article “Navigating UI3” (article 23954856027159) returned HTTP 404 in en-us and es-419 and could not be used.
- Unverified: iOS/Android export presets; not mentioned in the export-settings article retrieved.
Mechanism
Spatial model of a UI3 design file (reconstruction from the sources above; proportions not to scale):
┌──────────────────────────────────────────────────────────────────────────────┐
│ [A] Navigation bar (File · Assets · Tools · Agents tabs; file name; Minimize) │
├───────────────┬───────────────────────────────────────────┬──────────────────┤
│ [B] Left │ [C] Canvas │ [D] Right sidebar│
│ sidebar │ rulers (top/left, Shift R) │ zoom % ▾ share │
│ Pages │ infinite; Space+drag pans; Cmd/Ctrl+ │ ┌Design┬Proto─┐ │
│ Layers │ scroll zooms; marquee, smart guides, │ │ sections… │ │
│ (resizable │ snapping; Alt-hover measurements │ │ (resizable)│ │
│ width) │ │ └───────────┘ │
│ Assets tab │ ┌───────────────────────────────┐ │ │
│ (components) │ │ [E] Toolbar (floating, bottom) │ │ │
│ │ │ Move▾ Region▾ Shape▾ Pen▾ T C▾ │ │ │
│ │ │ Actions(⌘K) Dev Mode Draw │ │ │
│ │ └───────────────────────────────┘ │ │
└───────────────┴───────────────────────────────────────────┴──────────────────┘
Minimize UI (Cmd/Ctrl Shift \): A and B collapse; D reappears while a layer is selected.
Hide UI (Cmd/Ctrl \): A, B, D and E all hidden.
Design tab for a frame with auto layout selected (order reconstructed; see unverified note):
- Header row: layer name, “Create component” button, property-label toggle available from the zoom menu.
- Position: alignment row; X, Y; rotation; flip horizontal/vertical; Constraints (hidden for auto-layout children; expanded by default per the UI3 design post).
- Auto layout: flow (vertical, horizontal, wrap, grid); gap; padding (uniform / per side); alignment grid; advanced settings.
- Layout: W, H; resizing per axis (hug, fill, fixed); min/max width and height; Clip content; Layout guide (uniform grid, column, row).
- Appearance: opacity (placement inferred, unverified); corner radius with Independent corners and smoothing; blend mode; visibility.
- Fill: fill list (solid, gradient, image, video, pattern), colour picker with styles and variables.
- Stroke: colour, opacity, weight, position, per-side strokes, advanced (style, dash/gap, cap, join, miter angle).
- Effects: Glass, Drop shadow, Inner shadow, Layer blur, Background blur, Noise, Texture.
- Selection colors: present only for mixed-fill selections.
- Export: format, scale, suffix, per-format options; page export when nothing is selected.
Contextual variants:
- Text layer: Typography section (text style, family, weight, size, line height, letter spacing, alignment, type settings); text resizing controls in Layout.
- Vector layer: Enter opens vector edit mode with a secondary tool strip; Fill/Stroke/Effects remain.
- Main component: Properties section (Boolean, Instance swap, Text, Variant, Slot) placed above appearance attributes.
- Instance: instance menu with swap, reset, detach and “Go to main component”; component property controls.
- Multiple objects: alignment/distribution row; Selection colors when fills differ; mixed values otherwise (mixed-value rendering: unverified).
- Nothing selected: styles, local variables, canvas background colour, page export.
- Scale tool active: scale multiplier, W/H, anchor.
NUIF relevance
Borrow
- Five-region composition (navigation, left structure panel, centre canvas, right properties panel, bottom floating toolbar) as the test editor’s shell, because it is documented and stable since April 2025.
- The Design-tab section taxonomy (Position, Auto layout, Layout, Appearance, Fill, Stroke, Effects, Export) as the property-group vocabulary for the editor’s inspector and for the query surface in
spec/12-cli-api-and-automation.md. - Minimize/hide UI behaviour, resizable panels and the selection-driven reappearance of the properties panel, because they exercise shell state without touching document state (
apps/editor/ARCHITECTURE.md). - Contextual section switching by selection type (text, vector, component, instance, mixed) as test fixtures for selection-dependent inspector queries.
Adapt
- Replace Figma-specific section names where NUIF semantics differ (for example, “Auto layout” becomes the NUIF responsive-container model from
spec/04-layout.md), keeping the spatial slot but not the vendor vocabulary. - Keep the toolbar order but expose every tool as a semantic operation first, so pointer gestures translate into protocol operations before mutation.
- Present property labels always on (Figma’s optional labels) to make screenshots and accessibility-tree assertions deterministic.
Reject
- Comments, annotations and measurement tools of the Comment group; multiplayer cursors and avatars; Dev Mode and its Inspect/codegen panels; FigJam, Slides, Buzz and Sites modes; the Actions menu’s AI features and asset search; plugin/widget marketplace; Agents tab; version history and branching UI; presentation/prototype player beyond what a test needs; Figma Draw illustration tools (Glass, Noise, Texture effects, brush strokes); Community and library publishing. Reason: none of these are required for testing, import or export, and several depend on network services.
Open questions
- Which section order does the UI3 Design tab render for a plain frame, and does Auto layout appear as a collapsed row inside Layout or as its own section? Requires an in-app check or a source not yet retrieved.
- What are the default and minimum widths of the sidebars, and does the right sidebar width persist per file or per user?
- How should the test editor render “mixed” values for multi-selection without copying Figma’s exact presentation?
- Should the test editor implement the Scale tool (K) given that scale is a derived transform in NUIF rather than a property?
Figma public plugin/document node model
Document status:
reviewed. Canonical source.
Summary
Figma exposes document content through a versioned REST response and an
in-editor plugin API. The public model includes document/canvas containment,
frames, components, instances, vectors, text, auto layout, grid layout, paints,
styles and variables. The public contract is an API model, not the .fig file
encoding.
Evidence
GET /v1/files/:keyreturns a document node, component/style maps, schema version and file version.ids,depth,geometry=pathsandplugin_datacontrol projection. The endpoint requiresfile_content:readand is subject to plan-dependent rate limits. https://developers.figma.com/docs/rest-api/file-endpoints/#get-file and https://developers.figma.com/docs/rest-api/rate-limits/ (retrieved 2026-08-29).- The REST node catalogue distinguishes
DOCUMENT,CANVAS,FRAME,COMPONENT,INSTANCE, basic shapes, vectors and text. Properties are conditional on node type. https://developers.figma.com/docs/rest-api/file-node-types/ (retrieved 2026-08-29). - The official plugin typings define stable node IDs, ordered children,
geometry, paints, auto-layout sizing/alignment/padding/gap, grid tracks and
variable bindings. The plugin API is the writable document surface.
https://github.com/figma/plugin-typings/blob/master/plugin-api-standalone.d.ts
(
BaseNodeMixin,ChildrenMixin,LayoutMixin,AutoLayoutMixin,GridLayoutMixin,GeometryMixin; retrieved 2026-08-29). setPluginDatastores plugin-private string data with a 100 kB limit per entry. REST retrieval includes only requested plugin IDs or shared data. https://developers.figma.com/docs/plugins/api/properties/nodes-setplugindata/ (retrieved 2026-08-29).- The Variables API represents boolean, float, string and color values by mode; local variable read/write scopes can be plan-restricted. https://developers.figma.com/docs/rest-api/variables-endpoints/ (retrieved 2026-08-29).
NUIF relevance
Borrow stable foreign node IDs, explicit node kinds, ordered containment, component references, auto-layout fields, variable bindings and plugin data for correspondence metadata.
Adapt REST JSON as a bounded read fixture and use a plugin companion for writes. Every report must record file version, schema version, requested depth, geometry mode and omitted plugin-data namespaces. A first profile can map one page with frames, rectangles, ellipses and pinned text.
Reject undocumented .fig or multiplayer protocols as normative
dependencies. REST access is authenticated, rate-limited and primarily a read
surface; a credential-free bidirectional conformance gate requires checked-in
API fixtures and a separately tested plugin bridge.
FLIP perceptual difference evaluator for rendered images (LDR-FLIP, HDR-FLIP, NVIDIA reference implementation)
Document status:
reviewed. Canonical source.
Summary
FLIP (Andersson et al., Proc. ACM Comput. Graph. Interact. Tech. 3(2), Article 15, 2020) is a full-reference image difference evaluator built for the case where a reference and a test rendering are alternated (“flipped”) on the same display. It produces a per-pixel error map in [0, 1] and a set of pooled statistics. The model is parameterised by pixels per degree (PPD), which couples the metric to display size, resolution and viewing distance. Two parallel pipelines are combined: a colour pipeline (contrast-sensitivity filtering in an opponent colour space, Hunt-adjusted Lab*, HyAB distance, non-linear remapping) and a feature pipeline (edge and point detectors whose scale depends on PPD). HDR-FLIP (Andersson, Nilsson, Shirley, Akenine-Möller, Eurographics 2021 Short Papers, DOI 10.2312/egs.20211015) extends the method to high-dynamic-range inputs by compositing LDR-FLIP maps over a range of exposures. NVIDIA publishes a BSD-3-Clause reference implementation with C++, CUDA, Python (nanobind) and PyTorch entry points.
The source facts below come from the paper text, the FLIP.h header and the repository READMEs. The NUIF interpretation is confined to the “NUIF relevance” section.
Evidence
- Venue and identity: Proceedings of the ACM on Computer Graphics and Interactive Techniques, vol. 3, no. 2, Article 15, pp. 15:1–15:23, August 2020, DOI 10.1145/3406183 (paper header; Crossref record;
misc/LDRFLIP.txtin the repository). - Design target: the abstract states the map “approximates the difference perceived by humans when alternating between two images” (paper, abstract, p. 15:1).
- PPD model: Equation 1, Section 4.1.1, computes PPD from observer distance d, monitor width W_m (metres) and horizontal resolution W_p (pixels). The paper’s default setup is a 0.69 m × 0.39 m, 3840 × 2160 monitor viewed at 0.70 m, giving p = 67 PPD (p. 15:7). The header uses
calculatePPD(0.7f, 3840.0f, 0.7f)with a 0.7 m width (src/cpp/FLIP.h, line 118); both round to 67. - Colour pipeline: sRGB is linearised, converted to XYZ and to the YyCxCz opponent space; each channel is convolved with a Gaussian approximation of a contrast sensitivity function whose radius is r = ⌈3 σ_max p⌉ (Equation 6, Section 4.1.1); filtered colours are clamped to the RGB cube, converted to Lab* under D65 and Hunt-adjusted (Section 4.1.2); the HyAB distance ΔE_HyAB = |ΔL*| + sqrt(Δa² + Δb²) is used (Equation 8, Section 4.1.3) because Euclidean Lab* distances are only valid for small differences and rendering errors such as fireflies are large.
- Colour remapping constants: distances are raised to q_c = 0.7; the maximum Hunt-adjusted HyAB distance (blue vs. green) is c_h,max = 203, giving c_max = 41 after the power; the range [0, p_c c_max) maps linearly to [0, p_t) and [p_c c_max, c_max] to [p_t, 1], with p_c = 0.4 and p_t = 0.95 (Section 4.1.3, Figure 4). The header stores the same constants as
gqc = 0.7f,gpc = 0.4f,gpt = 0.95f,gw = 0.082f,gqf = 0.5f(FLIP.h, lines 128–132). - Feature pipeline: edge and point features are Gaussian first- and second-derivative responses on the achromatic channel, kernel radius ⌈3σ(w, p)⌉ in pixels, so feature scale follows PPD (Section 4.2.1). The feature difference is ΔE_f = (max(|‖∇R‖ − ‖∇T‖|, |‖∇²R‖ − ‖∇²T‖|) / √2)^q_f with q_f = 1/2 (Equation 9, Section 4.2.2).
- Combination: ΔE = (ΔE_c)^(1 − ΔE_f) (Equation 10, Section 4.3). A feature difference can only increase the colour difference; ΔE = 0 when filtered colours are identical; ΔE = 1 for the blue/green extreme or ΔE_f = 1. The parameters q_c, q_f, p_c, p_t “were chosen based on visual inspection” of many image pairs (Section 4.3).
- Pooling: Section 5 argues that pooling loses information and should be avoided where possible; it proposes a weighted histogram (bucket count multiplied by bucket-centre FLIP value, normalised per megapixel) and single-value summaries. The tool reports mean, weighted median, first and third weighted quartiles, min and max (
src/python/README.md, example output). - Validation: 42 subjects, 21 image pairs (11 rendered, 10 natural) at 67 PPD, comparing FLIP with Butteraugli, a CNN visibility metric, HDR-VDP-2, LPIPS, PieAPP, Euclidean RGB distance, S-CIELAB, SMAPE and SSIM; FLIP obtained the best average score (2.1) with non-overlapping 95% confidence intervals against the others on average (Section 6.2, pp. 15:19–15:20). SSIM scored best on two individual pairs (R2, N10).
- Critique of SSIM: the paper notes SSIM “does not consider viewing distance and pixel size” and yields uninterpretable negative values shown as a separate colour in its maps (Section 6.1, p. 15:17; Section 6, p. 15:14).
- HDR-FLIP: Eurographics 2021 Short Papers, DOI 10.2312/egs.20211015 (
misc/HDRFLIP.txt); it computes “a composite visualization over a number of low dynamic range error maps of exposure compensated and tone mapped image pairs” (NVIDIA publication page abstract). The header exposesstartExposure,stopExposure,numExposures(automatic when left at infinity/−1) and tone mappersreinhard,aces(default) and a third (Hable) (FLIP.h, lines 119–122, 138–144, 1395–1420). Since v1.7 automatic exposure handles references whose median luminance is 0 (repository README). - Tool chapter: Andersson, Nilsson, Akenine-Möller, “Visualizing and Communicating Errors in Rendered Images”, Ray Tracing Gems II, ch. 19, pp. 301–320, DOI 10.1007/978-1-4842-7185-8_19 (
misc/FLIP.txt; Crossref). - Implementations: single header
src/cpp/FLIP.hfor CPU and CUDA (-DFLIP_ENABLE_CUDA=ON), Python packageflip-evaluatorvia nanobind, PyTorch losssrc/pytorch/flip_loss.py(repository README). - Metric reproducibility caveat: the Python README states output “might differ slightly between the different operative systems”; the repository’s own tests compare means to six decimal places while “not all error map pixels are identical” across Windows, Linux and macOS (
src/python/README.md, “Python (API and Tool)”). - Practical thresholds observed in a Rust renderer: Vello’s smoke snapshot tests call
assert_mean_less_than(0.01); a known-issue reproduction uses0.001; the comparison helper rejects thresholds ≥ 0.1 as implausible for a passing test; PPD isnv_flip::DEFAULT_PIXELS_PER_DEGREE = 67.0(vello_tests/tests/smoke_snapshots.rs, lines 29, 47, 75, 119;vello_tests/tests/known_issues.rs, line 55;vello_tests/src/compare.rs, lines 45–48;nv-flip/src/lib.rs, line 15).
Mechanism
Inputs: reference R, test T (sRGB, same size), pixels per degree p
p = d * (W_p / W_m) * (pi / 180) # Eq. 1; 0.70 m, 3840 px, 0.69 m -> 67
Colour pipeline (Section 4.1)
R_lin, T_lin = sRGB^-1(R), sRGB^-1(T)
R_o, T_o = XYZ->YyCxCz(RGB->XYZ(.))
for c in {Yy, Cx, Cz}: R_o[c] = G_c(p) * R_o[c] # CSF-derived Gaussian, radius ceil(3*sigma_max*p)
clamp back to RGB cube, convert to L*a*b* (D65), apply Hunt adjustment to a*, b*
dE_hyab = |dL*| + sqrt(da*^2 + db*^2) # Eq. 8
e = dE_hyab ^ q_c # q_c = 0.7, c_max = 41
dE_c = e < p_c*c_max ? e * p_t/(p_c*c_max)
: p_t + (e - p_c*c_max)/(c_max - p_c*c_max) * (1 - p_t) # p_c = 0.4, p_t = 0.95
Feature pipeline (Section 4.2)
edge = |grad G_sigma(p) * Y|, point = |laplacian-like second derivative|
dE_f = ( max(|edge_R - edge_T|, |point_R - point_T|) / sqrt(2) ) ^ q_f # q_f = 0.5
Combination (Eq. 10)
FLIP = dE_c ^ (1 - dE_f) # per pixel, in [0, 1]
Pooling (Section 5)
weighted histogram; mean; weighted median; weighted quartiles; min; max
HDR-FLIP
for exposure in linspace(c_start, c_stop, N): map = LDR-FLIP(tonemap(R*2^exposure), tonemap(T*2^exposure))
composite = per-pixel maximum over exposures; exposure map records the argmax
The HDR compositing rule (per-pixel maximum with an exposure map) is stated in the Eurographics paper abstract as a composite over exposure-compensated maps; the exact aggregation is documented in the tool’s exposure-map output naming (src/python/README.md).
NUIF relevance
Borrow
- Use pooled FLIP mean at a fixture-declared PPD as the tolerance statistic for the non-normative GPU tier of the
rendersuite, because it is validated against human judgement of rendering artefacts and is already exercised in the Rust ecosystem vianv-flip. - Adopt the weighted-histogram report (not only the mean) in conformance reports, because the paper documents that single-value pooling discards localisation information.
Adapt
- Derive PPD from the evaluation context’s pixel ratio and a declared viewing model instead of the fixed 67, because NUIF fixtures are rendered at multiple device pixel ratios and the metric’s feature scale depends on p.
- Record the FLIP implementation version and platform with each result, because the metric itself is not pixel-identical across operating systems.
Reject
- Do not use FLIP as the gate for the deterministic CPU reference path, because exact pixel equality is the normative requirement there and FLIP would mask real semantic regressions such as off-by-one clipping.
- Do not use HDR-FLIP, because NUIF documents specify display-referred sRGB output rather than scene-referred radiance.
Implemented diagnostic boundary
The synthetic reconstruction-evaluation gate uses exact nv-flip 0.1.2 with
resolved nv-flip-sys 0.1.1, LDR input, arithmetic-mean pooling and the
reference default of 67 PPD. It records those values, platform sensitivity and
an evaluator digest in the typed report. The dependency is test-only. Exact
pixel, element, text, geometry and provenance fields remain independent, so a
low pooled error cannot hide a missing control. The adapter accepts only
same-sized opaque sRGB8; callers must explicitly composite transparency under
a recorded policy. This closes the wiring and alpha-ambiguity questions for
the fixture, not the threshold or cross-platform reproducibility questions.
Open questions
- Which per-fixture-class thresholds (text, thin strokes, gradients) are appropriate; Vello’s 0.01 mean is an engineering choice, not a published recommendation.
- Whether a pure-Rust FLIP port with a pinned evaluation order is required so that the metric itself is reproducible across CI platforms.
- Which declared compositing backgrounds should become separate transparent-fixture profiles; the current diagnostic refuses transparency rather than guessing.
Flutter box-constraint layout model
Document status:
reviewed. Canonical source.
Summary
Flutter’s box layout passes constraints from parent to child, returns sizes from child to parent and assigns child positions in the parent. The standard box layout is one pass and each render object chooses a size within its incoming constraints.
Evidence
- The Flutter constraint guide states the processing order as constraints down, sizes up and parent-assigned positions. Width and height requests can be constrained or ignored by ancestors. https://docs.flutter.dev/ui/layout/constraints (retrieved 2026-08-29).
- The guide identifies the one-pass limitation: a box can choose only within parent constraints, does not choose its global position and cannot determine geometry independently of the tree. https://docs.flutter.dev/ui/layout/constraints#limitations (retrieved 2026-08-29).
NUIF relevance
Borrow BoxConstraints-compatible min/max sizing, row/column flex and
parent-relative placement for a bounded lowering profile.
Adapt NUIF entities into a generated, profile-owned Dart widget subset. Stable identity metadata and source spans are required for reconciliation. Conformance requires a pinned Flutter engine, platform, pixel ratio and fonts.
Reject arbitrary Dart widget-tree import. Builders, state, inherited widgets, custom render objects, assets and platform plugins are executable semantics outside profile zero.
Requested font identity, replacement resources and item-level fallback fidelity
Document status:
reviewed. Canonical source.
Summary
A requested font, the stable asset that records its portability decision and
the bytes actually used for layout are different facts. NUIF now binds a text
item to a font asset by optional AssetId, retains the requested SHA-256 on the
text, and derives an effective SHA-256 from the asset. This represents exact,
substituted and unavailable outcomes without treating a family name as exact
identity or overwriting authored intent.
Evidence
- CSS Fonts Level 4 defines
font-familyas an ordered selection input and notes that a family name does not identify an individual face. Its fallback procedure may select a different installed font and can vary between user agents and operating systems. - The same specification distinguishes downloadable
@font-faceresources from installed fonts and selects a fallback when the intended resource is unavailable. This proves that requested family, selected face and resource availability are separate state. - CSS Font Loading Level 3 exposes
unloaded,loading,loadedanderrorstates for a face rather than pretending that every family request resolved. Source: https://www.w3.org/TR/css-font-loading-3/. - OpenType
OS/2.fsTypedescribes embedding signals for bytes; it does not identify which text items requested those bytes or which replacement was chosen. Source: https://learn.microsoft.com/en-us/typography/opentype/spec/os2#fstype.
Executable decision
TextContent.font_sha256 remains the requested exact identity.
TextContent.font_asset is an optional stable semantic reference:
- absent: legacy executable profile 0 uses the requested hash directly;
- exact: the referenced font asset’s resource hash must equal the requested hash;
- substituted: the requested hash is retained and the referenced asset’s resource hash becomes the declared effective replacement;
- unavailable: the referenced asset has no resource and remains linked to the affected text item.
Core validation rejects a missing asset, a non-font target, an exact binding whose digest differs from the request, or a bound usable asset without a valid resource digest. Resolution is pure and performs no filesystem, network or platform-font lookup.
Layout shapes with an available declared replacement and emits item-level
approximated fidelity. If the substitute is absent from the evaluation
context, or if the asset is unavailable, layout and rendering emit item-level
unsupported fidelity and rendering emits no false text command. Legacy
unbound profile-0 text retains its typed missing-context error.
The existing HTML, React, Svelte, SVG and Penpot profiles do not encode this binding. Their exporters therefore reject it at profile inspection instead of silently dropping it; their importers continue to create unbound text. The same audit found that those profiles and the scalar DTCG profile did not encode the document asset table at all, so every such exporter now rejects any non-empty asset table before serialization.
Alternatives rejected
- Family-name matching: ambiguous across faces and platforms and expressly insufficient for an exact-font claim.
- Replace the requested hash: loses authored intent and makes it impossible to say what was substituted.
- Global substitution map only: cannot represent different decisions for individual text items and weakens property-level fidelity.
- Fidelity text without a stable asset reference: reports loss but cannot connect an unavailable resource, policy evidence and affected item.
- Implicit system fallback: makes canonical output depend on the host and bypasses explicit resource authority.
Evidence boundary
cargo xtask gate-i-font packages and decodes substituted and unavailable
assets, verifies the binding survives, exercises layout with and without the
replacement in the context, and proves renderer command/fidelity behavior in
six blocking trials. The declared replacement is the already pinned Ahem
resource, so this establishes whole-text item semantics, not a general fallback
engine.
Cluster-level fallback, missing-glyph reporting, multiple faces per run, variable axes, feature-dependent substitution, shaping with arbitrary packaged font bytes and cross-platform raster equivalence remain separate work.
NUIF relevance
The binding keeps requested design intent, resource policy and the bytes used for evaluation as separate core facts. Thin adapters can reject or report the unsupported field without inventing host fallback, and layout/rendering derive the same item-level fidelity from one authoritative resolution function.
Fontations read-fonts and Skrifa OpenType stack
Document status:
verified. Canonical source.
Summary
Fontations is a safe Rust family for reading, writing and interpreting OpenType
fonts. read-fonts is its low-level no-copy/no-allocation reader; Skrifa adds
metadata, character maps, variation information and outlines. NUIF pins Skrifa
0.46.2 for both profile-zero outlines and static package-font metadata after
retiring ttf-parser for RUSTSEC-2026-0192. NUIF-owned sfnt validation remains
ahead of the library, and a committed HarfBuzz capture is the external metadata
oracle.
Evidence
- The repository identifies
read-fontsas a high-performance parser suitable for shaping and describes its access as allocation- and copy-free. Locator: repositoryREADME.md, “Structure”, retrieved 2026-08-30. - Skrifa exposes metrics, codepoint-to-glyph mapping, localized strings,
attributes, axes and TrueType/CFF/color/bitmap outline sources. Locator:
skrifa/README.md, “Features”, retrieved 2026-08-30. - Skrifa forbids unsafe code and says corrupted or malicious input should not
panic. Fontations maintains cargo-fuzz and OSS-Fuzz integration. Locators:
skrifa/README.md, “Panicking” and “Safety”; repositoryREADME.md, “Fuzzing”, retrieved 2026-08-30.
Mechanism
nuif-font constructs a Skrifa FontRef only after NUIF validates sfnt search
fields, table ordering, ranges, packing, padding and checksums. NUIF directly
reads required head, maxp and OS/2 fields, requires metric agreement and
owns the conservative embedding-bit policy. The conformance executable compares
the resulting units, glyph count, family, table inventory and normalized
Unicode coverage with a digest-bound hb-info 14.4.0 capture before it runs
package and policy trials.
Alternatives and decision
Fontations replaces the unmaintained production parser because it is already pinned for outlines, forbids unsafe code and maintains fuzzing infrastructure. NUIF does not use Skrifa as its own independent oracle: a pinned HarfBuzz capture provides external evidence, while direct sfnt reads catch disagreement in the required fields. FreeType remains valuable as a future native third oracle and browser stacks provide essential WOFF2 evidence, but neither is needed to define the smallest static sfnt baseline.
NUIF relevance
Borrow maintained metadata, character-map and outline access behind a single exact version pin.
Adapt parser results through NUIF limits and exact semantic ranges. A font parser does not own package resolution, shaping or policy.
Reject promoting the library’s broad feature surface into NUIF support without fixtures for each declared font category.
Open questions
- Compare NUIF and browser-selected cmaps for symbol, format 13 and variation sequence cases before a broader profile.
- Add FreeType or an external implementation as a third oracle only with a pinned build and measured sandbox boundary.
Coverage-guided fuzzing of structured inputs (libFuzzer, AFL++, cargo-fuzz, arbitrary, grammar fuzzing, parser resource limits)
Document status:
reviewed. Canonical source.
Summary
Coverage-guided fuzzers (libFuzzer, AFL++) mutate a corpus of byte inputs and keep mutations that reach new coverage. For structured inputs, three approaches exist: custom mutators that parse, mutate and re-serialise (libFuzzer LLVMFuzzerCustomMutator, AFL++ afl_custom_fuzz, libprotobuf-mutator, Grammar-Mutator); grammar-based generators with tree mutation and coverage feedback (Nautilus); and parametric generators that decode the fuzzer’s byte stream into a typed value so that byte mutations become structural mutations (Zest; Rust arbitrary with cargo fuzz). Parser fuzz targets must be deterministic, fast and bounded; libFuzzer enforces -timeout, -rss_limit_mb and -malloc_limit_mb, OSS-Fuzz flags inputs over about 25 seconds or 2.5 GB, and production parsers (usvg, roxmltree, serde_json, image, Skia) enforce nesting depth, node counts and allocation budgets in code.
For NUIF the codec, extension payload handling and path geometry are untrusted-input parsers under spec/11-security.md; the same Arbitrary-based generator can feed both fuzz targets and the trial-and-error loop.
Evidence
- libFuzzer target contract:
extern "C" int LLVMFuzzerTestOneInput(const uint8_t *Data, size_t Size); the target “must tolerate any kind of input”, “must notexit()”, “must be as deterministic as possible”, “must be fast”, and ideally not modify global state; narrower targets are better. https://llvm.org/docs/LibFuzzer.html, §Fuzz Target, retrieved 2026-08-29. - Coverage comes from SanitizerCoverage inline 8-bit counters via
-fsanitize=fuzzer; mutations that reach a previously uncovered path are added to the corpus. Same page, §Corpus. - Options:
-max_len(0 = guess from corpus),-len_control,-timeout(default 1200 s),-rss_limit_mb(default 2048),-malloc_limit_mb(single allocation cap),-dict,-use_value_profile,-jobs/-workers,-minimize_crash,-merge. Same page, §Options. - Structure-aware fuzzing: custom mutator
size_t LLVMFuzzerCustomMutator(uint8_t *Data, size_t Size, size_t MaxSize, unsigned int Seed)parses per grammar, mutates, re-serialises; libprotobuf-mutator withDEFINE_PROTO_FUZZERuses protobuf as the intermediate format (SQLite example). https://github.com/google/fuzzing/blob/master/docs/structure-aware-fuzzing.md, retrieved 2026-08-29. - AFL++: LTO instrumentation preferred;
-mmemory limit “highly recommend”;-ttimeout; dictionaries via-xandAFL_LLVM_DICT2FILE; CMPLOG/Redqueen viaAFL_LLVM_CMPLOG=1and-c;afl-cminandafl-tmin. https://github.com/AFLplusplus/AFLplusplus/blob/stable/docs/fuzzing_in_depth.md, retrieved 2026-08-29. - AFL++ custom mutator API:
afl_custom_init,afl_custom_fuzz,afl_custom_post_process,afl_custom_trim,afl_custom_havoc_mutation,afl_custom_queue_get;AFL_CUSTOM_MUTATOR_LIBRARY,AFL_CUSTOM_MUTATOR_ONLY. https://github.com/AFLplusplus/AFLplusplus/blob/stable/docs/custom_mutators.md, retrieved 2026-08-29. Paper: Fioraldi, Maier, Eißfeldt, Heuse, WOOT 2020, §3.2.1 (havoc probability 6%), §3.2.2 Input-To-State (https://aflplus.plus/papers/aflpp-woot2020.pdf, retrieved 2026-08-29). - Grammar-Mutator: AFL++ custom mutator for “highly-structured inputs” with JSON grammars, tree-based mutations (rules, random, random recursive, splicing) and tree-based trimming;
grammar_generator-<lang> 100 1000 ./seeds ./treescreates 100 seeds of max tree size 1000. https://github.com/AFLplusplus/Grammar-Mutator README, retrieved 2026-08-29. - cargo-fuzz: “a tool to invoke a fuzzer”, libFuzzer via
libfuzzer-sys, nightly required; commandscargo fuzz init|add|list|run|tmin|cmin|coverage|fmt;cargo fuzz coveragebuilds with-Cinstrument-coverageand writesfuzz/coverage/<target>/coverage.profdata; crash artefacts underfuzz/artifacts/<target>/. https://rust-fuzz.github.io/book/cargo-fuzz.html, /cargo-fuzz/tutorial.html, /cargo-fuzz/coverage.html and https://github.com/rust-fuzz/cargo-fuzz README, retrieved 2026-08-29. afl.rs alternative:cargo afl build,cargo afl fuzz -i in -o out <bin>,fuzz!macro (https://rust-fuzz.github.io/book/afl.html). fuzz_target!accepts|data: &[u8]|or|input: T|forT: Arbitrary, aninit:block, and an optional-> Corpusreturn (Corpus::Keep/Corpus::Reject); inputs whoseArbitrarydecoding fails are rejected. https://docs.rs/libfuzzer-sys/latest/libfuzzer_sys/macro.fuzz_target.html and https://rust-fuzz.github.io/book/cargo-fuzz/structure-aware-fuzzing.html, retrieved 2026-08-29.- arbitrary 1.4.2:
trait Arbitrary<'a>: Sized { fn arbitrary(u: &mut Unstructured<'a>) -> Result<Self>; fn arbitrary_take_rest(u: Unstructured<'a>) -> Result<Self>; fn size_hint(depth: usize) -> (usize, Option<usize>); fn try_size_hint(depth: usize) -> Result<(usize, Option<usize>), MaxRecursionReached> };#[derive(Arbitrary)]with thederivefeature. https://docs.rs/arbitrary/latest/arbitrary/trait.Arbitrary.html and https://github.com/rust-fuzz/arbitrary README, retrieved 2026-08-29. Unstructuredmethods:int_in_range(not necessarily uniform; returns range start on empty data),choose,choose_index,ratio,arbitrary_len(uses elementsize_hint, takes lengths “from the end of the data”),bytes,take_rest,arbitrary_iter. https://docs.rs/arbitrary/latest/arbitrary/struct.Unstructured.html andsrc/unstructured.rs, retrieved 2026-08-29.- Recursion behaviour:
size_hint::MAX_DEPTH = 20guards onlysize_hintcomputation (src/size_hint.rsline 4, lines 38–47);ArbitraryIter::nextdraws abool“keep going” flag that is false on exhausted data, soVec<T>and recursive children terminate when bytes run out (src/unstructured.rs,src/foreign/alloc/vec.rs,src/foreign/core/bool.rs). Retrieved 2026-08-29. NUIF interpretation: depth must be bounded explicitly inarbitraryimplementations. - Grammar fuzzing: derivation trees with expansion phases bounded by
min_nonterminals/max_nonterminals(https://www.fuzzingbook.org/html/GrammarFuzzer.html); greybox grammar fuzzing combines fragment or region mutation with coverage-guided seed selection (https://www.fuzzingbook.org/html/GreyboxGrammarFuzzer.html). Retrieved 2026-08-29. - Nautilus: combining context-free grammars with feedback-driven fuzzing outperforms AFL “by an order of magnitude”; mutations on derivation trees (random with a configurable maximum subtree size, rules, random recursive 2^n repetitions, splicing) plus subtree and recursive minimisation; AFL-style 64 KB bitmap. Aschermann et al., NDSS 2019, DOI 10.14722/ndss.2019.23412, §IV.A–C, §V (PDF https://www.ndss-symposium.org/wp-content/uploads/2019/02/ndss2019_04A-3_Aschermann_paper.pdf, retrieved 2026-08-29).
- Zest: “converts random-input generators into deterministic parametric generators”; mutations in the untyped parameter domain map to structural mutations; every parameter sequence yields a syntactically valid input if the generator does; the algorithm tracks total and valid coverage and saves inputs that add valid coverage; the XML generator bounds
MAX_DEPTHandMAX_CHILDREN. Padhye et al., ISSTA 2019, DOI 10.1145/3293882.3330576, Abstract, §3.1–3.2, Fig. 2 (PDF https://rohan.padhye.org/files/zest-issta19.pdf, retrieved 2026-08-29). - OSS-Fuzz: default engines libfuzzer, afl, honggfuzz, centipede; seed corpus
<target>_seed_corpus.zip,<target>.dict,<target>.optionswith[libfuzzer]keys such asmax_len,rss_limit_mb = 6000,timeout = 30; inputs over “~25 seconds or more than 2.5GB RAM” are reported as timeout or OOM bugs; Rust projects build withcargo fuzz build -Oonbase-builder-rust. https://google.github.io/oss-fuzz/getting-started/new-project-guide/, /new-project-guide/rust-lang/, /faq/, retrieved 2026-08-29. Ideal integration: targets live in the project repository, run in regression CI, ship dictionaries and seed corpora, must not hang or exhaust memory instantly. https://google.github.io/oss-fuzz/advanced-topics/ideal-integration/. - usvg parser limits (
crates/usvg/src/parser/svgtree/parse.rs,main, retrieved 2026-08-29):if depth > 1024 { return Err(Error::NodesLimitReached); }inparse_xml_node;if doc.nodes.len() > 1_000_000 { return Err(Error::NodesLimitReached); };useself-reference checkif link == node || link == origin;fix_recursive_patterns,fix_recursive_links,fix_recursive_fe_imagereplace self-referential paint/clip/mask/filter links. roxmltreeParsingOptions { allow_dtd, nodes_limit }andError::EntityReferenceLoop(depth limit 10, 255 references per reference). https://docs.rs/roxmltree/latest/roxmltree/, retrieved 2026-08-29. No fuzz directory exists in the resvg repository and no OSS-Fuzzprojects/resvgentry was found. - Skia:
fuzz/Fuzz.hwraps the byte buffer withnext<T>(),nextRange(min, max),exhausted();FuzzCanvas(Fuzz*, SkCanvas*, int depth = 9)returns whendepth <= 0or the buffer is exhausted, draws up to 2000 ops and recurses withdepth - 1into paints, image filters and pictures;fuzz/oss_fuzz/FuzzSVG.cpprejects inputs over 30,000 bytes and renders the SVG DOM to a 128×128 surface. https://github.com/google/skia,fuzz/Fuzz.h,fuzz/FuzzCanvasHelpers.h,fuzz/FuzzCanvasHelpers.cpp,fuzz/oss_fuzz/FuzzSVG.cpp,main, retrieved 2026-08-29. - image 0.25.10:
Limits { max_image_width, max_image_height, max_alloc }withmax_allocdefault 512 MiB andreserve/freeaccounting; fuzz targets such asfuzz/fuzzers/fuzzer_script_png.rscallimage::load_from_memory_with_format(data, ImageFormat::Png). https://docs.rs/image/latest/image/struct.Limits.html and https://github.com/image-rs/image/tree/main/fuzz, retrieved 2026-08-29. - serde_json:
remaining_depth: 128withcheck_recursion!returningRecursionLimitExceeded;disable_recursion_limitrequires theunbounded_depthfeature and the docs recommend another stack-overflow guard. https://docs.rs/serde_json/latest/serde_json/struct.Deserializer.html#method.disable_recursion_limit, retrieved 2026-08-29. - fontations:
cargo +nightly fuzz build -O --debug-assertions;fuzz_skrifa_outline.rsiterates glyphs over size, location, hinting and memory variants;helpers.rscaps variation axes at 5. https://github.com/googlefonts/fontations/tree/main/fuzz, retrieved 2026-08-29.
Mechanism
// Structure-aware target over the NUIF document model (synthesis; attributions inline)
#[derive(Arbitrary, Debug)]
struct FuzzDoc { root: FuzzNode, context: FuzzContext }
impl<'a> Arbitrary<'a> for FuzzNode { // manual impl: explicit depth bound (Zest Fig. 2; Skia depth = 9)
fn arbitrary(u: &mut Unstructured<'a>) -> Result<Self> { gen_node(u, 0) }
}
fn gen_node(u, depth) -> Result<FuzzNode> {
let n_children = if depth >= MAX_DEPTH { 0 } else { u.int_in_range(0..=MAX_CHILDREN)? };
... // arbitrary_len / arbitrary_iter stop on exhausted bytes
}
fuzz_target!(|doc: FuzzDoc| -> Corpus { // libfuzzer-sys; decoding failure auto-rejected
let bytes = encode_canonical(&doc.into_document()); // bounded by construction
if bytes.len() > MAX_INPUT { return Corpus::Reject; }
let decoded = decode_with_limits(&bytes, Limits { depth: 256, entities: 100_000, alloc: 256 MiB });
match decoded {
Err(e) if e.is_limit() => Corpus::Keep, // limits must fire, not overflow (usvg, serde_json)
Err(_) => Corpus::Reject,
Ok(d) => { assert_eq!(encode_canonical(&d), bytes); layout_with_budget(&d, &doc.context); Corpus::Keep }
}
});
Run configuration: cargo fuzz run codec_roundtrip -- -max_len=65536 -timeout=10 -rss_limit_mb=2048 -malloc_limit_mb=512 -dict=fuzz/nuif.dict -use_value_profile=1 -jobs=8; cargo fuzz cmin after each campaign; cargo fuzz tmin on crashes; cargo fuzz coverage to compare corpus coverage with the property-based generator.
Budget classes to enforce inside NUIF parsers, with source examples: nesting depth (usvg 1024; serde_json 128; roxmltree entity depth 10), node or entity count (usvg 1,000,000; roxmltree nodes_limit), input size (Skia 30,000 bytes for SVG; libFuzzer -max_len), allocation (image max_alloc 512 MiB; libFuzzer -malloc_limit_mb), reference cycles (usvg fix_recursive_*), and time (libFuzzer -timeout; OSS-Fuzz 25 s).
Invariants: the target is deterministic for a given input; every limit produces a typed error rather than a panic or stack overflow; decoded documents re-encode to identical bytes; extension payloads decoded from arbitrary bytes are preserved verbatim.
NUIF relevance
Borrow
cargo fuzz/libFuzzer raw byte targets for every untrusted parser and a parametric byte choice stream for valid semantic operations, matching thefuzz parsers/codecs and path geometrytechnique inconformance/PLAN.md.- Concrete limit values from usvg, serde_json and image as starting points for the bounds that
spec/11-security.mdrequires (depth, entity count, allocation, cycle detection). - libFuzzer’s target contract (deterministic, no exit, no global state) as the acceptance rule for every headless engine entry point exposed through
spec/12-cli-api-and-automation.md. - Zest’s valid-coverage feedback: keep corpus inputs that are valid documents and add coverage, reject structurally invalid ones so the corpus stays useful for the round-trip loop.
Adapt
- Depth bounding must be explicit in
Arbitraryimplementations becausearbitrary’sMAX_DEPTHguards only size hints; NUIF should generate nodes with a depth parameter as in Zest and Skia. - The same typed operation choice-stream mapper is reusable by deterministic trials and coverage-guided fuzzing; it delegates values and invariants to production operation types instead of serializing a second document model (Zest parametric generators; Hypothesis choice sequences).
- Extension payloads are opaque bytes and should be fuzzed for preservation, not for interpretation: the oracle is byte equality after round trip (
rfcs/0002-extension-preservation.md).
Reject
- Grammar-mutator custom mutators (AFL++ Grammar-Mutator, libprotobuf-mutator) are unnecessary while the Rust model already provides a typed generator; they add a second grammar to maintain.
- A raw-byte-only campaign with no valid corpus. NUIF deliberately retains raw malformed parser inputs but regenerates valid canonical/package/resource/source seeds from production fixtures so mutations reach post-parse relations.
Implemented decision
fuzz/ is a standalone workspace pinned to nightly 2026-08-28,
cargo-fuzz 0.13.2 and libfuzzer-sys 0.4.13. Five targets separate codec,
package/archive, PNG/font, static-source-adapter and valid operation concerns.
Each owns a generated target-specific corpus under ignored target/; corpora
are not shared across incompatible input selectors. CPU rendering is sampled
only after valid typed operations, while GPU execution is excluded from the
security fuzzer because it has a separate nondeterministic process/device risk
boundary. cargo xtask fuzz-smoke applies 10-second, 512-MiB allocation and
2-GiB RSS limits, records every target, and CI runs 512 inputs per target under
AddressSanitizer. Format resource limits remain normative where declared by
spec/11-security.md; campaign limits are implementation test budgets rather
than wire-profile requirements.
GitBook bidirectional Git synchronization and content structure
Document status:
verified. Canonical source.
Summary
GitBook Git Sync is bidirectional. Repository commits update a GitBook space,
and GitBook editor changes update the configured repository branch. GitBook
uses .gitbook.yaml to select one root and uses SUMMARY.md as the table of
contents. When no summary exists, GitBook can infer and later create or update
one from editor state.
Evidence
- GitBook states that Git Sync automatically synchronizes changes from its editor and commits from GitHub or GitLab. Locator: GitHub & GitLab Sync, “Overview”, retrieved 2026-08-30.
.gitbook.yamlselects a root directory; other paths are relative to that root. Locator: Content configuration, “Root”, retrieved 2026-08-30: https://gitbook.com/docs/getting-started/git-sync/content-configuration.SUMMARY.mdmirrors the GitBook table of contents and GitBook creates or updates it when content is edited in GitBook. Locator: same document, “Summary”, retrieved 2026-08-30.- GitBook warns that creating README files through its editor can produce duplicates, rendering conflicts and unpredictable precedence. Locator: Troubleshooting, “Be sure to only create readme files in your repo”, retrieved 2026-08-30: https://gitbook.com/docs/getting-started/git-sync/troubleshooting.
Mechanism
The GitBook space stores presentation and editing state. Synchronization maps that state to Markdown, README and summary files on a selected branch. This is useful when GitBook is an accepted authoring system, but it does not implement a read-only compilation boundary from scattered canonical repository sources.
NUIF relevance
Reject GitBook Git Sync for the canonical NUIF documentation. The project requires source changes to pass research, link, metadata and conformance checks in one repository transaction. A bidirectional service would add a second writer and service-owned publication state.
Open questions
- GitBook could consume a dedicated exported branch, but that branch would duplicate generated documentation and provide no required capability over Pages artifacts.
GitHub Pages publication through Actions artifacts
Document status:
verified. Canonical source.
Summary
GitHub Pages supports branch publication and custom GitHub Actions workflows.
The custom workflow path builds static files, uploads one Pages artifact and
deploys that artifact. A pull request can execute the build and omit deployment.
The generated site therefore does not require a committed gh-pages branch.
Evidence
- The publishing-source documentation states that a custom Actions workflow is appropriate when the project uses a generator other than Jekyll or does not want a branch containing compiled files. Locator: “About publishing sources”, lines 28–31, retrieved 2026-08-30.
- The documented workflow checks out the repository, builds static files,
invokes
actions/upload-pages-artifact, and deploys withactions/deploy-pagesonly for the default branch. Locator: “Publishing with a custom GitHub Actions workflow”, lines 84–90, retrieved 2026-08-30. - The deployment uses a
github-pagesenvironment, for which GitHub recommends a protection rule that restricts deployment to the default branch. Locator: same section, line 90, retrieved 2026-08-30.
Mechanism
The build job receives read-only repository contents and emits a static site
directory. actions/upload-pages-artifact transfers that directory between
jobs. A deployment job with pages: write and id-token: write publishes the
artifact through the github-pages environment. Generated files remain
workflow artifacts rather than source revisions.
NUIF relevance
Borrow the artifact deployment boundary. NUIF can keep Markdown, research
metadata and specification modules in their current paths while an xtask
compiler stages the presentation under target/.
Reject a generated-output branch. It would create a second review history without adding a distinct source artifact.
Open questions
- The custom domain and deployment protection rules remain repository settings.
- Immutable specification snapshots require a separate tagged-source policy; the first Pages site publishes the current default-branch view.
GitHub prerelease delivery, artifact provenance, and desktop signing boundaries
Document status:
verified. Canonical source.
Summary
Cargo accepts Semantic Versioning prerelease identifiers such as
0.1.0-alpha.1 and orders numeric prerelease components numerically. GitHub
Releases attach binary assets to a tag and distinguish prereleases from stable
releases. GitHub’s immutable-release procedure creates a draft, attaches all
assets, and publishes the draft; publication then prevents tag and asset
mutation when repository immutability is enabled. GitHub artifact attestations
bind an artifact digest to the workflow identity and source revision through
OpenID Connect.
Direct desktop distribution has a separate trust boundary. Apple notarization requires Developer ID signing, the hardened runtime, a secure timestamp, and a notary-service submission. Microsoft documents Artifact Signing as its recommended signing service for non-Store distribution and states that unsigned applications receive stronger SmartScreen warnings. Repository-hosted checksums and GitHub attestations establish origin and integrity, but they do not replace operating-system code signing.
NUIF uses tag-driven GitHub prereleases for the reference editor. The first tag
is v0.1.0-alpha.1. Five native-host jobs build versioned archives, record
package manifests, and attest the archives. A final job creates checksums, a
separate editor and MCP CycloneDX software bills of materials, and a release
manifest, uploads all files to a draft, and publishes the prerelease. Every external workflow action is
pinned to the full commit of a verified release, checkout credentials are not
persisted, and a pinned zizmor audit rejects regressions. The alpha artifacts
remain explicitly unsigned until platform credentials are configured and
reviewed.
Evidence
- Cargo requires three numeric version components and permits a hyphenated
prerelease whose period-separated numeric components compare numerically.
Locator: Cargo Book, The Manifest Format,
versionfield, lines 88–98, retrieved 2026-08-30: https://doc.rust-lang.org/cargo/reference/manifest.html#the-version-field. - GitHub recommends creating a draft, attaching all assets, and publishing the draft for immutable releases. Published immutable releases prevent tag and asset mutation and receive a release attestation. Locator: Immutable releases, “What immutable releases protect” and “Best practices for publishing immutable releases”, retrieved 2026-08-30: https://docs.github.com/en/code-security/concepts/supply-chain-security/immutable-releases.
- GitHub artifact attestations require
id-token: write,contents: read, andattestations: write;actions/attest@v4accepts asubject-pathfor binary provenance. Locator: Using artifact attestations to establish provenance for builds, binary example, retrieved 2026-08-30: https://docs.github.com/en/actions/how-tos/secure-your-work/use-artifact-attestations/use-artifact-attestations. - GitHub’s public runner table lists Ubuntu 24.04 on x86-64 and Arm64, Windows
2025 on x86-64, macOS 15 on Arm64, and
macos-15-intelon x86-64. Locator: GitHub-hosted runners reference, public repository table, lines 40–51, retrieved 2026-08-30: https://docs.github.com/en/actions/reference/runners/github-hosted-runners. - Apple requires a Developer ID certificate, hardened runtime, secure
timestamp, and valid executable signatures before notarization.
notarytoolandstaplersupport scripted distribution. Locator: Notarizing macOS software before distribution, “Prepare your software for notarization” and “Add a notarization step to your build scripts”, retrieved 2026-08-30: https://developer.apple.com/documentation/security/notarizing-macos-software-before-distribution. - Microsoft states that unsigned applications cannot transfer publisher reputation between releases and identifies Artifact Signing as the recommended non-Store signing service. Locator: SmartScreen reputation for Windows app developers, “Certificate options” and “Minimizing SmartScreen warnings”, updated 2026-05-06 and retrieved 2026-08-30: https://learn.microsoft.com/en-us/windows/apps/package-and-deploy/smartscreen-reputation.
- cargo-dist 0.32.0 generates release workflows and general Rust archives. The
NUIF editor already has native application layouts, package-local manifests,
semantic trials, and platform-specific smoke tests in
cargo xtask editor-package. Replacing that path would duplicate package policy without removing signing credentials or host verification. Locator: cargo-dist releasev0.32.0, published 2026-05-21 and retrieved 2026-08-30: https://github.com/axodotdev/cargo-dist/releases/tag/v0.32.0; NUIFxtask/src/main.rs,build_editor_packageandverify_editor_package. - cargo-cyclonedx 0.5.9 generates per-crate or per-binary CycloneDX documents
from Cargo metadata and the lock file. It honors
SOURCE_DATE_EPOCHand omits the random serial number for reproducible output. Locator: cargo-cyclonedx release0.5.9, published 2026-03-19 and retrieved 2026-08-30: https://github.com/CycloneDX/cyclonedx-rust-cargo/releases/tag/cargo-cyclonedx-0.5.9. - GitHub states that pinning an action to a full-length commit SHA is the only
immutable way to use an action. NUIF resolved the selected release tags
through the GitHub Git data API and records both the SHA and human-readable
release beside each
usesentry. Locator: GitHub Docs, Secure use reference, “Using third-party actions”, retrieved 2026-08-30: https://docs.github.com/en/actions/reference/security/secure-use. - zizmor 1.29.0 identifies mutable action references, persisted checkout
credentials, expression-to-shell interpolation, undocumented permissions,
and absent concurrency controls. After remediation,
zizmor 1.29.0 --pedantic .reports no findings; CI runs the same version throughzizmor-action0.6.2 with its action itself pinned by full SHA. Locator: zizmor audit documentation and action release, retrieved 2026-08-30: https://docs.zizmor.sh/audits/ and https://github.com/zizmorcore/zizmor-action/releases/tag/v0.6.2.
Mechanism
The editor version in apps/editor/Cargo.toml determines the only accepted
release tag: v followed by that exact version. cargo xtask release-check
rejects a mismatched tag, SemVer build metadata, or an uncommitted source tree.
The tag checkout runs the complete verification harness before native package
jobs start.
Each native job builds and tests on the target operating system and processor
architecture. cargo xtask editor-package produces a versioned archive and a
platform manifest containing the source revision, binary digest, archive
digest, smoke-test result, and signing status. actions/attest@v4 records
provenance for both files. Five additional native jobs build and exercise the
separately versioned stateless MCP binary, then attest its archive and manifest.
A second five-host matrix packages the standalone CLI only after its release
binary creates, validates, canonicalizes and inspects a reference document.
The publication job downloads all editor, binding, MCP and CLI artifacts,
requires five archives/manifests for each native product, and runs the attested
cargo-cyclonedx 0.5.9 binary with the tagged commit time as
SOURCE_DATE_EPOCH. It replaces the checkout path with /src, writes
SHA256SUMS, and combines editor packages, browser bindings, MCP services and
CLI tools into release-manifest.json. It attests all software bills of
materials and both
index files before using GitHub CLI to create or resume a draft, upload the
assets, and publish the prerelease.
Workflow dependencies are resolved separately from Cargo dependencies. All
uses references are full commit SHAs with release-version comments, checkout
sets persist-credentials: false, and shell steps receive GitHub context values
through environment variables rather than source interpolation. Default token
access is read-only; only the package and publication jobs receive the OIDC,
attestation, and release-write permissions they require. CI cancels superseded
runs and executes a pinned pedantic zizmor audit, making these constraints
enforceable rather than review conventions.
The macOS package separates the SemVer version from the bundle build number.
CFBundleShortVersionString receives the three-component base version, while
CFBundleVersion receives the numeric GitHub workflow run number. This avoids
placing the alpha.1 suffix in Apple’s numeric bundle fields. The package
manifest and archive name retain the full SemVer version.
NUIF relevance
Borrow GitHub’s draft-attach-publish sequence, artifact attestations, and native runner matrix. These mechanisms bind release assets to a reviewed tag without introducing a release-specific build service.
Adapt SemVer at the application boundary. The editor uses
0.1.0-alpha.1, while draft specification profiles and unpublished library
crates retain their existing version namespaces.
Reject cargo-dist for the first alpha. Its generic archive generation does not replace the existing macOS bundle, Linux desktop layout, Windows GUI wrapper, package manifest, semantic trial, or signing boundary.
Reject describing unsigned alpha artifacts as trusted desktop installations. Checksums and attestations provide integrity and provenance; Developer ID notarization and Windows publisher signing remain separate credentialed release stages.
Open questions
- Which organization identity and credential store will sign macOS and Windows artifacts after the alpha review?
- Whether immutable releases are enabled is a repository setting that requires user review; the workflow remains compatible by using a draft before publication.
- Whether the stable editor distribution uses direct archives, the Microsoft Store, a macOS disk image, or installer packages depends on signing and update policy that the alpha does not establish.
GitHub Wiki repository and indexing boundaries
Document status:
verified. Canonical source.
Summary
A GitHub Wiki has an independent Git repository whose live content comes from its default branch. Public editing can be restricted, but Wiki changes retain a history separate from the project repository. GitHub also restricts search engine indexing of most Wikis and directs projects that require indexing to GitHub Pages.
Evidence
- Local Wiki editing clones
https://github.com/OWNER/REPOSITORY.wiki.git; only changes pushed to the default branch become live. Locator: Adding or editing wiki pages, “Adding or editing wiki pages locally”, lines 58–68, retrieved 2026-08-30: https://docs.github.com/en/communities/documenting-your-project-with-wikis/adding-or-editing-wiki-pages. - Search engines index a public Wiki only when it has at least 500 stars and public editing is disabled. GitHub recommends Pages when indexing is needed. Locator: About wikis, lines 25–28, retrieved 2026-08-30.
- Wikis have a soft limit of 5,000 files, after which pages can become inaccessible. Locator: About wikis, lines 30–32, retrieved 2026-08-30.
Mechanism
Wiki page edits create commits in the Wiki repository. A generated mirror would copy project Markdown into that repository and create a second revision graph. Direct Wiki edits would create a second authoring surface with no atomic commit across code, conformance fixtures and their documentation.
NUIF relevance
Reject the Wiki as both canonical documentation and generated mirror. NUIF requires documentation claims to remain reviewable with the code, research records and conformance artifacts they describe.
Open questions
- A one-page Wiki redirect is technically possible but provides no capability that the repository homepage and Pages URL do not already provide.
KHR_interactivity portable behavior graphs
Document status:
reviewed. Canonical source.
Summary
KHR_interactivity adds portable, self-contained behavior graphs to glTF. Graph nodes model events, control flow, value operations and state while companion extensions add optional capabilities. The design explicitly considers constrained execution and graceful handling of unavailable companion operations.
Evidence
The current specification is a Release Candidate. It defines directed acyclic graphs, strictly typed value sockets, retained custom variables, static and dynamic implementation limits, and no-op replacement for unsupported extension operations. Its full runtime remains Turing-complete because operations may repeatedly activate flow, so time and execution budgets are still necessary. The 16 July 2026 announcement says it was submitted for ratification; that does not make it a ratified extension.
NUIF relevance
NUIF should use a separate behavior/state graph rather than embedding arbitrary scripts into visual nodes. The core behavior profile should remain declarative, bounded and capability-aware; richer runtime logic belongs in extensions or host application code.
Khronos glTF Validator report format, sample-asset corpus and extension prefix registry as a conformance kit
Document status:
reviewed. Canonical source.
Summary
The Khronos glTF ecosystem pairs a specification with three executable conformance artifacts. The glTF Validator (Dart, Apache-2.0) checks JSON schema conformance, reference validity, binary buffer contents, images, GLB container structure and a set of extensions, and writes a JSON report described by docs/validation.schema.json: uri, mimeType, validatorVersion, validatedAt, an issues block with counts (numErrors, numWarnings, numInfos, numHints), a truncated flag and messages each carrying code, severity (0 Error, 1 Warning, 2 Information, 3 Hint), message and either a JSON pointer or a GLB byte offset, plus an info block. ISSUES.md enumerates 159 codes across six categories (IoError, SchemaError, SemanticError, LinkError, DataError, GlbError). Unknown extensions are Information (UNSUPPORTED_EXTENSION), undeclared use is an Error (UNDECLARED_EXTENSION), and unknown properties are Warnings (UNEXPECTED_PROPERTY).
glTF-Sample-Assets is “a curated collection of glTF models that illustrate one or more features or capabilities of glTF”; each model has metadata.json (name, path, summary, tags, legal with SPDX license, screenshot) and is indexed in Models/model-index.json with tags core, extension, testing, showcase, video, written, pbrtest, issues and format variants (glTF, glTF-Binary, glTF-Embedded, glTF-Draco, glTF-Quantized, glTF-KTX-BasisU). CI runs the validator over every asset. The extension registry defines prefixes (KHR reserved for Khronos, EXT for multi-vendor, 99 registered vendor prefixes obtained by GitHub issue), naming rules, a five-stage status ladder (Proposal, Initial Draft, Review Draft, Release Candidate, Ratified) and the rule that extensions “can’t remove existing glTF properties or redefine existing glTF properties”.
Evidence
- Report schema:
uri,mimeType(model/gltf+jsonormodel/gltf-binary),validatorVersion(semver),validatedAt(date-time),issues.{numErrors,numWarnings,numInfos,numHints,messages,truncated}, messageseverityenum 0–3 with descriptions Error/Warning/Information/Hint,pointer(json-pointer) oroffsetrequired — https://github.com/KhronosGroup/glTF-Validator/blob/main/docs/validation.schema.json (retrieved 2026-08-29). - CLI: report written to
<asset_filename>.report.json, recursive directory validation, “Shell return code will be non-zero if at least one error was found”; options-o/--stdout,-r/--validate-resources,-t/--write-timestamp,-p/--absolute-path,-m/--messages,-a/--all,-c/--config,-h/--threads—README.md(main). - Config file:
max-issues,ignorelist,onlylist,overrideseverity map with “0 - Error, 1 - Warning, 2 - Info, 3 - Hint” —docs/config-example.yaml. - Implemented feature classes: JSON syntax and GLB correctness, schema properties, reference validity, Data URI, accessor values (NaN, invalid quaternions, indecomposable matrices),
accessor.min/max, sparse accessors, animation I/O, image NPOT and unsupported features, extension validation forEXT_texture_webp,KHR_animation_pointer(partial),KHR_lights_punctual,KHR_materials_anisotropy, … —README.md“Implemented features”. - Issue table: 159 codes; category sizes IoError 1, SchemaError 14, SemanticError 41, LinkError 52, DataError 35, GlbError 16 —
ISSUES.md(main, retrieved 2026-08-29). - Specific codes:
UNEXPECTED_PROPERTYWarning (line 20),INVALID_EXTENSION_NAME_FORMATWarning (42),NON_RELATIVE_URIWarning (73),UNKNOWN_ASSET_MAJOR_VERSIONError /UNKNOWN_ASSET_MINOR_VERSIONWarning (78–79),INCOMPLETE_EXTENSION_SUPPORTInformation (102),UNDECLARED_EXTENSIONError (128),UNEXPECTED_EXTENSION_OBJECTError (129),UNSUPPORTED_EXTENSIONInformation (131),UNUSED_OBJECTInformation (134) —ISSUES.md. - Validator test corpus:
test/base/data/<category>/<case>.gltfpaired with<case>.gltf.report.jsongolden reports (e.g.accessor/alignment.gltf.report.json,custom_property.gltf.report.json); categories accessor, animation, asset, buffer, buffer_view, camera, glb, image, json, material, mesh, node, root, sampler, scene, skin, texture and_datavariants;test/extfor extensions — repository listing (retrieved 2026-08-29). - Latest tags
2.0.0-dev.3.10,2.0.0-dev.3.8,2.0.0-dev.3.7; distribution via npmgltf-validator, hosted drag-and-drop tool at https://github.khronos.org/glTF-Validator —README.md, tags list. - Sample assets purpose and lists: Showcase, Complete, Testing (“intended to be used for testing of viewers, converts, and other software systems”), Core Only, Video Tutorials, Written Tutorials, PBR tests, Issues — https://github.com/KhronosGroup/glTF-Sample-Assets/blob/main/README.md (retrieved 2026-08-29).
- Provided forms: separate-resource
.gltf, embedded Data URI (to be avoided except for specific cases), binary.glb— same README, “Model Contents”. Models/model-index.jsonentries withlabel,name,screenshot,tags,variants(e.g.ABeautifulGamewithglTF,glTF-Binary,glTF-Binary-KTX-ETC1S-Draco;AlphaBlendModeTesttaggedcore,testing) — file head (retrieved 2026-08-29); 162 model directories underModels/(listing).- Per-model
metadata.jsonwithversion: 2,legal[](license, licenseUrl, artist, year, owner, what, spdx),tags,screenshot,name,path,summary,createReadme—Models/Box/metadata.json,Models/NegativeScaleTest/metadata.json. - Sample-assets CI installs validator 2.0.0-dev.3.10 and runs
./gltf_validator -r -a ./Models/, uploading**/*.report.json—.github/workflows/ci.ymllines 1–30. - The archived glTF-Sample-Models repository was replaced; unlicensed assets removed;
2.0renamedModels— sample-assets README “Obsolete Interface”. - Prefix registry:
KHRandEXTreserved; 99 registered vendor prefixes (ADOBE, AGI, AMZN, BLENDER, CESIUM, EPIC, GODOT, GOOGLE, MSFT, NV, OMI, UNITY, VRMC, …); request “by submitting an issue on GitHub” with prefix and vendor name — https://github.com/KhronosGroup/glTF/blob/main/extensions/Prefixes.md andextensions/README.mdlines 117–121 (retrieved 2026-08-29). - Naming rules: uppercase prefix plus underscore, lowercase snake-case, recommended
<PREFIX>_<scope>_<feature>—extensions/README.mdlines 184–191. - Status ladder table (Proposal, Initial Draft, Review Draft, Release Candidate, Ratified); Review Draft requires “At least one third party glTF implementation”; Release Candidate requires Sample Viewer and Validator support —
extensions/README.mdlines 66–75. - “Extensions can’t remove existing glTF properties or redefine existing glTF properties to mean something else”;
extensionsUsedvsextensionsRequired; required iff “a typical glTF loader would fail to load the asset in the absence of support” —extensions/README.mdlines 131–175. - Extension schemas “should allow additional properties”;
extrasis the application-specific escape hatch distinct from extensions —extensions/README.mdlines 178–215. - Current in-progress KHR list includes
KHR_interactivity(Release Candidate),KHR_gaussian_splatting(Release Candidate),KHR_texture_procedurals(Initial Draft) —extensions/README.mdlines 45–57.
Mechanism
The validator is a single-pass structural checker followed by link and data passes. Schema errors come from the JSON Schema of the core spec; semantic errors from cross-field constraints; link errors from index references between arrays (accessor → bufferView → buffer, node → mesh, …); data errors from decoding buffers and images; GLB errors from container parsing. Every finding is a coded issue with a fixed severity and a locator (JSON pointer for the JSON tree, byte offset for GLB). Severities are policy, not structure: a YAML config can override any code’s severity, ignore codes, or restrict to a subset, and a max-issues cap sets truncated. The exit code depends only on error count, so warnings never break a pipeline unless overridden to errors.
Extension handling encodes the ecosystem’s preservation contract. Any extension object must be declared in extensionsUsed (else Error). An extension the validator does not implement is reported as Information, not failure, so vendor extensions pass validation by default; partially implemented extensions are flagged as INCOMPLETE_EXTENSION_SUPPORT. Unknown properties outside extensions/extras are Warnings. This makes “opaque but declared” the validated state, and “opaque and undeclared” the failing state.
The test corpus is golden-report based: every input asset has a checked-in expected report, so validator changes are detected as report diffs. The sample-asset corpus is separately tagged by purpose; the testing tag marks assets that exercise a feature edge (alpha modes, negative scale, NPOT textures) and the core tag marks assets that need no extension. CI validates the entire corpus, and the extension status ladder requires validator and sample-viewer support before Release Candidate, closing the loop between spec, corpus and checker.
Prefix registration is deliberately lightweight: a GitHub issue reserves a namespace, and promotion from vendor to EXT requires multiple implementations while KHR requires Khronos ratification and IP coverage. Naming rules make ownership and scope parseable from the identifier.
NUIF relevance
Borrow
- Publish a JSON Schema for NUIF
validateoutput with counts per severity,truncated, coded messages, and a locator (entity/property path or byte offset for binary profiles), mirroringvalidation.schema.json. - Maintain a single issue-code table with fixed default severities and categories (schema, semantic, link, data, container), and require every diagnostic to cite a code.
- Make severity policy configurable (ignore, only, override, max-issues) while keeping exit status defined by error count, as the validator does.
- Treat “unknown but declared extension” as Information and “undeclared extension object” as Error, which is the executable form of NUIF’s used/required rule.
- Build the conformance fixture corpus with per-asset
metadata.json(SPDX license, tags, summary, variants) and a generated index; tag fixturescore,extension,testing,issues. - Store golden validation reports alongside fixtures so validator regressions surface as diffs.
- Adapt the prefix registry model to NUIF’s lowercase identifier grammar (
nuif.*,ext.*, collision-resistant vendor namespaces) and adopt the status ladder that requires validator and reference-viewer support before ratification.
Adapt
- glTF’s locator is a JSON pointer, which is path-based; NUIF locators must use stable semantic IDs with an optional path hint since NUIF identity is path-independent.
- The validator does not check preservation across a round trip; NUIF conformance must add a preservation suite (import → export → compare) that the glTF kit lacks.
- Format variants (
glTF-Binary,glTF-Embedded, …) map to NUIF profiles (nuif-text-0,nuif-cbor-0, package); NUIF should require every fixture in every profile with canonical-hash equality. extrasas an untyped escape hatch conflicts with NUIF’s requirement that extension data be namespaced; NUIF should route such data through a vendor extension instead.
Reject
- Dart as an implementation language is irrelevant; NUIF’s validator is the Rust CLI.
- Severity-only categorization without fidelity classes is insufficient; NUIF diagnostics must also carry the fidelity class (
lossless…unsupported) and the responsible pass. - The sample corpus’s mixed CC-BY/CC0 licensing complicates redistribution in test suites; NUIF fixtures should be CC0 or project-licensed.
Open questions
- Whether NUIF should reserve numeric severity values (0–3) for compatibility with tooling that consumes glTF-style reports.
- How to represent multi-profile locators (text line/column, CBOR byte offset, semantic ID) in one report entry without ambiguity.
- Whether the validator’s “unsupported extension is Information” default is safe for NUIF where an unrenderable required extension must be a conformance failure at render time but not at parse time.
- How NUIF’s extension status ladder should handle dialects that change lowering rules rather than add properties.
glTF core and extension registry model
Document status:
reviewed. Canonical source.
Summary
glTF keeps a focused base format and grows through registered extensions. extensionsUsed and extensionsRequired let consumers distinguish optional data from capabilities required for correct loading/rendering. Prefix governance separates Khronos, multi-vendor and vendor namespaces.
NUIF relevance
Borrow explicit used/required capability declarations and staged extension governance. NUIF additionally needs a normative opaque-preservation rule so an editor can round-trip unknown payloads without understanding them.
Godot text scene format, resource identity and missing-type preservation
Document status:
reviewed. Canonical source.
Summary
Godot’s .tscn/.tres formats are INI-like text serializations of a PackedScene (a flattened SceneState) or a Resource. A file declares a format version and a uid, lists external resources by (type, uid, path, id), internal sub-resources by (type, id), then nodes by name, type, parent path and property assignments, then signal connections. Node identity in the file is the scene-relative node path; resource identity is a per-file local id plus a project-wide uid:// that survives renames and moves. Inherited scenes and instanced sub-scenes are stored as sparse overrides against a base PackedScene. Since pull request (PR) #60597 (merged 2022-05-05, milestone 4.0) nodes and resources whose class is not registered are loaded into MissingNode/MissingResource placeholders that record every assigned property and are written back under their original class name on save, so unknown types survive load/save round trips. Local ids for sub-resources are generated randomly and then cached, which gives diff stability only after the first save.
Evidence
- File descriptor
[gd_scene format=3 uid="uid://..."];format=3for Godot 4.x,format=2for 3.x;load_stepsis deprecated; five sections (descriptor, external resources, internal resources, nodes, connections) “should appear in order”. Godot docs (latest), “TSCN file format”, retrieved 2026-08-29. [ext_resource type="Texture2D" uid="uid://ccbm14ebjmpy1" path="res://gradient.tres" id="2_eorut"];[sub_resource type="CapsuleShape3D" id="CapsuleShape3D_fdxgg"]; node heading[node name="PlayerCamera" type="Camera" parent="Player/Head" ...]; valid node keys includeinstance,instance_placeholder,owner,index,groups,node_paths; the root “must not have a parent= entry”, direct children useparent="."; comments start with;and are discarded on save. Same page.uidis “a unique string-based identifier representing the scene” enabling tracking when moved. Same page.ResourceUID: UIDs “allow the engine to keep references between resources intact, even if files are renamed or moved”;create_id()is random and unique among loaded UIDs;create_id_for_path()is deterministic, “seeded with the provided path and project name”;id_to_text()yieldsuid://...;set_id()rebinds a UID to a new path. Godot docs (stable), classResourceUID, retrieved 2026-08-29.- Text loader falls back to path when the UID is unknown:
WARN_PRINT("...invalid UID: " + uidt + " - using text path instead: " + path); ext_resource lines are written withuid=only whenResourceSaver::get_resource_id_for_pathreturns a valid id.scene/resources/resource_format_text.cpp, master, retrieved 2026-08-29. - Sub-resource ids: if
res->get_scene_unique_id()is empty, a new id<class>_<generate_scene_unique_id()>is generated until unused; duplicates are cleared and regenerated; ext_resource ids are<counter>_<scene_unique_id>underTOOLS_ENABLED. Same file. FORMAT_VERSION = 4(“PackedByteArray can be base64 encoded, and PackedVector4Array was added”) andFORMAT_VERSION_COMPAT = 3(“save as version 3 if not using PackedVector4Array or no big PackedByteArray”); loading refusesformat_version > FORMAT_VERSIONwith “Saved with newer format version”.scene/resources/resource_format_text.hand.cpp, master, retrieved 2026-08-29.#define PACKED_SCENE_VERSION 3; inSceneState::instantiate, whenClassDB::instantiatefails,missing_node = memnew(MissingNode); missing_node->set_original_class(snames[n.type]); missing_node->set_recording_properties(true); node = missing_node;.scene/resources/packed_scene.cpp, master, retrieved 2026-08-29.- On pack (
SceneState::_parse_node):MissingNode *missing_node = Object::cast_to<MissingNode>(p_node); if (missing_node != nullptr) { nd.type = _nm_get_string(missing_node->get_original_class(), name_map); }, so the original class name is written back. Same file. MissingNode::_setinserts any property whilerecording_propertiesis true and otherwise only updates existing keys;_get_property_listenumerates recorded properties with their runtime Variant type; configuration warnings: “This node was saved as class type ‘%s’, which was no longer available when this scene was loaded.”scene/main/missing_node.cpp, master, retrieved 2026-08-29.MissingNodeis “An internal editor class intended for keeping the data of unrecognized nodes” withoriginal_class,original_scene,recording_properties,recording_signals;MissingResourcelikewise withoriginal_classandrecording_properties; both warn that properties can be freely modified in code regardless of intended type. Godot docs (stable), classesMissingNodeandMissingResource, retrieved 2026-08-29.- Resource loader path: when
ResourceLoader::is_creating_missing_resources_if_class_unavailable_enabled(), an unknown class yieldsMissingResourcewithset_recording_properties(true), later disabled; properties that could not be set are stored underMETA_MISSING_RESOURCES.resource_format_text.cpp, master. - Motivating defect: issue #57427 (2022-01-29, neikeq) shows a node of a missing GDExtension type reverting to
Node, losing custom properties, the type attribute and signal connections on save; closed by PR #60597. GitHub issue #57427, retrieved 2026-08-29. - PR #60597 “Implement missing Node & Resource placeholders” (reduz, opened 2022-04-28, merged 2022-05-05, milestone 4.0): on save “both binary and text formats recognize these placeholders and convert them back to their original types”; the aim is that “missing types no longer cause data loss”. GitHub PR #60597, retrieved 2026-08-29.
recording_signalswas added toMissingNodeby PR #105449 (merged 2025-10-10); commitfdecca2f18onscene/main/missing_node.cpp. GitHub API query, retrieved 2026-08-29.- Counter-evidence on robustness: issue #99863 (2024-11-30, v4.3.stable) reports entire scenes turning into
MissingNodewithout a reproduction; closed and archived without a maintainer root cause. GitHub issue #99863, retrieved 2026-08-29. - Scene-unique names: a node renamed with a leading
%or marked “Access as Unique Name” is addressable as%Namefrom within the same scene; lookups are cached; access from other scenes goes through an intermediate node (get_node("%Sword/%Hilt")). Godot docs (stable), “Scene Unique Nodes”, retrieved 2026-08-29. PackedScene.pack()“Packs the path node, and all owned sub-nodes”;instantiate()triggers child scene instantiation;GEN_EDIT_STATE_MAIN_INHERITEDexists “for the case where the scene is being instantiated to be the base of another one”. Godot docs (latest), classPackedScene, retrieved 2026-08-29.SceneStatestores abase_scene_idxinto the variants array for inherited scenes,NO_PARENT_SAVED, and flagsFLAG_ID_IS_PATHandFLAG_INSTANCE_IS_PLACEHOLDER.packed_scene.cpp, master.
Mechanism
Serialization model. A PackedScene holds a SceneState: string tables (names), a variants array (property values, including references to external PackedScenes and sub-resources), and a node table where each node record has name, type, parent index, owner, instance reference, an ordered property list of (name index, variant index) pairs, groups, and connection records. pack() walks the tree from the root, including only nodes owned by the root (nodes created by instanced sub-scenes are represented by their instance root plus overrides, not expanded). The text writer emits the state as [node ...] sections in tree order, with parent expressed as a scene-relative path; node order within a parent is implicit in the section order and can be pinned with index.
Identity. Three identities coexist. Nodes are identified by path from the scene root (name-based, order-independent, changed by rename or reparent). Sub-resources and external resources get file-local ids of the form <Class>_<5 random alphanumerics>; the id is cached on the resource (scene_unique_id) so later saves reuse it, but a fresh resource or a duplicate gets a new random id. Files are identified project-wide by a 64-bit uid stored in the .import or resource file and mirrored in ext_resource uid=; the loader prefers the uid and falls back to path with a warning, so moves and renames do not break references as long as the uid cache is current.
Inheritance and overrides. An inherited scene stores a reference to its base PackedScene in the variants array (base_scene_idx) and then only the nodes and properties that differ: added nodes with their parent path into the base, and property assignments on base nodes. An instanced sub-scene inside a scene is a node with instance=ExtResource(...); child nodes of the instance are addressable for overrides only when the instance is marked editable, and overrides are again stored as sparse property assignments by path. Resolution (instantiate) instantiates the base first, then applies the derived state’s nodes and properties in order. Authored state is therefore the override set; the resolved tree exists only in memory.
Unknown types. Loading is class-name driven. When ClassDB cannot instantiate the recorded class, the loader creates a placeholder whose _set records every incoming (name, value) pair verbatim while recording_properties is on. Because the packer asks the node for get_property_list() and its values, the placeholder reports exactly the recorded pairs and the packer substitutes original_class for the type name, so the written section is byte-equivalent in content to the original (ordering and formatting are regenerated by the writer). Verified: preservation on save is implemented in SceneState::_parse_node (node path) and in the text saver’s MissingResource handling (resource path), not only at load time. Limits: placeholders are editor-facing (the docs warn users to ignore them), value types are not validated, signals were not recorded until PR #105449 in 2025, and there is at least one unresolved field report (#99863) of scenes collapsing into placeholders.
Diff stability. Text output is regenerated on every save from the in-memory state: comments are dropped, ids are reused when cached, and section order follows tree order. The format attribute is written as 3 unless a feature requiring 4 is used, so files do not churn on format version. Random suffixes on ids mean two users adding a sub-resource independently will produce distinct ids and a textual merge will not falsely unify them, at the cost of non-reproducible output for freshly created resources.
NUIF relevance
Borrow
- The two-phase “record everything you cannot interpret, write it back under the original type” placeholder design as a concrete realization of opaque-preservation for whole entities, including the observable rule that the packer treats placeholders identically to real nodes.
- Path-independent project-wide
uidfor files separate from file-local ids for embedded resources, with a documented fallback to path and a warning; NUIF asset references should carry the same (stable id, path hint) pair. - Writing the compatible format version whenever the newer features are unused, which keeps forward compatibility maximal without a separate export step.
Adapt
- Node identity by path is insufficient for NUIF (spec/02 forbids path-dependent identity); the sparse-override structure can be kept but keyed by stable entity IDs so that renames and reparents inside the base do not orphan overrides.
- Random local ids with post-hoc caching should become deterministic ids derived from the entity’s stable ID so canonical output is reproducible from the first save.
- Placeholder entities should carry a fidelity status (
preserved_unrenderable) and a declared origin namespace rather than an editor-only warning. - Record signals/relations of unknown entities from the start; Godot’s three-year gap before
recording_signalsshows why preservation must cover relations as well as properties.
Reject
- Type-less recording (the placeholder stores runtime Variant types inferred from values); NUIF preservation must retain the serialized encoding of unknown data rather than reinterpret it through the host type system.
- Implicit ordering by section order with an optional
indexescape hatch; NUIF containment order must be explicit and merge-safe.
Open questions
- Whether
MissingNoderound trips preserve property order and formatting well enough that a no-op load/save of a scene with unknown types yields an empty textual diff; not verified from retrieved sources. - The generation algorithm and entropy of
generate_scene_unique_id()and the collision behaviour on merge when two branches create the same suffix. - Whether the docs’
format=3statement will be updated forFORMAT_VERSION = 4in master, and how older editors handle a version-4 file (loader refuses withERR_FILE_UNRECOGNIZED).
Golden-master (characterization) and snapshot testing with insta, Jest and deterministic rendering baselines
Document status:
reviewed. Canonical source.
Summary
A characterization test records the observed behaviour of existing code as its oracle so that later changes are detected; it documents actual, not desired, behaviour (Feathers). Snapshot testing is the same idea automated: the first run stores the output, later runs diff against it. Jest 14 (2016) popularised the practice for UI trees and introduced the review-and-update workflow (toMatchSnapshot, -u, --ci). The Rust crate insta provides file and inline snapshots, serialised snapshot macros, redactions and filters for nondeterministic content, a with_settings! scope, an INSTA_UPDATE policy and the cargo insta review workflow. Grey-literature studies identify fragility, lack of context, large snapshots and blind approval as the main drawbacks. Deterministic visual baselines in browsers rely on the Ahem font, per-platform baselines, disabled animations and perceptual pixel thresholds.
For NUIF, snapshots are the storage form of the canonicalization, layout and render suites: canonical text, resolved box tables and images. The evidence below determines how to keep them deterministic and reviewable.
Evidence
- Characterization testing documents “your system’s actual behavior, not check for the behavior you wish your system had”; a production system “becomes its own specification”. Feathers, https://michaelfeathers.silvrback.com/characterization-testing, retrieved 2026-08-29. Book: Working Effectively with Legacy Code, Prentice Hall 2004, ISBN 0131177052, chapter 13 (chapter text not retrieved).
- Wikipedia equates characterization test with “Golden Master Testing” and notes such tests verify observed behaviour, not correctness. https://en.wikipedia.org/wiki/Characterization_test, retrieved 2026-08-29.
- ApprovalTests: “Also known as Golden Master Tests or Snapshot Testing”;
*.approved.*files are committed,*.received.*files are transitory; a reporter opens a diff tool on failure only. https://github.com/approvals/ApprovalTests.cpp/blob/master/doc/README.md and https://github.com/approvals/ApprovalTests.Net README, retrieved 2026-08-29. - Jest 14.0 (2016-07-27) introduced
toMatchSnapshot()withreact-test-renderer, storingpretty-formatoutput in.snapfiles and updating withjest -u. https://jestjs.io/blog/2016/07/27/jest-14, retrieved 2026-08-29. - Jest docs: snapshots live in
__snapshots__/*.snap;--updateSnapshot/-ure-records failing snapshots;--cifails instead of writing new snapshots; as of Jest 20 snapshots are not written on CI without--updateSnapshot;toMatchInlineSnapshot(); property matchers such asexpect.any(Date)are checked before the snapshot is written. https://jestjs.io/docs/snapshot-testing and https://jestjs.io/docs/cli, retrieved 2026-08-29. - Jest best practices: treat snapshots as code and review them; resist regenerating snapshots instead of examining root causes (
no-large-snapshotslint); tests must be deterministic (mockDate.now()); use descriptive names. https://jestjs.io/docs/snapshot-testing §Best Practices. - insta 1.48.0 stores file snapshots as
snapshots/<module>__<name>.snapnext to the test; pending snapshots are.snap.new(inline:.pending-snap); header is YAML withsource,expression,input_file, separated from the body by---; files are normalised to LF before diffing. https://docs.rs/insta/latest/insta/, https://insta.rs/docs/snapshot-types/, https://insta.rs/docs/snapshot-files/, retrieved 2026-08-29. - Macros:
assert_snapshot!,assert_debug_snapshot!,assert_json_snapshot!,assert_compact_json_snapshot!,assert_compact_debug_snapshot!,assert_yaml_snapshot!,assert_ron_snapshot!,assert_csv_snapshot!,assert_toml_snapshot!,assert_binary_snapshot!(experimental, compared byte for byte),with_settings!,glob!; featuresyaml,json,ron,csv,toml,redactions,filters,glob. https://docs.rs/insta/latest/insta/index.html#macros, retrieved 2026-08-29. - Inline snapshots use a trailing
@"..."argument; after review the tool rewrites it to@r###"..."###. https://insta.rs/docs/quickstart/ and https://insta.rs/docs/snapshot-types/. INSTA_UPDATE:auto(default:noon CI,newotherwise),new(write.snap.newpending review),always(write.snap, bypass review),unseen(alwaysfor new,newfor existing),no(never write),force(rewrite even if passing).INSTA_FORCE_PASS=1lets tests pass to collect multiple snapshots;INSTA_OUTPUT∈ {diff, summary, minimal, none};INSTA_WORKSPACE_ROOToverrides cargo-based root detection. https://docs.rs/insta/latest/insta/ and https://insta.rs/docs/advanced/, retrieved 2026-08-29.with_settings!({sort_maps => true}, { ... });Settingssetters:set_sort_maps(“forceful sorting of maps before serialization”),set_snapshot_path(defaultsnapshots),set_prepend_module_to_snapshot,set_snapshot_suffix(parameterised tests),set_description,set_info,set_omit_expression,set_redactions,set_filters,set_strip_ansi_escape_codes,set_input_file,set_comparator. https://insta.rs/docs/settings/ and https://docs.rs/insta/latest/insta/struct.Settings.html, retrieved 2026-08-29.- Redactions (feature
redactions) are a third macro argument{ "selector" => replacement }with selectors.key,["key"],[index],[],[start:end],.*,.**; helpersinsta::dynamic_redaction(|value, path| ...),insta::sorted_redaction()for unordered collections andinsta::rounded_redaction(3)for floats. https://insta.rs/docs/redactions/, retrieved 2026-08-29. - Filters (feature
filters) are regex replacements applied to the string form, e.g.(r"\b[[:xdigit:]]{32}\b", "[UID]"), for content that is inherently textual. https://insta.rs/docs/filters/, retrieved 2026-08-29. cargo instasubcommands:review(aliasverify),accept,reject,testwith--review,--accept,--accept-unseen,--check,--force-update-snapshots, and--unreferenced∈ {ignore, warn, reject, delete, auto};pending-snapshots;show. https://insta.rs/docs/cli/ andcargo-insta/src/cli.rsonmaster, retrieved 2026-08-29.- YAML is the recommended serialiser “because YAML is human readable and excellent at diffing because it is line based”. https://insta.rs/docs/serializers/, retrieved 2026-08-29.
- Pitfalls: a grey-literature review of 50 documents finds fragility (28%), lack of context (22%), large snapshots (16%), manual verification (12%) and flakiness (6%); “blindly updating the test results” is the named failure mode; mitigations are code review (26%), treating snapshots as code (22%) and small snapshots (14%). Cruz, Rocha, Valente, “Snapshot testing in practice: Benefits and drawbacks”, JSS 204 (2023) 111797, DOI 10.1016/j.jss.2023.111797, §2, §4.2 Table 3, §4.3 Table 4 (PDF https://homepages.dcc.ufmg.br/~mtov/pub/2023-jss-snapshot.pdf, retrieved 2026-08-29).
- Dodds (2017) quotes Searls: developers “will sooner just nuke the snapshot and record a fresh passing one”, and states that snapshots beyond a few dozen lines suffer maintenance issues. https://kentcdodds.com/blog/effective-snapshot-testing, retrieved 2026-08-29.
- Playwright: screenshots differ across browsers and platforms “due to different rendering, fonts and more” and must be generated in the same environment; file names carry browser and platform suffixes;
toHaveScreenshotdefaultsanimations: "disabled",caret: "hide",scale: "css",threshold0.2 (YIQ perceived colour difference), optionalmaxDiffPixels,maxDiffPixelRatio,mask,stylePath. https://playwright.dev/docs/test-snapshots and https://playwright.dev/docs/api/class-pageassertions#page-assertions-to-have-screenshot-1, retrieved 2026-08-29. - pixelmatch v5.3.0
threshold0.1 default using YIQ colour difference (Kotsarenko and Ramos 2010) with anti-aliasing detection; the current README cites OKLab instead. https://github.com/mapbox/pixelmatch/blob/v5.3.0/README.md and https://github.com/mapbox/pixelmatch, retrieved 2026-08-29. - WPT reftest fuzzy matching:
<meta name="fuzzy" content="maxDifference=15;totalPixels=300">with inclusive ranges and per-reference prefixes. https://web-platform-tests.org/writing-tests/reftests.html, retrieved 2026-08-29. - Ahem font: “well defined glyphs of precise sizes and shapes”; em-square exactly square; baseline 0.2em above bottom; X is a 1em square, p a 0.2em rectangle below baseline, É a 0.8em rectangle above, space transparent. https://web-platform-tests.org/writing-tests/ahem.html, retrieved 2026-08-29. Chromium: “Use the Ahem font to reduce the variance introduced by the platform’s text rendering system”; pixel baselines are
-expected.pngwithplatform/<PLATFORM-VERSION>fallback chains. https://chromium.googlesource.com/chromium/src/+/main/docs/testing/writing_web_tests.md and web_test_baseline_fallback.md, retrieved 2026-08-29. - expect-test 1.5.1 offers
expect![[...]]inline snapshots updated withUPDATE_EXPECT=1and lists insta as the more complete alternative. https://docs.rs/expect-test, retrieved 2026-08-29.
Mechanism
Snapshot assertion lifecycle (insta):
assert_*_snapshot!(name?, value, redactions?):
text = serialize(value, format) # yaml/json/ron/csv/toml/debug/display
text = apply_redactions(text, selectors) # structured, before serialization
text = apply_filters(text, regexes) # string level
path = snapshot_path / (module__name[suffix]).snap
if exists(path) and body(path) == text: pass
else match INSTA_UPDATE:
no -> fail
new -> write path.snap.new; fail (pending review)
always -> write path.snap; pass
unseen -> exists ? new : always
force -> write path.snap regardless
review: cargo insta review walks *.snap.new, shows diff, accept/reject moves or deletes
Determinism checklist for NUIF snapshots, with attribution:
- Stable key ordering:
BTreeMapinnuif_core::Documentalready sorts; usesort_mapsfor anyHashMap(insta settings). - Float normalisation:
rounded_redaction(n)or, preferably, round in the canonical encoder so that the snapshot equals the canonical text (insta redactions;spec/08-serialization.mdnumeric normalisation). - Redact or fix generated identifiers and timestamps; NUIF should instead generate IDs from the seed so no redaction is needed (Jest property matchers; JSS D14).
- Pin fonts: ship Ahem or an equivalent metric-defined font for text fixtures (WPT Ahem; Chromium; Taffy and Yoga fixtures embed Ahem).
- Fix viewport, scale factor, writing direction, locale and theme through
nuif_layout::EvaluationContext(Playwrightscale,deviceScaleFactor). - No animations or carets in rendered snapshots (Playwright defaults).
- Per-platform image baselines only where the rasteriser is platform dependent; prefer a CPU reference path so one baseline suffices (Chromium fallback chains as the case to avoid).
- Perceptual tolerance declared per fixture (
maxDifference;totalPixels, YIQ/OKLab threshold), never global (WPT fuzzy; pixelmatch). - Never write snapshots on CI (
INSTA_UPDATE=no, Jest--ci). - Keep snapshots small and named; review in code review; prune with
--unreferenced=delete(Jest best practices; JSS Table 4; cargo-insta).
NUIF relevance
Borrow
- insta file snapshots in YAML for canonical documents, resolved box tables and fidelity reports, with
with_settings!suffixes for the 360/768/1440 viewport matrix (instaset_snapshot_suffix,sort_maps). - The
INSTA_UPDATE=noon CI pluscargo insta reviewworkflow as the approval gate for the golden structural fixtures named inconformance/PLAN.md. - Ahem-style metric fonts for text fixtures and WPT-style
maxDifference;totalPixelsfuzzy declarations for render fixtures where exact pixels are not normative. - Snapshot header metadata (
description,info) to carry implementation version, capability profile, fixture ID and evaluation context as required byconformance/PLAN.md.
Adapt
- Redactions should be unnecessary for canonical NUIF output; if a snapshot needs redaction, the encoder or the seed handling is nondeterministic and should be fixed instead.
- Image snapshots must record the comparison policy (metric, threshold, pixel budget) in the machine-readable report, not only in test code.
- Snapshot churn is a signal: the report should count snapshot updates per change so that over-approval is measurable (JSS 2023 fragility finding).
Reject
- Large whole-document snapshots of rendered DOM or HTML export output; export tests should snapshot the fidelity report and a normalised structural view, not raw formatter output.
- Platform-specific baseline fallback chains; NUIF should require a deterministic CPU reference renderer instead.
Open questions
- Which of the canonical text profile or YAML
Debugoutput should be the snapshot body, given that the canonical text is itself the normative artefact? - Can
insta::Settings::set_comparatorhost a tolerance-aware comparator for box tables so that resolved layouts are snapshotted with declared epsilon rather than exact text? - How should image snapshots be stored in the repository without bloating history: content-addressed assets in the
.nuifpackage or Git LFS?
Sources of nondeterminism in GPU rendering and determinism tiers for conformance
Document status:
reviewed. Canonical source.
Summary
GPU rasterisation is specified so that several results are permitted for the same input. WGSL does not fix a rounding mode, allows reassociation and fusion of floating-point operations, allows subnormal flushing, permits implementations to assume no NaN or infinity at runtime, gives only ULP bounds for transcendental functions, and gives no error bound at all for derivatives and determinants. WebGPU defines pixel-centre sampling and a standard multisample pattern but leaves pixel-centre-on-edge inclusion, line rasterisation and polygon barycentrics for more than three vertices implementation-dependent; Vulkan only guarantees standard sample locations when standardSampleLocations is reported. Independent of the API, IEEE-754 arithmetic is non-associative, so any change in reduction order (thread count, workgroup scheduling, atomics) or in compiler contraction (FMA) changes the low-order bits. Reproducible summation is possible but requires algorithms that renderers do not use. Rendering projects therefore stratify: a CPU reference path with fixed evaluation order for exact comparison, platform-keyed baselines for known-good variants, and perceptual or count-and-delta tolerances for GPU output.
Evidence
- WGSL rounding: “No rounding mode is specified. An implementation may round an intermediate result up or down.” (WGSL §15.7.2 Differences from IEEE-754).
- WGSL finite-math assumption: “Implementations may assume that overflow, infinities, and NaNs are not present during shader execution”, and in that case an overflowing runtime expression yields “an indeterminate value of the target type”; implementations may also ignore the sign of zero (WGSL §15.7.2).
- WGSL flush-to-zero: “Any inputs or outputs of operations listed in § 15.7.4 Floating Point Accuracy may be flushed to zero”; other operations must preserve subnormals (WGSL §15.7.2).
- WGSL reassociation and fusion: “An implementation may reassociate operations.” and “An implementation may fuse operations if the transformed expression is at least as accurate as the original formulation.” (WGSL §15.7.5).
- WGSL accuracy table (f32):
x + y,x - y,x * ycorrectly rounded;x / y2.5 ULP;exp3 + 2|x| ULP;inverseSqrt2 ULP;cosabsolute error at most 2⁻¹¹ on [−π, π];logabsolute error 2⁻²¹ on [0.5, 2];min/maxon two subnormals may return either input;dpdx/dpdy/fwidthanddeterminantare listed as “Infinite ULP” with notes that implementations “should provide a pragmatically useful” function (WGSL §15.7.4.1). - WGSL derivatives: invocations in a quad “collaborate to compute approximate partial derivatives”; a derivative call in non-uniform control flow returns “an indeterminate value” (WGSL §15.6.2).
- WGSL portability statement: “WGSL sometimes permits several possible behaviors for a given feature. This is a portability hazard” (WGSL §1, Technical Overview).
- WGSL data races are dynamic errors that “may or may not be detectable” (WGSL §2.3 and §6.5.7 notes on data races).
- WebGPU sampling: with multisampling disabled, fragments are at pixel centres (fract(C) = (0.5, 0.5)) and “If a pixel center is on the edge of the polygon, whether or not it’s included is not defined” (WebGPU §23.2.5.4 Polygon Rasterization).
- WebGPU multisample pattern: “Implementations must use the standard sample pattern for the given multisample.count”; count 1: (0.5, 0.5); count 4: (0.375, 0.125), (0.875, 0.375), (0.125, 0.625), (0.625, 0.875) (WebGPU §23.2.5 Rasterization). The same section’s polygon step still describes per-pixel sample locations as “implementation-defined” (§23.2.5.4), an inconsistency in the current draft text.
- WebGPU lines and polygons: “The exact algorithm used for line rasterization is not defined, and may differ between implementations” (§23.2.5.2); barycentrics for polygons with more than three vertices are “implementation-dependent” (§23.2.5.3).
- WebGPU invalid data: GPU handling of NaN and infinity in resources is “subject to the accuracy of the GPU hardware implementation of the IEEE-754 standard”; subnormals “may be either preserved or replaced by -0.0 or +0.0”; NaN or Infinity “may be replaced by an indeterminate value” (WebGPU §2.1.5 Invalid Data).
- WebGPU texture LOD: implicit level-of-detail derivation is illustrated only by a non-normative reference to the Vulkan LOD operation (WebGPU GPUSampler section note).
- Vulkan sample locations: standard locations for 1, 2, 4, 8 and 16 samples apply only “If the standardSampleLocations member of VkPhysicalDeviceLimits is VK_TRUE”; otherwise locations are implementation-dependent; the 4-sample table matches WebGPU’s (Vulkan specification, Rasterization → Multisampling).
- NVIDIA whitepaper (Whitehead, Fit-Florea, “Precision & Performance: Floating Point and IEEE 754 Compliance for NVIDIA GPUs”): FMA rounds once whereas separate multiply and add round twice (§2.3); rn((A + B) + C) and rn(A + (B + C)) differ (§2); “Different math libraries cannot be expected to compute exactly the same result for a given input” (§5); changing the number of threads in a parallel reduction “rearranges parentheses” and gives different but equally valid results (§5.3); compiler flags
-ftz,-prec-div,-prec-sqrt(and-fmad) change results (§4.4) (docs.nvidia.com/cuda/floating-point). - Reproducible reductions: Demmel and Nguyen, “Parallel Reproducible Summation”, IEEE Transactions on Computers 64(7):2060–2070, 2015, DOI 10.1109/TC.2014.2345391; Collange, Defour, Graillat, Iakymchuk, “Numerical reproducibility for the parallel reduction on multi- and many-core architectures”, Parallel Computing 49:83–97, 2015, DOI 10.1016/j.parco.2015.09.001 (Crossref records). Both establish order-independent summation at extra cost.
- Observed consequences in renderers: Vello attributes non-zero GPU/CPU differences to “fast math on the GPU or different precisions” (
vello_tests/src/compare.rs) and platform “fast math” on Apple (vello_tests/README.md); resvg excludes a gradient fixture for “a SIMD rounding difference” even on the CPU (crates/resvg/tests/gen-tests.py); the FLIP tool’s own error maps are not pixel-identical across operating systems (NVlabs/flip src/python/README.md); WebRender renders CI reftests with OSMesa “to get consistent rendering across platforms” and still annotatesfuzzy-if(platform(swgl),...)(gfx/wr/README.md;wrench/reftests/text/reftest.list).
Mechanism
Sources of variance (S) and where each is permitted
S1 rounding/contraction: WGSL 15.7.2 (no rounding mode), 15.7.5 (reassociate, fuse); FMA one rounding vs two
S2 transcendental ULP: WGSL 15.7.4.1 (exp, log, cos, /, inverseSqrt bounds; derivatives unbounded)
S3 subnormals/NaN/Inf: WGSL 15.7.2 flush-to-zero; finite-math assumption -> indeterminate values
S4 reduction order: thread count / workgroup schedule / atomics order change the parenthesisation of sums
S5 coverage: pixel-centre-on-edge undefined (WebGPU 23.2.5.4); line algorithm undefined (23.2.5.2)
S6 multisampling: standard pattern required by WebGPU; Vulkan only with standardSampleLocations
S7 texture filtering/LOD: LOD selection non-normative; filtering precision unspecified
S8 compiler/driver: naga/tint/dxc/metal compilers apply different contractions and reassociations
Determinism tiers (NUIF interpretation)
Tier 0 bit-exact: CPU reference, fixed evaluation order, no FMA or pinned FMA, single or deterministic multithreading;
policy: |a - b| == 0 for all channels (resvg, vello_cpu f32 tolerance 0)
Tier 1 bounded: same algorithm, different SIMD/thread schedule or u8 pipeline;
policy: per-channel |a - b| <= t (t = 1..2) and count(diff) <= n (WPT/WebRender/Gold fuzzy)
Tier 2 perceptual: GPU backends across adapters and drivers;
policy: mean FLIP(ppd) < tau (Vello 0.01), plus baseline keyed by (backend, adapter class, driver)
Result record must carry: renderer id+version, tier, backend, adapter/driver, pixel ratio, fixture id, context hash
Reproducible-by-construction rules for Tier 0
sum in fixed order (no atomics-based accumulation); avoid dpdx/fwidth; avoid transcendental functions in coverage;
quantise anti-aliasing coverage to a fixed grid; avoid MSAA (use analytic or supersampled area coverage);
disable fast-math flags; avoid subnormal-dependent branches
NUIF relevance
Borrow
- Adopt the three-tier stratification (bit-exact CPU, bounded per-channel, perceptual GPU) with mandatory result metadata, because every surveyed project converged on some form of it and the WebGPU/WGSL texts make a single-tier exact policy impossible for GPU output.
- Adopt WebGPU’s standard sample pattern and pixel-centre rule as the definition of coverage in spec/05 where NUIF specifies anti-aliasing semantics, because it is the only normative sample geometry shared by WebGPU and Vulkan (when the limit is present).
Adapt
- Specify NUIF’s normative coverage as area coverage computed in a fixed evaluation order on the CPU path rather than as MSAA, because the pixel-centre-on-edge rule is undefined and MSAA sample positions are only conditionally standard.
- Turn the WGSL accuracy table into a fixture design rule: render fixtures should not depend on the low-order bits of
exp,log,cos, derivatives or determinants, because those are the operations with loose or unbounded error.
Reject
- Do not attempt bit-exact conformance across GPU backends, because reassociation, fusion, flush-to-zero and indeterminate values are permitted by the shader language itself.
- Do not use reproducible-summation algorithms in the interactive renderer, because their cost is unjustified when a CPU reference path already provides Tier 0.
Open questions
- Whether
wgpu/nagaexposes or could expose a “no contraction” or “strict” shader compilation option for the Vello backend to shrink Tier 2 variance. - How to detect at runtime whether an adapter uses standard sample locations (Vulkan reports it; Metal and D3D12 mappings through wgpu are not documented here).
- Whether the WebGPU draft’s inconsistency between “must use the standard sample pattern” and “locations, which are implementation-defined” is resolved in a later editor’s draft.
HarfBuzz shaping and Unicode text algorithms
Document status:
reviewed. Canonical source.
Summary
HarfBuzz maps Unicode code points to ordered glyph IDs, clusters, advances and offsets based on font data and shaping rules. Unicode UAX #9/#14/#29 define bidi, line-break and segmentation foundations, but actual line choice and font rendering remain higher-level concerns.
NUIF relevance
Canonical documents store semantic text, runs, font references and layout intent. Shaped glyph runs are resolved/cache data tied to evaluator context and font hashes; they must not replace source text.
Houdini procedural networks, cooking, dirty propagation and digital assets
Document status:
reviewed. Canonical source.
Summary
Houdini stores a scene as networks of operator nodes with parameters and wires; this network is the authored intent and is what a .hip file persists. Geometry, images and simulation states are produced by cooking: a pull-based, memoized evaluation in which a node recomputes only when it is asked for its data and is marked out of date. Parameter edits dirty every dependent node transitively; extra (non-wire) dependencies must be declared on every cook because they are cleared during dirty propagation. Digital assets (HDAs) package a subnetwork as a reusable operator type identified by namespace::name::version, with instances locked to the definition by default and explicit unlock/save/match operations. hython runs the same object model headlessly.
Evidence
- “In Houdini, cooking refers to evaluating the nodes in the networks to compute the state of the scene in the current frame”; update modes Auto Update, On Mouse Up, Manual with Force Update. SideFX docs, “Cooking” (
basics/cooking.html), retrieved 2026-08-29. - “Nodes are never cooked unless they are asked for their data”; a node recooks only when asked and “itself is out of date”; recooking “propagates up the cook chain”; “If Houdini at any point encounters a node that is up to date, then no further cooking will be done”; cooking is framed as functional evaluation without side effects; SOP implementers override
SOP_Node::cookMySop(). HDK docs, “Cooking” (_h_d_k__op_basics__overview__cooking.html), retrieved 2026-08-29. - “When a parameter changes, everything in the graph that depends on the parameter’s data is dirtied accordingly”; parameters are dependencies by default; other-node data must be declared via
OP_Node::addExtraInput(); extra inputs “are cleared as soon as they are traversed upon the dirty propagation”, so the call must happen on every cook;DOP_Parent::simMicroNode()tracks simulation dependencies. HDK docs, “Dependencies” (_h_d_k__op_basics__overview__dependencies.html), retrieved 2026-08-29. hou.OpNode.cook(force=False, frame_range=())“Asks or forces the node to re-cook”;needsToCook(time=hou.time())“Asks if the node needs to re-cook”;isTimeDependent(for_last_cook=False): a time dependent node “is re-evaluated every time the frame changes”;cookCount()counts cooks in the session;matchesCurrentDefinition(),allowEditingOfContents(propagate=False),isLockedHDA(). SideFX docs, classhou.OpNode, retrieved 2026-08-29.hythonis a Python shell that adds$HHPtosys.path, importshou, loads.hipfiles passed on the command line, accepts%-prefixed hscript, and checks out a Houdini Batch licence (falling back to FX). SideFX docs, “Command line scripting” (hom/commandline.html), retrieved 2026-08-29.- HDA internal names are
[namespace::]name[::version]; a version “can only contain numbers and periods”; without a version Houdini “selects the node with the highest version number”; scoped and un-namespaced definitions take precedence in a documented order;HOUDINI_OPNAMESPACE_HIERARCHYoverrides. SideFX docs, “Namespaces and versions” (assets/namespaces.html), retrieved 2026-08-29. - A digital asset is created by converting a subnetwork; multiple assets can share a
.hdalibrary; “you can’t change the internal name without recreating the asset”; assets can be saved embedded in the HIP file. SideFX docs, “Create a digital asset” (assets/create.html), retrieved 2026-08-29. - Locked instances “match the current definition”; “Allow editing of contents” unlocks; “Save node type” writes to the library; “Match current definition” relocks and discards; while unlocked, “Other instances of the same asset will get the same changes, but the original definition of the asset still exists on disk”. SideFX docs, “Editing digital assets” (
assets/edit.html), retrieved 2026-08-29. - Asset Manager: black names use the current definition, yellow means a newer definition exists elsewhere, red means not current; “Use This Definition” pins a definition; priority options for index files, HIP-embedded and latest-date definitions; “Safeguard Operator Definitions” removes unlock menu items. SideFX docs, “Asset Manager window” (
ref/windows/optypemanager.html), retrieved 2026-08-29.
Mechanism
Authored state is the operator graph: node types, parameter values (which may be expressions over time and other parameters), wires, and flags. Resolved state is the cook output per node per OP_Context (time plus evaluation options), held in per-node caches. Dirtying is push-based: a parameter or input change marks the node and walks outgoing edges, marking dependents; declared extra inputs are consumed during this walk, which is why they are re-declared on every cook. Cooking is pull-based: a consumer (viewport display flag, renderer, script) requests data; if the node is dirty it requests its inputs recursively, recomputes and clears its dirty flag. Time-dependent nodes are dirtied on every frame change. Update modes only change when the UI issues pull requests. The invariant is memoized purity: identical inputs and parameters at a given context yield identical outputs, so caches can be trusted until dirtied.
Digital assets are operator type definitions stored outside the scene. An instance stores its type name and parameter values; its internal subnetwork is not persisted while locked, so the HIP references the definition and resolves contents at load. Unlocking copies the definition’s contents into the instance (a local override of the whole subnetwork); saving pushes the copy back as the new definition; matching discards the copy. Definition selection is name-based with version ordering and configurable precedence, so two libraries can supply the same type name and the resolved definition depends on environment and Asset Manager preferences.
NUIF relevance
Borrow
- Separation of persisted authored graph from cached cooked output, with per-context caches invalidated by dependency dirtying; this is the model for NUIF’s resolved layer, where layout and text shaping results are caches keyed by evaluation context (spec/04, RFC 0003).
- Explicit dependency declaration for non-structural dependencies (token references, expressions) and the rule that such dependencies are re-established on every evaluation.
- Version in the type name with numeric ordering and highest-wins default for component library resolution.
Adapt
- Push-dirty/pull-cook fits NUIF layout evaluation, but NUIF must record which context a resolved value belongs to and never overwrite authored values (spec/02), whereas Houdini caches are anonymous per node.
- HDA lock/unlock is whole-subnetwork override; NUIF instance overrides are per property and per slot, so the analogue is a typed override set with an explicit “detached from definition” state that fidelity accounting can report.
Reject
- Definition resolution dependent on environment variables and editor preferences; NUIF component references must resolve to a specific identity and version recorded in the document.
- Loss of the unlocked subnetwork’s relation to its definition beyond a type name; NUIF requires provenance records linking overrides to the definition version they were authored against.
Open questions
- Whether Houdini persists per-node cook caches to disk in any format that could be compared with NUIF resolved caches; the retrieved docs describe in-memory caching only.
- How PDG/TOPs work-item dependencies differ from OP-level dirtying (not retrieved; the
tops/cooking.htmlpage was located but not fetched).
Hydra scene index, render delegate abstraction, dirty tracking and image-diff testing
Document status:
reviewed. Canonical source.
Summary
Hydra is Pixar’s imaging framework whose stated goal is “to decouple scene processing from rendering, and both from the application”. Hydra 1.0 used three abstractions: HdSceneDelegate (adapter to a client scene graph), HdRenderIndex (a flattened representation of the scene that tracks changes through HdChangeTracker dirty bits and orchestrates prim sync) and HdRenderDelegate (the backend that creates rprims, sprims and bprims and owns a resource registry). Renderers are discovered at run time as HdRendererPlugin instances through the Plug system. Hydra 2.0 replaces scene delegates with HdSceneIndexBase (a queryable prim tree of nested data sources), replaces the render index with a graph of filtering scene indices, and replaces coarse dirty bits with hierarchical HdDataSourceLocator invalidation; legacy delegates and render delegates are wrapped by adapter classes.
Pixar tests Hydra at two levels. Core hd tests are C++ unit tests (testHdSceneIndex, testHdDirtyBitsTranslator, testHdMergingSceneIndex, testHdDataSourceLocator, …) that use a recording observer to assert exact notification sequences. Imaging tests (testUsdImagingGL*) render a stage offscreen to PNG and compare against checked-in baselines using OpenImageIO idiff with per-test pixel and percentage thresholds passed from CMake through cmake/macros/testWrapper.py.
Evidence
- Design goal and definitions of scene index, scene index observer, filtering scene index, scene index plugin — Hydra 2.0 Getting Started Guide, section “What is Hydra 2.0”, https://openusd.org/release/api/_page__hydra__getting__started__guide.html (v26.08 docs, retrieved 2026-08-29).
- Mapping from 1.0 to 2.0:
HdSceneDelegate→ scene indices,HdRenderIndex→ scene index graph viaHdSceneIndexPluginRegistry,HdRenderDelegate→HdRenderer; adaptersHdRenderDelegateAdapterRendererandHdRenderIndexAdapterSceneIndex; env togglesHD_ENABLE_SCENE_INDEX_EMULATION,USDIMAGINGGL_ENGINE_ENABLE_SCENE_INDEX— same guide, section “How does Hydra 2.0 compare to Legacy Hydra 1.0?”. HdSceneIndexBaseinterface:GetPrim,GetChildPrimPaths,_SendPrimsAdded/Removed/Dirtied/Renamed; invariant that add/remove notices must match traversal viaGetChildPrimPaths— same guide, section “The Scene Index API”;pxr/imaging/hd/sceneIndex.hlines 48–121, 181–207.HdSceneIndexObserverentries:AddedPrimEntryacts as resync if the path exists;RemovedPrimEntryis a subtree;DirtiedPrimEntrycarriesHdDataSourceLocatorSetinterpreted hierarchically;PrimsRenamedis an optimization over remove+add — same guide.- Data source model:
HdContainerDataSource::GetNames/Get,HdVectorDataSource,HdSampledDataSource::GetValue(shutterOffset)andGetContributingSampleTimesForInterval— same guide, section “Prim Data”. HdRenderIndexdocumented as “a flattened representation of the client scene graph”, tied to a singleHdRenderDelegate, tracking changes viaHdChangeTracker, orchestrating “syncing”; now “only used for emulation purposes” —pxr/imaging/hd/renderIndex.hlines 63–107;SyncAll,GetChangeTracker,InsertRprim,InsertSceneIndex,_emulationSceneIndex,_mergingSceneIndex,_terminalSceneIndex— lines 204–590.HdChangeTracker“Tracks changes from the HdSceneDelegate, providing invalidation cues to the render engine”; flags accumulate until the resource is next required —pxr/imaging/hd/changeTracker.hlines 6–14;RprimDirtyBitsincludesClean,InitRepr,Varying,DirtyPoints,DirtyTopology,DirtyPrimvar,DirtyTransform,DirtyVisibility,DirtyNormals,DirtyMaterialId,DirtyInstancer,DirtyRenderTag,AllDirty; version countersGetSceneStateVersion,GetRprimIndexVersion,GetVisibilityChangeCount,GetRenderTagVersion— same header (retrieved via WebFetch 2026-08-29).HdRenderDelegatepure virtualsGetSupportedRprimTypes/SprimTypes/BprimTypes,GetResourceRegistry,CreateRprim/Sprim/Bprim/Instancer,CreateRenderPass,CommitResources(HdChangeTracker*); optionalGetRenderSettingDescriptors,GetCapabilities,IsPauseSupported,IsParallelSyncEnabled—pxr/imaging/hd/renderDelegate.hlines 106–551.HdRenderParamis “an opaque (to core Hydra) handle” passed to prims during sync —renderDelegate.hlines 43–45.HdRendererPlugin: “dynamically discovered and loaded at run-time using the Plug system”, singleton per library,IsSupported(reasonWhyNot)—pxr/imaging/hd/rendererPlugin.hlines 23–54.HdSceneDelegateis “Adapter class providing data exchange with the client scene graph” —pxr/imaging/hd/sceneDelegate.hline 400–402.- Core unit tests registered in
pxr/imaging/hd/CMakeLists.txt:testHdSceneIndex,testHdDirtyBitsTranslator,testHdDirtyList,testHdMergingSceneIndex,testHdDataSourceLocator,testHdDataSource,testHdSortedIds*,testHdTimeSampleArray,testHdExtCompDependencySort(listing retrieved 2026-08-29). testHdSceneIndex.cppdefinesRecordingSceneIndexObserverwithEventType_PrimAdded/Removed/Dirtied, hashes events and compares event vectors andGetChildPrimPathsresults via_CompareValue—pxr/imaging/hd/testenv/testHdSceneIndex.cpplines 100–606.- Image tests:
pxr_register_testacceptsIMAGE_DIFF_COMPARE,WARN,WARN_PERCENT,HARD_WARN,FAIL,FAIL_PERCENT,HARD_FAIL,PERCEPTUAL,DIFF_COMPARE,EXPECTED_RETURN_CODE,TESTENV,ENV,PRE_COMMAND,POST_COMMAND—cmake/macros/Public.cmakelines 772–784, 892–946. testWrapper.py_imageDiffshells out toidiff(idiff.exeon Windows) with-warn,-warnpercent,-hardwarn,-fail,-failpercent,-hardfail,-p; return codes 0 OK, 1 warning, 2 failure, 3 size mismatch, 4 file error; only 0 and 1 pass; failing pairs copied to--failures-dir—cmake/macros/testWrapper.pylines 199–260.- Baseline lookup
_resolvePathchecks anon-specificsubdirectory before the platform baseline directory; text diffs use systemdiff --strip-trailing-cr(fc.exeon Windows) —testWrapper.pylines 113–197. - Concrete thresholds:
testUsdImagingGLBasicDrawingusesFAIL 0.2 FAIL_PERCENT 0.5 PERCEPTUAL;testUsdImagingGLInstancing_instancedCubesusesFAIL 0.01 FAIL_PERCENT 0.005 WARN 0.02 WARN_PERCENT 0.0025;testUsdImagingGLPurposecompares four images withFAIL 0.1 FAIL_PERCENT 10 PERCEPTUAL—pxr/usdImaging/usdImagingGL/CMakeLists.txt(retrieved 2026-08-29). - Test binaries
testUsdImagingGLBasicDrawing,...Highlight,...PickAndHighlight,...InstancePicking,...Resync,...SurfaceShader,...SublayerOperations,...Purpose,...PopOut,...TextureResync; tests skipped on macOS, Windows, headless and static builds — same CMakeLists. - Baselines are checked-in PNGs, e.g.
pxr/usdImaging/usdImagingGL/testenv/testUsdImagingGLBasicDrawing/baseline/testUsdImagingGLBasicDrawing.png,_refined.png,_shadersAnim_001.png; theusdImagingGL/testenvtree holds 119 entries (listing retrieved 2026-08-29). testUsdImagingGLBasicDrawing.cppselects the renderer plugin viaSetRendererPlugin(_GetRenderer())and writes the color AOV withWriteToFile(_engine.get(), HdAovTokens->color, imageFilePath)— lines 84–307.
Mechanism
Hydra 1.0 is a retained-mode pipeline. The application inserts prims into the render index by type id and path; the render delegate instantiates backend-specific HdRprim/HdSprim/HdBprim objects for the types it advertises. Scene edits do not rebuild the scene; the scene delegate marks dirty bits on the change tracker. When a task executes, the render index computes the set of prims needing sync, calls Sync on each with the delegate as data source and the dirty bits as the work list, then clears the bits. Version counters (scene state, index versions, visibility, render tags) let consumers detect coarse changes without walking prims. CommitResources gives the backend a barrier after sync. This is the sync-not-regenerate pattern: invalidation is fine-grained, pull-based and cleared on consumption.
Hydra 2.0 replaces the render index with a scene index graph. Each scene index is both a query surface (GetPrim, GetChildPrimPaths) and a notification source; filtering scene indices compose as a chain, each observing the previous. Invalidation is addressed by HdDataSourceLocator paths into nested container data sources and interpreted hierarchically, replacing the fixed dirty-bit vocabulary with an open, structured one. Sampled data sources expose time-varying values through shutter offsets and contributing sample times so renderers can reconstruct motion blur without a separate API. Emulation classes wrap old delegates and render delegates so both worlds interoperate during the transition.
Testing has an exact tier and a tolerance tier. The exact tier records observer events from a scene index under scripted mutations and compares them to expected sequences, and checks that traversal agrees with notifications (the documented invariant). The tolerance tier runs a renderer plugin offscreen, writes an AOV to disk and delegates comparison to idiff with declared per-pixel thresholds (FAIL), fraction-of-pixels thresholds (FAIL_PERCENT), an absolute per-pixel ceiling (HARD_FAIL) and an optional perceptual metric. Baselines are per-platform directories with a non-specific fallback, and thresholds are declared per test rather than globally.
NUIF relevance
Borrow
- Separate a queryable scene abstraction from backends, with backends declaring supported prim types and capabilities, as
HdRenderDelegate::GetSupportedRprimTypesandGetCapabilitiesdo. - Use pull-based, hierarchical invalidation (locator sets over nested data sources) between the NUIF resolved model and render backends instead of regenerating the render scene on each edit.
- Require the notification/traversal consistency invariant (adds and removes must equal a fresh traversal) and test it with a recording observer, as
testHdSceneIndexdoes. - Declare image-comparison tolerances per fixture (
FAIL,FAIL_PERCENT,HARD_FAIL, perceptual) and keep per-platform baselines with a platform-neutral fallback directory. - Treat the render-scene boundary as an emulation point: adapters allow old and new backends to coexist during migration, which NUIF should plan for around ADR 0003.
Adapt
- Hydra’s shutter-offset sampling model maps to NUIF evaluation contexts (viewport, theme, state); NUIF should parametrize data-source queries by context rather than by time.
- Dirty bits are backend-facing; NUIF must additionally map invalidation back to authored entities for fidelity and provenance, which Hydra does not attempt.
- Hydra’s baselines are opaque PNGs; NUIF conformance should also compare structured render plans (vector display lists) before rasterization so failures are attributable.
idiffthresholds are chosen per test by hand; NUIF should derive tolerances from the normative text-rendering and anti-aliasing allowances in spec/05 and record them in fixtures.
Reject
- The Plug-system dynamic discovery of renderer plugins is not needed for NUIF conformance; backends can be static Rust trait implementations.
- Hydra skips image tests on macOS, Windows, headless and static builds; NUIF must run its reference renderer headlessly on all platforms because the headless QA contract is normative.
- The 1.0 fixed dirty-bit enum should not be copied; NUIF should start from locator-style structured invalidation.
Open questions
- Which perceptual metric
idiff -pimplements and whether an equivalent is acceptable for text-heavy UI renders where small shifts are semantically significant. - How to express NUIF evaluation-context dimensions in a locator scheme so a change to a token or breakpoint invalidates exactly the dependent resolved values.
- Whether NUIF render backends should expose a
CommitResources-style barrier or rely on immutable snapshot handoff. - How much of the 2.0 filtering scene index pattern maps onto NUIF lowering passes (authored → resolved → render scene) versus onto editor-side view transforms.
Rust GUI toolkits compared for a headless-testable native editor (iced, Slint, GPUI, Makepad, Floem, Dioxus Native, egui, Masonry)
Document status:
reviewed. Canonical source.
Summary
Eight Rust GUI stacks were examined against the requirements in apps/editor/ARCHITECTURE.md and apps/editor/QA.md: the editor must be driven headlessly by the same engine as the CLI, must produce deterministic snapshots, and must be testable without synthetic pointer input. The comparison criteria are rendering backend, headless harness maturity, accessibility tree, text stack, docking/panels, WASM target, licence and MSRV. Every toolkit except Makepad (crates.io release) now ships a headless harness of some kind. Only egui (egui_kittest), Masonry (masonry_testing), Slint (i-slint-backend-testing) and Blitz (blitz-test-harness) expose a semantic tree that tests can query by role or label; of these, egui and Masonry use AccessKit as the query surface, Slint uses its own element tree with AccessKit only at the platform boundary, and Blitz uses DOM selectors with AccessKit behind a feature. Only Masonry, Floem and Blitz render through Vello. Every toolkit’s MSRV (1.88 to 1.96) exceeds the NUIF pin of 1.85.0.
NUIF interpretation: the requirement set favours Masonry (Vello, Parley, AccessKit, CPU screenshot harness, owned tree) for a Rust-native editor, with egui_kittest as the reference for query ergonomics. iced and Slint have credible headless simulators but weaker or non-AccessKit semantic surfaces; GPUI demonstrates that a large editor can be tested headlessly with deterministic executors but ships no accessibility-tree test surface and is tied to Zed’s release cadence.
Evidence
- iced 0.14.0 released 2025-12-07; workspace
rust-version = "1.88",license = "MIT",edition = "2024"; default features includewgpuandtiny-skia(software renderer);cosmic-text = "0.15",wgpu = "27.0",winit = "0.30"; noaccesskitdependency appears in the workspace manifest. Locator:Cargo.tomllines 25-31, 158-238 at tag 0.14.0;CHANGELOG.mdline 9. - iced_test 0.14.0: “A library for testing iced applications in headless mode”;
simulator(view),Simulator::{new, with_settings, with_size, find(selector), point_at, click(selector), tap_key, typewrite, simulate(events), snapshot(theme) -> Snapshot, into_messages};Snapshot::{matches_image(path), matches_hash(path)};snapshot()renders atscale_factor = 2.0throughRenderer::screenshotand requirescore::renderer::Headless. Selectors:&str(text),String,widget::Id,Point,selector::id,selector::is_focused. Locator:test/Cargo.toml;test/src/simulator.rslines 26-350;selector/src/lib.rslines 14-154, tag 0.14.0. - iced ships
pane_grid(PaneGrid) iniced_widget; README lists “Cross-platform support (Windows, macOS, Linux, and the Web)”. Locator:widget/src/lib.rslines 28, 73 at tag 0.14.0;README.mdline 31. - Slint 1.17.1 released 2026-07-07 (crates.io); master is 1.18.0 with
rust-version = "1.92",license = "GPL-3.0-only OR LicenseRef-Slint-Royalty-free-2.0 OR LicenseRef-Slint-Software-3.0". Locator:Cargo.tomllines 78-84, master;LICENSE.md. - Slint renderers:
femtovg(OpenGL ES 2.0),skia, software; “Web using WebAssembly” section in README. The winit backend has anaccessibilityfeature pullingaccesskit = "0.24"andaccesskit_winit = "0.33". Core text uses Parley by default (default = ["std", "unicode", "shared-parley"],accessibility-textlinksparley/accesskit). Locator:README.mdlines 55, 136-137;internal/backends/winit/Cargo.tomllines 58, 108-109;internal/core/Cargo.tomllines 81-92. - Slint testing backend (i-slint-backend-testing 1.17.1):
init_integration_test_with_mock_time,init_integration_test_with_system_time,init_no_event_loop,mock_elapsed_time;ElementQuery::{from_root, match_descendants, match_id, match_type_name, match_inherits, match_accessible_role, match_predicate, find_first, find_all};ElementHandle::{find_by_accessible_label, find_by_element_id, accessible_role, accessible_label, accessible_value, set_accessible_value, invoke_accessible_default_action, invoke_accessible_increment_action, mock_single_click, mock_drag, scroll, size, absolute_position}; modulesmcp_serverandsystest. Locator:internal/backends/testing/lib.rslines 13-103;search_api.rslines 227-1091, master; docs.rs i_slint_backend_testing 1.17.1. - Slint screenshot tests:
tests/screenshotswithsoftwareandskiadrivers, test cases generated bybuild.rsfromscreenshots/cases, markersSLINT_SCALE_FACTOR=,BASE_THRESHOLD=,ROTATION_THRESHOLD=; fonts pinned viaSLINT_DEFAULT_FONT/SLINT_FONT_PATH;SLINT_CREATE_SCREENSHOTS=1writes references; default colour-difference threshold 0.1. Locator:tests/screenshots/{main.rs,build.rs,testing.rs}, master;docs/testing.md. - gpui 0.2.2 (2025-10-22),
license = "Apache-2.0", “hybrid immediate and retained mode, GPU accelerated”; README requires “the latest version of stable Rust”; macOS renders with Metal (objc2-metal) and text needsfont-kit(Zed fork); Linux/FreeBSD needwayland/x11; Windows uses Win32 and DirectWrite;taffy = "=0.13.0";accesskit.workspace = trueand ana11yexample exist. Zed repository licence file is GPL (LICENSE-GPL). Locator:crates/gpui/Cargo.tomllines 3, 9, 50, 96, 120-131, 256;crates/gpui/README.md;LICENSE-GPL, main commit e3adf43. - GPUI test infrastructure:
TestAppContext::{build, single, add_window, add_empty_window -> VisualTestContext, simulate_keystrokes, simulate_input, dispatch_action, run_until_parked, simulate_window_resize, simulate_prompt_answer, write_to_clipboard, read_from_clipboard, windows};VisualTestContext::{simulate_mouse_move, simulate_mouse_down, simulate_mouse_up, simulate_click(position, modifiers), simulate_modifiers_change, simulate_resize, window_title};#[gpui::test]acceptsseed,seeds,iterations,retries,on_failure;TestPlatformrecords prompts and windows. Locator:crates/gpui/src/app/test_context.rslines 21-883;crates/gpui_macros/src/test.rslines 14-88;crates/gpui/src/platform/test/platform.rslines 26-217. - Zed editor tests:
crates/editor/src/editor_tests.rscontains 550#[gpui::test]attributes;EditorTestContext::{set_state(marked_text), set_selections_state, assert_state_with_diff, simulate_keystroke, run_until_parked, buffer_text, display_text, pixel_position}. Locator:crates/editor/src/editor_tests.rs(count via grep);crates/editor/src/test/editor_test_context.rslines 37-439. - Zed workspace has
crates/workspace/src/dock.rsandpane.rs(docking implemented in the application, not in gpui). Locator: repository tree, main. - Makepad: crates.io
makepad-widgets1.0.0 (2025-05-13);devbranch widgets crate is 2.0.0 (MIT OR Apache-2.0), repository licence MIT; README: “A cross-platform UI runtime for native and web targets”, “Rust-first framework with a scriptable UI DSL”; noaccesskitpath in the tree. Locator:widgets/Cargo.toml;README.md; tree grep, dev branch commit (pushed 2026-08-29). - Makepad test crate
libs/makepad_test0.1.0 (path dependency, unpublished):#[makepad_test] fn t(app: TestApp),Selector::id,.wait_visible(),.click(),.fill(),.wait_text(),.wait_value(),app.press_return(); “drive the app through the existing Studio protocol in headless mode”; documented invocation uses--test-threads=1;MAKEPAD_TEST_VISIBLE=1targets a running Studio at127.0.0.1:8001. Locator:libs/makepad_test/README.md,examples/counter/tests/ui.rs, dev branch. - Floem: crates.io 0.2.0 (2024-11-14); main manifest 0.2.0,
rust-version = "1.91",license = "MIT"; renderers vger, vello, AnyRender Skia (GPU) with tiny-skia CPU fallback;parley = "0.7.0"; Taffy layout;src/headless.rsexposesHeadlessHarness::new(view)andpointer_down(x, y);src/platform/wasm_stubs.rsexists; noaccesskitdependency. Locator:Cargo.tomllines 29-97;README.md“Features”;src/headless.rslines 1-46; tree grep, main. - Dioxus Native/Blitz: dioxus-native 0.7.10 (2026-07-31; 0.8.0-alpha.1 pre-release); blitz workspace main is 0.3.0-beta.2 (
blitz-dom0.2.4 stable),license = "MIT OR Apache-2.0",rust-version = "1.91.0"; dependenciesanyrender_vello 0.14.0,anyrender_vello_cpu 0.17.0,anyrender_skia 0.11.0,parley 0.11.1,taffy 0.14.0,accesskit 0.24viaaccesskit_xplat; dioxus-native featureaccessibilityenables AccessKit; README lists “Accessibility using AccessKit” as an intended goal and states beta status. Locator:Cargo.tomllines 34-137 (blitz main);packages/native/{src/lib.rs,Cargo.toml}(dioxus main);README.md“Status”, “Goals”. blitz-test-harness: “Headless test harness for Blitz documents”;Harness::from_html,Harness::from_component,pump/tickwith a controlled animation clock, selectors, layout rects, hit-testing, tree dumps, input synthesis routed “through the real event-dispatch pipeline, without requiring a window”; “No window, GPU, or compositor is required”. Locator:packages/blitz-test-harness/src/lib.rslines 1-14, main.- egui and Masonry rows: see nuif:research:egui-and-egui-kittest and nuif:research:masonry-xilem-and-linebender-test-harness (MSRV 1.95 and 1.88/1.96; AccessKit-native harnesses; Vello only in Masonry).
Mechanism
Comparison table. “unverified” marks cells not confirmed against a primary source in this retrieval; all other cells cite the Evidence section above.
| Toolkit (version) | Rendering backend | Headless harness | Accessibility tree | Text stack | Docking / panels | WASM target | Licence | MSRV |
|---|---|---|---|---|---|---|---|---|
| egui 0.36.1 | egui-wgpu (wgpu 30), also glow (unverified) | egui_kittest: AccessKit queries, pointer/key synthesis, wgpu CPU-adapter snapshots, UPDATE_SNAPSHOTS | AccessKit every frame; kittest queries by role/label/value | epaint glyph atlas, Context::set_fonts; no shaping pipeline (unverified) | egui_dock 0.21.1, egui_tiles 0.17.1 (external crates) | yes (eframe web demo, README) | MIT OR Apache-2.0 | 1.95 |
| Masonry/Xilem 0.4.0 (main b81d8d7) | Vello on wgpu; main adds imaging_vello_cpu/hybrid/skia | masonry_testing TestHarness: event injection, virtual time, accesskit_consumer::Tree, assert_render_snapshot!, MASONRY_TEST_BLESS | AccessKit TreeUpdate per redraw; access_node(WidgetId), accessibility_click_on | Parley (Fontique, HarfRust, Skrifa, ICU4X) | none built in (unverified) | xilem_web targets DOM, not Masonry (unverified) | Apache-2.0 | 1.88 (0.4.0), 1.96 (main) |
| iced 0.14.0 | wgpu 27 or tiny-skia software | iced_test Simulator: click(selector), typewrite, snapshot(theme) with matches_image/matches_hash | none (no accesskit dependency) | cosmic-text 0.15 | pane_grid widget | web listed in README | MIT | 1.88 |
| Slint 1.17.1 (master 1.18.0) | femtovg (GLES2), Skia, software renderer | i-slint-backend-testing: ElementQuery/ElementHandle, mock time, mock_single_click, invoke_accessible_default_action; CI screenshot drivers with thresholds | own element tree with accessible_* properties; AccessKit 0.24 at winit backend (accessibility feature) | Parley (shared-parley default) | none verified | yes (README “Web using WebAssembly”) | GPL-3.0-only OR Slint Royalty-free 2.0 OR Slint commercial | 1.92 |
| GPUI 0.2.2 (Zed main e3adf43) | Metal (macOS); Linux/Windows renderer (unverified: blade) | TestAppContext/VisualTestContext: deterministic executor, run_until_parked, keystroke/mouse simulation, seeded #[gpui::test] with iterations/retries; 550 editor tests | accesskit dependency and a11y example; no tree query API in test contexts (unverified beyond grep) | font-kit (Zed fork), CoreText/DirectWrite | Zed workspace::{dock, pane} (application level) | no (unverified) | Apache-2.0 (crate); Zed app GPL/AGPL | “latest stable” |
| Makepad widgets 1.0.0 (dev 2.0.0) | own shader-based renderer (README) | makepad_test 0.1.0 (unpublished, dev): TestApp, Selector::id, wait_text, headless via Studio protocol, --test-threads=1 | none (no accesskit in tree) | own (unverified) | Studio app has panels (unverified) | yes (README “native and web”) | MIT (repo); MIT OR Apache-2.0 (crates) | unverified |
| Floem 0.2.0 (main) | vger, Vello, AnyRender Skia; tiny-skia CPU fallback | floem::headless::HeadlessHarness (pointer synthesis) | none (no accesskit dependency) | Parley 0.7.0 | none verified | wasm stubs only (unverified) | MIT | 1.91 |
| Dioxus Native 0.7.10 / Blitz 0.3.0-beta.2 | AnyRender: Vello, Vello CPU, Vello Hybrid, Skia | blitz-test-harness: from_html/from_component, pump/tick, selectors, layout rects, hit-testing, input synthesis | AccessKit 0.24 via accesskit_xplat behind accessibility feature | Parley 0.11.1 | none verified | dioxus-web is a separate DOM renderer (unverified) | MIT OR Apache-2.0 | 1.91 |
Harness capability matrix relevant to QA.md items (interpretation): semantic queries without pointer input are available in egui (AccessKit), Masonry (AccessKit), Slint (own tree), Blitz (DOM selectors); iced selects by visible text or widget ID; GPUI dispatches actions and keystrokes but selects by view handles; Makepad selects by element id over a protocol; Floem selects by coordinates.
NUIF relevance
Borrow
- Masonry’s combination (owned tree, Vello scene output, AccessKit tree, CPU screenshot harness) as the baseline architecture for a Rust-native editor, because it satisfies headless-testable, Vello-compatible and accessibility-tree-driven simultaneously.
- egui_kittest’s query ergonomics (
get_by_role_and_label,click_accesskit) as the API model for a NUIF harness, because they are the most direct expression of QA item 2 and QA item 3 without pointer synthesis. - GPUI’s seeded, iterated, deterministic-executor test attribute as the model for NUIF operation-sequence tests, because it shows that editor-scale interaction tests can be deterministic under a controlled scheduler.
Adapt
- Slint’s marker-driven screenshot thresholds (
BASE_THRESHOLD=in the fixture) can be transposed to NUIF render fixtures, because conformance/PLAN.md requires declared tolerances per fixture rather than a global threshold. - Blitz’s
pump/tickclock and iced’sinto_messagesdrain suggest a NUIF harness API where every interaction yields the list of protocolOperations it produced, because ARCHITECTURE.md requires gestures to become semantic operations before mutation.
Reject
- iced as the editor shell, because it has no accessibility tree in 0.14.0 and its selectors are text- or ID-based, which cannot satisfy role/relationship queries.
- Slint for the reference editor, because the GPL/royalty-free/commercial triple licence conflicts with the
Apache-2.0 OR MITworkspace policy for a vendor-neutral reference implementation, and its DSL owns the widget tree rather than a Rust API. - Makepad and Floem for a headless-first editor, because neither exposes an accessibility tree and their harnesses are coordinate- or protocol-driven.
- GPUI as a dependency, because it tracks “the latest version of stable Rust”, is versioned with Zed’s monorepo, and offers no accessibility-tree query API for tests.
Open questions
- Whether the NUIF MSRV pin (1.85.0) will be raised; every candidate requires 1.88 or newer, so the editor cannot be added to the current workspace without a toolchain decision.
- Whether Blitz (HTML/CSS document model with Vello, Parley, Taffy, AccessKit) is a viable canvas host for NUIF documents, given that NUIF already lowers to CSS-compatible layout through Taffy.
- Whether GPUI’s accessibility integration exposes an AccessKit tree to tests in a later release, which would change its column.
- Whether iced will add AccessKit (an open topic in the iced repository; not verified here).
Industry Foundation Classes interoperability standard
Document status:
reviewed. Canonical source.
Summary
IFC is a large ISO-standardized cross-vendor information model with modular core/shared/domain/resource schemas and multiple machine representations. Its history demonstrates both the durability and complexity cost of broad semantic interchange.
NUIF relevance
Borrow rigorous schemas, profiles and machine-readable listings; avoid uncontrolled ontology expansion. NUIF should prove a small implementable core before adding domain-specific semantics.
InferUI robust relational layout synthesis (OOPSLA 2018), with Scout (CHI 2020) and Rewire (CHI 2018)
Document status:
reviewed. Canonical source.
Summary
InferUI takes a set of views with absolute positions on one device (the “input specification”) and synthesises an Android ConstraintLayout program: one horizontal and one vertical constraint per view, drawn from 26 constraint types (relative alignment, baseline, circular, fixed-size centring with bias, and dynamic-size centring), plus a size mode per axis. The rendering semantics of ConstraintLayout is written as linear equations, and synthesis is a satisfiability query in Z3 over booleans, integers and reals. Because a single device underdetermines the layout, the paper adds (i) robustness: the candidate layout is rendered symbolically on a list of additional devices and six properties (order, margin, centring, aspect-ratio preservation, pixel-perfectness, inside-screen) must hold, and (ii) a probabilistic model trained on ConstraintLayout files from top-500 GitHub and Google Play applications that scores candidate constraints and turns the query into maximum-score satisfiability with top-K candidate pruning. Single-device synthesis is always exact by construction; on held-out device sizes 86.5% (GitHub) and 92.3% (Play Store) of views generalise, 62% of synthesised constraints match the developer’s, and synthetic user feedback resolves the rest with 0/1/2/3+ corrections in 63/25/8/4% of cases. Unguided synthesis times out beyond about ten views; guided multi-device synthesis succeeds on 98.7% of 2-3-view layouts and 41.7% of 16-19-view layouts. Scout (CHI 2020) addresses a different task, generating many layout alternatives from designer-authored high-level constraints (grouping, order, emphasis, alternates, repeats) compiled to low-level Z3 constraints with branch-and-bound enumeration. Rewire (CHI 2018) infers editable vector objects from screenshots; only its abstract was retrieved. NUIF interpretation follows.
Evidence
InferUI, “Robust Relational Layout Synthesis from Examples for Android” (DOI 10.1145/3276526, Article 156, 29 pages; author PDF retrieved 2026-08-29):
- Input: N views with absolute positions (top-left and bottom-right points) inside a content frame ρ; output: N sizes and N horizontal plus N vertical constraints such that
v ⊨ ψ_layout(ρ, c_h, c_v, s). Source: §3 “Input Specification”, §5 “Problem Statement”. - Target: Android ConstraintLayout only (version 1.0.2 for cross-checking). Source: §1, §8.
- View representation: five handle points
⟨x_L, x_R, y_T, y_B, y_baseline⟩; constraint tuple⟨type, A, B, C, m_L, m_R, bias, α, r⟩. Source: §4, Figure 5. - Constraint classes (Table 1): relative positioning ℛ_LL/ℛ_LR/ℛ_RL/ℛ_RR (and top/bottom analogues), baseline ℛ_B, circular ℛ_C (angle and distance), fixed-size centring ℱ_* with bias, dynamic-size centring 𝒟_* (view spans between two anchors); 26 types in total; size mode Fixed or MatchConstraint, the latter exactly when a 𝒟 constraint is used. Source: §4, Table 1, §5.
- A one-pixel Android solver bug is modelled explicitly for ℱ_LR/ℱ_RL against the content frame. Source: §4, discussion of Table 1.
- Rendering semantics
ψ_layout = φ_position ∧ φ_size ∧ φ_constraintsas linear equations. Source: Figure 7, Figure 8. - Single-device synthesis formula
ψ_single_synwith guardsg_i^k ⇒ ⟦c_i^k⟧, exactly-one guard per view per axis (Z3PbEq), acyclicity via integer distance variables; solver Z3 4.6.0, one-minute timeout. Source: §5, Figure 9, footnote 2, §8. - Multi-device:
ψ_multi_syn = ψ_single_syn(ρ, v) ∧ ⋀_k ψ_gen(d_k, v, c, s); devices are an input list (or a maximum resize ratio); “input specification still consists of absolute view positions v only for a single device ρ”. Source: §6, Figure 10. - Robustness properties (§6.1):
φ_preserve_order(pairwise handle ordering),φ_preserve_margins(distances in a set of common values such as 16 and multiples of 8),φ_preserve_centering,φ_preserve_aspect_ratio(only for ratios in {16/9, 3/2, 4/3, 1/1, 3/4, 2/3}),φ_pixel_perfect(non-negative integer handles),φ_inside_screen. - Probabilistic model:
P(c, ρ, v) = (1/Z) ∏_k P_fk(c | f_k(c, v))^{w_k}with MLE and additive smoothing; features margins, bias, distance, size (16 px buckets), orientation, type, intersection count, plus a regulariser on distinct constants and views; trained in under a second on all developer-written constraints. Source: §7, Table 2, Equation 1, §8. - Guided search: top K = 5 candidates per view; on UNSAT add 10 more per view from the unsat core; top-5 sufficient in 69% of cases. Objective is maximum Σ score (Figure 12). Source: §7, §8.
- Results (Table 5, views generalising on held-out devices 341×518 to 384×640 dp after synthesis at 360×640 dp): GitHub 12.6% (single), 69.4% (single + guided), 86.5% (multi + guided); Play Store 12.9%, 75.5%, 92.3%. Vertical generalisation is higher than horizontal in every configuration. Source: §8.2.
- Developer-constraint match 62%; max-sat versus sat improves view generalisation by 35% and constraint match by 20% for single + guided. Source: §8.3.
- Scalability (Table 4): unguided
ψ_syntimes out in 87.2% of cases beyond about ten views;ψ_multi_synunguided works only below four views;ψ_multi_syn + guidedsucceeds for 98.7% of layouts with 2-3 views and 41.7% with 16-19 views; runtimes 44 ms to 3 s. Source: §8.1. - UNSAT causes: views cannot fit a smaller screen; robustness properties “too restrictive” when views are “centered simply by chance”. Source: §8.1.
- Property violations in synthesised layouts without robustness (Table 6): for example
¬φ_inside_screenin 86% (single) versus 52.3% (single + guided) of cases; violations also found in real applications. Source: §8.2. - Feedback: user moves or resizes rendered views; changed views become additional input; evaluated with a synthetic user derived from developer constraints: 0/1/2/3+ rounds in 63/25/8/4% of cases; overall multi-device generalisation 89%. Source: §6.2, §8.4.
- Limitations stated: returns a single most likely layout; feedback study synthetic; no threats-to-validity section. Source: §8.4, §9-10.
Scout, “Rapid Exploration of Interface Layout Alternatives through High-Level Design Constraints” (DOI 10.1145/3313831.3376593; arXiv:2001.05424v1 retrieved 2026-08-29):
- High-level constraints: grouping, order (important/unimportant, first/last), emphasis (low/normal/high), alternate groups, repeat groups, Keep/Prevent feedback. Design variables: layout grid (margin, columns 2-4, gutter, column width), baseline grid, per-group alignment (six values), arrangement (horizontal, vertical, balanced rows/columns), padding, per-element x, y and precomputed size triples in 4 px steps. Source: §“Scout System”, Table 1.
- Compilation: groups to alignment/arrangement/padding plus visual-hierarchy inequalities; order to ordering or bounding-box constraints; emphasis to size and relative-size/area constraints; repeats to equal arrangement across subgroups; every layout also satisfies in-bounds, pairwise non-overlap and 48×48 minimum touch targets, citing InferUI’s robustness properties. Source: §“Constraint Solving”.
- Solver: Z3 inside a modified branch-and-bound that assigns one variable at a time, backtracks on infeasibility, randomises assignment order and adds a blocking clause after each layout; size triples precomputed because Z3 “does not efficiently compute multiplication constraints”. Source: §“Constraint Solving”.
- Throughput: 20 solver threads, typically 15 layouts of 9 elements per request in under 5 s (Ryzen 7 1800X). Source: §“Implementation”.
- Quality model: per-group size, balance and alignment scores, area-weighted with density, adapted from Riegler and Holzmann. Source: §“Quality Model”.
- Study: 18 designers, within-subjects against Adobe XD; Scout layouts 12% more spatially diverse (p < 0.027), +35% for non-professionals, expert-rated quality not significantly different (5.37 vs 5.73). Source: §“Evaluation”, Tables 1-2.
Rewire, “Interface Design Assistance from Examples” (DOI 10.1145/3173574.3174078, CHI 2018 pp. 1-12; DOI verified via Crossref 2026-08-29; only the abstract was retrieved): the abstract states that Rewire “automatically infers a vector representation of screenshots where each UI component is a separate object with editable shape and style properties”. No mechanism or numbers are recorded here.
Mechanism
InferUI (§4-7):
View v = ⟨x_L, x_R, y_T, y_B, y_baseline⟩
Constraint = ⟨t ∈ 𝒞 (26 types), A, B, C ∈ View, m_L, m_R ∈ ℤ≥0, bias ∈ [0,1], α, r⟩
Size = ⟨t_h, t_v ∈ {Fixed, MatchConstraint}, width, height⟩
ψ_layout(ρ, c_h, c_v, s) = φ_position ∧ φ_size ∧ φ_constraints (linear)
ψ_single_syn = ψ_layout ∧ φ_valid ∧ φ_acyclic ∧ ⋀_i (Σ_k g_i^k = 1) ∧ ⋀_{i,k} (g_i^k ⇒ ⟦c_i^k⟧)
ψ_multi_syn = ψ_single_syn(ρ, v) ∧ ⋀_{d ∈ devices} ( ψ_layout_syn(d, v_d, c, s) ∧ φ_robust(v, v_d) )
φ_robust = φ_order ∧ φ_margins ∧ φ_centering ∧ φ_aspect_ratio ∧ φ_pixel_perfect ∧ φ_inside_screen
Objective = max Σ_i score_i, score_i = P(c_i^k, v) (max-sat over guards)
Search = top-K (K=5) candidates per view; on UNSAT add 10 from unsat core; repeat
P(c, ρ, v) = (1/Z) ∏_k P_fk(c | f_k(c, v))^{w_k}
Scout:
high-level constraints (group, order, emphasis, alternate, repeat, keep/prevent)
→ low-level linear constraints over x, y, size triples, grid variables
→ branch-and-bound over variables with Z3 feasibility checks + blocking clauses
→ N diverse layouts, ranked by quality model (size, balance, alignment, density)
NUIF relevance
- Borrow: The six robustness properties are a concrete, checkable definition of “the inferred layout generalises”; nuif:experiment:layout-inference should rank candidate stack/flex/grid/constraint reconstructions by these properties on held-out viewport widths.
- Borrow: Scoring inferred constraints with a probability (
P(c, v)) and recording it is exactly the “inference confidence” that NUIF provenance must retain for reconstructed intent; the 62% developer-match figure shows why such confidence must never be reported as lossless. - Borrow: Treating device sizes as an explicit input list of evaluation contexts, with the synthesised layout rendered symbolically on each, matches NUIF’s context-keyed resolved snapshots.
- Adapt: The 26 ConstraintLayout types map onto NUIF’s
constraintfamily (edge equalities with margins, centring with bias, size modes) but NUIF must keep them as portable relations with identities and strengths rather than Android attribute names. - Adapt: Scout’s high-level constraints (grouping, order, emphasis, repeat) are close to authored intent in NUIF’s
stackfamily; Scout’s compilation shows how intent can be lowered to linear constraints when aconstraintevaluator is the target. - Adapt: The single-device input assumption should be replaced by multi-context observations as in nuif:research:reverse-layout-inference; InferUI’s own data show generalisation rising from 12.6% to 86.5% only when extra devices constrain the search.
- Reject: Exactly one constraint per view per axis; NUIF constraint layouts require multiple simultaneous relations (min/max, aspect ratio, distribution) and the restriction is an Android encoding choice.
- Reject: The Android-trained probabilistic prior as a NUIF default; the feature set (16 px margins, multiples of 8) encodes platform conventions and must remain a pluggable adapter heuristic.
Open questions
- InferUI’s robustness properties are stated for absolute-position views; which of them survive translation to flex/grid families where order and centring are structural rather than numeric?
- The paper’s device range is narrow (341-384 dp width); generalisation to responsive breakpoints (360, 768, 1440 in nuif:experiment:v0-responsive-card) is untested.
- Scout enumerates alternatives; NUIF import needs a single ranked result with alternatives retained as provenance - how many candidates are worth storing?
- Rewire’s screenshot-to-vector inference was not examined in detail; whether its component segmentation can seed NUIF entity identity for raster imports remains open.
IPLD DAG-CBOR strictness rules and content identifiers as a hash-bearing CBOR profile
Document status:
reviewed. Canonical source.
Summary
DAG-CBOR is the IPLD codec that encodes the IPLD Data Model in CBOR for content addressing. Its specification states that “DAG-CBOR requires that there exist a single, canonical way of encoding any given set of data, and that encoded forms contain no superfluous data that may be ignored or lost in a round-trip decode/encode.” The strictness section fixes seven rules: only tag 42 (CID links) is permitted and decoders must reject other tags; the RFC 8949 §4.2 rules are applied with shortest integer and length arguments, keys “sorted in (byte-wise) lexical order, including their major type 3 and length” (therefore length first), and no indefinite-length items; only major-type-7 minors 20, 21, 22, 25, 26 and 27 are usable; floats “must always encoded in 64-bit, double-precision form”; NaN, Infinity and -Infinity “must not be accepted as they do not appear in the IPLD Data Model”; -0.0 “should not appear or be accepted” and zero is always 0x0000000000000000; encoders and decoders handle a single top-level item. Decoders “should reject encoded forms not adhering to” the rules but “may relax strictness requirements by default” for historical data. Links are CIDs prefixed with the identity multibase byte 0x00 inside a byte string under tag 42. A CID is a typed content address (content-type, content-address) whose bytes include the codec multicodec, so identical data under two codecs yields two identifiers. The IPLD Data Model defines floats as IEEE 754 values “excluding special values such as NaN, Infinity and -Infinity” and notes that DAG-CBOR restricts integers to the signed 64-bit range.
Evidence
- Strictness introduction: “DAG-CBOR requires that there exist a single, canonical way of encoding any given set of data, and that encoded forms contain no superfluous data that may be ignored or lost in a round-trip decode/encode.” https://ipld.io/specs/codecs/dag-cbor/spec/ section “Strictness” (retrieved 2026-08-29).
- Rule 1: “Use no tags other than the CID tag (42). A valid DAG-CBOR encoder must not encode using any additional tags and a valid DAG-CBOR decoder must reject objects containing additional tags as invalid.” Same section (retrieved 2026-08-29).
- Rule 2: apply “the ‘Deterministically Encoded CBOR’ rule suggestions defined in section 4.2 of RFC 8949”; “a valid DAG-CBOR decoder should reject encoded forms not adhering to the following rules”: integer encoding “must be as short as possible”, lengths of major types 2–5 as short as possible, tag 42 as short as possible, “The keys in every map must be sorted in (byte-wise) lexical order, including their major type 3 and length. Therefore, the keys are sorted by length first.”, “Indefinite-length items are not supported, only definite-length items are usable.” Same section (retrieved 2026-08-29).
- Rule 3: “The only usable major type 7 minor types are those for encoding Floats (minors 25, 26, 27), False (minor 20), True (minor 21) and Null (minor 22).” Rule 4: “Floating point values must always encoded in 64-bit, double-precision form, regardless of whether they can be represented as half (16) or single (32) precision.” Rule 5: “IEEE 754 special values NaN, Infinity and -Infinity must not be accepted as they do not appear in the IPLD Data Model.” Rule 6: “The floating point value -0.0 should not appear or be accepted”, zero “always be encoded as 0x0000000000000000”. Rule 7: a single top-level CBOR object. Same section (retrieved 2026-08-29).
- Decoder relaxation: “DAG-CBOR decoders may relax strictness requirements by default” to accept historical data. Same page (retrieved 2026-08-29).
- Links: “the Multibase identity prefix (0x00) is prepended to the binary form of a CID and this new byte array is encoded into CBOR as a byte-string (major type 2), and associated with CBOR tag 42”; the identity prefix “must not be omitted”. Section “Links” (retrieved 2026-08-29).
- IPLD Data Model kinds: floats are “roughly what you’d expect from IEEE 754 floats, but excluding special values such as NaN, Infinity and -Infinity”; “Some codecs, such as DAG-CBOR, will assume that integers must be within the 64-bit signed range and reject anything larger”; bytes “are not considered to have any character encoding”. https://ipld.io/docs/data-model/kinds/ (retrieved 2026-08-29).
- CID:
<cidv1> ::= <CIDv1-multicodec><content-type-multicodec><content-multihash>; “A CID is a self-describing content-addressed identifier… a typed content address: a tuple of (content-type, content-address)”. https://github.com/multiformats/cid (retrieved 2026-08-29).
Mechanism
DAG-CBOR selects, for each Data Model value, one byte sequence: no tag other than 42, shortest heads, length-first key order (the RFC 7049 canonical order that RFC 8949 §4.2.3 retains only for compatibility), fixed binary64 floats, and a value set that excludes NaN, the infinities and negative zero. The content identifier is the multihash of that byte sequence prefixed by the codec code. Determinism is therefore established at the codec boundary, and cross-codec identity is not claimed: DAG-JSON of the same value has a different CID. Byte strings are Data Model bytes; the codec never interprets their content, so nested payloads survive strict decoding unchanged. Decoder strictness is “should reject” with an explicit allowance for lenient defaults, which the specification justifies by deployed data that predates the rules.
NUIF relevance
Borrow
- The value-set restriction pattern: exclude NaN and the infinities and collapse negative zero at the data-model level, so that every remaining value has one encoding.
- The single-top-level-item rule and the “no tags except the ones the profile defines” rule for
nuif-cbor-0. - Opaque byte strings as the carrier for content the codec must not interpret (CID links in DAG-CBOR; extension payloads in NUIF).
- Including the encoding profile identifier in any content-addressed identifier NUIF publishes, following the CID structure, so that a
nuif-cbor-0hash is never confused with a hash of a later profile.
Adapt
- NUIF geometry cannot exclude NaN from arithmetic, but it can exclude NaN from authored properties; a NUIF validator rejects NaN and infinities at property-set time, as DAG-CBOR does at decode time.
- “May relax strictness by default” is acceptable for a general reader but not for the hash path; NUIF needs two decoder modes with the strict mode mandatory for canonical hashing.
Reject
- Length-first key order and always-binary64 floats; RFC 8949 §4.2.1 and the current IETF drafts specify bytewise-lexicographic order and shortest float width.
- The signed 64-bit integer limit as a data-model rule; NUIF integers stay within the CBOR major type 0/1 range and do not need the extra restriction.
Open questions
- Whether NUIF content identifiers should be multihash-encoded CIDs (codec plus hash) or a NUIF-specific tuple; interoperability with IPFS tooling is the only reason to prefer CIDs.
- Whether the “single top-level item” rule holds for
.nuifpackages, which contain several records, or whether the package manifest is the single item and records are byte strings addressed from it.
Jetpack Compose constraint and single-pass layout model
Document status:
reviewed. Canonical source.
Summary
Jetpack Compose lays out a UI tree by passing constraints to children, measuring each child, deciding parent size and then placing children. Standard layout prohibits measuring a child more than once; intrinsic measurement and subcomposition are separate mechanisms.
Evidence
- The layout basics define a single pass in which parents initiate measurement, constraints descend, and resolved sizes and placement instructions return up the tree. Measurement and placement are distinct sub-phases. https://developer.android.com/develop/ui/compose/layouts/basics#the-layout-model (retrieved 2026-08-29).
- Custom layout requires measuring children, deciding size and placing children. Compose rejects ordinary multi-pass child measurement. https://developer.android.com/develop/ui/compose/layouts/custom (retrieved 2026-08-29).
- Modifier order changes the constraint and layout nodes wrapped around a composable. Equivalent visible output does not imply equivalent authored modifier structure. https://developer.android.com/develop/ui/compose/layouts/constraints-modifiers (retrieved 2026-08-29).
NUIF relevance
Borrow bounded min/max constraints, intrinsic queries, row/column alignment and the measurement/placement separation for a lowering profile.
Adapt NUIF stack/flex semantics into a generated, profile-owned Kotlin DSL subset with stable identity comments or modifiers. Compile and screenshot tests require a pinned Android Gradle Plugin, Compose version, SDK and font set.
Reject arbitrary Kotlin/Compose import as a lossless document operation. Composable execution, state, modifier order, subcomposition and platform resources require runtime evaluation and cannot be recovered from resolved geometry alone.
JSON Patch (RFC 6902), JSON Pointer (RFC 6901) and JSON Merge Patch (RFC 7396) versus identity-addressed operations
Document status:
reviewed. Canonical source.
Summary
RFC 6902 defines a JSON Patch as an ordered array of operations add, remove, replace, move, copy and test, each addressing its target with a JSON Pointer (RFC 6901). test compares the addressed value with a supplied value and fails the patch on inequality, which makes it a precondition mechanism. A patch is applied sequentially and atomically: any failing operation makes the whole patch unsuccessful. Array positions are addressed by index or by - (append), and add shifts later elements. RFC 7396 (JSON Merge Patch, obsoleting RFC 7386) defines a recursive object overlay in which null deletes a member, arrays and non-object values are replaced wholesale, and a literal null cannot be stored. Both formats are path-addressed: a concurrent reorder or insert invalidates array indices, move cannot express “move entity X” independently of X’s current path, and neither format defines an inverse. Identity-addressed operations, as used in nuif-protocol, remove path dependence but still carry an index in Insert and Move.
Evidence
- RFC 6902, April 2013, Standards Track, Bryan and Nottingham; operations defined in §4:
add(§4.1) inserts into an array at the index shifting later elements, or appends with-; “The specified index MUST NOT be greater than the number of elements in the array”;remove(§4.2) requires the target to exist;replace(§4.3) equals remove then add;move(§4.4): “The ‘from’ location MUST NOT be a proper prefix of the ‘path’ location”;copy(§4.5);test(§4.6) compares strings byte-wise, numbers numerically, arrays element-wise, objects member-wise regardless of order. https://www.rfc-editor.org/rfc/rfc6902.html (retrieved 2026-08-29). - RFC 6902 §5 error handling: if an operation fails, “evaluation of the JSON Patch document SHOULD terminate and application of the entire patch document SHALL NOT be deemed successful”; HTTP PATCH is atomic. (retrieved 2026-08-29).
- RFC 6901, April 2013: reference tokens separated by
/,~escaped as~0and/as~1(§3); array tokens are base-10 digits without leading zeros or the single character-denoting the position after the last element (§4); “This specification does not define how errors are handled” (§7). https://www.rfc-editor.org/rfc/rfc6901.html (retrieved 2026-08-29). - RFC 7396, October 2014, Hoffman and Snell, obsoletes RFC 7386; MergePatch pseudocode in §2 (transcribed below); “It is not possible to patch part of a target that is not an object, such as to replace just some of the values in an array”; non-object patches replace the entire target; explicit
nullvalues in the target cannot be expressed (§1). https://www.rfc-editor.org/rfc/rfc7396.html (retrieved 2026-08-29). RFC 7386 text retrieved for comparison (https://www.rfc-editor.org/rfc/rfc7386.html, retrieved 2026-08-29); the RFC 7396 header shows the obsoletion.
Mechanism
JSON Patch application (RFC 6902 §3-5):
apply(doc, ops):
for op in ops (in order):
target := resolve(doc, op.path) -- RFC 6901; array index or "-"
match op.op:
add : insert at index (shift right) | append if "-" | set/replace member
remove : target MUST exist; array elements shift left
replace : target MUST exist; remove then add
move : from MUST exist; from not a proper prefix of path; remove(from) then add(path, value)
copy : add(path, value at from)
test : fail unless value(target) == op.value (type-specific equality)
on failure: abort; entire patch unsuccessful
JSON Merge Patch (RFC 7396 §2):
MergePatch(Target, Patch):
if Patch is an Object:
if Target is not an Object: Target = {}
for each Name/Value in Patch:
if Value is null: remove Name from Target if present
else Target[Name] = MergePatch(Target[Name], Value)
return Target
else: return Patch
Path dependence (NUIF interpretation of the source rules): an operation {"op":"move","from":"/children/3","path":"/children/0"} denotes whichever element occupies index 3 at application time; a concurrent add at /children/1 shifts the intended element to index 4, and test can only detect the mismatch by comparing the entire element value. Identity addressing (Move { entity: EntityId, .. }) names the element regardless of position; the remaining position-dependent component is the destination index, which the same concurrent insert also shifts.
Inverse computation: neither RFC defines an inverse; remove and replace discard the prior value, so an inverse requires the applier to record it, which is the memento/inverse-recording problem treated in nuif:research:command-pattern-undo-and-event-sourcing.
NUIF relevance
Borrow
testas the canonical shape of a precondition: an operation-level guard that compares an addressed value with an expected value and aborts the patch on mismatch, matching spec/06 “Preconditions MAY guard expected prior values”.- Sequential, atomic patch semantics (RFC 6902 §5) for NUIF transactions: all operations of a transaction succeed or the transaction is not applied.
- The
moveprefix rule (§4.4) as the path-form of NUIF’s acyclicity precondition forMove.
Adapt
- Replace path addressing with entity identity everywhere, and replace the destination
indexinInsertandMovewith an order anchor (preceding sibling ID or order key) so that operations remain valid under concurrent sibling edits. - Merge-patch style overlay semantics are usable only for unordered property maps of a single entity (set/unset property), never for containment sequences.
- Record removed values (or reference the base revision’s content hash) in
Remove/SetPropertyso that patches are invertible; RFC 6902 leaves this to the application.
Reject
- JSON Merge Patch for structural edits: arrays are replaced wholesale and
nullis overloaded as delete, which conflicts with NUIF unset semantics and explicit null property values. - JSON Pointer array indices as the identity of children in any NUIF patch encoding.
Open questions
- Whether a NUIF
test-style precondition should compare by value or by content hash of the addressed subtree; the latter is cheaper for large subtrees but couples preconditions to the canonical encoding. - Whether a JSON Patch projection of NUIF patches (for tooling interoperability) should be generated against a fixed base snapshot with indices resolved at generation time, and marked as non-mergeable.
Lens laws from Foster et al. (TOPLAS 2007) and Boomerang (POPL 2008) to symmetric, edit and delta lenses
Document status:
reviewed. Canonical source.
Summary
Foster et al. define a lens as a pair of partial functions get (l↗ : C ⇀ A) and putback (l↘ : A × C ⇀ C) and call it well-behaved when GetPut (l↘(l↗ c, c) ⊑ c) and PutGet (l↗(l↘(a, c)) ⊑ a) hold; the optional PutPut law (l↘(a′, l↘(a, c)) ⊑ l↘(a′, c)) defines very well-behaved lenses and is deliberately not required because map, flatten, merge and conditionals fail it. Boomerang restates lenses for strings with a total create and laws GetPut, PutGet, CreateGet, and introduces dictionary (resourceful) lenses that align chunks by key rather than position; the corresponding weakening of obliviousness is the EquivPut law (quasi-obliviousness). Hofmann, Pierce and Wagner’s symmetric lenses replace get/put by putr and putl over a complement C with laws PutRL and PutLR, prove that asymmetric lenses embed and that every symmetric lens factors as two asymmetric lenses back to back, and identify alignment as an explicit non-goal. Edit lenses make edits first-class: a module is a set with a monoid of edits acting partially on it, an edit lens translates edits through a complement while preserving a consistency relation, and Theorem 7.1 gives a one-to-one correspondence with state-based symmetric lenses under the overwrite monoid. Diskin, Xiong and Czarnecki’s delta lenses take model spaces as categories whose arrows are deltas; a well-behaved delta lens satisfies incidence laws (GetInc, PutInc1, PutInc2), identity laws (GetId, PutId) and PutGet, while very well-behaved additionally satisfies GetGet and PutPut on deltas; Theorem 5 recovers a well-behaved state-based lens from a delta lens plus a differencing function and Theorem 6 recovers PutPut only conditionally on the differencing being composition-compatible. NUIF interpretation follows.
Evidence
Foster, Greenwald, Moore, Pierce, Schmitt, “Combinators for bidirectional tree transformations”, ACM TOPLAS 29(3), Article 17, May 2007 (DOI 10.1145/1232420.1232424 verified via Crossref 2026-08-29; author preprint from cis.upenn.edu retrieved 2026-08-29, preprint page numbers cited):
- Definition 3.1 (Lenses): partial get
l↗ : V ⇀ Vand partial putbackl↘ : V × V ⇀ V. Source: §3, p. 6. - Definition 3.2 (Well-behaved lenses):
l ∈ C ⇌ Aiffl↗(C) ⊆ A,l↘(A × C) ⊆ C, GetPutl↘(l↗ c, c) ⊑ c, PutGetl↗(l↘(a, c)) ⊑ a, wheref(x) ⊑ ymeansf(x)is undefined or equalsy. Source: §3, p. 6. - PutPut
l↘(a′, l↘(a, c)) ⊑ l↘(a′, c)is “optional”; a well-behaved lens also satisfying it is very well behaved. Source: §3, p. 7. - “we will not require PutPut because some of our lens combinators-in particular, map, flatten, merge, and conditionals-fail to satisfy it for reasons that seem pragmatically unavoidable.” Source: §3, p. 7; map counterexample §5 p. 22 (modifying a child differs from deleting and re-adding it); flatten counterexample §9 pp. 47-48.
- Definition 3.3 (Totality):
l ∈ C ⇐⇒ AifC ⊆ dom(l↗)andA × C ⊆ dom(l↘); footnote 3 notes well-behavedness “is rather trivial in the absence of totality”. Source: §3, p. 7. - Definition 3.7 (Oblivious):
l↘(a, c) = l↘(a, c′)for alla, c, c′; Lemma 3.9: a total oblivious lens has bijective get; conversely every bijection induces a total oblivious lens; §11 notes every oblivious lens is very well behaved. Source: §3, p. 8; §11, p. 59. - Ω (“missing”) as the argument to putback when no concrete view exists; conventions
l↗Ω = Ω,l↘(Ω, c) = Ω; Lemma 3.20. Source: §3 “Dealing with Creation”, p. 11. - Composition
(l; k)↘(a, c) = l↘(k↘(a, l↗ c), c); Lemma 4.3 (well-behavedness preserved), Lemma 4.4 (totality preserved). Source: §4, p. 13. - Combinators: id, compose, const, hoist, plunge, fork/xfork, filter, prune, add, focus, rename, map, wmap, copy, merge, ccond/acond/cond, list combinators (hd, tl, list_map, rotate, group, concat, list_filter), flatten, pivot, join. Source: §§4-9.
- Foundations: well-behaved lenses correspond to Gottlob-Paolini-Zicari dynamic views and very well behaved lenses to Bancilhon-Spyratos constant-complement translators; footnote 9: with total components the laws including PutPut “characterize the set C as isomorphic to A × B for some B”. Source: §10, pp. 50-51.
- The paper explicitly rejects choosing a minimal translation by an ordering (Johnson-Rosebrugh-Dampney) in favour of the programmer specifying the update policy with the view definition; Buneman-Khanna-Tan intractability of inferring minimal view updates is cited. Source: §10, p. 51.
- Framing is state-based, not trace-based: “we are interested here in the final tree a′, not the particular sequence of edit operations”. Source: §2, footnote 1, p. 5.
Bohannon, Foster, Pierce, Pilkiewicz, Schmitt, “Boomerang: Resourceful Lenses for String Data”, POPL 2008, pp. 407-419 (DOI 10.1145/1328438.1328487 verified via DBLP 2026-08-29; author preprint retrieved):
- Basic lens:
get ∈ C → A,put ∈ A → C → C,create ∈ A → Cwith lawsput (get c) c = c(GetPut),get (put a c) = a(PutGet),get (create a) = a(CreateGet); total components; laws are part of the definition. Source: §1, footnote 1. - Positional Kleene-star put “mangles” reordered output, “a show-stopper for many of the applications”. Source: §1, §2.
- Dictionary lens: components get, parse (concrete to skeleton plus dictionary), key, create, put;
key Eandmatch ⟨l⟩combinators; Theorem 3.1: a dictionary lens coerces to a basic lens satisfying the basic laws. Source: §3. - Quasi-obliviousness:
c ∼ c′ ⟹ put a c = put a c′(EquivPut) for an equivalence∼on C; every dictionary lens is quasi-oblivious with respect to key-respecting chunk reordering; oblivious iff the maximal equivalence is total; very well behaved iff constant complement. Source: §4. - “Very well behavedness is a strong condition and imposing it on all lenses would prevent writing many useful transformations”; the alternative “is disallowing deletions”. Source: §4.
- Typing: unambiguous concatenation and unambiguous iteration of regular languages, decidable (Fact 2.1); required for well-definedness and well-behavedness (
lambigcounterexample). Source: §2.
Hofmann, Pierce, Wagner, “Symmetric Lenses”, POPL 2011, pp. 371-384 (DOI 10.1145/1926385.1926428 verified via DBLP; author preprint retrieved):
- Definition 2.1:
ℓ ∈ X ↔ Yhas complementC,missing ∈ C,putr ∈ X × C → Y × C,putl ∈ Y × C → X × C, with PutRLputr(x, c) = (y, c′) ⟹ putl(y, c′) = (x, c′)and PutLR symmetric. Source: §2. - Symmetric PutPut variants “appear too strong to be desirable in practice”. Source: §2.
- Definition 3.2 (lens equivalence) via a relation on complements; needed because associativity of composition and other laws hold only up to equivalence. Source: §3.
- Definition 4.2 (composition,
C = k.C × ℓ.C); symmetric lenses form a category with equivalence classes as arrows; Theorem 5.1: no categorical products; tensor product is symmetric monoidal. Source: §§4-5. - Definition 9.1: asymmetric lens
ℓembeds asℓ^symwith complement{f ∈ Y → X | ∀y. get(f(y)) = y}andmissing = create; Theorem 9.4: every symmetric lens factors as(k1^sym)^op ; k2^sym. Source: §9. - “One important non-goal of the present paper is dealing with the (critical) issue of alignment”; deltas or edit monoids suggested as future work. Source: §2, §11.
Hofmann, Pierce, Wagner, “Edit Lenses”, POPL 2012, pp. 495-508 (DOI 10.1145/2103656.2103715 verified via Crossref; author preprint from dmwit.com retrieved):
- Motivation: prior lenses “only consider edits of the form ‘overwrite the whole structure’”; complements hold small alignment information. Source: abstract, §1-2, Figure 1.
- Definition 3.2 (monoid action, partial), Definition 3.3 (module
⟨X, init_X, ∂X, ⊙_X⟩), Definition 3.4 (stateful monoid homomorphism). Source: §3. - Definition 3.5 (symmetric edit lens): complement
C,init ∈ C, homomorphisms⇛ : ∂X × C → ∂Y × Cand⇚, consistency relationK ⊆ X × C × Ywith(init_X, init, init_Y) ∈ Kand preservation: if(x, c, y) ∈ K,dx xdefined and⇛(dx, c) = (dy, c′), thendy yis defined and(dx x, c′, dy y) ∈ K(and symmetrically). Source: §3. - Theorem 3.7 (totality on consistent states); Definition 3.8 and Theorem 3.9 (equivalence via bisimulation). Source: §3.
- List module generators
mod(p, dx),ins(i),del(i),reorder(f),fail; mapping lens carries insertions, deletions and reorderings across unchanged; container lenses (Theorem 5.7,T(ℓ)functorial). Source: §§4-5. - Theorem 7.1: with the overwrite monoid,
|−|and∂give a one-to-one correspondence between equivalence classes of edit lenses and state-based symmetric lenses. Source: §7.
Diskin, Xiong, Czarnecki, “From State- to Delta-Based Bidirectional Model Transformations: the Asymmetric Case”, Journal of Object Technology 10 (2011) 6:1-25 (DOI 10.5381/jot.2011.10.1.a6 printed; retrieved 2026-08-29):
- Definition 1 restates state-based well-behaved lenses (GetPut, PutGet) and very well-behaved (PutPut). Source: §2.1.
- Two failures of state-based lenses: composed lenses using different alignment keys turn a rename into delete plus insert (P1, §2.2); PutPut fails for “a quite reasonable transformation” because differencing, not propagation, is non-compositional (P2, §2.3, §3.1 equations (3)-(4)).
- Definition 3 (model space: a connected category whose arrows are deltas); Definition 4 (delta lens
(A, B, get, put)withgeta graph morphism andput : B₁ × A₀ → A₁). Source: §4.1-4.2. - Laws (Figure 9): GetInc, PutInc1 (
put(b, A)defined iffA.get₀ = source(b)), PutInc2 (source(put(b, A)) = A); GetId, PutId (id_A = put(id_B, A)); PutGet (get₁(put(b, A)) = b); GetGet, PutPut (put(b; b′, A) = put(b, A); put(b′, A′)). Well-behaved = incidence + identity + PutGet; very well-behaved adds GetGet and PutPut. GetPut is not required on deltas; PutId is described as the identity-preservation content of GetPut. Source: §4.2. - Theorem 2 and Theorem 3: composition preserves (very) well-behavedness; delta lenses form a category. Source: §4.3.
- Definition 6 (differencing with DifInc, DifId), Theorem 5 (every well-behaved delta lens plus differencing yields a well-behaved state-based lens), Theorem 6 (very well-behaved yields conditional PutPut when
dif(B, B″) = dif(B, B′); dif(B′, B″)); a leap-day example shows PutPut can still fail on deltas. Source: §4.4.
Mechanism
State-based asymmetric lens (TOPLAS 2007, Definitions 3.1-3.3):
get l↗ : C ⇀ A
put l↘ : A × C ⇀ C
GetPut l↘(l↗ c, c) ⊑ c
PutGet l↗(l↘(a, c)) ⊑ a
PutPut l↘(a′, l↘(a, c)) ⊑ l↘(a′, c) -- optional; "very well behaved"
total C ⊆ dom(l↗) ∧ A × C ⊆ dom(l↘)
oblivious l↘(a, c) = l↘(a, c′) ⟹ get bijective (Lemma 3.9), very well behaved
Boomerang basic and dictionary lenses (POPL 2008 §1, §3-4):
get : C → A ; put : A → C → C ; create : A → C
GetPut put (get c) c = c
PutGet get (put a c) = a
CreateGet get (create a) = a
EquivPut c ∼ c′ ⟹ put a c = put a c′ -- quasi-oblivious w.r.t. key-preserving reorderings
Symmetric lens (POPL 2011, Definition 2.1):
ℓ : X ↔ Y = (C, missing ∈ C, putr : X×C → Y×C, putl : Y×C → X×C)
PutRL putr(x, c) = (y, c′) ⟹ putl(y, c′) = (x, c′)
PutLR putl(y, c) = (x, c′) ⟹ putr(x, c′) = (y, c′)
embedding of asymmetric ℓ: C = {f : Y → X | get ∘ f = id}, missing = create
factorisation: every symmetric lens = (k1^sym)^op ; k2^sym
Edit lens (POPL 2012, Definitions 3.3, 3.5):
module ⟨X, init_X, ∂X, ⊙⟩ : monoid ∂X acting partially on X
lens (C, init, ⇛ : ∂X×C → ∂Y×C, ⇚ : ∂Y×C → ∂X×C, K ⊆ X×C×Y)
⇛, ⇚ stateful monoid homomorphisms (identity ↦ identity, composition threaded through C)
(init_X, init, init_Y) ∈ K
(x, c, y) ∈ K ∧ dx x defined ∧ ⇛(dx, c) = (dy, c′) ⟹ dy y defined ∧ (dx x, c′, dy y) ∈ K
Delta lens (JOT 2011, Definition 4, Figure 9):
model spaces A, B : categories, arrows = deltas
get : A → B graph morphism (get₀ on models, get₁ on deltas) ; put : B₁ × A₀ → A₁
GetInc, PutInc1, PutInc2 incidence (put applies only to the matching base model)
GetId get₁(id_A) = id_B ; PutId put(id_B, A) = id_A
PutGet get₁(put(b, A)) = b
GetGet get₁(a; a′) = get₁(a); get₁(a′) ; PutPut put(b; b′, A) = put(b, A); put(b′, A′)
well-behaved = incidence + identity + PutGet ; very well-behaved = + GetGet + PutPut
NUIF relevance
- Borrow: NUIF’s patch model (ordered operations with base snapshot identity and preconditions) is a delta lens
put: PutInc1/PutInc2 are the base-snapshot precondition and the requirement that the resulting source patch applies to exactly that base; PutId is the requirement that a no-op design edit yields an empty source patch; PutGet is the requirement that re-lowering the patched source reproduces the design delta. These three, not the state-based laws, are the laws NUIF should require of alosslessadapter. - Borrow: Foster et al.’s explicit refusal to define “minimal” updates by an ordering (TOPLAS §10) and Boomerang’s argument that very well-behavedness would disallow deletions justify NUIF requiring well-behaved (PutId, PutGet, incidence) plus retentiveness (nuif:research:retentive-lenses) rather than PutPut; NUIF’s “minimal source patch” should be defined as retention of unaffected source regions, which edit lenses achieve by translating
ins/del/reorder/modedits rather than by minimising a metric. - Borrow: Symmetric lenses justify NUIF’s system-level symmetry: source code and design each hold private data (comments, formatting, tokens, layout intent), so the complement
Cis the correspondence record; Theorem 9.4 shows this can still be implemented as two directional adapters back to back. - Adapt: Boomerang’s key-based alignment (dictionary lenses, EquivPut) becomes NUIF’s stable entity identity: adapters must align by identity and fall back to structural matching only when identity is absent, which is exactly the quasi-oblivious regime.
- Adapt: Theorem 5/6 of the delta-lens paper give the conformance test for adapters that expose only state: compute deltas by differencing, check PutId and PutGet, and test PutPut only for delta pairs whose differencing composes.
- Reject: Requiring PutPut (very well-behaved) of NUIF adapters; the TOPLAS combinators map, merge and conditionals - all needed for component instantiation and conditional layout - fail it for pragmatic reasons.
- Reject: Purely state-based (overwrite) synchronisation in the protocol; Theorem 7.1 of Edit Lenses shows the state-based view is recoverable from edit lenses, but the converse loses alignment (P1/P2 in the delta-lens paper), which is the failure mode “regenerate instead of sync” that nuif:claim:sync-not-regenerate names.
Open questions
- Which subset of NUIF operations forms a monoid with a partial action in the edit-lens sense, given that
create/delete/moveandset propertydo not commute and transactions group them? - Retentive lenses are stated for state-based lenses; is there a delta-lens formulation of retentiveness that NUIF can adopt directly for patch replay?
- Boomerang’s unambiguity typing has no analogue for tree-structured source; can tree-sitter grammars (nuif:research:tree-sitter) provide the equivalent guarantee that a correspondence record identifies a unique source span?
- The delta-lens laws assume a single base model per put; NUIF three-way merge with concurrent design and source edits needs a multi-source generalisation (Diskin’s later multiary delta lenses were not reviewed here).
libtest-mimic, datatest-stable, trybuild, expect-test and directory-driven fixture conventions (resvg, Taffy, rust-analyzer, Slint)
Document status:
reviewed. Canonical source.
Summary
Cargo test targets with harness = false replace libtest with a user-supplied main. libtest-mimic re-implements libtest’s CLI (filters, --list, --skip, --exact, --ignored, --test-threads, --format) so that a main can enumerate Trials from a fixture directory at run time while remaining compatible with cargo test and cargo-nextest. datatest-stable wraps this in a harness! macro that maps one file per test. trybuild is the same idea specialised to compile-fail fixtures with .stderr references. expect-test provides inline or file-backed expectations updated in place with UPDATE_EXPECT=1. Large Rust projects converge on the same conventions: fixtures are files, expectations live beside inputs, regeneration is an explicit environment variable, references are pinned against font and renderer inputs, and generated tests are never edited by hand.
NUIF interpretation: conformance/ should be a set of harness = false integration crates that enumerate fixture directories with libtest-mimic, produce one test per fixture (nextest then isolates each in a process), keep input, expected and context files together, and regenerate references only through one documented variable.
Evidence
- Cargo: “The
harnessfield indicates that the--testflag will be passed torustc”; withharness = false“you are responsible for defining amain()function”; “Each integration test results in a separate executable binary, andcargo testwill run them serially”; the working directory of each test is the package root. Locator: Cargo book “Cargo Targets” (harness, integration tests) and “cargo test” (working directory), retrieved 2026-08-29. - libtest-mimic 0.8.2 (2026-03-16): usage
[[test]] name = "mytest" path = "tests/mytest.rs" harness = false;Arguments::from_args(),Trial::test(name, runner),Trial::bench,Trial::ignorable_test,with_kind,with_ignored_flag,libtest_mimic::run(&args, tests).exit(); run withcargo test --test mytest. Locator:src/lib.rslines 13-52, 104-221;CHANGELOG.md“0.8.2”. - Supported arguments:
--include-ignored,--ignored,--test,--bench,--list,--nocapture(no-op),--show-output,--exact,--quiet,--test-threads,--logfile,--skip(repeatable),--color,--format, positional filter;Arguments::{is_ignored, is_filtered_out}added in 0.8.2. Locator:src/args.rslines 19-160;CHANGELOG.md. - Known differences from libtest: “Output capture and
--nocapture: simply not supported”; no--format=junit. Locator:src/lib.rslines 54-71. examples/tidy.rswalks a directory recursively and createsTrial::test(relative_path, move || check_file(&path)).with_kind("tidy")per.rsfile. Locator:examples/tidy.rs.- nextest requires libtest-mimic 0.4.0 or 0.5.2+; a custom harness “MUST support being run with
--list --format terse” printing<TEST_NAME>: testper line; datatest-stable is the reference example and with nextest “each test case is represented as a separate test, and is run as a separate process in parallel”. Locator: nextestsite/src/docs/design/custom-test-harnesses.md; datatest-stableREADME.md. - datatest-stable 0.3.3 (2026-03-31), passively maintained, Rust ^1.72:
datatest_stable::harness! { { test = my_test, root = "path/to/fixtures", pattern = r".*" }, }; test signaturesfn(&Path) -> Result<()>,fn(&Utf8Path) -> Result<()>,fn(&P, String),fn(&P, Vec<u8>);rootrelative to the crate root; recursive traversal;patternis a regex (fancy_regex); fixtures can be embedded withinclude_dir!. Locator:README.md“Usage”;src/lib.rslines 44-210. - trybuild 1.0.120 (2026-08-03):
TestCases::new().compile_fail("tests/ui/*.rs")and.pass(path); expected compiler output in adjacent*.stderr; on mismatch the actual output is written into awipdirectory for manual promotion. Locator:README.md“Compile-fail tests”, lines 97-134. - test-generator 0.3.1 (2022-12-08, no release since):
#[test_resources("res/*/input.txt")]generates one test per glob match; suggested layoutres/setN/{input.txt, expect.txt}; requiresbuild.rsto re-run on resource changes. Locator:README.md. - expect-test 1.5.1 (2024-12-21):
expect![["..."]]andexpect_file!["./path"];Expect::assert_eq,assert_debug_eq;UPDATE_EXPECT=1patches source files in place usingfile!,line!,column!; leading indentation is stripped. Locator:src/lib.rslines 1-80. - rust-analyzer: parser fixtures in
crates/parser/test_data/{lexer,parser}/{ok,err,inline}/NNNN_name.rswith sibling.rastexpectations;tests.rsiterates the directories and callsexpect_file![case.rast].assert_eq(&actual); multi-file fixtures use//- /main.rsmetadata comments parsed bytest-utils::Fixture; style guide requires minimal snippets, unindented raw strings,cov_markmarks, and forbids#[should_panic]. Locator:crates/parser/src/tests.rslines 10-76;crates/parser/test_data/parser/ok/0001_struct_item.{rs,rast};crates/test-utils/src/fixture.rslines 1-40;docs/book/src/contributing/style.md“Minimal Tests”, “Marked Tests”, “#[should_panic]”. - resvg: fixtures are 200x200-viewBox SVGs where “Each test must test only a single issue”, every element has an
id, titles are unique; references are rendered at width 300 with pinned fonts (--skip-system-fonts --use-fonts-dir tests/fontsand explicit family mappings) and optimised withoxipng -o 6 -Z;crates/resvg/tests/svgmirrorsresvg-test-suite/svg, whiletests/pngholds regression renders, not reference renders. Locator:crates/resvg/tests/README.md. - resvg runner:
render_innerreadstests/<name>.svgandtests/<name>.png, regenerates whenMAKE_REFis set (invoking oxipng), writes side-by-side diff images totests/diffs/, and counts pixels exceedingDIFF_THRESHOLD. Locator:crates/resvg/tests/integration/main.rslines 36-176. - Taffy: HTML fixtures in
test_fixtures/;scripts/gentestdownloads Chrome for Testing and ChromeDriver (getchrome), drives Chrome through WebDriver (fantoccini), and regeneratestests/generated/wholesale (just gentest), then runscargo fmt; “You should not manually update the tests intests/generated”; fixtures prefixedxare disabled; benchmarks are generated from the same fixtures; hand-written tests live intests/hand_written/and sharetests/commonhelpers (new_test_tree, measure functions). Locator:CONTRIBUTING.mdlines 26-83;scripts/gentest/src/main.rslines 11-67, 157;tests/common/src/lib.rs. - Slint:
.slintfixtures carry marker comments (SLINT_SCALE_FACTOR=,BASE_THRESHOLD=,ROTATION_THRESHOLD=),build.rsgenerates tests intoOUT_DIRfromscreenshots/cases, fonts are pinned throughSLINT_DEFAULT_FONT/SLINT_FONT_PATH, andSLINT_CREATE_SCREENSHOTS=1writes references; syntax tests regenerate withSLINT_SYNTAX_TEST_UPDATE=1; the test crates live in a separatetests/workspace. Locator:tests/screenshots/{build.rs,testing.rs};docs/testing.md, master.
Mechanism
Custom harness skeleton for a fixture directory (libtest-mimic 0.8):
// conformance/layout/tests/fixtures.rs ([[test]] name = "fixtures", harness = false)
use libtest_mimic::{Arguments, Failed, Trial};
fn main() -> std::process::ExitCode {
let args = Arguments::from_args();
let root = std::path::Path::new(env!("CARGO_MANIFEST_DIR")).join("fixtures");
let mut trials = Vec::new();
for case in walk_case_dirs(&root) { // one directory per fixture
let id = case.strip_prefix(&root).unwrap().display().to_string();
let ignored = case.join("SKIP").exists();
trials.push(Trial::test(format!("layout::{id}"), move || run_case(&case))
.with_kind("layout").with_ignored_flag(ignored));
}
libtest_mimic::run(&args, trials).exit_code()
}
fn run_case(dir: &std::path::Path) -> Result<(), Failed> {
let input = std::fs::read_to_string(dir.join("input.nuif"))?;
let context: EvaluationContext = toml::from_str(&std::fs::read_to_string(dir.join("context.toml"))?)?;
let actual = engine.layout(&decode(&input)?, &context)?;
let actual_text = canonical_layout_text(&actual);
let expected_path = dir.join("expected.layout.txt");
if std::env::var_os("NUIF_UPDATE_EXPECT").is_some() { std::fs::write(&expected_path, &actual_text)?; return Ok(()); }
let expected = std::fs::read_to_string(&expected_path)?;
if expected != actual_text { return Err(format!("mismatch in {}", dir.display()).into()); }
Ok(())
}
Properties of this design (from the sources): --list --format terse is provided by libtest-mimic, so nextest can run each Trial in its own process; --skip, --exact and the positional filter select fixtures by their path-derived names; Trial::with_ignored_flag implements SKIP markers without deleting fixtures; the working directory is the package root, so relative paths resolve; test names must be stable and unique because nextest keys results and JUnit cases by name.
Recommended layout for conformance/ fixture crates (interpretation combining the sources):
conformance/
Cargo.toml # member of the workspace; [[test]] harness = false per suite
README.md, PLAN.md
fixtures/
<suite>/ # model | canonicalization | extensions | layout | render | operations | merge | provenance | adapter | security
<fixture-id>/ # stable slug; directory name is the test name
input.nuif # authored input (text form) or input.bin for binary-only cases
context.toml # evaluation context: viewport, scale, fonts, capability profile, tolerances
expected.<kind>.txt|json # canonical text/JSON expectation (layout boxes, diagnostics, canonical form)
expected.png # only for render suite; rendered by the CPU reference path, oxipng-optimised
ops.json # operations suite: transaction list; inverse and replay derived, not stored
meta.toml # title (unique), issue reference, tolerance overrides, disabled = true|false
fonts/ # pinned fonts referenced by context.toml; no system fonts
tests/
<suite>.rs # libtest-mimic harness; one Trial per fixture directory
generated/ # browser-differential cases regenerated by an xtask; never edited by hand
Regeneration protocol: a single variable (NUIF_UPDATE_EXPECT=1, mirroring UPDATE_EXPECT, MAKE_REF, MASONRY_TEST_BLESS, SLINT_CREATE_SCREENSHOTS) rewrites expectations; missing expectations fail and write *.new files; diff artifacts go to an ignored diffs/ directory and are uploaded by CI; generated suites are regenerated wholesale by an xtask and committed separately.
NUIF relevance
Borrow
- libtest-mimic as the harness for every
conformance/suite, because it preservescargo testfilters and nextest process isolation without a code generator. - The resvg fixture discipline (one issue per fixture, unique title, ids on every element, pinned fonts, lossless-compressed references) for the render and layout suites, because conformance/PLAN.md requires reproducible results with declared tolerances.
- rust-analyzer’s sibling-expectation layout (
NNNN_name.rs+.rast) andUPDATE_EXPECTsemantics, because they keep inputs and expectations reviewable side by side in diffs. - Taffy’s “generated tests are never edited by hand” rule for browser-differential layout fixtures, because it separates oracle changes from engine changes.
Adapt
- datatest-stable’s one-file-per-test model must become one-directory-per-test, because NUIF fixtures need
input,contextandexpectedfiles together and a fixture ID in the report (conformance/PLAN.md “fixture ID and evaluation context”). - expect-test inline expectations are appropriate for unit tests in crates, not for conformance fixtures, which must be language-neutral files usable by non-Rust implementations.
- trybuild’s
wipdirectory convention translates to*.newfiles next to expectations, matching egui and Masonry.
Reject
- test-generator, because it has had no release since 2022-12-08 and relies on
build.rsre-run hints rather than run-time enumeration. - Editing generated differential fixtures by hand, because both Taffy and Slint document that generated suites are overwritten wholesale.
#[should_panic]and stdout capture as fixture assertions, because libtest-mimic does not capture output and rust-analyzer’s style guide rejectsshould_panicin favour of explicit results.
Open questions
- Whether nextest’s per-process execution overhead is acceptable for thousands of small fixtures, or whether suites should batch fixtures per process for local runs and isolate only in CI.
- Whether conformance fixtures should be embedded with
include_dir!for a self-containednuif conformanceCLI subcommand, in addition to directory enumeration. - Whether a JSON test-result schema should be emitted by the harness directly (QA item 10), since libtest-mimic lacks
--format=junitand nextest’s JUnit only covers pass/fail/time.
Sibling order representation for authored documents; fractional indexing, list CRDT positions and anchor-based operations compared
Document status:
reviewed. Canonical source.
Summary
Three families represent the order of siblings under a parent. Fractional indexing assigns each child a key from a dense ordered set (a fraction in (0, 1) written as a string over a base-95 or base-62 alphabet); insertion between two neighbours picks a key strictly between their keys, so a reorder is one register write. List CRDTs (RGA, Logoot, LSEQ, YATA, Fugue) assign each element an identifier that is either a dense path (Logoot, LSEQ) or a reference to an existing element plus a Lamport-style identifier (RGA, YATA, Fugue); the order is recovered by a deterministic traversal that requires tombstones for deleted elements. Anchor-based operations (“insert after element x”) are the operation form used by RGA, Automerge, Yjs and Fugue and, in Penpot, as an optional after-shape alternative to an integer index.
The interleaving anomaly (Kleppmann, Gomes, Mulligan and Beresford, PaPoC 2019) affects every dense-identifier scheme, fractional indexing included: two concurrent runs inserted at one gap can be shuffled element by element. RGA and YATA are proved forward non-interleaving; Weidner and Kleppmann’s FugueMax is proved maximally non-interleaving (forward and, where achievable, backward). Figma and the fractional-indexing library accept interleaving and unbounded key growth; neither source describes rebalancing. USD does not store keys at all: each layer holds an ordered child list plus a sparse reorder nameChildren statement, and composition appends names weakest-to-strongest and applies each layer’s reorder in turn.
Source statements are reported in ## Evidence; the NUIF interpretation follows in ## NUIF relevance.
Evidence
Fractional indexing.
- Figma stores position as “a fraction between 0 and 1 exclusive”; “Each index is stored as a string”; base 95 over printable ASCII with the leading “0.” omitted; insertion “set[s] the index for the new object to the average index of the two objects on either side”; “Averaging between two identical indices doesn’t work”, resolved by the server “generating and assigning a unique position to the second insert operation”; “Index length can grow over time”, accepted because “the number of reordering operations is bounded by user activity”; “Merging new elements from multiple clients may interleave them”, accepted because “the new objects likely don’t overlap”. No rebalancing is described. E. Wallace, “Realtime Editing of Ordered Sequences”, https://www.figma.com/blog/realtime-editing-of-ordered-sequences/, retrieved 2026-08-29.
- Parent link and position are one property “so they update atomically”; the server “reject[s] parent property updates that would cause a cycle”. E. Wallace, “How Figma’s multiplayer technology works”, 2019-10-16, retrieved 2026-08-29 (see nuif:research:figma-multiplayer-and-rendering-engineering).
- Wallace’s later algorithm note: objects are sorted “by their positions (using object id as a tie-breaker)”; “add a random offset to the end of the fraction during each insert operation” to avoid identical keys with high probability; “If two peers both simultaneously insert a run of objects at the same location, the resulting objects may be interleaved. So this algorithm is not appropriate in situations where object adjacency is critical”; “Index length can become long in pathological scenarios”; “Floating-point numbers are insufficient”. https://madebyevan.com/algos/crdt-fractional-indexing/, retrieved 2026-08-29.
rocicorp/fractional-indexing4.0.0 (CC0-1.0):generateKeyBetween(a, b)andgenerateNKeysBetween(a, b, n); default digits0-9A-Za-zwith integer-part headsA-Z/a-z; keys “sort correctly using ordinary lexicographic comparison because the digits do”;localeCompare“will give an incorrect ordering”; repeated insertion at one gap lengthens keys (a0,a1,a2,Zz,a1Vin the README example); jitter is not in the core library and is delegated tonathanhleung/jittered-fractional-indexing.README.mdlines 4, 29, 61, 91, 114, 135, 172–178;package.jsonlines 3, 34; retrieved 2026-08-29.- Jittered variant: after computing the unjittered midpoint, “with 50% probability each, we either generate a key between the original lower bound
aand themidpoint, or a key between themidpointand the original upper boundb”, repeatedjitterBitstimes; the README gives a birthday-bound example of “~4.5% chance of collision” at 30 bits and 10 000 concurrent keys. https://github.com/nathanhleung/jittered-fractional-indexing README, retrieved 2026-08-29.
List CRDTs.
- RGA: Roh, Jeon, Kim, Lee, “Replicated abstract data types: Building blocks for collaborative applications”, J. Parallel Distrib. Comput. 71(3), 354–368, 2011, DOI 10.1016/j.jpdc.2010.12.006 (dblp record retrieved 2026-08-29). In Attiya et al.’s formulation an insertion is a triple (a, t, r) with r the timestamp of the reference (predecessor) character; deletions retain tombstones; multiple insertions anchored to one character “are sorted in descending timestamp order”. Kleppmann et al., PaPoC 2019, §3 (PDF p. 3).
- Logoot: Weiss, Urso, Molli, ICDCS 2009, pp. 404–412, DOI 10.1109/ICDCS.2009.75. LSEQ: Nédelec, Molli, Mostéfaoui, Desmontils, DocEng 2013, pp. 37–46, DOI 10.1145/2494266.2494278 (dblp records retrieved 2026-08-29). Both assign “a unique position identifier from a dense ordered set”; identifiers “are paths through a tree”. PaPoC 2019, §2.
- YATA: Nicolaescu, Jahns, Derntl, Klamma, “Near Real-Time Peer-to-Peer Shared Editing on Extensible Data Types”, GROUP 2016, DOI 10.1145/2957276.2957310 (Semantic Scholar record retrieved 2026-08-29). Yjs: “Everything inserted in a Yjs document is given a unique ID, formed from a ID(clientID, clock) pair”; an item “stores a reference to the IDs of the preceding and succeeding item” in
originandoriginRight; a deleted item “is flagged as deleted”; “No data is kept on when an item was deleted”.INTERNALS.mdlines 7–13, 45–50, 62–69, 104–108, yjs main, retrieved 2026-08-29. Conflict resolution inItem#integrate: items with equaloriginare ordered byo.id.client < this.id.client(case 1) and by transitive origin membership (case 2).src/structs/Item.jslines 168–245, retrieved 2026-08-29. - Automerge: “When you insert elements into a list the insert operation references the ID of the element you are inserting after”; concurrent inserts after one element are resolved by “arbitrarily choose one to insert first and then insert the other immediately afterwards”; no list move operation is documented. https://automerge.org/docs/reference/under-the-hood/merge-rules/, retrieved 2026-08-29. The Rust crate exposes
Cursor { Start, End, Op(OpCursor) }, “An identifier of a position in a Sequence”; automerge 0.11.0, https://docs.rs/automerge/latest/automerge/enum.Cursor.html, retrieved 2026-08-29. - Fugue: Weidner and Kleppmann, “The Art of the Fugue: Minimizing Interleaving in Collaborative Text Editing”, arXiv 2305.00583 (v1 2023-04-30, v2 2023-11-17, v3 2025-10-21). Algorithm 1 (PDF p. 6):
ID := (RID × N) ∪ {null}; a node is(id, value, parent, side);insert(i, x)takesleftOrigin(the (i−1)-th value) andrightOrigin(“next node after leftOrigin in the tree traversal that includes tombstones”); the node becomes a right child ofleftOriginif it has no right children, otherwise a left child ofrightOrigin; on delivery siblings are ordered bynode.id <;deletesetsvalue ← ⊥. “We cannot remove a deleted element’s node entirely: it may be an ancestor to non-deleted nodes” (p. 6). Definition 2 (forward non-interleaving) and Definition 4 (maximal non-interleaving, conditions (1)–(3)), §5.2 (p. 8); Definition 6: FugueMax “visits right-side siblings in the reverse order of their right origins, breaking ties using the lexicographic order of their IDs”, §5.3 (p. 9); Theorem 9: FugueMax is maximally non-interleaving, §5.4. Table 1 (p. 3): Logoot, LSEQ and Treedoc interleave forward and backward; RGA is proved forward non-interleaving and interleaves backward; Yjs is proved forward non-interleaving, backward one-replica unproven, backward multi-replica interleaves; Fugue and FugueMax are proved non-interleaving in all three columns. Evaluation (§6, Tables 2–3, p. 11–12): on a 260 k-operation trace Fugue used 2.4 MB, 46 network bytes per operation and 94 k operations/s; Yjs 13.6.8 used 3.3 MB, 29 bytes/op, 39 k ops/s; Automerge-Wasm 0.5.0 used 126 bytes/op and 52 k ops/s. - Interleaving anomaly: Kleppmann, Gomes, Mulligan, Beresford, “Interleaving anomalies in collaborative text editors”, PaPoC 2019, DOI 10.1145/3301419.3323972 (author PDF https://martin.kleppmann.com/papers/interleaving-papoc19.pdf, retrieved 2026-08-29). §2: Logoot and LSEQ “suffer from this problem”; Figure 3 shows the anomaly with rational-number identifiers, the model that fractional indexing instantiates. §2.1 adds clause 1(d) to the strong list specification: for concurrent insertion sets X and Y at one location, “either all X insertions appear before all Y insertions … or vice versa, but they are never interleaved”. §3: RGA “does not suffer from” the character-level anomaly under sequential insertion but exhibits a “lesser” anomaly for non-sequential insertion; §3.1 proposes a 4-tuple (a, t, r, e) with e the set of timestamps of prior insertions at the same reference, and a session-grouping order, stated as a conjecture. The Fugue paper (§3.2) reports that this 2019 definition “cannot be satisfied by any algorithm” and that the proposed fix is flawed.
mweidner037/list-positions(MIT) implements Fugue for application lists:Position = { bunchID: string; innerIndex: number }, each bunch carriesBunchMetadescribing “its location in the tree”; “if two users concurrently insert a (forward or backward) sequence at the same place, their sequences will not be interleaved”;lexicographicString(pos)yields strings whose lexicographic order matches the list order; fractional indexing is described as “a related but less general idea”. README, retrieved 2026-08-29.- Tree move with list order: the move-operation paper delegates sibling order to “an additional list CRDT for each branch node, e.g. using RGA [14] or Logoot [15]”, carried in the metadata field, so a reorder is a move with unchanged parent (nuif:research:crdt-tree-move-operation, PDF §3.7). Kleppmann, “Moving Elements in List CRDTs”, PaPoC 2020, DOI 10.1145/3380787.3393677, shows delete-and-reinsert duplicating an element under concurrent moves.
Design and scene documents.
- Penpot: a parent’s children are an ordered vector of shape ids (
(:shapes shape)),common/src/app/common/types/container.cljcget-direct-children; the:mov-objectschange schema carriesparent-id,shapes,index(optional int) andafter-shape(optional); at applicationindexis computed as(or (some-> (d/index-of (:shapes parent) after-shape) inc) index)and the shapes are inserted withd/insert-at-indexor appended; a move is rejected when the target is a descendant of the moved shape, and skipped when the parent no longer exists (“race condition when an inflight move operations lands when parent is deleted”).common/src/app/common/files/changes.cljclines 229–239, 735–770, 815–823, develop branch, retrieved 2026-08-29.:add-objlikewise carries an optional integerindex(lines 189–198, 595–600). - OpenUSD:
SdfPrimSpec::SetNameChildrenOrder(names)“Given a list of (possibly sparse) child names, authors a reorder nameChildren statement for this prim. The reorder statement can modify the order of name children during composition. This order doesn’t affect GetNameChildren(), InsertNameChild(), SetNameChildren(), et al.”;ApplyNameChildrenOrder“employs the standard list editing operation for ordered items in a ListEditor”;InsertNameChild(child, index)documents thatindex“is ignored except for range checking”.pxr/usd/sdf/primSpec.hlines 160–210, release branch, retrieved 2026-08-29.SdfListOpTypehasExplicit, Added, Deleted, Ordered, Prepended, Appended, andApplyOperationsis well defined only when neither operand “use[s] the ‘ordered’ or ‘added’ item lists” (pxr/usd/sdf/listOp.hlines 30–36, 205–217). Composition:PcpComposeSiteChildNamesiterates layers withTF_REVERSE_FOR_ALL(weakest first), appends names not yet seen, then applies the layer’sPrimOrderfield withSdfApplyListOrdering(pxr/usd/pcp/composeSite.cpplines 487–540);_ComposePrimChildNamesperforms a “Reverse strength-order traversal (weak-to-strong)” over prim-index nodes (pxr/usd/pcp/primIndex.cpplines 5648, 5669–5671, 5783–5791). The glossary states list-edited elements “always resolve to a set” with no repetition (https://openusd.org/release/glossary.html, “List Editing”, retrieved 2026-08-29).
Mechanism
Fractional keys. Order is the lexicographic order of keys; a key is a variable-length numeral in a fixed alphabet. Insertion between keys a < b computes a key k with a < k < b; when a and b share a long common prefix the new key is at least one digit longer, so n repeated insertions at the same gap produce keys of length Θ(n) digits. Two concurrent insertions at one gap can produce equal keys, so a tie-break (object id, server rewrite, or random jitter) is required, and two concurrent runs at one gap sort by key value, not by run, which is the interleaving anomaly of Figure 3 in the PaPoC 2019 paper. A key is a per-child register: reorder and reparent are one register write, moves commute with unrelated writes, and two concurrent moves of one child resolve by last-writer-wins on the register with no conflict signalled. Keys are replica-generated; the same visible order admits infinitely many key assignments, so a document’s hash depends on editing history unless keys are renormalised, and no retrieved source specifies renormalisation.
Anchor-based positions with tombstones (RGA, YATA, Fugue). An insertion names an existing element (left origin; YATA and Fugue also a right origin) and carries a unique identifier (replica, counter). The visible order is a depth-first traversal of the origin tree with a fixed sibling rule: descending identifier (RGA), client identifier (YATA case 1), identifier order among siblings (Fugue), or reverse right-origin order then identifier (FugueMax). Deleted elements stay as tombstones because later insertions may name them; Fugue states this explicitly. Insertions commute under causal delivery; order is a pure function of the operation set. Forward non-interleaving holds for RGA, YATA and Fugue; maximal non-interleaving for FugueMax. Metadata per element is one identifier plus one or two origin references; save size in Fugue’s benchmark was 60 % of the literal text.
Anchor-based operations over a plain array. The operation carries after: Option<EntityId> (RGA’s reference element) and the canonical document stores the resolved array. Two operations with different anchors commute; two insertions with the same anchor need a tie-break rule to be replay-deterministic; a move whose anchor was concurrently deleted needs a fallback. The collaboration profile supplies the tie-break and the fallback through a list CRDT whose tombstones are profile data; a sequential patch supplies them through preconditions and typed conflicts. Penpot’s after-shape and Automerge’s Cursor::Op are instances of this form.
USD reorder statements. Each layer stores the ordered child list it authored plus a sparse ordering list; composition appends new names in weakest-to-strongest order and applies each stronger layer’s reorder to the running list. Order is derived, deterministic and free of replica identifiers, and the ordered-items list op is not closed under composition, which is why USD excludes it from ApplyOperations.
Comparison against the evaluation criteria (sources as above).
| Criterion | Integer index (current NUIF, Penpot) | Fractional key (Figma, rocicorp) | List-CRDT identifier (RGA, YATA, Fugue) | Anchor operation over ordered array (proposed) |
|---|---|---|---|---|
| Commutativity of independent operations | No; indices shift | Yes (register per child) | Yes under causal delivery | Yes when anchors differ; same-anchor case needs a tie-break |
| Canonical form without replica metadata | Yes | No; keys are history-dependent, jitter is random | No; tombstones and identifiers required | Yes; array only |
| Human-readable text form | Position implicit in listing | Opaque strings (a1V) | Opaque identifier pairs | Position implicit in listing; operation names a neighbour |
| Interleaving of concurrent runs | Not applicable (sequential) | Interleaves (PaPoC 2019 Fig. 3; Wallace) | RGA/YATA forward-safe; FugueMax maximal | Delegated to profile CRDT; sequential patches never interleave |
| Growth bound | None | Key length Θ(n) at one gap; no rebalancing specified | One identifier per element; tombstones retained | None in canonical form |
| Replay determinism | Order-sensitive | Deterministic given tie-break | Deterministic | Deterministic given anchor resolution and same-anchor tie-break |
| Detection of two moves to one slot | Index collision, ambiguous | Register last-writer-wins, silent | Register over positions (Kleppmann 2020), silent | Same-anchor moves are detectable and reportable as a typed conflict |
NUIF relevance
Borrow
- The RGA/Automerge operation form: an insertion or move names the sibling it follows, not an integer position;
Option<EntityId>withNonemeaning “first child” is the RGAheadvalue. - The USD principle that the canonical document stores a resolved ordered list and no order keys, so
nuif-text-0lists children in order and the hash covers the array, not replica-generated strings. - Fugue’s tree formulation (left origin, right origin, identifier tie-break) as the list CRDT the collaboration profile should specify, since it is the only candidate with proved forward and backward non-interleaving and a published, benchmarked implementation.
- The move-operation paper’s rule that a reorder is a move with unchanged parent and new order metadata, so one operation type covers reparent and reorder.
- Penpot’s precondition that a move whose parent no longer exists is dropped with a diagnostic rather than applied to a stale index.
Adapt
- Same-anchor concurrency: in a sequential patch the two insertions are applied in patch order (no ambiguity); in a three-way merge the merge tool orders same-anchor insertions from different branches by branch precedence declared in the merge input and reports
OrderAmbiguousas an informational diagnostic; in the collaboration profile the CRDT’s sibling rule applies. NUIF must state all three rules; no retrieved source covers the three-way case. - Anchor deleted concurrently: the collaboration profile resolves through tombstones (RGA semantics); a sequential patch or three-way merge whose
afteranchor is absent from the target parent fails the operation with a typed conflictAnchorMissingthat carries the intended anchor and the last known neighbours, matching spec/06’s requirement to surface typed conflicts rather than pick winners. - Two concurrent moves of one entity are a semantic conflict in NUIF (
MoveConflict { entity, targets: [(parent, anchor); 2] }), not a last-writer-wins register write as in Figma; the profile may converge structurally on one winner but must retain the conflict object (spec/10). - Fractional keys may still appear as a profile-internal or transport optimisation (Figma’s single-register reparent-and-reorder) provided checkpoint materialisation strips them; they must not appear in
nuif-coretypes.
Reject
- Integer
indexinInsertandMove(currentcrates/nuif-protocol/src/lib.rs): non-commutative and replay order-sensitive; Penpot’safter-shapeshows the migration path. - Fractional keys in the canonical form: history-dependent hashes, unbounded key growth without a specified rebalance, interleaving, and jitter that introduces randomness into a document that must be deterministic (spec/08).
- List-CRDT identifiers in the canonical form: require tombstones and replica identifiers that spec/10 assigns to profile data.
- Logoot and LSEQ for the profile: interleave in every column of Fugue’s Table 1.
Open questions
- Whether the collaboration profile should mandate FugueMax or accept any algorithm satisfying Definition 2 (forward non-interleaving); Yjs and Automerge satisfy only the weaker property, and requiring FugueMax excludes both without an adapter layer.
- Branch precedence for same-anchor insertions in three-way merge: by branch identity, by operation identifier, or by entity identifier; each is deterministic but none has a source in the retrieved literature.
- Whether
AnchorMissingshould fall back to the nearest surviving predecessor when the patch carries an ordered anchor chain (Fugue’s right origin suggests a two-anchor formbetween: (Option<EntityId>, Option<EntityId>)); the cost is a larger operation and a second failure mode. - The Fugue paper’s claim that the PaPoC 2019 non-interleaving definition is unsatisfiable was not independently verified here; only the Fugue paper’s statement (§3.2) was retrieved.
Bounded live Chromium capture transport and executable baseline
Document status:
verified. Canonical source.
Summary
The first executable live-capture profile should use the already pinned Chrome for Testing build through a small synchronous Chrome DevTools Protocol (CDP) client. The decision is intentionally narrower than choosing a general browser automation framework. NUIF needs exact DOM snapshot tables, a computed-style whitelist, platform-font usage, response bodies, an accessibility tree and a reference screenshot from one known Chromium build. A bounded loopback WebSocket transport supplies those operations without adding a second browser version authority or moving capture semantics into a framework object model.
This is not a general claim that raw CDP is better than Playwright or WebDriver BiDi. Playwright is the stronger current choice for cross-browser application testing. WebDriver BiDi is the stronger standards-track transport to watch for portable remote control. Neither currently improves this profile’s required Chromium-specific evidence while preserving the repository’s existing browser pin and one-process Rust boundary.
Evidence
- The CDP project documents JSON command/event domains and warns that the
tip-of-tree protocol changes frequently without a backwards-compatibility
guarantee. It also documents
DevToolsActivePort,/json/listand the page WebSocket endpoint used here. NUIF therefore pins Chrome for Testing 152.0.7977.64, records the reported product and protocol version, and treats a protocol change as a gate failure rather than accepting a floating schema. Locator: https://chromedevtools.github.io/devtools-protocol/, retrieved 2026-08-31. DOMSnapshot.captureSnapshotreturns a flattened DOM/layout table plus a requested computed-style whitelist. CSS exposes actualgetPlatformFontsForNodeusage, Network exposes bounded response bodies, Accessibility exposes the computed tree and Page exposes screenshots. Locators, retrieved 2026-08-31: https://chromedevtools.github.io/devtools-protocol/tot/DOMSnapshot/, https://chromedevtools.github.io/devtools-protocol/tot/CSS/, https://chromedevtools.github.io/devtools-protocol/tot/Network/, https://chromedevtools.github.io/devtools-protocol/tot/Accessibility/ and https://chromedevtools.github.io/devtools-protocol/tot/Page/.Page.getResourceTreeandPage.getResourceContentare explicitly marked experimental by CDP. They are therefore only a pinned-build fallback after passiveNetwork.getResponseBody, not a portable capture contract. The Fetch domain was also evaluated and rejected for this first profile: its response-stage mode pauses every matched request until the client continues it, so it changes the loading path being observed. Locators: https://chromedevtools.github.io/devtools-protocol/tot/Page/ and https://chromedevtools.github.io/devtools-protocol/tot/Fetch/, retrieved 2026-08-31.- Playwright versions require particular browser binaries and normally use its installation command/cache. That is useful when Playwright owns the test matrix, but NUIF already content-pins and installs the browser used by Gate C and the WebAssembly smoke test. Adding Playwright here would create a second browser compatibility and download lifecycle without replacing the CDP-only evidence calls. Reconsider it when Firefox/WebKit live capture becomes an implemented gate. Locator: https://playwright.dev/docs/browsers, retrieved 2026-08-31.
- WebDriver BiDi is a W3C Working Draft on the Recommendation track with a Web-platform test suite and implementation report. It is the preferred portability watch path, but the current draft’s remote-control/network/script surface does not replace Chromium’s flattened DOM/layout snapshot or platform-font-use calls. A future adapter should share NUIF observation types rather than force CDP vocabulary into the standard transport. Locator: https://www.w3.org/TR/webdriver-bidi/, 29 June 2026 Working Draft, retrieved 2026-08-31.
- Tungstenite provides a synchronous RFC 6455 client with explicit maximum
frame/message configuration and no default TLS feature. That matches a
sequential
ws://127.0.0.1debugger socket. Tokio-tungstenite, chromiumoxide or headless_chrome would add async/runtime or browser-object layers without changing the evidence contract. TLS is deliberately absent because remote debugger endpoints are rejected. NUIF pins 0.29.0 rather than current 0.30.0: the upstream changelog says 0.30 adds rejection of non-compliant clients on the server side and updates Rand/SHA/MSRV. This adapter is exclusively a client to pinned Chrome; 0.29 preserves the needed API and removes four duplicate Digest-family version lines from the resolved graph. Locators: https://github.com/snapview/tungstenite-rs and https://github.com/snapview/tungstenite-rs/blob/master/CHANGELOG.md, retrieved 2026-08-31.
Mechanism
cargo xtask gate-j-live installs or reuses the exact browser lock, then
accepts four complete isolated-profile captures against a bounded concurrent
keep-alive loopback fixture: 360 px, 768 px, held-out 900 px and a repeated
360 px run. An incomplete browser/network outcome is never filled in: the
harness records it and permits at most three fresh-profile attempts for that
viewport. The adapter:
- creates a fresh temporary profile and accepts only its loopback debugger;
- caps discovery HTTP; WebSocket frame, message and write-buffer bytes; event count and aggregate bytes; command count; connected capture time; DOM nodes; per-node and total font uses; resource count; per-resource bytes and total retained response bytes, with base64 length checked before decode;
- enables only the required CDP domains and pins viewport/DPR,
en-US, UTC, screen media, light color scheme and reduced motion; - waits for the lifecycle
loadevent carrying the exact navigation loader ID, disables animation/transition/caret phases, fixes scroll at zero, waits two animation frames, awaits every image decode plusdocument.fonts.readyand waits two final frames so ready assets are reflected in layout/font-use evidence; - captures flattened DOM/layout/background style, computed accessibility, actual platform-font use and exact HTTP response bodies, using bounded Network response bodies first and the post-load Page resource tree/content cache as fallback; it drains pending events to a bounded quiet point and accepts a bounded PNG only after two consecutive screenshots are identical;
- replaces opaque CDP backend IDs with deterministic preorder identities;
- strips URL query/fragment data before a capture can be serialized and never requests cookies, storage or request-header fields from CDP; and
- carries the structured runtime context into the canonical observation bundle.
The fixture injects independent query, cookie, storage, Authorization and custom-header canaries after navigation. The server proves the query and request values arrived; in-page code proves storage round-tripped and returns the already-awaited probe response body. That body is rejected if it reflects any canary, then retained under its same-origin URL. The gate scans serializable capture, observation, proposal and package bytes for all five values. This is a specific non-retention regression test, not a claim that arbitrary response content is free of application secrets.
Executable result
The gate requires exact repetition of raw capture, normalized observations, proposal/package bytes and narrow-viewport screenshot. All five declared response bodies must have exactly the expected SHA-256 set. The custom Ahem font must be reported as an actually used downloaded font, main/button role and name must occur in the accessibility tree, and screenshot dimensions must equal the declared viewport. The report records per-viewport and total attempt counts; only a complete exact fixture can be accepted, and three failed attempts are a blocking gate error.
For the declared responsive fixture, linear geometry prediction fitted to the 360/768 px observations must have lower aggregate absolute error at 900 px than copying the 360 px geometry as a one-screenshot freeform baseline. The machine report stores both errors and every raw count. This result validates only the fixture and falsifies a broken multi-viewport path; it is not evidence of broad responsive-layout inference accuracy.
Security and non-claims
An isolated profile avoids ambient browser cookies, storage and extensions. The adapter owns no general login/profile import path. Response bodies remain untrusted inert package resources and are not executed by package readers. Captured pages still execute inside Chromium and may make network requests, so the caller must authorize the target and apply environment/network policy.
Cross-origin opaque bodies, arbitrary application state, local host font bytes, source-map correlation, canvas/WebGL semantics, video/worklet state and hostile page determinism remain explicit omissions. Browser crashes, protocol drift and budget violations fail the capture atomically. Cross-browser reproduction, authenticated-site capture and a real licensed corpus remain future gates.
Revisit conditions
Adopt Playwright or an equivalent higher-level runner when a real multi-engine matrix is funded and its browser pins can become the single matrix authority. Add WebDriver BiDi when its implemented domains can produce the portable observation subset. Do not retain a raw CDP command merely because it exists; every added domain must have a typed output, byte/cardinality ceiling, secret policy and executable negative case.
NUIF relevance
The result turns browser capture from a provider-input sketch into one real
ports-and-adapters consumer of the same observation, package and proposal
contracts used by later reconstruction work. Exact source/runtime evidence
stays distinguishable from inferred NUIF semantics, and the browser adapter
does not enter nuif-core or redefine the HTML source-synchronization profile.
Open questions
- Which smallest matched-style and stylesheet/source-correspondence set is useful enough to add without making one capture unbounded?
- How should opaque frame/resource evidence be represented across CDP and WebDriver BiDi without treating missing source as screenshot equivalence?
- Which cross-OS differences remain after the exact browser/context pin, and which belong in environment compatibility rather than capture fidelity?
- Can a licensed real-page corpus preserve sensitive response bodies safely enough to evaluate the source-backed route outside synthetic fixtures?
LoRA low-rank parameter adaptation
Document status:
reviewed. Canonical source.
Summary
LoRA freezes pretrained weights and learns low-rank update matrices in selected layers. It can reduce trainable parameter count and keep task adapters separate from the base model. That makes it a plausible packaging and experimentation technique after NUIF has task-specific data and evaluation.
LoRA is not a reconstruction architecture, dataset-quality method or accuracy guarantee. A low-rank adapter can efficiently learn the wrong target just as a full fine-tune can.
Evidence
- arXiv:2106.09685 and the ICLR 2022 OpenReview paper define a frozen base matrix with a trainable low-rank decomposition added to its update.
- The paper reports large reductions in trainable parameters and avoids extra inference latency from a separate serial adapter path when weights are merged.
- Results are model/task specific. They do not establish that every vision encoder/decoder, multimodal projector or operation grammar has a low-rank task update.
Mechanism
For a frozen weight matrix W, LoRA learns BA with rank r and uses
W + scale * BA during the forward pass. The adapter artifact therefore depends
on the exact base-model identity, target modules, rank, scaling and training
configuration.
NUIF relevance
Borrow separable, content-addressed task adapters and controlled rank/module ablations.
Adapt the artifact manifest to pin base model, tokenizer/image processor,
operation-schema version, renderer/evaluator version, dataset revision and
license/provenance. Never store adapters in a .nuif document.
Reject “use LoRA” as a research conclusion before an untuned baseline and a frozen evaluation suite exist.
Open questions
- Which modules need adaptation for screen grounding versus operation decoding?
- Does a small adapter preserve general visual/OCR capability better than a full fine-tune on narrow synthetic layouts?
- Can one adapter cover both initial synthesis and corrective operations without negative transfer?
Lottie and Rive portable animation/runtime models
Document status:
reviewed. Canonical source.
Summary
Lottie specifies a JSON-based animated-vector document with extensible additional data. Rive serializes artboards, shapes, animation and state machines into a compact binary runtime format designed for forward evolution.
NUIF relevance
Animation/state-machine semantics should be modular rather than mixed into base geometry. NUIF should define stable behavior graph concepts and allow richer animation dialects to lower into them; it should not duplicate either runtime format wholesale.
LPIPS learned perceptual image similarity
Document status:
reviewed. Canonical source.
Summary
LPIPS compares normalized deep feature activations and was calibrated/evaluated against human perceptual judgments on image distortions. It complements raw pixel and classical structural metrics, but it is model- and weight-dependent and does not measure UI structure, text correctness or editability.
Evidence
- CVPR 2018 defines distance as a spatial average of weighted squared distances between normalized feature activations across network layers.
- The paper evaluates linear calibration, full tuning and training-from-scratch variants on perceptual judgments; metric identity includes the backbone and weights, not merely the label “LPIPS.”
- The study concerns image-patch perceptual similarity. It does not establish a threshold for UI reconstruction or resistance to metric gaming.
Mechanism
Two images are passed through the same fixed feature network. Per-layer activations are channel-normalized, optionally channel-weighted, compared and spatially averaged. A reproducible report must pin preprocessing, resolution, backbone, weights, library version and reduction.
NUIF relevance
Borrow LPIPS as one non-normative visual diagnostic in a metric ensemble.
Adapt thresholds only after correlation with human UI judgments and property-level errors is measured. Report it beside raw pixel difference, FLIP, SSIM, text/geometry/structure/resource metrics.
Reject LPIPS as the sole reward or correctness boundary; a full-page screenshot embedded as one image could score well while containing no editable semantics.
Open questions
- Which backbone and preprocessing correlate best with UI differences after controlling for text antialiasing?
- How susceptible is the selected metric to adversarial or degenerate reconstructions in the operation-search loop?
- Does it add ranking value beyond FLIP and property-level measures?
macOS Metal dependency and Rust uninhabited-static future incompatibility
Document status:
verified. Canonical source.
Summary
The previous macOS editor graph resolved masonry_winit -> imaging_wgpu -> wgpu 28.0.0 -> wgpu-hal 28.0.1 -> metal 0.33.0 -> block 0.1.6. Rust 1.98 compiled
that graph but reported block as future-incompatible. wgpu pull request 5641
replaced metal-rs with objc2-metal and block2; wgpu 29 contains that
migration. The reviewed refpath/xilem commit eabfe0a updates the NUIF
Masonry revision to the wgpu 29 API without patching the Objective-C blocks ABI.
NUIF pins its immediate descendant 1b96eb8 by full SHA; that descendant
replaces abandoned font dependencies without changing the graphics migration.
The active editor graph resolves imaging_wgpu 0.0.2 -> wgpu 29.0.4 -> objc2-metal 0.3.2 -> block2 0.6.2.
block 0.1.6 and metal-rs are absent from the lock file and active dependency
graph. The fork remains a maintenance boundary until the equivalent migration
is available from the selected upstream Xilem revision.
Evidence
- wgpu pull request 5641 replaced
metalwithobjc2-metal,blockwithblock2, andcore-graphics-typeswithobjc2-core-graphics. The pull request merged on 2026-01-28. Locator: dependency and source-file changes, retrieved 2026-08-30: https://github.com/gfx-rs/wgpu/pull/5641. wgpu-hal 29.0.4declaresblock2 0.6.2,objc2-metal 0.3.2, andobjc2-quartz-core 0.3.2for the Metal backend. Locator: the macOS target dependencies inwgpu-hal/Cargo.tomlat tagv29.0.4, retrieved 2026-08-30: https://github.com/gfx-rs/wgpu/blob/v29.0.4/wgpu-hal/Cargo.toml.refpath/xilemcommiteabfe0a92ff5ab0e26515383fdeaf288672b3e88updates the imaging dependencies and the three affected wgpu API call sites. Locator: commit diff, retrieved 2026-08-30: https://github.com/refpath/xilem/commit/eabfe0a92ff5ab0e26515383fdeaf288672b3e88.- The active pin
1b96eb8db3f88f85db1a3594d80d3480b29392fbhaseabfe0aas its sole parent and replaces abandoned font dependencies. Locator: commit metadata and diff, retrieved 2026-08-30: https://github.com/refpath/xilem/commit/1b96eb8db3f88f85db1a3594d80d3480b29392fb. cargo tree -p nuif-editor -i blockreports no matching package after the migration.cargo tree -p nuif-editorresolves wgpu 29.0.4, objc2-metal 0.3.2, and block2 0.6.2 from the active lock file (2026-08-30).cargo report future-incompatibilitiesreports that no reports are available after rebuildingnuif-editoragainst the migrated graph with rustc 1.98.0 (2026-08-30).cargo test -p nuif-editor --features editor-automationpasses 14 tests against the pinned fork commit. A macOS Metal window smoke test reaches the event loop and presents without a surface error (2026-08-30).- Upstream metal-rs commit
9ed9fe9still declaresblock 0.1.6and its README now deprecates the crate in favor ofobjc2-metal. The review-onlyrefpath/metal-rsbranchmove-to-block2, commit7e0a178, replaces the production dependency withblock2 0.6.2, migrates the typed callbacks, and replaces the shared-event layout mutation with block2’s explicit ABI encoding. No pull request was opened and NUIF does not depend on this fork. Locator: https://github.com/refpath/metal-rs/tree/move-to-block2, retrieved 2026-08-30. - The review fork passes the upstream crate checks and tests on Rust 1.82, its
declared MSRV. Real macOS probes completed both an
MTLSharedEventnotification and a command-buffer completion handler. Its normal dependency graph contains block2 and no rust-block; the all-target development graph still reaches rust-block through the deprecatedcocoa 0.26dev dependency. This makes the branch useful for review, but not a complete modernization of the deprecated objc stack.
Mechanism
masonry_winit constructs the wgpu instance, acquires each surface texture,
and accesses the Metal presentation layer during macOS live resize. wgpu 29
changes the instance descriptor from a borrowed value to an owned value and
returns CurrentSurfaceTexture instead of Result<SurfaceTexture, SurfaceError>. The fork updates those call sites and maps Timeout and
Occluded to a skipped frame, as specified by the wgpu 29 variant
documentation. The live-resize path calls the objc2 selector spelling exposed
by objc2-quartz-core. The dependency update selects wgpu-hal’s objc2 Metal
backend, so metal-rs and rust-block no longer participate in compilation or
linking. Locator: masonry_winit/src/vello_util.rs and
masonry_winit/src/event_loop_runner.rs in refpath/xilem commit eabfe0a;
CurrentSurfaceTexture in wgpu 29.0.4, retrieved 2026-08-30:
https://docs.rs/wgpu/29.0.4/wgpu/enum.CurrentSurfaceTexture.html.
Decision boundary
Borrow wgpu 29’s objc2 Metal backend as the maintained replacement for metal-rs and rust-block.
Adapt the selected Xilem revision through a full-SHA fork pin. Each fork update includes the NUIF editor tests, the reverse dependency trace, and a macOS Metal window smoke test.
Contain the direct metal-rs experiment in the review-only
refpath/metal-rs branch. It demonstrates a small block2 migration but changes
callback-facing Rust types and cannot remove rust-block from the legacy Cocoa
development graph. NUIF therefore does not pin or ship the experiment.
NUIF relevance
Borrow Cargo’s future-incompatibility report and reverse dependency tree as the reproducible diagnostics for the pinned toolchain and lock file.
Adapt dependency-update review so a macOS editor-stack change verifies the
absence of block, the presence of the expected block2 path, and a package
smoke test in addition to the workspace checks.
Reject treating the fork as an indefinite divergence. The pin is removed when the selected upstream Xilem revision provides an equivalent wgpu version.
Open questions
- Which upstream Xilem revision first provides a compatible wgpu 29 or later dependency graph?
- When does
imaging_skiapublish a release that no longer requires wgpu 28 for its optional GPU backend?
Editor stack decision: Masonry on imaging with Vello and AccessKit, re-verified against the 0.4.0 release and the main branch
Document status:
reviewed. Canonical source.
Summary
Masonry has one release in the twelve months to 2026-08-29 (0.4.0, 2025-10-29, MSRV 1.88) and a main branch (b81d8d7, 2026-08-28, MSRV 1.96) that differs from that release in the paint model, the renderer abstraction, the dependency set and the widget inventory. At 0.4.0, Widget::paint receives a vello::Scene (Vello 0.6) and the test harness screenshots through wgpu. On main, Widget::paint receives an imaging::Painter, a Canvas widget records into an imaging::record::Scene, and masonry_testing rasterizes through imaging_vello_cpu without a GPU. The release notes for 0.3.0 and 0.4.0 describe the software as alpha-quality with major breaking changes expected; no changelog file exists on main. Xilem is a view layer over Masonry with incomplete coverage of Masonry’s widgets. An empirical cargo metadata resolution of Masonry main together with Vello 0.10, Parley 0.11 and AccessKit 0.25 produces two copies of Vello, wgpu, Parley and AccessKit and four of accesskit_consumer.
The alternatives re-checked here do not change the ranking from the earlier records: egui-wgpu callbacks draw into egui’s own render pass; Floem still has no accessibility tree; Blitz exposes <canvas> custom paint sources but its harness crate is unpublished and its document model is HTML/CSS; GPUI now builds an AccessKit tree per frame but its test contexts expose no tree query and the crate requires the latest stable toolchain.
NUIF interpretation: the proposed stack (Masonry, Vello, AccessKit) is confirmed with three corrections. The editor targets Masonry main pinned by git revision, not 0.4.0, because only main has the CPU harness and the Canvas widget. The canvas integration is a lowering from the NUIF render scene to imaging commands, not the injection of a vello::Scene, because main has no entry point for a Vello scene. Xilem is deferred; the editor uses Masonry’s widget tree directly. The editor is confined to apps/editor, follows Masonry’s dependency versions for every type that crosses the widget boundary, and exposes the harness through NUIF’s own session-driver trait.
Evidence
Masonry and Xilem releases, MSRV and stability
- crates.io versions of
masonry: 0.1.0 (2022-11-30, MIT), 0.1.1 and 0.1.2 (2023-02-05, Apache-2.0, MSRV 1.65), 0.2.0 (2024-05-07), 0.3.0 (2025-05-10, MSRV 1.86), 0.4.0 (2025-10-29, MSRV 1.88).masonry_core,masonry_testing,masonry_winit,xilem,xilem_coreare all at 0.4.0. GitHub releases: v0.1.0 (2024-05-07), v0.3.0 (2025-05-10), v0.4.0 (2025-10-29). Locator: crates.io API/api/v1/crates/masonry;gh api repos/linebender/xilem/releases, retrieved 2026-08-29. - Workspace manifests: tag v0.3.0
rust-version = "1.86",vello = "0.5.0",wgpu = "24.0.3",parley = "0.4.0",accesskit = "0.19.0",accesskit_winit = "0.27.0",winit = "0.30.10"(lines 33-59). Tag v0.4.0rust-version = "1.88",vello = "0.6.0",parley = "0.6.0",accesskit = "0.21.1",accesskit_winit = "0.29.2",accesskit_consumer = "0.31.0",winit = "0.30.12"(lines 36-72); release notes state wgpu 26. Main b81d8d7rust-version = "1.96",license = "Apache-2.0",imaging = "0.0.1",imaging_wgpuwith featurewgpu-28,imaging_vello,imaging_vello_hybrid,imaging_vello_cpu,vello = "0.8.0",wgpu = "28.0.0",kurbo = "0.13.1",parley = "0.8.0"(featureaccesskit),peniko = "0.6.1",winit = "0.30.13",accesskit = "0.24.0",accesskit_winit = "0.32.2",accesskit_consumer = "0.35.0"(lines 38-91). Locator:Cargo.tomlat each ref. - Release notes v0.3.0: “This is alpha-quality software. There are plenty of missing features and other issues.” Release notes v0.4.0: “This release has an MSRV of 1.88”; the software is described as alpha-quality, “We expect to continue active development, including making major breaking changes”, and “we plan to start keeping a changelog after this release”. Locator: GitHub release bodies v0.3.0, v0.4.0.
- No
CHANGELOG.mdexists at the repository root or undermasonry/at b81d8d7 (root listing:.clippy.toml,ARCHITECTURE.md,AUTHORS,Cargo.lock,Cargo.toml,LICENSE,README.md,docs, crate directories). Milestones:0.3.0closed,0.4.0open with one issue; no later milestone. Locator:gh api repos/linebender/xilem/contents;gh api repos/linebender/xilem/milestones?state=all. - README main: “An experimental Rust architecture for reactive UI”; “Xilem is a UI framework, whereas Masonry is a toolkit for building UI frameworks”; “This version of Masonry has been verified to compile with Rust 1.96 and later”. Locator:
README.mdlines 5, 34, 140;masonry/README.mdline 211. - Linebender blog, “Linebender in 2026 Q1” (2026-04-19): “Masonry has moved to imaging as an abstraction over the 2D rendering engine”; new widgets “Svg, Divider, CollapsePanel, StepInput, RadioButtons, Switch, Clip, Split”; “Masonry now has a new layout system”; “Masonry is using ui-events for more of the integration with system capabilities, including IME”; “Because imaging supports a wide variety of back-ends, Masonry can now operate in a wider variety of environments, including Vello CPU for rendering not requiring a GPU”. No later status post exists; the 2026 posts are dated 2026-04-19, 2026-07-11, 2026-08-08 and 2026-08-12 and the last three concern fearless_simd and hyperbezier curves. Locator: https://linebender.org/blog/tmil-25/; https://linebender.org/blog/ index. Zulip announcements were not retrieved (unverified).
Paint model and canvas embedding
- v0.4.0:
fn paint(&mut self, ctx: &mut PaintCtx<'_>, _props: &PropertiesRef<'_>, scene: &mut Scene)withuse vello::Scene. Locator:masonry_core/src/core/widget.rslines 13, 271 at v0.4.0. - Main:
fn paint(&mut self, ctx: &mut PaintCtx<'_>, props: &PropertiesRef<'_>, painter: &mut Painter<'_>)andpost_paintwith the same signature;use crate::imaging::Painter;masonry_core/srccontains novello::path (grep, 0 matches);masonry_corere-exportsimaging(pub use imaging;). Locator:masonry_core/src/core/widget.rslines 20, 393-408;masonry_core/src/lib.rsline 82, b81d8d7. Canvaswidget (main only; absent from the v0.4.0 widget listing): “A widget allowing custom drawing. A canvas takes a painter callback; every time the canvas is repainted, that callback is run with animagingrecord::Scene”;Canvas::update_scene(this: &mut WidgetMut<Self>, f: impl FnOnce(&mut MutateCtx, &mut Scene, Size))clears the scene, runs the callback and requests a render;with_alt_text; actionCanvasSizeChanged { size }. Locator:masonry/src/widgets/canvas.rslines 1-80, b81d8d7.imaging0.0.1 (2026-05-21, MSRV 1.92, Apache-2.0 OR MIT, “This is the initial release”): “backend-agnostic 2D imaging recording + streaming API” withPainterstreaming into anyPaintSinkandrecord::Sceneretaining an owned command stream.Paintermethods includereplay(&record::Scene),fill,fill_rect,stroke,glyphs,blurred_rounded_rect,draw_image(ImageBrushRef, Affine),push_clip,push_group,record_mask,with_masked_group;record::Scene::append_transformed. Locator: forest-rs/imagingimaging/README.md,imaging/CHANGELOG.md,imaging/src/painter.rslines 112-500,imaging/src/record.rsline 766, commit 89b364b.imaging_vellolowersrecord::Sceneinto avello::Scene(VelloSceneSink::new(&mut vello::Scene, surface_clip)) and renders native Vello scenes; the crate documentation states that “Semanticimaging::record::Scenevalues can be lowered to native Vello scenes”. No path from an existingvello::Sceneinto animagingsink was found.imaging_vello0.0.2 depends onvello ^0.7.0or^0.8.0;imaging_wgpu0.0.1 offerswgpu ^27.0.1or^28.0.0;imaging_vello_cpu0.0.2 depends onvello_cpu ^0.0.9. Locator:imaging_vello/src/lib.rslines 6-63,imaging_vello/src/scene_sink.rslines 15-65; crates.io dependency endpoints.masonry_imaging(main, unpublished): “owns the bridge between Masonry paint output and concrete imaging backends”, exposesvello,vello_hybrid,vello_cpu(headless only) andskiamodules and “host-neutral texture rendering helpers for writing into caller-provided WGPU targets”. Locator:masonry_imaging/src/lib.rslines 7-26, 50-59.
Harness on main
TestHarnessholdsrenderer: Option<VelloCpuRenderer>fromimaging_vello_cpu, created lazily asVelloCpuRenderer::new(1, 1);render() -> RgbaImage;redraw() -> (VisualLayerPlan, TreeUpdate). Locator:masonry_testing/src/harness.rslines 30, 150, 532, 563, 576, b81d8d7.access_nodewrites a rawu64into anaccesskit_consumer::NodeIdunder#[expect(unsafe_code)], citing AccessKit issue 701. AccessKit issue 701 (“ConsumerNodeIdpublic API”) was closed 2026-04-13;accesskit_consumer0.36.0 (2026-05-11) added “Allow looking up nodes by LocalNodeId and TreeId (#707)”. Masonry main pinsaccesskit_consumer = "0.35.0", so theunsafeblock persists there. Locator:harness.rslines 592-604;gh api repos/AccessKit/accesskit/issues/701;accesskit_consumer/CHANGELOG.mdsection 0.36.0.- Tests at b81d8d7: 252
#[test]attributes acrossmasonry/src,masonry_core/srcandmasonry/tests; 207 reference PNG files undermasonry/screenshots. Locator: shallow clone,grep -rh "#\[test\]" | wc -l,ls masonry/screenshots | wc -l.
Widget inventory
- v0.4.0
masonry/src/widgets/: align, button, checkbox, flex, grid, image, indexed_stack, label, portal, progress_bar, prose, scroll_bar, sized_box, slider, spinner, split, text_area, text_input, variable_label, virtual_scroll, zstack (21 modules). Main adds badge, badged, canvas, collapse_panel, disclosure_button, divider, pagination, passthrough, radio_button, radio_group, resize_observer, selector, selector_item, step_input, svg, switch (37 modules). Locator:gh api repos/linebender/xilem/contents/masonry/src/widgetsat v0.4.0 and main. - Tracking issue #1710 (2026-03-31) lists every widget above as available in Masonry; Xilem views are missing for Align, Pagination, Selector and StepInput and marked uncertain for DisclosureButton, Passthrough, ScrollBar, SelectorItem and TextArea. Locator: issue body table.
Splitbuilder:split_axis,split_fraction,split_point(SplitPoint::{Fraction, FromStart, FromEnd}),min_lengths,bar_thickness,min_bar_area,draggable,solid_bar; a drag testdrag_moves_split_pointexists. Locator:masonry/src/widgets/split.rslines 21-175, 852.- Layers:
LayerStackis “the top-level stack of visible layers owned by RenderRoot”; “Other layers can represent tooltips, menus, dialogs, etc.”; a tooltip layer exists inmasonry/src/layers/tooltip.rs. No menu, tab strip, tree view, colour picker or drag-and-drop facility was found by name (grep forDragAndDrop,ContextMenu,Popup,Tooltipovermasonry/src/widgetsandmasonry_core/src/corereturns onlySplit::draggable, slider drag and window drag helpers). Locator:masonry_core/src/app/layer_stack.rslines 17-23;masonry/src/layers/. - Text:
TextInput“does not support newlines entered by the user, although pre-existing newlines are handled correctly” and wraps aTextArea; IME events are modelled asIme::{Enabled, Disabled, Preedit(text, span), Commit}on the core event type. Locator:masonry/src/widgets/text_input.rslines 22-33;masonry_core/src/core/events.rslines 189-222.
Open issues relevant to a canvas-heavy editor (linebender/xilem, open on 2026-08-29; 105 open issues in total)
- #388 (2024-06-12) first-class text editing widget; #266 (2024-05-05) caret movement and editing actions; #1417 (2025-10-14) undo and redo in text inputs; #1341 (2025-08-15) most text widgets lack tests; #1562 (2026-01-08) high CPU when focusing a text box on Linux; #1581 (2026-01-16)
Splitbar area does not receive exclusive pointer events; #918 (2025-04-04) memory usage; #685 (2024-10-17) safety rails for widgets with many children; #1264 (2025-08-03) scale factor tracking; #1451 (2025-11-05) wasmAtomics.wait. Locator:gh api search/issuesqueriesime,text input,split,performance large,panic.
Dependency resolution probe
- A scratch manifest (edition 2024, cargo 1.97.1) depending on
masonry,masonry_testing,masonry_winitat git rev b81d8d7 plusvello 0.10,vello_cpu 0.2,parley 0.11,taffy 0.14,harfrust 0.13,accesskit 0.25,accesskit_consumer 0.39,proptest 1.11,libtest-mimic 0.8,ciborium 0.2,insta 1.48resolves to 549 packages. Duplicates:vello{0.8.0, 0.10.0};vello_cpuandvello_common{0.0.7, 0.2.0};wgpu{28.0.0, 29.0.4};wgpu-core{28.0.1, 29.0.4};accesskit{0.24.1, 0.25.0};accesskit_consumer{0.35.0, 0.36.0, 0.38.0, 0.39.0};parleyandfontique{0.8.0, 0.11.1};skrifa{0.40.0, 0.44.0};harfrust{0.5.2, 0.12.0, 0.13.3}. Single versions:peniko0.6.1,kurbo0.13.1,winit0.30.13,taffy0.14.0,imaging0.0.1,ui-events0.3.0. Highestrust-versionin the graph: 1.96 (Masonry crates,tree_arena,linebender_include_doc_path), then 1.92 (imagingcrates). Locator: scratchcargo metadata --format-version 1, 2026-08-29.vello0.10.0 depends onwgpu ^29.0.3(crates.io dependency endpoint).
Alternatives re-checked
- egui 0.36.1 (2026-08-07, MSRV 1.95, MIT OR Apache-2.0).
egui_wgpu::CallbackTrait::paint(&self, info: PaintCallbackInfo, render_pass: &mut RenderPass<'static>, callback_resources: &CallbackResources)issues “draw commands into the same wgpu::RenderPass that is used for all other egui elements”;prepare(&self, device, queue, screen_descriptor, egui_encoder, callback_resources) -> Vec<CommandBuffer>runs before that pass andfinish_prepareafter allpreparecalls. Locator: docs.rs egui-wgpuCallbackTrait. A Vello scene therefore cannot be drawn inpaint(Vello requires compute passes); it would be rendered to a texture inprepareand sampled inpaint(interpretation; no retrieved example demonstrates it). egui repository issue #8411 (closed 2026-08-11) mentions updatingvello_cpu; the use site was not examined (unverified). - Floem: crates.io 0.2.0 (2024-11-14, MSRV 1.80); main manifest 0.2.0 with
rust-version = "1.91",license = "MIT",parley 0.7.0,taffy 0.9.2, optionalvellofeature throughfloem_vello_renderer; accessibility issues #8 (2023-04-14) and #973 (2025-11-11) remain open. Locator: floemCargo.tomllines 29-150;gh api search/issues q="repo:lapce/floem accesskit". - Blitz main 0.3.0-beta.2 (
rust-version = "1.91.0",MIT OR Apache-2.0,anyrender 0.13.0,anyrender_vello 0.14.0,anyrender_vello_cpu 0.17.0,parley 0.11.1,taffy 0.14.0,accesskit 0.24): a<canvas src="<u64>">element is queued asSpecialOp::LoadCustomPaintSourceand stored asSpecialElementData::Canvas(CanvasData { custom_paint_source_id }).blitz-test-harnessis a path crate not published on crates.io. TheCustomPaintSourcetrait definition in the anyrender repository was not located (unverified). Locator: blitzCargo.tomllines 34-137;packages/blitz-dom/src/mutator.rslines 39, 947-949, 1219-1230; crates.io lookupblitz-test-harness(not found). - GPUI: crates.io 0.2.2 (2025-10-22, Apache-2.0, no
rust_version); README: “pre-1.0. There will often be breaking changes between versions. You’ll also need to use the latest version of stable Rust”; Zedrust-toolchain.tomlpins 1.97.1. Main hascrates/gpui/src/window/a11y.rs: “Every frame, we build a TreeUpdate and send it to the platform-specific adapter” with node IDs derived fromGlobalElementId;crates/gpui/src/app/test_context.rscontains noaccesskitora11ysymbol (grep, 0 matches). Locator:crates/gpui/README.mdline 8;a11y.rslines 1-60;test_context.rs.
Licence facts
- Masonry
Cargo.tomlinheritslicense.workspace = truewith workspacelicense = "Apache-2.0";masonry/LICENSEis the Apache License Version 2.0 text; the repository hasLICENSEandAUTHORSand noNOTICEfile.tree_arena0.2.0,accesskit_winit0.34.0 andwinit0.30.13 are Apache-2.0 only;accesskitandaccesskit_consumerare MIT OR Apache-2.0;vello,vello_cpu,parley,fontique,peniko,kurbo,imaging,ui-events,anymore,understory_virtual_list,resvgare Apache-2.0 OR MIT. Locator:masonry/Cargo.tomllines 1-10; repository root listing; crates.iolicensefields.
Mechanism
Integration paths available on Masonry main (interpretation, each element cited above):
interactive path
nuif-render RenderScene --lowering--> imaging::Painter / PaintSink (nuif "imaging" render delegate)
| Canvas::update_scene(record::Scene)
v
Masonry RenderRoot --redraw()--> (VisualLayerPlan, accesskit::TreeUpdate)
| imaging_vello (vello 0.8, wgpu 28) on screen
| imaging_vello_cpu (vello_cpu 0.0.7) in masonry_testing
headless and snapshot path
nuif-render CPU reference --> pixmap bytes --> Painter::draw_image (no shared crate version)
Type-crossing rule derived from the probe: a type that crosses the widget boundary must be a single version. accesskit::TreeUpdate and ActionRequest cross between Masonry and the NUIF harness, so the editor harness uses Masonry’s AccessKit line (0.24 at b81d8d7), not 0.25. imaging types cross through Canvas, so the NUIF imaging delegate uses Masonry’s imaging line (0.0.1). vello::Scene never crosses because main offers no entry point; NUIF’s own Vello backend (0.10, wgpu 29) is therefore a feature that the editor build disables, and the duplicate copies of Vello, wgpu and Parley in the probe disappear from the editor binary. Pixel buffers cross as bytes, so the CPU reference path keeps its own vello_cpu version.
Xilem’s position: xilem diffs a view tree into Masonry widget mutations; its view coverage lags the widget set (issue #1710) and the editor needs explicit WidgetId to EntityId maps for the accessibility surface. Building on Masonry directly removes one layer whose identity allocation is internal to the framework.
Churn model: between v0.4.0 and b81d8d7 the Widget trait’s paint signature, the renderer abstraction, the layout system and the dependency set changed, with no changelog; the release interval was 172 days (0.3.0 to 0.4.0) and 304 days from 0.4.0 to the retrieval date without a release. A git-revision pin with a single “toolkit bump” commit per update is the only reproducible way to consume main under --locked.
NUIF relevance
Borrow
- Masonry main at a pinned git revision as the editor shell, because it is the only candidate that produces a Vello-compatible scene plan and an AccessKit
TreeUpdatefrom oneredraw()and rasterizes headlessly throughimaging_vello_cpu. Canvas::update_scenewith animaging::record::Sceneas the canvas contract, because it is the sanctioned custom-drawing widget and its recorded scene can be validated and replayed by the harness.Split(draggable,min_lengths),CollapsePanel,PortalwithScrollBar,VirtualScroll,Selector,StepInput,TextInput,Switch,RadioGroup,Flex,Grid,ZStack,Alignas the widget basis for UI-SPEC regions A to E and the Design sections, because each exists on main and is listed as available in issue #1710.- The
LayerStackand tooltip layer for the floating toolbar, command palette and dialogs, because the layer model is documented for “tooltips, menus, dialogs”.
Adapt
- ADR 0006’s “Vello (interactive rendering)” becomes “imaging with the Vello backend”:
nuif-rendergains animagingrender delegate that lowersRenderSceneintoPaintercalls; the standalone Vello backend remains for the CLI and for tier 3 experiments and is disabled in the editor build. - ADR 0006 gating decision 4 (versions pinned together) is extended: the editor crate declares
accesskit,accesskit_consumer,imaging,parleyandkurboat Masonry’s versions and no other NUIF crate depends on them; the harness comparator wrapsmasonry_testing::TestHarnessbehind the NUIF session-driver trait. - Widgets absent on main are composed in the editor crate: layers tree from
VirtualScroll,DisclosureButtonandFlexwith reparenting implemented as protocolMoveoperations triggered by pointer events on the canvas widget rather than a toolkit drag-and-drop protocol; tool group menus and the command palette from layers plusTextInputandVirtualScroll; colour and token controls fromStepInput,Slider,SelectorandCanvas; keyboard bindings from a shortcut table in the application, since Masonry has none. - Shell screenshot tests use tier 2 tolerances (per-channel delta of at most 1) rather than Masonry’s exact comparator, because the harness rasterizes with
vello_cpu0.0.7 throughimaging_vello_cpuwhile the NUIF reference path is separate.
Reject
- Masonry 0.4.0 for the editor, because it lacks
Canvas, paints into avello::Sceneof Vello 0.6, screenshots only through wgpu and pins AccessKit 0.21; none of the harness requirements inconformance/HARNESS.mdis met by that release. - Xilem for the reference editor at this stage, because its view coverage lags Masonry and the editor’s state is NUIF state addressed by entity identity; the option stays open for demonstrations.
- egui as a Vello host, because
CallbackTrait::paintis confined to egui’s render pass and the text and vector stack would be duplicated, as the earlier record concluded. - Blitz as the editor shell, because the harness crate is unpublished, the custom paint trait could not be verified, and the UI-SPEC panels would be authored in HTML and CSS against a beta document engine.
- GPUI, because tree queries are absent from its test contexts and its README requires the latest stable toolchain.
Open questions
- Whether a Masonry 0.5.0 release with the
imagingpaint model, and an MSRV at or below the NUIF policy value, will be tagged before the editor crate is added; no milestone or announcement exists as of 2026-08-29. - Whether
imagingwill offer an ingestion path for an existingvello::Scene, which would allow the standalone NUIF Vello backend to feed the canvas without a second lowering. - Whether the
Splitpointer exclusivity defect (#1581) affects the resizable panels B and D in practice; a fixture in the editor harness is required. - Whether Masonry will move to
accesskit_consumer0.36 or later, removing theunsafenode lookup inmasonry_testing. - Whether Xilem’s
Canvasview and its state diffing can be adopted later without changing the harness contract.
Masonry, Xilem and the masonry_testing TestHarness
Document status:
reviewed. Canonical source.
Summary
Masonry is a retained-mode widget-tree manager: a RenderRoot owns the tree, runs rewrite passes (layout, compose, accessibility), and emits a per-redraw VisualLayerPlan (Vello scenes) and an AccessKit TreeUpdate. Xilem is a reactive view layer that diffs view trees into Masonry widget mutations. The masonry_testing crate provides TestHarness, a windowless host that injects pointer, text, window and accessibility events, advances a virtual clock, exposes the widget tree and the accesskit_consumer::Tree, rasterises frames and compares them against PNG references with assert_render_snapshot!. At v0.4.0 the harness rasterises with Vello on wgpu; on main it rasterises with imaging_vello_cpu (no GPU), and the workspace has moved to an imaging abstraction with Vello, Vello Hybrid, Vello CPU and Skia backends.
NUIF interpretation: Masonry’s architecture (owned tree, centralised focus/pointer/accessibility, explicit passes, harness that reads both the scene plan and the accessibility tree) is the closest structural match to the editor described in ARCHITECTURE.md, and its CPU-render snapshot path matches ADR 0003. The costs are API churn (0.4.0 versus main differ in renderer, dependency versions and MSRV), Apache-2.0-only licensing, and an MSRV (1.88 at v0.4.0, 1.96 on main) above the NUIF pin.
Evidence
- Published versions (crates.io, 2026-08-29): masonry, masonry_core, masonry_testing, masonry_winit, xilem, xilem_core all 0.4.0 (2025-10-29); GitHub release
v0.4.02025-10-29. Locator: crates.io API;gh api repos/linebender/xilem/releases/latest. - v0.4.0 workspace:
edition = "2024",rust-version = "1.88",license = "Apache-2.0",vello = "0.6.0",parley = "0.6.0"(featureaccesskit),winit = "0.30.12",accesskit = "0.21.1",accesskit_winit = "0.29.2",accesskit_consumer = "0.31.0". Locator:Cargo.tomllines 32-72 at tag v0.4.0. - main workspace (commit b81d8d7):
rust-version = "1.96", new membermasonry_imaging,imaging = "0.0.1"withimaging_wgpu(wgpu-28),imaging_skia,imaging_vello,imaging_vello_hybrid,imaging_vello_cpu;vello = "0.8.0",wgpu = "28.0.0",parley = "0.8.0",accesskit = "0.24.0",accesskit_consumer = "0.35.0". Locator:Cargo.tomllines 9, 40, 48-91, main. - Masonry README: “Masonry gives you a platform-independent manager, which owns and maintains a widget tree”; built on Imaging (default Vello and wgpu), Parley, AccessKit; “not opinionated about what your user-facing abstraction will be”; backends
masonry_winitandmasonry_android_view. Locator:masonry/README.md, cargo-rdme section, main. - masonry_testing crate docs: the harness can “Simulate any external event which Masonry handles”, “Control the flow of time”, “Take screenshots”; screenshots are compared against
screenshots/<name>.pngadjacent toCargo.toml;MASONRY_TEST_BLESS=1updates references; files are losslessly compressed with Oxipng under a size limit; backend featuresimaging_vello,imaging_vello_hybrid,imaging_vello_cpu,imaging_skia. Locator:masonry_testing/src/lib.rslines 11-58, main. - main
masonry_testing/Cargo.tomldepends onaccesskit_consumer,image(png),imaging_vello_cpu,masonry_core,oxipng 9.1.5; v0.4.0 instead depends onfutures-intrusive,pollsterand usesmasonry_core::vello::util::{RenderContext, block_on_wgpu}. Locator:masonry_testing/Cargo.tomllines 21-27 (main) and 21-30 (v0.4.0);harness.rslines 35-36 (v0.4.0). TestHarnessfields includerender_root: RenderRoot,access_tree: accesskit_consumer::Tree,renderer: Option<VelloCpuRenderer>,action_queue,clipboard,title.TestHarnessParams { window_size, background_color, root_padding, scale_factor, panic_on_rewrite_saturation, max_screenshot_size },DEFAULT_SIZE = 400x400, default max screenshot 8 KiB. Locator:masonry_testing/src/harness.rslines 146-262, main.- Public harness methods (main):
create,create_with_size,create_with,process_window_event,process_pointer_event,process_text_event,process_access_event(ActionRequest),render() -> RgbaImage,redraw() -> (VisualLayerPlan, TreeUpdate),access_tree(),access_node(WidgetId),mouse_move,mouse_button_press,mouse_button_release,mouse_wheel,mouse_click_on(id, button),mouse_move_to,scroll_into_view,accessibility_click_on(id),keyboard_type_chars,press_tab_key,focus_on,set_focus_fallback,animate_ms,set_disabled,root_widget,get_widget_with_id,get_widget,take_records_of,inspect_widgets,edit_root_widget,edit_widget,pop_action,cursor_icon,has_ime_session,ime_rect,clipboard_contents,window_size,title,save_render_snapshot,check_render_snapshot. Locator:harness.rslines 342-1100, main. At v0.4.0mouse_click_on(id)takes no button argument (line 641). assert_render_snapshot!(harness, name)expands tocheck_render_snapshot(env!("CARGO_MANIFEST_DIR"), name, false); a missing reference writes<name>.new.pngand panics; a mismatch writes<name>.diff.png;SKIP_RENDER_TESTSskips comparison but still runs the paint pass. Locator:harness.rslines 205-247, 1100-1180, main.- Image comparison is exact:
get_image_diffreturnsNoneonly when the maximum per-channel distance is 0 and sizes match. Locator:masonry_testing/src/screenshots.rslines 22-36, main. redraw()callsrender_root.redraw()andaccess_tree.update_and_process_changes(tree_update, &mut NoOpTreeChangeHandler).render()composes theVisualLayerPlanlayers into one VelloScenewith padding and callsVelloCpuRenderer::render_source(&mut scene, width, height). Locator:harness.rslines 532-584, main.access_nodecontains anunsafeNodeId conversion citing AccessKit issue #701 (“No public API exists for modifying/creating a accesskit_consumer::NodeId”). Locator:harness.rslines 591-608, main.- Helper widgets:
Recorder/Recording/Recordcapture widget method calls viaTestWidgetExt::record;ModularWidget,WrapperWidget; assertionsassert_any,assert_all,assert_none,assert_debug_panics_inner. Locator:masonry_testing/src/lib.rslines 60-80. - Vello 0.10.0 (2026-08-14), vello_cpu 0.2.0 and vello_hybrid 0.2.0 (2026-08-07), parley 0.11.1 (2026-08-16) on crates.io; Vello main README states verification “with Rust 1.88 and later” and that MSRV bumps are not breaking changes. Locator: crates.io API;
vello/README.md“Minimum supported Rust Version”. - Parley’s stack is Fontique (enumeration/fallback), HarfRust (shaping), Skrifa (glyph outlines/metrics), ICU4X. Locator:
parley/README.md“The Parley text stack”. - Vello CPU “is a 2D graphics rendering engine written in Rust, for devices with no or underpowered GPUs” with
RenderContext,Pixmap,RenderMode::OptimizeSpeed/OptimizeQuality. Locator:sparse_strips/vello_cpu/README.md, main.
Mechanism
Ownership and passes: RenderRoot owns the widget arena; external events enter through process_window_event, process_pointer_event, process_text_event, process_access_event. Each returns Handled and schedules rewrite passes; panic_on_rewrite_saturation turns pass loops into test failures. redraw() runs the remaining passes and returns both outputs of a frame:
#![allow(unused)]
fn main() {
let (visual_layers, tree_update) = harness.redraw(); // VisualLayerPlan, accesskit::TreeUpdate
let tree: &accesskit_consumer::Tree = harness.access_tree();
let node = harness.access_node(widget_id).unwrap(); // accesskit_consumer::Node
assert_eq!(node.role(), accesskit::Role::Button);
harness.accessibility_click_on(widget_id); // ActionRequest { action: Click, .. }
let (action, source) = harness.pop_action::<ButtonPress>().unwrap();
assert_render_snapshot!(harness, "button_pressed"); // screenshots/button_pressed.png
}
Invariants observable from the source: the accessibility tree is rebuilt from the same frame that produces the scene, so semantic assertions and pixel assertions refer to one state; time is virtual (animate_ms), so animations converge deterministically; screenshot references are exact-match PNGs compressed by oxipng preset 5 with an 8 KiB default ceiling that forces small, focused images.
Renderer dependence: at v0.4.0 the harness allocates a wgpu RenderContext and blocks on the GPU (block_on_wgpu), so screenshot tests require a wgpu adapter in CI; on main the harness allocates VelloCpuRenderer::new(1, 1) lazily and needs no GPU. The imaging abstraction lets the application choose imaging_vello, imaging_vello_hybrid, imaging_vello_cpu or imaging_skia per build.
Xilem layer: views are values; xilem_core diffs view trees and applies WidgetMut edits to Masonry; the harness tests Masonry widgets directly and Xilem apps through their root widget (the crate docs point to “the tests in Masonry’s examples”).
NUIF relevance
Borrow
- The two-output frame (
VisualLayerPlan,TreeUpdate) as the harness contract, because NUIF’sEngine::build_render_sceneplus a semantic tree can be exposed the same way and asserted in one test. TestHarnessParams(fixed window size, scale factor, background, root padding) as explicit evaluation context, because QA item 4 requires layout “at explicit contexts” and conformance/PLAN.md requires fixture ID and evaluation context in every result.- The bless protocol (
MASONRY_TEST_BLESS=1,.new.png,.diff.png, size ceiling, lossless compression), because it constrains repository growth while keeping exact references. vello_cpuas the deterministic rasteriser for screenshot conformance, because ADR 0003 asks for a CPU/reference backend andimaging_vello_cpudemonstrates it inside a harness.
Adapt
access_node(WidgetId)andaccessibility_click_on(WidgetId)address Masonry widget IDs; NUIF needs the same operations keyed byEntityId, so a mapping from entity to widget/accessibility node must be maintained by the editor shell.- The exact-pixel comparator is appropriate for widget chrome; NUIF render conformance declares tolerances (conformance/PLAN.md), so a thresholded comparator must wrap or replace
get_image_diff. - The
Recorderwidget pattern can become a NUIF operation recorder that captures protocolOperations issued by the shell, satisfying QA item 3 (inverse/replay logs).
Reject
- Treating any Masonry, Vello or Parley type as NUIF state, because ADR 0003 states “No Vello internal data type is normative NUIF state”.
- Building against
main(rust-version 1.96, unpublishedimaging 0.0.1) for a reference implementation, because the NUIF pin is 1.85.0 and unpublished dependencies break reproducible--lockedbuilds. - Relying on the v0.4.0 GPU screenshot path in CI, because it requires a wgpu adapter and reintroduces the cross-machine nondeterminism egui’s README catalogues.
Open questions
- When the
imagingabstraction andmasonry_testingCPU path will ship in a tagged release, and what MSRV that release will carry relative to NUIF’s pin. - Whether AccessKit issue #701 (public
NodeIdconstruction inaccesskit_consumer) has been resolved in accesskit_consumer 0.39, which would remove theunsafeblock inaccess_node. - Whether Masonry’s layout passes can host NUIF’s
LayoutSnapshot(authored to resolved boxes computed by nuif-layout) without a second layout, or whether the canvas must be a single custom widget that paintsRenderScenedirectly. - Whether Apache-2.0-only licensing of the toolkit is acceptable for the reference editor given the workspace’s
Apache-2.0 OR MITpolicy.
MaterialX shader generation and render test suite across backends
Document status:
reviewed. Canonical source.
Summary
MaterialX validates its node graph standard by generating shader code for each renderable element in a document corpus (resources/Materials/TestSuite, organized by library: stdlib, pbrlib, nprlib, bxdf) with every registered code generator (genglsl, genosl, genoslnetwork, genmdl, genessl, genmsl, genslang) and then compiling and, where a backend is available, rendering the generated code. The MaterialXTest executable (Catch-based, driven by ctest or test tags such as [genglsl], [renderglsl], [renderosl], [rendermsl]) reads _options.mtlx, a MaterialX document that parametrizes targets, light rigs, render size, geometry, IBL paths, render test paths and exclusions. Outputs are generated source, per-language logs (generation, implementation coverage, document validation, profiling, render) and rendered images.
There are no reference images and no automated pixel tolerance gate in the core render tests: pass/fail is compile-and-render success plus implementation-count checks. Cross-backend agreement is assessed by a separate report generator (python/MaterialXTest/tests_to_html.py) that lays out images from two or three targets side by side and, if Pillow is installed, computes per-pair RMS difference images with an optional RMS filter; CI on the extended macOS build produces an HTML/PDF “MaterialX_RenderComparison” artifact comparing msl and osl. Viewer and graph-editor screen captures are also produced in CI as artifacts without comparison.
Evidence
- Test categories and tags: core (
Document.cpp,Element.cpp, …), I/O (XmlIo.cpp), shader generation ([genshader],[genglsl],[genosl],[genmdl],[genmsl]), render ([rendercore],[renderglsl],[renderosl],[rendermsl]) —source/MaterialXTest/README.mdsections 1–3.3 (main, retrieved 2026-08-29). - Render setup enabled by
MATERIALX_TEST_RENDER; OSL requiresMATERIALX_OSL_BINARY_OSLC,MATERIALX_OSL_BINARY_TESTRENDER,MATERIALX_OSL_INCLUDE_PATH; MDL requiresMATERIALX_MDL_SDK_DIR; MSL 2.0+ on macOS — same README, “Per-Language Render Setup”. - Output logs
gen<language>_<target>_generatetest.txt,_implementation_check.txt,_render_doc_validation_log.txt,_render_profiling_log.txt,_render_log.txt; the render log references per-material error files — same README, “Test Outputs”. _options.mtlxTestSuiteOptionsnodedef:overrideFiles,lightFiles(light_rig_test_2.mtlx),targets(genglsl,genosl,genoslnetwork,genmdl,genessl,genmsl,genslang),checkImplCount,shaderInterfaces(1 reduced, 2 complete, 3 both),renderSize(512x512),dumpUniformsAndAttributes,dumpGeneratedCode,renderGeometry(sphere.obj),enableDirectLighting,enableIndirectLighting,radianceIBLPath,irradianceIBLPath,renderTestPaths,renderTestExcludeFiles,outputDirectory,enableTracing(Perfetto) —resources/Materials/TestSuite/_options.mtlx(main).- Test suite corpus layout by library and element category (e.g.
stdlib/math/{math,math_operators,transform,trig,vector_math}.mtlx),Geometry,Images,Utilities(testrender utilities and light configuration); “each file is parsed to determine renderable elements” and code is “compiled, and/or rendered” —resources/Materials/TestSuite/README.md. ShaderRenderTesterbase class withvalidate(optionsFilePath),runTest(TestSuiteOptions),loadOptions, virtualsaveImagedefaulting tofalse;RenderProfileResult { elementsTested, success };TestRunLogger,TestRunProfiler,TestRunTracer—source/MaterialXTest/MaterialXRender/RenderUtil.hlines 42–301.- Render test subdirectories per backend:
MaterialXRenderGlsl,MaterialXRenderOsl,MaterialXRenderMsl,MaterialXRenderMdl,MaterialXRenderSlang; generatorsMaterialXGenGlsl,MaterialXGenOsl,MaterialXGenMdl,MaterialXGenMsl,MaterialXGenSlang—source/MaterialXTest/listing (retrieved 2026-08-29). - CI:
ctest --output-on-failure, Python tests (MaterialXTest/main.py,genshader.py,mxspec.py compareagainst specification markdown,mxformat.py --upgrade), shader validation viagenerateshader.py --target msl --validator "xcrun metal ...", render captures withMaterialXView --captureFilenameandMaterialXGraphEditor --captureFilenameuploaded as artifacts —.github/workflows/main.ymllines 258–350. - Render comparison report step (extended macOS build with OSL):
tests_to_html.py -i1 Materials -l1 msl -l2 osl -d --order-from _options.mtlx -o MaterialX_RenderComparison.html, optionally printed to PDF with headless Chrome —.github/workflows/main.ymllines 351–370. tests_to_html.py: “Install pillow via pip to enable image differencing and statistics”;computeDiffusesImageChops.differenceand returnssum(diffStat.rms) / (3.0 * 255.0); options-l1/-l2/-l3target languages,-d/--diff,-e/--error“Filter out results with RMS less than this”,-ttimestamps —python/MaterialXTest/tests_to_html.pylines 9–81, 191–202.mxspec.py comparechecks that specification markdown node tables and*_defs.mtlxagree for stdlib, pbrlib and nprlib —.github/workflows/main.ymllines 267–269.- Latest release v1.39.5 (2026-05-22), license Apache-2.0 (GitHub releases API and repository, retrieved 2026-08-29).
Mechanism
The suite is a differential test over code generators with a shared oracle of “compiles and renders”. For each document under the configured paths, MaterialX enumerates renderable elements (materials, node graph outputs), instantiates each generator named in targets, generates source under one or more shader-interface modes, and records whether every node used has an implementation for that target (checkImplCount). Where a backend toolchain exists, the source is compiled (GLSL via OpenGL, OSL via oslc and testrender, MDL via the SDK, MSL via Metal, Slang) and rendered into a fixed camera/geometry/light configuration to a fixed size; the image is written next to the source material or into outputDirectory. Failures are compile or render errors, missing implementations, or document validation errors; timings are logged for profiling. Options live in a MaterialX document, so the test harness is configured in the format under test.
Agreement across backends is reviewed, not asserted. tests_to_html.py walks the output tree, pairs images by material and target language, computes normalized RMS of the absolute difference when Pillow is available, renders difference images, and emits an HTML table ordered by the renderTestPaths list; -e filters pairs below an RMS threshold to surface only divergent materials. CI publishes this report as an artifact for human inspection. Separately, mxspec.py performs a structural check that specification tables and library definitions do not drift, which is the only automated oracle relating the normative text to the implementation.
NUIF relevance
Borrow
- Configure the conformance harness with a NUIF document (the
_options.mtlxpattern) so evaluation contexts, backends and fixture paths are expressed in the format under test. - Run every fixture through every lowering/backend pair and log implementation coverage per construct (
checkImplCount) so unsupported constructs are enumerated rather than discovered. - Generate a cross-backend comparison report with normalized RMS and difference images, ordered by fixture list and filterable by threshold, as a review artifact distinct from pass/fail gates.
- Check normative text against machine-readable definitions (
mxspec.py compare) to keep spec tables and schema files synchronized. - Emit per-backend structured logs (generation, coverage, validation, profiling, render) with per-fixture error files referenced from the summary log.
Adapt
- MaterialX has no golden images; NUIF must combine MaterialX’s cross-backend differential report with Hydra-style thresholded golden comparisons for the reference renderer, since UI rendering has normative pixel expectations for geometry and text placement.
- RMS over the whole image is insensitive to small localized errors typical of text and hairline rendering; NUIF should add region- or glyph-level metrics.
- Human-reviewed HTML reports are adequate for shading research but not for a headless QA contract; NUIF reports must be machine-readable first with HTML as a projection.
Reject
- Treating “compiles and renders” as the pass criterion for a render suite; NUIF’s render suite must assert declared tolerances.
- Optional dependency (Pillow) gating whether comparison statistics exist; NUIF comparison must be a required part of the reference implementation.
- Platform-specific backends (Metal, OSL toolchain) as prerequisites; NUIF’s CPU reference backend must run everywhere.
Open questions
- Whether a normalized RMS threshold has any defensible meaning for UI fixtures, or whether per-element structural comparisons should replace image metrics entirely except for the final rasterization suite.
- How to keep a cross-backend comparison meaningful when backends legitimately differ (font hinting, anti-aliasing) without masking real defects.
- Whether NUIF should adopt a “future updates” style option (
applyFutureUpdates-like) to test fixtures under pending dialect versions before ratification.
MaterialX open graph standard for cross-renderer content
Document status:
reviewed. Canonical source.
Summary
MaterialX is a platform-independent open standard for graph-based material/look content across authoring applications and renderers, with standard nodes plus extensible node definitions.
NUIF relevance
It is useful precedent for a graph vocabulary that stays independent of one renderer while supporting domain extensions and multiple implementations.
Stateless MCP agent adapter over the authoritative NUIF core
Document status:
verified. Canonical source.
Summary
MCP 2026-07-28 replaced its protocol handshake and hidden session with a
stateless core. Each request carries protocol and client metadata, optional
server/discover reports server support, and application state must be an
explicit value or handle. Tool schemas use JSON Schema 2020-12 and structured
tool results remain machine-readable. Roots, sampling and protocol logging are
deprecated; a stdio server logs only on stderr and reserves stdout for MCP
messages.
That architecture matches NUIF only when MCP remains a process adapter. The
canonical model, validation, hashing and semantic patch behavior stay in the
Rust core. nuif-mcp-tools-0 therefore exposes four pure tools over inline
canonical text: validate, inspect, canonicalize and apply-patch. Applying a
patch transforms a supplied value and returns a new canonical value; it does
not mutate a host file or keep a hidden document session. The first profile has
no resources, prompts, tasks, sampling, roots, HTTP, OAuth, filesystem,
network, host-document or credential authority.
The implementation uses the official rmcp 3.1.4 server and its generated
schemas rather than reproducing a newly changed protocol by hand. It pins the
only supported protocol revision to 2026-07-28 and uses a current-thread Tokio
runtime. A NUIF-owned reader limits a JSON line before the SDK’s stdio
transport can accumulate it; document, patch, transaction and operation limits
then apply at progressively more semantic layers.
Evidence
- The final 2026-07-28 announcement removes
initialize/initializedandMcp-Session-Id, requires request metadata for the stateless lifecycle, and introduces optionalserver/discover. It recommends explicit application handles rather than transport-hidden state. Locator: The 2026-07-28 Specification, “No handshake or sessions”, retrieved 2026-08-30: https://blog.modelcontextprotocol.io/posts/2026-07-28/. - The same announcement marks roots, sampling and logging deprecated. The release-candidate detail identifies stderr as the stdio logging replacement and records full JSON Schema 2020-12 for tool input and output schemas. Locator: “Roots, Sampling, and Logging Are Deprecated” and “Full JSON Schema 2020-12 for Tools”, retrieved 2026-08-30: https://blog.modelcontextprotocol.io/posts/2026-07-28-release-candidate/.
- The transport contract uses newline-delimited UTF-8 JSON-RPC, requires stdout to contain only protocol messages, and permits logging on stderr. Locator: MCP specification, Transports, “stdio”, retrieved 2026-08-30: https://modelcontextprotocol.io/specification/2025-06-18/basic/transports. The final 2026 revision retained stdio while replacing the HTTP lifecycle.
- The official Rust SDK describes
rmcpas its protocol crate over Tokio. Its current roadmap reports complete dated 2025-11-25 and 2026-07-28 client and server conformance suites, while separately tracking features that those suites do not exercise. Locator:ROADMAP.md, “Conformance” and “Spec features without conformance scenarios”, retrieved 2026-08-30: https://github.com/modelcontextprotocol/rust-sdk/blob/main/ROADMAP.md. rmcp3.1.4 declares Rust 1.88, Apache-2.0, optional protocol features and an officialtransport-ioserver surface. NUIF enables only macros, server and stdio; HTTP, client, OAuth, base64, tasks and provider features are absent. Locator: crates.io package metadata and the locked manifest, retrieved 2026-08-30: https://crates.io/crates/rmcp/3.1.4.rmcp3.1.4’sAsyncRwTransport::receivecallsread_untilinto a reusable vector without applying theJsonRpcMessageCodec::new_with_max_lengthoption. The NUIF wrapper therefore must bound the reader itself before transport parsing. Locator:crates/rmcp/src/transport/async_rw.rsat tagrmcp-v3.1.4,receiveandJsonRpcMessageCodec, retrieved 2026-08-30: https://github.com/modelcontextprotocol/rust-sdk/blob/rmcp-v3.1.4/crates/rmcp/src/transport/async_rw.rs.
Mechanism
The host launches nuif-mcp with piped stdin and stdout. A bounded asynchronous
reader permits at most 4 MiB before a newline and closes the connection on the
first excess byte. The official router verifies the 2026 request envelope,
decodes each tool’s generated object schema and dispatches into a stateless
handler. The handler accepts at most 1 MiB each of inline document and patch
text. It decodes documents through CanonicalText, measures patch cardinality
through nuif-protocol, and delegates mutation to a temporary nuif-api
Session. Successful values use declared output schemas; execution failures
are tool errors with stable NUIF_* prefixes so an agent can correct input.
All tools declare read-only, non-destructive, idempotent and closed-world annotations because they transform caller-owned values without external side effects. These are truthful hints, not an authorization mechanism. A host must still choose whether to launch the process and whether to pass its output into a privileged product API.
The full NUIF document is deliberately not mirrored into MCP types. Typed MCP records describe only arguments and bounded results; canonical document and patch strings cross the boundary. This keeps the protocol shell replaceable and makes direct Rust, CLI, WASM and MCP results comparable by canonical hash.
NUIF relevance
Choose the official Rust SDK over a handwritten JSON-RPC loop. The 2026 revision changed lifecycle, request metadata, result discrimination, schema rules and server-to-client interaction together. Reimplementing that surface would create protocol work unrelated to NUIF semantics.
Choose in-process Rust over a TypeScript or Python SDK sidecar. Those SDKs are valid alternatives for applications already implemented in those languages, but a NUIF sidecar would need a second private RPC to the Rust core, duplicate packaging and add another failure boundary. Rust SDK tier status is therefore a monitored delivery risk, not a reason to duplicate the core.
Choose stdio first over Streamable HTTP. Local development clients already own process launch and authorization, while stdio adds no listener, origin, TLS, OAuth or multi-tenant policy. A future HTTP profile is a distinct security product and must define authentication, authorization, request concurrency, tenant isolation, origin validation, rate limits and deployment observability before code is enabled.
Reject file-path tools in profile zero. They make a model-selected string
an ambient filesystem capability and prevent deterministic cross-surface
tests. Large documents and .nuif packages remain available through the CLI,
WASM or direct API where the host explicitly owns bytes and resources.
Open questions
- Which real MCP hosts correctly consume 2026-07-28 stateless stdio requests, output schemas and tool annotations must be proven by a pinned live-client matrix; SDK conformance alone is not host-product evidence.
- The 1 MiB inline text limit is conservative and must be retained or changed from measured agent workloads, not raised merely to match the 16 MiB codec envelope.
- A resource-handle extension may become useful for large documents, but it requires an explicit host-owned capability grant and lifecycle rather than a server-invented path namespace.
- A remote MCP service remains out of scope until its separate security and operations RFC is accepted.
mdBook static documentation and generated navigation
Document status:
verified. Canonical source.
Summary
mdBook 0.5.4 builds searchable static documentation from Markdown. The
SUMMARY.md file determines inclusion, order, hierarchy and source paths.
Preprocessors can transform Markdown before rendering, and alternate backends
can consume the same book representation.
Evidence
- mdBook describes itself as a command-line Markdown book generator with integrated search, syntax highlighting, themes, preprocessors and backends. Locator: mdBook 0.5.4 introduction, lines 12–26, retrieved 2026-08-30.
- mdBook requires a strictly formatted
SUMMARY.md; without that file there is no book. Locator: SUMMARY.md, lines 13–16, retrieved 2026-08-30: https://rust-lang.github.io/mdBook/format/summary.html. - Preprocessors modify raw Markdown before it reaches the renderer. Locator: Configuring Preprocessors, lines 13–24, retrieved 2026-08-30: https://rust-lang.github.io/mdBook/format/configuration/preprocessors.html.
Mechanism
NUIF generates a temporary mdBook source directory and SUMMARY.md from a
validated documentation catalog. The catalog owns identity, status and
navigation. mdBook owns HTML presentation and client-side search. A renderer
change therefore does not move or duplicate canonical Markdown.
NUIF relevance
Borrow mdBook’s Rust-native static renderer and search implementation.
Adapt its source-root assumption by staging canonical documents under
target/docs-src. The staging transformation removes YAML frontmatter from the
rendered body and rewrites repository-relative links while retaining the source
path in the generated catalog.
Reject a committed SUMMARY.md. It would duplicate the catalog’s inclusion
and ordering state.
Open questions
- Research faceting and backlink presentation may require generated index pages or a later renderer. The catalog boundary permits that change without a content migration.
Metamorphic testing and metamorphic relations for graphics, layout and round-trip oracles
Document status:
reviewed. Canonical source.
Summary
Metamorphic testing (MT) checks necessary relations between the outputs of two or more related executions instead of checking one output against an expected value. It was proposed by Chen, Cheung and Yiu in 1998 as a way to derive new test cases from successful ones when no practical test oracle exists. Segura et al. (2016) surveyed 119 papers and report computer graphics and compilers among the most common application domains. Donaldson et al. (OOPSLA 2017) applied MT to graphics shader compilers: semantics-preserving transformations produce a family of variant shaders that must render the same image; deviations are detected with a tolerant histogram metric and reduced by reversing transformations. The later spirv-fuzz tool records transformations as a protobuf sequence, replays them, and shrinks failing sequences with delta debugging.
For NUIF, MT supplies the oracle for the trial-and-error loop where no reference implementation exists: encode-decode fixpoints, operation-then-inverse identity, translation equivariance of resolved boxes, and semantics-preserving document rewrites. The relation classes below are labelled as source-derived or as NUIF synthesis.
Evidence
- MT was motivated by the oracle problem: the 1998 report states that test oracles are “pragmatically unattainable in most situations”. Chen, Cheung, Yiu, HKUST-CS98-01, Abstract; arXiv 2002.12543, https://arxiv.org/abs/2002.12543, retrieved 2026-08-29.
- The 1998 report derives follow-up test cases from an input-output pair and the errors typically associated with the program; construction and checking must cost strictly less than executing the program. Same report, §2 Preliminaries.
- Canonical shortest-path example: reversing the query (y, x, G) must return the same distance, and splitting at an intermediate vertex must give distances that sum to the original. Same report, §3.3 and Table 3. The 1998 report does not use the phrase “metamorphic relation”; the formal term appears in later work.
- Segura et al. define a metamorphic relation as a relation among a series of inputs x1..xn (n > 1) and their outputs, and distinguish it from an invariant because it relates different executions. Segura, Fraser, Sanchez, Ruiz-Cortés, IEEE TSE 42(9), 2016, DOI 10.1109/TSE.2016.2532875, §2 (author preprint https://personal.us.es/sergiosegura/files/papers/segura16-tse.pdf, retrieved 2026-08-29).
- The survey covers 119 papers from 1998 to 2015; among case-study papers the leading domains are web services (16%), computer graphics (12%), simulation and modelling (12%) and embedded systems (10%). Same preprint, §1 and §5.1.
- The survey lists compilers as a domain, citing equivalence-preservation relations (replacing an expression by an equivalent one) and the EMI work that found 147 confirmed GCC/LLVM bugs. Same preprint, §5.1.10.
- MR construction is described as typically manual, with composition and automatic generation as reviewed alternatives. Same preprint, §4.2.
- The 2018 review gives Definition 1: an MR is a necessary property of f over a sequence of two or more inputs and their outputs, R ⊆ X^n × Y^n; it also states that not all MRs are equality relations. Chen et al., ACM Computing Surveys 51(1), 2018, DOI 10.1145/3143561, §2.2 Definition 1 and §3 Concept 3 (course mirror PDF https://homes.cs.washington.edu/~rjust/courses/CSE503/2021_02_12-reading2.pdf, retrieved 2026-08-29).
- Donaldson et al. state that GLSL is deliberately under-specified (denormal flushing, rounding, optimisation-induced differences), which makes pixel-exact comparison against a reference impossible. Donaldson, Evrard, Lascu, Thomson, PACMPL 1(OOPSLA), Art. 93, 2017, DOI 10.1145/3133917, §2.2 (open PDF https://www.doc.ic.ac.uk/~afd/homepages/papers/pdfs/2017/OOPSLA.pdf, retrieved 2026-08-29).
- The metamorphic form used is p(fI(x)) = fO(p(x)) checked with a tolerant equality; for a semantics-preserving fI, fO is the identity. The paper assumes P is deterministic and notes that MT finds bugs but cannot prove absence. Same paper, §2.3.
- Three phases: variant generation, detection of deviant variants, reduction of deviant variants. Same paper, §3.
- Opaque values come from a uniform
injSwitchset to (0.0, 1.0) at run time, yielding expressions T, F, 0 and 1 the compiler cannot fold. Same paper, §3.1.1. - Six transformation families: dead code injection, dead jump injection, live code injection, expression mutation (e + 0, e * 1, T ? e : d), vectorisation, and control-flow wrapping (single-iteration loops,
if(T){C}). Transformations compose and are “easy to reverse during reduction”. Same paper, §3.1.2 and §3.1.3. - Image comparison uses the chi-squared distance between histograms (OpenCV
compareHist, HSV,HISTCMP_CHISQR) with threshold 100 chosen empirically; pixel-per-pixel comparison is rejected. Same paper, §3.2 and §5.2. - Reduction reverses random subsets of applied transformations until a minimal set is reached; it converges to a local minimum and is described as similar to delta debugging. Same paper, §3.3.
- Results: more than 60 distinct bugs across 17 GPU and driver configurations; §5.4 tabulates 71 logged issues. False positives were judged by three human raters over 975 reductions. Same paper, Abstract, §5.3, §5.4.
- GraphicsFuzz docs describe the pipeline:
glsl-generateproduces shader families from reference and donor shaders; workers render on devices;glsl-reduceshrinks with an interestingness script; default metricHISTOGRAM_CHISQR, threshold 100.0, alternativePSNR; reduction kinds includeABOVE_THRESHOLD,NO_IMAGE,IDENTICAL. https://github.com/google/graphicsfuzz,docs/glsl-fuzz-intro.md,docs/glsl-fuzz-walkthrough.md,docs/glsl-fuzz-reduce.md,docs/glsl-reduce-intro.md, master branch, retrieved 2026-08-29. The repository was archived on 2025-12-08. - spirv-fuzz records each transformation as a protobuf message (
spvtoolsfuzz.proto), writes.transformationsand.transformations_json, and has FUZZ, REPLAY and SHRINK modes;--shrink=<input.transformations> -- <interestingness_test>where the script returns 0 iff the binary is interesting;--replay-range,--shrinker-step-limit,--donors,--force-render-red; the default pass selection is swarm testing. KhronosGroup/SPIRV-Tools,tools/fuzz/fuzz.cppusage text (lines 60–176) anddocs/spirv-fuzz.md, main branch, retrieved 2026-08-29. - Transformation-based testing makes reduction and deduplication cheap: if transformations are small and independent, delta debugging shrinks the transformation subsequence, and the bug is reported as a delta between the original and a minimally transformed program. Donaldson et al., PLDI 2021, DOI 10.1145/3453483.3454092, Abstract, §2.1, §3.4 (open PDF https://www.doc.ic.ac.uk/~afd/papers/2021/PLDI.pdf, retrieved 2026-08-29).
- The PLDI 2021 reducer skips subsequences whose preconditions fail and halves chunk size until no single transformation can be removed; a set of facts (DeadBlock, Synonymous, Irrelevant, LiveSafe) is maintained to justify transformations. Same paper, §2.1, §3.2, §3.4.
- Janus (ICSE 2025) applies a delta-consistency oracle to browsers: for two HTML files differing by a minor modification, two browsers must agree on whether the renderings differ. Only the repository description was readable: https://github.com/ChijinZ/janus-browser-fuzzer, retrieved 2026-08-29; bug counts unverified.
Mechanism
Definitions (from Segura 2016 §2 and Chen 2018 §2.2):
- A metamorphic relation R for a program P relates n ≥ 2 inputs and their outputs: R(x1..xn, P(x1)..P(xn)).
- A metamorphic test case is a source test case plus follow-up test cases derived from it by an input transformation fI; the check is R over the executions.
- R need not be an equality; subset, ordering and tolerant relations are admitted.
Transformation-based MT (from OOPSLA 2017 §3 and PLDI 2021 §2–3):
generate(reference, seed):
T = [] # ordered transformation log
facts = {} # justifications (dead block, synonym, ...)
repeat until budget:
t = choose_transformation(seed, facts) # each t has precondition + effect
if precondition(t, program, facts):
program = apply(t, program); facts = update(facts, t); T.append(t)
return program, T
check(reference, variant):
img_r = run(reference); img_v = run(variant)
return distance(hist(img_r), hist(img_v)) > threshold # tolerant, not pixel-exact
reduce(reference, T, interesting):
# delta debugging over T; program is rebuilt from reference each time
n = 2
while |T| >= 2:
for chunk in split(T, n):
T' = T \ chunk
if replayable(T') and interesting(replay(reference, T')):
T = T'; n = max(n - 1, 2); break
else:
if n >= |T|: break
n = min(2n, |T|)
return T
Invariants of the method:
- Every transformation is semantics-preserving modulo a declared tolerance (floating-point noise in shaders).
- Transformations are recorded, replayable by seed and reversible; the artifact under test is never edited directly, so reduced variants remain valid by construction.
- The oracle is a relation between executions, so no expected image is stored.
Relation classes for NUIF. Each class is tagged as source-derived (S) or NUIF synthesis (N).
- Equivalence-preserving rewrite (S: OOPSLA 2017 §2.3, Segura §5.1.10). Wrapping a subtree in a no-op container, splitting a text run, or renaming entities must leave resolved boxes and rendered images within tolerance.
- Encode→decode→encode fixpoint (N; matches the byte-stability criterion in
conformance/fixtures/v0-responsive-card/README.md). For canonical encoders E and decoders D: E(D(E(d))) = E(d) and canonicalize is idempotent. The relation is exact, not tolerant. - Operation-then-inverse identity (N; reversal of recorded transformations in OOPSLA 2017 §3.3 is the closest source). For a transaction t with inverse t⁻¹ produced per
spec/06-operations-and-patches.md: canon(apply(t⁻¹, apply(t, d))) = canon(d), and the inverse must satisfy its preconditions. - Commutativity of independent operations (S: Chen 1998 §3.3 reversal; N for layout). Operations on disjoint subtrees applied in either order must yield identical canonical hashes.
- Translation and scale equivariance (N; general non-identity fO form from OOPSLA 2017 §2.3). Translating a freeform root by (dx, dy) must translate every resolved box by (dx, dy); scaling the viewport for a fully proportional layout must scale boxes proportionally within tolerance.
- Additivity (S: Chen 1998 Table 3; N for layout). In a stack family, the resolved extent of a container equals the sum of child extents plus gaps and padding.
- Monotone or subset relations (S: Chen 2018 Concept 3; N for documents). Removing an entity never increases the set of drawn commands; adding an opaque extension never changes resolved boxes.
- Round-trip through an adapter (N). For an adapter export X and import Y over the representable subset: canon(Y(X(d))) = canon(d), fidelity report entries must explain every deviation, and opaque extension bytes must be identical.
- Delta consistency across engines (S: Janus repository). For d and d’ differing by one operation, NUIF and a browser must agree on whether resolved boxes changed.
NUIF relevance
Borrow
- The three-phase structure generate variants → detect deviants → reduce by removing transformations maps directly onto NUIF operation logs, which are already ordered, serialisable and invertible (OOPSLA 2017 §3; PLDI 2021 §2).
- Record every generated variant as a replayable transformation sequence plus seed rather than as a mutated document, so reduction preserves validity (spirv-fuzz
.transformations,--replay,--shrink). - Use a declared tolerant metric for images and an exact metric for canonical bytes and hashes, mirroring the split between under-specified rendering and specified encoding (OOPSLA 2017 §2.2, §3.2).
- Treat MT as a bug-finding oracle with no completeness claim and pair it with differential and reference-model oracles (OOPSLA 2017 §2.3).
Adapt
- Replace the histogram chi-square metric with a perceptual metric bounded by the tolerances declared in
spec/00-conformance.md; the shader metric ignores spatial position, which is unacceptable for layout. - Extend equivalence-preserving rewrites with document-specific facts (entity is unreferenced, extension is opaque, subtree is invisible) analogous to the spirv-fuzz fact set, so that preconditions keep variants valid.
- Add exact-equality relations (fixpoint, idempotence, inverse identity) that shader testing does not need because compilers have no canonical form.
Reject
- Manual MR discovery per feature at survey scale (Segura §4.2) is too slow; NUIF should derive relations mechanically from the operation and layout family definitions.
- Human-rated false-positive adjudication (OOPSLA 2017 §5.3) is not automatable; NUIF should encode the tolerance policy in the report instead.
Open questions
- Which layout families admit exact translation and scale equivariance, and which require tolerance because of rounding to device pixels?
- How should tolerant relations be expressed in the machine-readable report so that a threshold change is visible as a policy change rather than as a test change?
- Does the inverse-operation relation hold for
Moveacross component instances with overrides, or must the relation be weakened to canonical equality modulo derived caches? - Can delta consistency against a browser be made stable when the browser and NUIF differ in sub-pixel snapping?
MLIR multi-level intermediate representation and dialect conversion
Document status:
reviewed. Canonical source.
Summary
MLIR deliberately supports multiple abstraction levels and domain-specific dialects in one framework, with explicit legality targets and rewrite-based lowering. Its text, in-memory and compact serialized representations demonstrate that logical IR semantics need not be tied to a single storage encoding.
Evidence
Primary references: MLIR Language Reference, Dialect Conversion documentation, and MLIR rationale. Dialect conversion separates conversion targets, rewrite patterns and type conversion.
NUIF relevance
Borrow the ideas of dialect namespaces, explicit lowering passes, validation and partial legality. Do not copy SSA/control-flow machinery: NUIF is an authored document model, not compiler code IR.
Model-agnostic screenshot reconstruction, evaluation and adaptation plan
Document status:
reviewed. Canonical source.
Summary
Screenshot-to-NUIF is an inverse problem, not ordinary file parsing. A static image is compatible with many different scene graphs, layout programs, fonts, resources, responsive rules, accessibility structures and behaviors. The correct product is therefore a reconstruction pipeline that emits one validated editable hypothesis, alternative hypotheses and calibrated evidence—not a claim that it recovered the unavailable authored source.
The recommended system combines deterministic computer vision/OCR, optional UI grounding and a replaceable vision-language reasoner with NUIF’s typed operations, validator, layout solver and renderer. The reasoner proposes; the core decides whether a transaction is valid; the renderer produces measurable outcomes; a bounded correction loop improves the proposal. No model provider, training framework or base checkpoint appears in the NUIF specification.
Training is deliberately later than evaluation. First implement an untuned baseline and closed loop, freeze a leak-resistant benchmark, and determine which errors remain. Only then compare prompting/tool use, supervised tuning, low-rank adaptation, quantized low-rank adaptation and sequence-level distillation under the same data and evaluator.
Evidence synthesis
What existing work supports
- Design2Code’s 484 real pages demonstrate persistent element-recall and layout errors in screenshot-conditioned frontend generation. This supports explicit element/geometry metrics and a real-world holdout.
- Pix2Struct shows that screenshot-to-structured-markup pretraining combines useful OCR, language and layout signals. It does not recover original source.
- ScreenAI shows value in an intermediate screen annotation containing element type, location, OCR and icon/image descriptions.
- OmniParser shows that a compact detector/OCR/caption pipeline can improve GUI grounding, while also demonstrating that component and weight licenses must be resolved per revision.
- ReverseORC and InferUI show why multiple viewports and held-out contexts are needed to distinguish layout hypotheses and assess generalization.
- DCGen reports improvements from hierarchical screenshot segmentation. This supports multi-scale crops and region-specific proposals rather than one downsampled full-screen prompt.
- VisRefiner is early evidence for learning from target/render differences and corrective edits. Its preprint status requires reproduction before adoption.
- LoRA and QLoRA reduce adaptation resource costs in their studied settings; neither supplies domain correctness or an evaluation method.
- Sequence-level distillation supports learning a smaller sequence generator from teacher outputs. Validated NUIF transactions are a safer target than raw teacher text.
- Calibration and selective-prediction research shows why raw confidence is not enough and why explicit abstention should be evaluated through risk/coverage.
- Dataset/model documentation research supplies reporting structure but does not replace rights review, privacy controls or executable tests.
What the evidence does not support
The reviewed evidence does not justify any of these claims:
- one screenshot identifies the original layout or behavior;
- current general-purpose vision models can reconstruct arbitrary interfaces with extreme precision without deterministic tools and iteration;
- a high perceptual-similarity score implies an editable or accessible result;
- an image crop recovers the original image asset;
- visual font matching identifies exact font bytes or embedding permission;
- low-rank or quantized fine-tuning is automatically more accurate;
- a synthetic screenshot/HTML corpus is representative of real authored tools;
- one closed model, one open model or one UI detector should become normative;
- a research alpha label certifies screenshot reconstruction quality.
Problem contract
The public task receives a set of EvidenceInput values and returns a
ReconstructionResult:
EvidenceInput
screenshots[]: bytes + viewport + DPR + crop/state/time metadata
optional observations[]: OCR, regions, accessibility, source capture
optional known resources[]: digest-pinned images/fonts
requested profile + budgets
ReconstructionResult
validated document or no-result
accepted operation log
immutable resources + derived-resource records
fidelity report
decision-level provenance and calibrated confidence
retained alternatives/abstentions
evaluation report and exact pipeline identity
Source-backed browser observations and screenshot-only inputs use this same
result type but carry different evidence classes. A source-backed field can be
lossless only within a declared adapter subset and only if its source bytes or
stable host semantics support the claim. A screenshot-inferred field cannot be
lossless; its best possible classification is representable or
approximated with inference provenance.
Architecture
Evidence normalization
screenshots, contexts, optional source observations, known resources
|
v
Replaceable observation providers
OCR/baselines | regions/edges/colors | UI grounding | repetition/assets
|
v
Typed ObservationGraph
coordinates, candidates, confidence, evidence regions, provider versions
|
v
Replaceable proposal engine
hierarchy + semantic kinds + layout hypotheses + typed NUIF operations
|
v
Core transaction boundary
schema validation -> operation validation -> apply -> document validation
|
v
Deterministic layout and render
declared contexts -> scene -> reference pixels + diagnostics
|
v
Difference and property evaluators
text | elements | tree | geometry | resources | visual | accessibility
|
v
Bounded corrective-operation loop
accept only improvements satisfying invariants and non-regression policy
The core, renderer and operation grammar are authoritative. Observation providers and proposal models are ports with capability manifests. A provider can be replaced without changing the file format or operation semantics.
Observation graph
Every observation has:
- stable run-local identity;
- provider kind, artifact digest and version;
- source screenshot digest, coordinate space and region;
- predicted type/value with alternatives;
- raw and calibrated confidence;
- relationships such as contains, aligns, repeats, overlaps and possible-parent;
- evidence class: observed-source, observed-pixels, inferred, user-confirmed;
- privacy classification and retention policy.
OCR stores polygons/baselines and Unicode candidates separately from inferred font/style. Region providers store visible boundaries rather than semantic objects. UI grounding labels remain proposals. Repetition detection can suggest components or stack/grid structure but cannot assert them without evaluation.
Coordinate normalization records viewport pixels, device pixels, crop origin, page scale and NUIF logical units. No provider is allowed to mix coordinate spaces implicitly.
Proposal interface and typed operations
The proposal engine never mutates core structs directly. It emits a bounded transaction using the same semantic operations as CLI, editor and adapters. The first reconstruction grammar should cover:
- create entity with temporary/proposed identity;
- set semantic kind, name and accessibility evidence;
- establish parent/sibling anchors;
- set text and text-style candidates;
- bind image or font asset candidates;
- set geometry/paint;
- set layout family and typed constraints;
- attach inference provenance and alternatives;
- replace a prior inferred value through an explicit corrective operation.
Arbitrary code, scripts, extension blobs and raw unbounded JSON are excluded from model output. Unsupported properties produce an abstention or explicit opaque observation; they do not bypass the validator.
The initial proposal and every correction is atomic. A stale expected revision, invalid graph, resource mismatch or budget excess rejects the complete transaction and becomes feedback. The engine cannot gradually corrupt a valid document while searching.
Deterministic and learned responsibilities
Deterministic code should own:
- image decoding, coordinate conversion and basic color/edge statistics;
- OCR-provider invocation contract and candidate normalization;
- validation, canonicalization and operation application;
- layout solving, scene lowering and reference rendering;
- raw/perceptual difference maps and property-level metrics;
- resource hashing, package validation and provenance storage;
- loop limits, acceptance policy and reporting.
Learned or heuristic providers may own:
- text detection/recognition candidates;
- region and icon/image classification;
- grouping, hierarchy and semantic-label proposals;
- layout-family/constraint ranking;
- initial transaction generation;
- selection of corrective operations from structured differences.
This boundary keeps measurable rules out of model weights and prevents a model change from redefining conformance.
Multi-scale and multi-context reconstruction
Small text and controls are lost when an entire high-resolution screenshot is reduced to a model’s fixed input. The baseline therefore supplies:
- a full-screen overview with normalized coordinates;
- deterministic hierarchical regions;
- overlapping high-resolution tiles with shared coordinate transforms;
- focused crops for uncertain text/icons;
- multiple viewports or interaction states when available.
Duplicate observations across tiles are merged by geometry and content evidence, retaining disagreements. Multiple viewport screenshots share proposed semantic identities; candidate layout programs are ranked by how well they predict held-out contexts, not only by fit to the input viewport.
Resources and provenance
Screenshot-only reconstruction can recover only visible samples. It handles resources as follows:
- an exact known image/font resource is bound only when supplied bytes match a declared digest or source-backed capture provides the bytes;
- a crop from the screenshot is a derived image whose provenance includes screenshot digest, crop polygon, scale and any alpha/masking procedure;
- vector tracing is a derived approximation with algorithm/version and visual error, not the original vector;
- generated inpainting or upscaling is never canonical source recovery and requires an explicit accepted transformation policy;
- font appearance yields ranked candidates or substitution, never original font identity or redistribution permission;
- inaccessible resources remain unavailable with item-level fidelity.
A degenerate “one image covering the page” output is a valid screenshot asset but fails the editable-reconstruction profile unless the user explicitly asks for a flat image document.
Closed-loop correction
For each candidate document and evaluation context:
- validate and apply the proposed transaction;
- render through the pinned reference path;
- compare target and result structurally and visually;
- localize differences to observations, entities and properties where possible;
- propose one bounded correction transaction;
- accept it only if validity remains true, the declared objective improves and no protected metric regresses beyond tolerance;
- stop on success, no improvement, repeated state, budget or iteration limit.
The objective is a vector, not one scalar. Selection can use a lexicographic or Pareto policy: document validity and required content first, then text and structural errors, then geometry/resources/accessibility, then perceptual appearance, then simplicity/editability. Every scalarization is recorded.
Loop state is content-addressed so repeated documents are detected. Tool calls, renders and difference maps are cached by input and tool identity. Parallel candidate evaluation is allowed; acceptance order is deterministic.
Benchmark design
Source-backed capture and screenshot reconstruction have separate suites.
Synthetic exact suite
Generate canonical NUIF documents across the supported profile, render them at several contexts and retain exact model/operation/resource labels. Include adversarial near-duplicates that look similar but require different hierarchy or layout. Exact expected data permits:
- document/operation validity rate;
- entity precision, recall and F1;
- parent/sibling and tree-edit distance;
- property accuracy by kind;
- geometry error and intersection-over-union;
- text character/word error, region recall and baseline error;
- exact resource-digest recall where bytes are supplied;
- held-out viewport layout error;
- calibrated confidence and abstention quality;
- reference-pixel, FLIP, SSIM and pinned LPIPS diagnostics;
- latency, peak RAM/VRAM, iteration count and cost.
Real screenshot suite
Use licensed, consented or otherwise documented inputs and human-reviewed target annotations. Do not pretend the original author’s full intent is known. Score visible elements/text/geometry, edit-task success, responsive observations when captured, accessibility evidence, resource provenance honesty and human visual ranking. Preserve ambiguity rather than forcing one gold structure where several reconstructions are equally supported.
Source-backed suite
For browser or host captures, compare preserved source/resource bytes, correspondence, resolved observations and held-out contexts. Source-backed results are not mixed into screenshot-only accuracy without a label because the available evidence is materially different.
Splits and leakage controls
Split by originating project/domain and also by template, component family, font family, resource family and generator seed. Near-duplicate screenshot or DOM/resource hashes cannot cross splits. Benchmark pages never become distillation examples. A public test set may expose inputs but keeps enough private or rotating evaluation to detect overfitting.
Metric policy
No single image metric is a correctness oracle:
- raw pixel difference catches exact raster changes but overreacts to benign platform text differences;
- FLIP models perceptual visibility under declared display parameters;
- SSIM is a classical structural diagnostic with known limitations;
- LPIPS adds learned perceptual features but pins a model and can be gamed;
- OCR/text metrics catch glyph-content errors hidden by aggregate appearance;
- geometry and tree metrics catch editable-structure errors;
- resource/provenance metrics detect false source-recovery claims;
- task edits and held-out viewport renders measure usability of the result.
Visual metrics are computed over the whole image and property-local masks. A large correct background must not hide missing small controls. Scores are reported with distributions and confidence intervals, not only one mean.
The implemented evaluator contract currently provides exact micro-rate evidence and deterministic per-example scored/unscored, mean and nearest-rank p50/p95 aggregation. It intentionally does not invent a confidence interval from its synthetic fixtures. The real benchmark must predeclare a suitable cluster-aware or bootstrap uncertainty method after the sampling unit and corpus design are fixed.
Baseline and ablation ladder
The experiment order is:
- deterministic segmentation/color/edge/repetition plus OCR, no VLM;
- one-shot vision-language proposal from full screenshot;
- proposal with normalized observation graph;
- hierarchical/multi-scale crops;
- multi-viewport layout ranking;
- deterministic render-difference correction loop;
- best available evaluated teacher pipeline;
- tuned student or task adapter;
- distilled student using only validated accepted traces.
Each stage uses identical held-out inputs and budgets. An added component stays only if it improves a predeclared metric without unacceptable regressions in validity, calibration, latency, memory, licensing or maintainability.
Training-data construction
Do not train from raw model transcripts. Store a versioned ReconstructionTrace:
- input screenshot/resource hashes and contexts;
- observation graph and exact provider artifacts;
- proposal model/base/processor/tool versions;
- initial transaction and validator diagnostics;
- intermediate documents, renders and localized differences;
- corrective transactions, acceptance/rejection reason and objective vector;
- final canonical document/package hashes and fidelity report;
- human confirmations/corrections where present;
- rights, privacy, retention and split-group metadata.
Training targets are derived only from validated accepted transitions. Rejected proposals can support preference/error classification if their retention is permitted, but are never silently treated as desired output.
Data sources, in priority order:
- synthetic canonical NUIF renders with perfect labels and controlled perturbations;
- project-owned or permissively licensed source-backed pages with exact resources and multi-context observations;
- contributor-provided examples under explicit training consent;
- separately licensed public datasets after revision-level review.
Authenticated/private captures default to no training and no telemetry. Secret scanning and redaction happen before any retained training record, and redaction itself is recorded as a transformation.
Adaptation and distillation decision
Training is justified only when all conditions hold:
- the untuned closed-loop baseline is reproducible;
- a frozen holdout and error taxonomy exist;
- repeated errors are plausibly learnable rather than missing core semantics;
- enough rights-cleared, high-quality traces exist;
- an adapted model has a clear deployment target and maintenance owner;
- success and rollback thresholds are predeclared.
Compare these options in order:
- prompt/tool/schema changes;
- retrieval or few-shot examples;
- supervised full or partial tuning where feasible;
- LoRA with rank/module ablation;
- QLoRA only when memory pressure justifies it for the selected architecture;
- sequence-level distillation into a smaller student after a stronger teacher and accepted-trace corpus exist.
The teacher is the best evaluated pipeline under a declared budget, not a brand
name. A teacher may combine deterministic tools and a model. A student is
replaceable and is evaluated from scratch on the same frozen holdout. Model
weights, adapters, processors and dataset snapshots are versioned artifacts
outside nuif-core and outside .nuif documents.
Confidence and review policy
Raw provider confidence is calibrated on a disjoint calibration split for each decision type. Reports include reliability and risk/coverage curves under normal and shifted conditions. Automatic application requires both:
- the operation is valid and within the profile; and
- calibrated expected risk is below the profile threshold.
Otherwise the system retains alternatives, produces an explicit abstention or asks for review. User confirmation becomes provenance; it is not retroactively described as model certainty.
Security and privacy
- All image, package, observation, operation, entity, text and iteration sizes are bounded before the corresponding expensive stage.
- Model outputs are untrusted inputs parsed by a strict operation decoder.
- Generated scripts, URLs and external resource requests are inert; no implicit network fetch or code execution occurs.
- Tool and renderer processes may be isolated with time, memory and GPU limits.
- Screenshot text may contain credentials, personal data or proprietary content; retention is opt-in and purpose-limited.
- Prompt injection embedded in a screenshot is visual content, never an instruction that can override the operation schema or tool policy.
- Training and inference dependencies carry a locked artifact/license bill of materials; one component’s license is not generalized to a whole pipeline.
Deployment boundaries
The reconstruction engine is an optional service or library beside the core:
nuif-core / operations / renderer / evaluator deterministic authority
nuif-reconstruct orchestration and ports
observation providers replaceable local/remote tools
proposal provider replaceable model service/runtime
model artifacts separately versioned and licensed
Local inference can keep screenshots private; remote inference requires an explicit data-transfer policy. Browser/WASM and constrained-device consumers use the resulting validated package and do not embed the training stack.
NUIF relevance
This plan keeps probabilistic reconstruction outside the normative core while making its outputs testable through the same operation, validation, rendering, resource and fidelity contracts used by deterministic adapters. It supplies a research path for screenshot import without weakening the distinction between authored facts, source-backed observations and inferred hypotheses. Model choice, adaptation method and deployment runtime remain replaceable; only the typed evidence and result contracts are candidates for specification.
Promotion gates
Screenshot reconstruction remains experimental until:
- deterministic baseline, one-shot model and closed-loop variants run from one command and emit complete reports;
- the frozen synthetic and real suites have documented rights and leak-resistant splits;
- every automatic result is valid, provenance-complete and confidence-calibrated;
- visual improvements do not come from flattening editable structure;
- an independent evaluator reproduces the main reported results;
- at least one real editing workflow demonstrates that reconstructed semantics are more useful than a flat screenshot;
- resource, privacy, security and license audits pass for the selected runtime.
No alpha tag on the editor or core advances these gates. A future model artifact may have its own experimental version, model card and benchmark report.
Falsifiers
Narrow or stop the approach if:
- a flat screenshot or overfit absolute-position tree consistently beats the editable model under the chosen objective;
- held-out viewport performance does not improve over freeform layout;
- the correction loop cycles or improves appearance while degrading protected semantic metrics;
- confidence cannot be calibrated enough to support useful automatic coverage;
- tuned students fail to beat the untuned tool-augmented baseline after fair cost controls;
- rights-cleared data is insufficient for the claimed deployment domain;
- model/provider churn makes results irreproducible without freezing unsafe or unmaintainable dependencies.
Open questions
- What is the smallest observation taxonomy that improves reconstruction across Web, desktop and mobile screenshots without becoming a second design schema?
- Should the first operation decoder use a constrained grammar, tool calls or a two-stage typed AST, and which has the lowest invalid/repair rate?
- How should equivalent but structurally different valid reconstructions be represented in training and evaluation?
- Which held-out edit tasks best measure real authoring usefulness?
- Can responsive rules be inferred robustly from two or three viewports, or is source-backed evidence required for practical accuracy?
- Which confidence events support safe automatic application and which should remain suggestions by design?
- What performance tier makes local reconstruction practical on developer hardware without choosing a normative model?
Datasheets and model cards for reconstruction artifacts
Document status:
reviewed. Canonical source.
Summary
Datasheets for Datasets and Model Cards for Model Reporting establish a useful minimum documentation discipline for training corpora and released models. A reconstruction system needs both: dataset records explain why examples exist, how they were collected and where they should not be used; model records explain intended use, evaluated conditions, limitations and performance.
These documents improve transparency but do not establish that data collection, training or model output is lawful, representative or safe. NUIF should combine them with content-addressed artifact manifests, exact license review, consent policy, leak-resistant splits and executable evaluation reports.
Evidence
- Gebru et al., Datasheets for Datasets, arXiv:1803.09010 / CACM 2021, proposes documenting motivation, composition, collection, preprocessing, distribution, maintenance and recommended uses.
- Mitchell et al., Model Cards for Model Reporting, arXiv:1810.03993 / FAT* 2019, proposes documenting intended uses, factors, metrics, evaluation data, training data and performance/limitations across relevant conditions.
- Neither publication makes documentation a substitute for measurement or governance; the artifacts communicate how a dataset/model was constructed and evaluated.
Mechanism
Every dataset snapshot receives a content digest and datasheet. Every model or adapter receives a content digest and model card. A training-run manifest binds base model, processors, operation-schema version, dataset splits, code revision, hyperparameters, seeds, hardware, evaluator and resulting artifact digests.
Private/authenticated captures are excluded from training by default. Opt-in is recorded per source; credentials and secret-bearing observations are never training features. Takedown and deletion procedures address retained examples and future datasets; limitations of already-released irreversible artifacts are stated plainly.
NUIF relevance
Borrow the two complementary reporting templates.
Adapt them to UI-specific concerns: source/capture rights, font and image licenses, template-family leakage, sensitive text, accessibility content, geographic/language coverage, renderer version and operation-schema version.
Reject “openly reachable” as permission to train, mutable dataset names as reproducible identity and an aggregate model score without per-condition error and calibration reporting.
Open questions
- What minimum evidence establishes permission for synthetic, public Web, contributor-provided and host-exported examples?
- How can source withdrawal be propagated to future dataset releases and retraining schedules?
- Which UI strata must be reported before a model can claim general rather than profile-specific usefulness?
NUIF naming collision reconnaissance
Document status:
reviewed. Canonical source.
Summary
NUIF is not unique. Public search finds at least:
- an old/unreleased Nexus User Input Framework name in PW New Media material;
- Necessary Undertaker Identification Framework (NUIF), a 2024 image-text matching research method/repository;
- unrelated financial/academic acronym uses.
No reviewed result appears to be an active authored-interface interchange standard, but this search is not a legal trademark clearance.
NUIF relevance
Keep nuif as the working repository/project identifier. Do not freeze the expanded standards name, trademark policy or final file-format branding until a proper legal/name clearance and standards-governance review occurs. A later rename must not affect stable document semantics or namespace/version identifiers.
OCI content descriptors and verification-before-consumption
Document status:
reviewed. Canonical source.
Summary
The OCI Image Specification uses descriptors to identify arbitrary byte content by media type, digest and size. Optional URLs are retrieval hints, not identity. A consumer verifies size and digest before expensive interpretation. This is a stronger resource-reference pattern for NUIF than paths or URLs alone.
Evidence
- OCI Image Specification
descriptor.md, Properties, definesmediaType,digestandsizeas the required descriptor fields;urls, annotations and platform information are optional. - Digests defines the digest as a content identifier calculated from the exact bytes. Implementations must support SHA-256 verification; its encoding is lowercase hexadecimal.
- Verification says content from untrusted sources should have its size checked and digest recalculated before consumption, and advises against heavy processing before verification.
- URLs do not replace the descriptor digest. The bytes retrieved from any location still have to satisfy the declared descriptor.
Mechanism
ResourceDescriptor {
media_type: string,
digest: "sha256:" + 64 lowercase hex digits,
size: u64,
locations: [package path or explicitly permitted external locator],
}
resolve location -> enforce declared/implementation size -> hash bytes
-> compare digest -> dispatch by media type
Semantic assets refer to a stable asset identity. The asset refers to one immutable resource descriptor. Replacing its content updates that binding but does not require replacing every semantic reference to the asset.
NUIF relevance
Borrow the required descriptor triple and verification ordering. Digest and declared size are also useful before archive expansion, image decode and font parsing.
Adapt optional URLs into a resolver policy that is disabled by default. Package paths are normalized locators only; an implementation must compare the bytes to the descriptor regardless of where they were found.
Reject OCI image layering, registries, platform manifests and container execution semantics. NUIF needs the descriptor pattern, not an OCI image.
Open questions
- Should additional digest algorithms be syntactically preservable but unsupported, or rejected by the first package profile?
- Which resource metadata is semantic and hashed with the document, and which metadata is package-only provenance?
- Can range-addressable resources be added without weakening whole-byte digest verification?
OCR detection and recognition as separate reconstruction observations
Document status:
reviewed. Canonical source.
Summary
Screenshot reconstruction needs both text localization and transcription. TextOCR provides dense polygon/word annotations for detection and recognition; TrOCR demonstrates a transformer encoder-decoder for cropped text recognition. Neither is specifically a UI typography recovery system. The useful pattern is to keep OCR as a replaceable observation provider with its own benchmarks, coordinates, confidence and language coverage.
Evidence
- TextOCR, CVPR 2021, reports 28,134 real images and 903,069 non-empty annotated words, with polygons for arbitrary-shaped text and explicit annotation/audit procedures.
- TextOCR focuses mainly on scene text; its distribution, shapes and language policy do not match desktop/mobile UI text exactly.
- TrOCR, AAAI 2023, DOI 10.1609/AAAI.V37I11.26538, uses pretrained image and text transformers for text recognition and explicitly leaves text detection to a separate stage.
- Both model families can hallucinate or normalize text. Exact NUIF content requires comparing recognized text to pixels/source evidence and retaining uncertainty rather than silently correcting it with a language prior.
Mechanism
screenshot -> text detector -> polygons/baselines/crop transforms
-> recognizer -> Unicode candidates + confidence
-> grouping/reading order -> text observations
-> typography and layout inference (separate task)
Character error rate, word error rate, region precision/recall and baseline geometry are measured independently. Font family, size, weight, line height and letter spacing are not OCR outputs unless a separate estimator supplies them.
NUIF relevance
Borrow dense text-region evaluation and separable detection/recognition.
Adapt evaluation to UI-scale text, multiple scripts, antialiasing modes, icons mixed with glyphs, truncation, clipping and overlapping regions. Preserve multiple candidates when confidence is low.
Reject one OCR implementation as normative, language-model autocorrection as source truth and inferred font identity from glyph appearance alone.
Open questions
- Which open, redistributable OCR candidates provide the best UI text accuracy per latency/VRAM on the frozen benchmark?
- How should ligatures, icon fonts, emoji and variable-font glyphs be classified?
- Can line baselines and advances be estimated accurately enough to improve text fitting before the exact font resource is known?
OmniParser screen-region detection and icon captioning
Document status:
reviewed. Canonical source.
Summary
OmniParser is a compact screen-parsing pipeline for GUI agents. It combines OCR, interactable-region detection and icon captioning to provide grounded regions to a downstream vision-language model. This is useful as an optional observation provider, especially for small controls, but its target is action grounding rather than full visual reconstruction.
Dependency and weight licensing must be checked at the exact selected revision.
The current repository describes its newer icon_detect_v3 as based on an
MIT-licensed YOLOv9 implementation, while earlier Ultralytics-based detectors
retain their original AGPL terms; caption models are described separately. A
generic NUIF pipeline cannot inherit one blanket license assumption.
Evidence
- Microsoft Research describes two curated tasks: interactable icon detection and icon functional description, implemented with complementary detection and captioning models.
- The reported benchmarks focus on agent grounding/navigation such as ScreenSpot, Mind2Web, Android-in-the-Wild and WindowsAgentArena, not design reconstruction or resource recovery.
- The current repository README distinguishes the license provenance of the newer detector, earlier detectors and captioning models. Exact weights and revisions still require an artifact manifest before distribution.
- The parser uses OCR and icon regions; it does not recover original DOM, CSS, font files, image assets or responsive constraints.
Mechanism
screenshot -> OCR text boxes
-> detector regions/interactivity
-> caption selected icon crops
-> deduplicate/serialize grounded screen elements
-> downstream model
The detector, OCR engine and caption model can be benchmarked independently and their observations can be passed to a model without incorporating their code or weights into the NUIF core.
NUIF relevance
Borrow modular region proposals, OCR fusion and icon crop captioning as an
optional ObservationProvider.
Adapt every output to typed boxes with model/version/license identity, confidence and source region. Evaluate non-interactive visual elements too.
Reject making one implementation mandatory, treating “interactable” as a complete UI element taxonomy, or importing any model weights without a locked bill of materials and compatible redistribution terms.
Open questions
- Does the provider improve final NUIF structural and visual measures over a strong OCR/CV baseline at acceptable latency and VRAM?
- How stable are region identifiers across nearby viewports and states?
- Can its icon captions be calibrated well enough to remain observations rather than accidental behavior assertions?
W3C Open UI Community Group component anatomy research
Document status:
reviewed. Canonical source.
Summary
Open UI researches common component anatomy, parts, states and behaviors across web frameworks and proposes targeted improvements to HTML/CSS/accessibility APIs. It intentionally does not define a universal visual document/editor standard.
NUIF relevance
Reuse terminology and accessibility/component research where compatible and coordinate rather than creating conflicting control vocabulary. NUIF’s scope is orthogonal: authored portable design documents and their implementations.
OpenFig reverse engineering of the Figma .fig Kiwi format
Document status:
reviewed. Canonical source.
Summary
OpenFig documents .fig as a ZIP containing a Kiwi-serialized canvas.fig whose schema is embedded and evolves with Figma versions. Its tooling can parse and encode documents and package .fig archives. Independent Grida research similarly extracts current Kiwi schemas.
NUIF relevance
This makes high-fidelity Figma adapters technically feasible, but the embedded schema is vendor-controlled and mutable. Reverse-engineered internals are compatibility evidence only, never a dependency or canonical semantic source for NUIF.
OpenPencil programmable editor, scene graph, Figma codec and DOM/CSS conversion
Document status:
reviewed. Canonical source.
Summary
OpenPencil exposes its editor engine, scene graph, .fig/Kiwi codec, DOM/CSS conversion, CLI, RPC and MCP surfaces. It proves a modern design editor can treat programmatic control as a first-class surface.
NUIF relevance
Borrow the operational lesson: editor actions, document querying, linting and export must be available headlessly. NUIF differs by making a neutral authored model canonical instead of centering compatibility with an existing vendor model.
OpenTimelineIO adapters, per-schema versioning with upgrade/downgrade functions and UnknownSchema preservation
Document status:
reviewed. Canonical source.
Summary
OpenTimelineIO (OTIO) is an interchange format and API for editorial cut information maintained under the Academy Software Foundation. Its canonical serialization is JSON in which every object carries an OTIO_SCHEMA label of the form Name.Version. Each schema is versioned independently; the C++ TypeRegistry holds per-schema upgrade functions keyed by target version and downgrade functions keyed by source version. On read, an object whose version is older than the registered one is upgraded by applying upgrade functions in order (gaps permitted); an object whose version is newer than the registered one is rejected with SCHEMA_VERSION_UNSUPPORTED. On write, a caller may pass target_schema_versions (or select a family/label such as OTIO_CORE:0.14.0 via OTIO_DEFAULT_TARGET_VERSION_FAMILY_LABEL) and the writer downgrades dictionaries step by step (no gaps allowed).
Objects whose schema name is not registered are instantiated as UnknownSchema, which stores the original schema name, original version and the raw dictionary, and writes them back verbatim; the test suite asserts equivalence after a serialize/deserialize round trip. Adapters are plugins declared in plugin_manifest.json (Adapter.1 entries with name, filepath, suffixes) and discovered via the OTIO_PLUGIN_MANIFEST_PATH environment variable or setuptools entry points; they implement any of read_from_file, read_from_string, write_to_file, write_to_string. Adapter test conventions rely on sample_data fixtures, JSON baselines and equivalence assertions (assertIsOTIOEquivalentTo, assertJsonEqual) with explicit disk-to-memory-to-disk round-trip tests.
Evidence
- Schema label format
"SimpleClass.2";schema_name()/schema_version()accessors;@otio.core.upgrade_function_for(SimpleClass, 2)and C++TypeRegistry::instance().register_upgrade_function(name, 2, fn);@otio.core.downgrade_function_from(SimpleClass, 2)— https://opentimelineio.readthedocs.io/en/latest/tutorials/versioning-schemas.html (retrieved 2026-08-29). - “upgrade functions will be called in order, but they need not cover every version number” and “Downgrade functions must be called in order with no gaps” —
docs/tutorials/versioning-schemas.mdlines 321–323 (main). schema_version_targetsargument on serialization: schemas above the target “will be converted to AnyDictionary and run through the necessary downgrade functions before being serialized” —docs/tutorials/versioning-schemas.mdline 124.- Families and labels:
OTIO_COREfamily with labels such as0.14.0,0.15.0; custom families in.plugin_manifest.jsonunderversion_manifests;otio.versioning.fetch_map("OTIO_CORE", "0.15.0"); env varOTIO_DEFAULT_TARGET_VERSION_FAMILY_LABEL=OTIO_CORE:0.14.0;otioconvert -A target_schema_versions=...— same tutorial. CORE_VERSION_MAP.cppis generated bymake version-map, is “part of the unit tests suite”, and maps label → {schema → version}, e.g.0.14.0hasClip 1,Marker 2;0.15.0hasClip 2—src/opentimelineio/CORE_VERSION_MAP.cpplines 1–60.TypeRegistry::_instance_from_schema: unregistered name →new UnknownSchema(schema_name, schema_version); newer version →ErrorStatus::SCHEMA_VERSION_UNSUPPORTED; older version → iterateupgrade_functionswhereschema_version <= e.first <= registered—src/opentimelineio/typeRegistry.cpplines 360–420.- Registered upgrades/downgrades in core:
register_upgrade_function(Marker::Schema::name, 2/3, ...),register_upgrade_function(Clip::Schema::name, 2, ...),register_downgrade_function(Marker, 3),register_downgrade_function(Clip, 2)—typeRegistry.cpplines 96–220. UnknownSchemaclass:UnknownSchema(std::string const& original_schema_name, int original_schema_version),original_schema_name(),original_schema_version(),data(),is_unknown_schema()override —src/opentimelineio/unknownSchema.h.UnknownSchema::read_fromswaps the reader dictionary into_dataand erasesOTIO_SCHEMA;write_towrites every stored key;_schema_name_for_referencereturns the original name so the label is re-emitted —src/opentimelineio/unknownSchema.cpplines 8–45.- Writer downgrade path:
_downgrade_version_manifest,_downgrade_dictionary, error “No downgrader function available for …” when a step is missing —src/opentimelineio/serialization.cpplines 112–114, 421–506, 1040–1052. tests/test_unknown_schema.py: fixture embeds"OTIO_SCHEMA": "MyOwnDangSchema.3"insidemedia_reference.metadata;test_serialize_deserializeassertsassertIsOTIOEquivalentTo(self.orig, test_otio);test_unknown_to_dictchecksdatareturns a copy and nested known schemas (RationalTime) are decoded — lines 9–100.tests/test_version_manifest.py:test_fetch_map,test_env_variable_downgrade,test_two_version_manifests— lines 65–131.- Adapter contract: implement
read_from_file,read_from_string,write_to_file,write_to_string; manifest{"OTIO_SCHEMA": "PluginManifest.1", "adapters": [{"OTIO_SCHEMA": "Adapter.1", "name", "filepath", "suffixes"}]}; discovery viaOTIO_PLUGIN_MANIFEST_PATHoropentimelineio.pluginsentry points — https://opentimelineio.readthedocs.io/en/latest/tutorials/write-an-adapter.html. - Core adapter tests:
tests/test_adapter_plugin.pyloadstests/baselines/adapter_example.json, checkshas_feature("read_from_file"), argument pass-through (suffix=3), media-linker hooks, manifest lookup by suffix/name, env-var path deduplication and manifest ordering — lines 22–302; baselines directory holdsempty_*.jsonper schema andadapter_plugin_manifest.plugin_manifest.json(listing retrieved 2026-08-29). otio_test_utils.OTIOAssertionsprovidesassertJsonEqual(with trailing-decimal-zero normalization) andassertIsOTIOEquivalentTo—src/py-opentimelineio/opentimelineio/test_utils.pylines 15–31.- External adapter convention (CMX3600):
tests/sample_data/*.edlfixtures;test_edl_round_trip_mem2disk2mem(write_to_string → read_from_string →assertJsonEqual),test_edl_round_trip_disk2mem2disk(read → write to tmp → read →assertIsOTIOEquivalentTo),test_regex_flexibility— https://github.com/OpenTimelineIO/otio-cmx3600-adaptertests/test_cmx_3600_adapter.pylines 16–286. - Adapters were moved out of core into separate packages bundled by
OpenTimelineIO-Plugins(otio-aaf-adapter,otio-cmx3600-adapter,otio-fcp-adapter,otio-svg-adapter, …) — https://github.com/OpenTimelineIO/OpenTimelineIO-Pluginspyproject.tomllines 17–26. - License Apache-2.0 —
LICENSE.txt; tagsv0.18.1,v0.18.0,v0.17.0(retrieved 2026-08-29).
Mechanism
Serialization is a typed dictionary encoding. Every SerializableObject writes its fields plus an OTIO_SCHEMA label; nested objects recurse, so schema labels appear at every level including inside free-form metadata maps. Deserialization dispatches on the label: the registry finds the _TypeRecord for the name, constructs the object, and if the on-disk version is lower it mutates the raw AnyDictionary through each registered upgrade function whose target version lies in the interval before calling read_from. Upgrades are therefore dictionary-to-dictionary transforms independent of the C++ class layout, and may skip versions. Reading a newer version than the library knows is a hard error rather than a best-effort parse.
Downgrading is symmetric but stricter. The writer receives a schema_version_map (from an explicit argument, an environment-selected family label, or a plugin manifest’s version_manifests). For any object whose registered version exceeds the target, the writer serializes it to a dictionary and applies downgrade functions one version at a time until it reaches the target, failing loudly if a step is missing. The CORE_VERSION_MAP snapshot ties human-readable labels (library releases) to full schema-version sets and is regenerated and unit-tested whenever schemas change, giving a reproducible target for “write for OTIO 0.14.0”.
Unknown schemas are preserved by construction. When the name lookup fails the registry constructs UnknownSchema with the original name and version, and read_from takes ownership of the entire dictionary minus the label. Nested known objects inside that dictionary are still decoded, so the preserved blob is a typed tree rather than raw bytes. On write the original label is re-emitted with the stored fields, which yields an idempotent round trip; the object is inert (no upgrade functions run on it) and reports is_unknown_schema() so tools can flag it.
Adapters are thin translators between an external format and the in-memory object model; the object model, not the adapter, owns versioning and preservation. Tests fix the contract from both ends: JSON baselines for the OTIO side and sample files for the foreign side, with equivalence checks after mem→disk→mem and disk→mem→disk cycles.
NUIF relevance
Borrow
- Version each NUIF schema/dialect construct independently and encode
name.versionon every serialized node, so migration is local rather than a monolithic document version bump. - Implement migrations as ordered dictionary transforms registered per target version, with gap-tolerant upgrades and gap-free downgrades, and reject newer-than-known versions explicitly.
- Publish a generated, unit-tested label → schema-version-set manifest (the
CORE_VERSION_MAPpattern) sonuif migrate --target <label>is reproducible. - Model foreign or unregistered constructs as a first-class
UnknownSchema-like value that keeps name, version and payload, decodes nested known values, re-emits verbatim and is queryable via anis_unknownpredicate. - Adopt the adapter test convention of paired fixtures (foreign sample data plus canonical JSON baselines) with both round-trip directions asserted through a canonical equivalence function.
Adapt
- OTIO preserves unknowns structurally (typed dictionary); NUIF additionally needs byte-exact preservation for extension payloads whose encoding is not NUIF’s own, per spec/07 and ADR 0004.
- OTIO’s downgrade discards data silently inside downgrade functions; NUIF downgrades must emit fidelity records (
approximated,unsupported) per dropped field. - Family/label selection via an environment variable is convenient for scripts but should be an explicit CLI/API argument in NUIF’s headless contract, not ambient state.
- OTIO unknown objects do not participate in upgrades; NUIF should allow dialects to register migrations for previously unknown namespaces when they become known.
Reject
- JSON with embedded schema labels at every node is verbose and lacks a canonical form; NUIF keeps
nuif-text-0/nuif-cbor-0canonicalization rules rather than free-form JSON. - Adapter discovery via setuptools entry points and manifest environment paths is Python-specific; NUIF adapters are Rust crates or WASM components with declared capabilities.
- Equivalence assertions that normalize trailing decimal zeros ad hoc should be replaced by NUIF numeric canonicalization rules in spec/08.
Open questions
- Whether per-node schema labels are needed in the binary profile or can be replaced by a document-level dialect version table plus per-entity type references.
- How to compose migrations across dialects when one dialect’s downgrade changes a construct another dialect references.
- Whether NUIF should support “read newer than known” via opaque preservation of the unknown fields instead of OTIO’s hard failure.
- How to keep the unknown-object equivalence check stable under canonicalization when preserved payloads contain floating-point values.
OpenType embedding permissions and reproducible font resources
Document status:
reviewed. Canonical source.
Summary
Reproducible text requires exact font bytes, but possessing or using a font does
not automatically permit redistribution inside a portable package. OpenType’s
OS/2.fsType flags provide machine-readable embedding signals—installable,
restricted, preview/print, editable, no-subsetting and bitmap-only—but those
flags are part of a wider license context and must not be treated as a complete
legal decision engine.
Evidence
- OpenType 1.9.1
OS/2,fsTypedefines mutually exclusive usage permissions: installable (0), restricted (2), preview/print (4) and editable (8). - Bit 8 forbids subsetting and bit 9 permits only embedded bitmaps. Reserved bits and historical version differences mean parsers must validate the table version and length rather than assuming one modern layout.
- Versions 0 through 2 historically permitted multiple usage bits with a least-restrictive interpretation, while version 3 made those bits mutually exclusive. The first NUIF profile deliberately rejects ambiguous historical combinations instead of silently selecting a permission.
- The specification says embedding-aware applications must not embed fonts whose permissions do not allow embedding or alter the flags, and notes that rights are granted by the font vendor.
- CSS Fonts Level 4 defines face selection, variation axes, feature settings and font fallback as separate inputs to rendered text. CSS Font Loading Level 3 exposes document font readiness; capture before fonts settle is not a stable observation.
Mechanism
A NUIF font asset points to exact bytes when policy permits and records at least: media type, SHA-256, face or collection index, names used for matching, variation axes, selected features, character coverage and embedding-policy evidence. A text run still records the shaping inputs required by its profile.
The packaging policy is explicit:
portable: exact bytes embedded and permitted for the declared use;private_authoring: bytes retained only in an access-controlled workspace, not a distributable package;linked: expected digest and resolver hint recorded, no implicit fetch;substituted: replacement bytes and item-level fidelity recorded;unavailable: metrics/evidence may be retained, but no false exactness claim.
NUIF relevance
Borrow the fsType signal and exact OpenType face/variation metadata.
Adapt it into a conservative policy decision that also accepts explicit license metadata and user/admin policy. The file-format validator reports the facts; it does not provide legal advice.
Reject family name as font identity, system-font discovery for exact
profiles, silent fallback, automatic embedding from a browser’s platform-font
name, and a claim that fsType == 0 alone proves redistribution rights.
Implemented narrow baseline
nuif-opentype-static-single-0 accepts only one checksummed, canonically packed
TrueType-outline sfnt face. It rejects TTC, CFF/CFF2, variable, color, bitmap,
SVG and WOFF/WOFF2 sources. Package validation compares face, family names,
static axis state and exact Unicode coverage, then requires matching fsType,
a non-empty license expression and an explicit approved embedding review.
cargo xtask gate-i-font accepts four static TrueType fixtures, compares Skrifa
0.46.2 against a committed hb-info 14.4.0 metadata capture on the pinned Ahem
resource, proves package byte fixpoint and resource retention, and runs 20
malformed/unsupported, 10 policy and six portability trials. Real TTC, CFF,
variable, COLR, embedded bitmap, CBDT and sbix fixtures prove fail-closed
exclusion. This is an automated baseline, not completion of the broader
font-resource experiment.
Open questions
- Which explicit license-expression vocabulary is reliable enough to augment
fsTypewithout pretending to automate legal interpretation? - Can a portable profile subset a font only when both the license signal and shaping corpus permit it, while retaining an audit link to the source digest?
- How should variable-font instancing be represented when the original file may not be redistributed but a licensed derived instance may be?
OpenUSD composition strength ordering, crate binary format, flattening and validation
Document status:
reviewed. Canonical source.
Summary
OpenUSD composes a stage from layers connected by composition arcs. Opinions are resolved per layer stack in a fixed strength order, now spelled LIVERPS in the glossary (Local, Inherits, VariantSets, rElocates, References, Payloads, Specializes); Inherits and VariantSets recurse with Specializes excluded. Composition is non-destructive: authored opinions remain in their layers and the stage produces a composed view. Payloads are weaker than references and may be deferred at load; instancing shares composed prototypes for prims marked instanceable that carry direct arcs. A prim’s typeName is plain metadata; unregistered type names compose and round-trip unchanged, and a fallbackPrimTypes layer-metadata dictionary lets older software map unknown types to known ones. Properties outside any schema are marked custom, a category the USD headers equate with Alembic userProperties.
The crate format (.usdc) is a versioned binary container with an eight-byte magic, a table of contents of six named sections (TOKENS, STRINGS, FIELDS, FIELDSETS, PATHS, SPECS), 64-bit value representations with inline/array/compressed flag bits, write-time deduplication tables for values and arrays, integer compression for structural sections, LZ4 for bulk data and mmap or pread access with zero-copy arrays. The software version is 0.15.0 but new files default to 0.8.0 and are upgraded only when a feature requires it; saving an existing file preserves its version. usdcat --flatten and UsdStage::Flatten lower a composed stage to a single arc-free layer, and the glossary states this loses sharing and multiplies data. Validation moved from a Python complianceChecker to a plugin-based UsdValidation framework with typed error severities and fixers; usdchecker is now a C++ front end over that framework.
Evidence
- LIVERPS ordering and the recursion rule “we ignore Specializes arcs while recursing” — https://openusd.org/release/glossary.html#liverps-strength-ordering (v26.08 docs, retrieved 2026-08-29).
- Layer stack definition: “ordered set of layers resulting from the recursive gathering of all SubLayers of a Layer, plus the layer itself as first and strongest”; arcs target layer stacks, not layers — https://openusd.org/release/glossary.html#layer-stack.
- Sublayers are the arc that builds layer stacks and accept layer offsets — https://openusd.org/release/glossary.html#sublayers.
- Root layer stack: session layer plus root layer; edit targets can only target root-layer-stack prim specs — https://openusd.org/release/glossary.html#root-layer-stack.
- Payloads are recorded but not traversed under
UsdStage::InitialLoadSet::LoadNone, and are weaker than references — https://openusd.org/release/glossary.html#payload. - Instancing shares composed prims across instances and forfeits per-instance overrides beneath the instance root — https://openusd.org/release/glossary.html#instancing;
instanceablerequires a direct composition arc on the prim — https://openusd.org/release/glossary.html#instanceable. - VariantSets are “a switchable reference”; a variant may contain arbitrary scene description and further arcs — https://openusd.org/release/glossary.html#variantset, #variant.
- Relocates arc: layer-metadata path mapping for non-destructive rename/reparent across arcs — https://openusd.org/release/glossary.html#relocates.
- All arcs except subLayers are list-editable (prepend, append, remove, reset) — https://openusd.org/release/glossary.html#composition-arcs.
- Flatten: text flattening “will generally produce extremely large files” because referenced assets are uniquely baked; crate mitigates via deduplication;
UsdStage::Flatten,usdcat --flatten,UsdUtilsFlattenLayerStack,usdcat --flattenLayerStack— https://openusd.org/release/glossary.html#flatten. - Toolset flag text for
usdcat --flatten,--flattenLayerStack,--usdFormat usda|usdc,usddiff -f,usdtree --flatten— https://openusd.org/release/toolset.html. - Crate glossary:
.usdcis “losslessly, bidirectionally convertible to the .usda text format”; crate reads only a small index at open and defers big data; mmap or pread selectable at runtime — https://openusd.org/release/glossary.html#crate-file-format. - Crate version history comment listing 0.0.1 through 0.15.0 (0.4.0 compressed structural sections, 0.5.0/0.6.0 compressed arrays, 0.7.0 64-bit array sizes, 0.8.0 payload list ops, 0.9.0 timecode, 0.10.0 pathExpression, 0.11.0 relocates, 0.12.0–0.15.0 splines and ArrayEdits) —
pxr/usd/sdf/crateFile.cpplines 384–411 at commit ee47c679. OLDEST_SUPPORTED_VERSION "0.0.1",OLDEST_CURRENT_VERSION "0.8.0",DEFAULT_NEW_VERSION "0.8.0"; env settingUSD_WRITE_NEW_USDC_FILES_AS_VERSIONdocumented as “saving edits to an existing file preserves its version” —crateFile.cpplines 157–168.RequestWriteVersionUpgrade(Version, reason)promotes the write version only when a value type requires it (e.g. 0.8.0 for payload layer offsets, 0.9.0 timecode, 0.11.0 relocates, 0.12.0/0.13.0/0.15.0 splines) —crateFile.cpplines 1062–1073, 1506–1642.SdfFileVersion::CanRead/CanWritepredicates —pxr/usd/sdf/fileVersion.hlines 76–83;CrateFile::Versionis an alias ofSdfFileVersion—crateFile.hline 268.- Bootstrap struct:
uint8_t ident[8]; // "PXR-USDC",uint8_t version[8],int64_t tocOffset—crateFile.hlines 479–484;USDC_IDENT = "PXR-USDC"—crateFile.cppline 434. - Section names TOKENS, STRINGS, FIELDS, FIELDSETS, PATHS, SPECS and
_KnownSections—crateFile.cpplines 266–275; writer emits them in that order then the bootstrap — lines 2897–2906. ValueRepbit layout:_IsArrayBit = 1<<63,_IsInlinedBit = 1<<62,_IsCompressedBit = 1<<61,_IsArrayEditBit = 1<<60,_PayloadMask = (1<<48)-1—crateFile.hlines 84–89.- Index types
FieldIndex,FieldSetIndex,PathIndex,StringIndex,TokenIndex;Field {TokenIndex, ValueRep};Spec {PathIndex, SdfSpecType, FieldSetIndex}—crateFile.hlines 230–234, 507–562. - Write-time deduplication tables
_valueDedup,_arrayDedup,_arrayEditDedupkeyed by value hash —crateFile.cpplines 1090, 1700–1760, 1903–1907; times deduplicated by ValueRep — line 1343. - Structural sections compressed with
Sdf_IntegerCompression(CompressToBuffer) —crateFile.cpplines 3057–3179, 3365–3382; bulk reps and token data compressed withTfFastCompression(LZ4 wrapper) — lines 3072–3074, 3413–3414. - Access paths:
_MmapStream(zero-copy arrays),_PreadStream; env settingsUSDC_MMAP_PREFETCH_KB,USDC_ENABLE_ZERO_COPY_ARRAYS,USDC_USE_ASSET—crateFile.cpplines 171–190, 599–712, 2332–2420. - Deprecation warning for files older than 0.8.0 (
PXR_USDC_EMIT_DEPRECATION_WARNINGS) —crateFile.cpplines 192–195. - Tools
usddumpcrate(“Write information about a usd crate (usdc) file”) andusdupdatecrateexist underpxr/usd/bin/— repository listing at ee47c679;usddumpcrate.pyline 33. - Text format:
SdfUsdaFileFormatwith version token “1.0”,GetMinInputVersion/GetMaxOutputVersion,SaveToFile“starting with the loaded layer’s file version and upgrading as needed” —pxr/usd/sdf/usdaFileFormat.hlines 29, 69–123. UsdPrim::GetTypeNamereturns “the composed type name as authored and may not represent the full type”;SetTypeNamewritesSdfFieldKeys->TypeNamemetadata —pxr/usd/usd/prim.hlines 192–204.- IsA schema membership derives from
typeName; a prim subscribes to at most one IsA schema — https://openusd.org/release/glossary.html#isa-schema. - Fallback prim types:
fallbackTypescustomData inschema.usda,UsdStage::WriteFallbackPrimTypes, and “prims with the unrecognized type name will be treated as having the effective schema type of the first recognized type in the list” — https://openusd.org/release/api/_usd__page__object_model.html (section “Fallback Prim Types”). UsdProperty::IsCustom: thecustommodifier “serves the same function as Alembic’s ‘userProperties’” for ad hoc client data outside any schema —pxr/usd/usd/property.hlines 179–185.- UsdValidation framework: validators with metadata (name
pluginName:validatorName, keywords, schemaTypes, isSuite),UsdValidationContextrunning validators in parallel, error types None/Error/Warn/Info, error sites, fixers —pxr/usdValidation/usdValidation/README.md;enum class UsdValidationErrorType { None, Error, Warn, Info }—pxr/usdValidation/usdValidation/error.hlines 37–42. usdcheckeris C++ (pxr/usdValidation/bin/usdchecker/usdchecker.cpp): options--includeKeywords,--noAssetChecks,-t, --strict(“Return failure code even if only warnings are issued”),--variantSets,--variants,--disableVariantValidationLimit,--rootPackageOnly,--skipVariants,--dumpRules; default behaviour validates “all possible combinations of variant selections” — lines 56–140;Warnescalates to failure only understrict— lines 217–218.pxr/usd/usdUtils/complianceChecker.pyno longer exists on the release branch (HTTP 404 at ee47c679, 2026-08-29); validators live underpxr/usdValidation/{usdGeomValidators,usdShadeValidators,usdSkelValidators,usdUtilsValidators,...}.
Mechanism
Value resolution for “strongest wins” fields walks the prim index in LIVERPS order within each layer stack. Local opinions are consulted across the sublayer-expanded stack first. Inherits and VariantSets targets are then composed recursively with Specializes suppressed, so a specialized base can never override an inherited or variant opinion. Relocates remap remote paths into the local namespace before References and Payloads are followed. Specializes is consulted last. Because arcs target layer stacks and every arc except subLayers is a list op, a downstream layer can prepend, append, remove or reset arcs without touching upstream files. This yields the non-destructive property: the union of all authored opinions is preserved as data, and the composed result is a pure function of that data plus load state (payload inclusion, variant selections, session layer).
Type identity is metadata. typeName composes like any other field; the schema registry maps it to a UsdPrimTypeInfo when known. Unknown type names produce prims with IsA false for every schema but with all authored properties intact. fallbackPrimTypes is a forward-compatibility contract: the writer records substitutes so an older reader treats the prim as the first recognized fallback. Schema-less properties survive as custom properties. Together these define USD’s opaque-preservation behaviour: preservation is structural (unknown tokens and fields round-trip through Sdf) rather than a byte-level blob.
Crate is index-first. The bootstrap at offset 0 holds the magic, a three-byte semantic version and the TOC offset; the TOC lists named sections with start and size. TOKENS and STRINGS are interned pools; PATHS is a compressed path tree; FIELDS pairs a token index with a 64-bit ValueRep; FIELDSETS are index runs terminated by a sentinel; SPECS map a path index and spec type to a field set. A ValueRep encodes the type enum, an inline flag (small values stored in the 48 payload bits), array and compressed flags, and otherwise a file offset. On write, every non-inlined value and array is hashed into deduplication tables so identical payloads share one offset, which is why the glossary claims crate flattening beats text flattening. Structural integer arrays use Sdf_IntegerCompression; bulk reps and token blocks use LZ4. Readers map or pread the file and materialize values on request; numeric arrays whose in-file layout matches memory are exposed zero-copy from the mapping. Versioning is monotone and feature-gated: a writer starts at the default (0.8.0) or the file’s existing version and calls RequestWriteVersionUpgrade only when a value type demands a newer encoding; readers accept any version from 0.0.1 and warn below 0.8.0.
Flatten is an explicit lowering. UsdStage::Flatten evaluates composition (including load and variant state) and emits one layer with no arcs, unique namespaces per referenced instance, and resolved opinions; UsdUtilsFlattenLayerStack is a weaker lowering that collapses only the sublayer stack and keeps references, payloads and variants intact. Neither is invertible.
Validation is a registry of named validators with keyword and schema-type metadata; a context selects validators (optionally including ancestor schema types), runs them in parallel, and returns errors with severity, sites and optional fixers. The CLI enumerates variant combinations by default and maps Warn to a non-zero exit only in strict mode.
NUIF relevance
Borrow
- Adopt a fixed, documented strength order for NUIF override sources (local, instance override, component variant, token theme, library reference) so resolution is a pure function of authored data, as LIVERPS makes USD composition deterministic.
- Adopt list-edit semantics (prepend, append, remove, reset) for relationship lists so downstream documents can non-destructively edit upstream composition, matching USD arcs.
- Adopt index-first binary layout with interned tokens, hashed value deduplication and a bootstrap-plus-TOC so the
nuif-cbor-0successor can support lazy random access and cheap flattening. - Adopt feature-gated, monotone encoding versions with a conservative default write version and “save preserves version”, which is how crate avoids forcing upgrades on consumers.
- Adopt the two-tier lowering distinction (flatten layer stack versus flatten everything) as named NUIF lowering passes with fidelity records, since USD documents the losses of each.
- Adopt a validator registry with severity enum, sites, keywords and fixers as the model for NUIF
validatediagnostics and auto-fix hooks.
Adapt
- USD’s unknown-type preservation is structural (typeName as metadata plus
customproperties); NUIF needs the same structural rule plus byte-level preservation for foreign extension payloads that have no NUIF value model. fallbackPrimTypesis a per-document forward-compatibility map; NUIF should generalize it to dialect-level fallback declarations attached toextensions_used.- Payload-style deferred loading maps to NUIF component libraries and asset references, but NUIF must define load state as part of the evaluation context so resolved output is reproducible.
- Instancing’s loss of per-instance overrides beneath the instance root is a precedent for NUIF instance override scoping; NUIF should keep overrides addressable but classify their cost.
- Variant combination validation in
usdcheckeris a model for NUIF responsive/theme context matrices, but NUIF should bound the matrix explicitly rather than via a hidden default limit.
Reject
- The 3D namespace model (prim paths, specifiers, kinds, purposes) is not adopted; NUIF identity is path-independent and semantic.
- Crate’s reliance on mmap and platform I/O is unsuitable as a normative NUIF requirement; NUIF should specify the logical layout and keep transport choices implementation-defined.
- The absence of a per-prim source provenance record in the flattened output is a gap NUIF must not replicate; flattening in NUIF must emit correspondence records.
Open questions
- Whether a NUIF text profile can guarantee lossless bidirectional conversion with a binary profile in the presence of opaque byte extensions, as usda and usdc do for USD values.
- How to define the analogue of
fallbackPrimTypesfor NUIF dialect constructs without allowing a fallback to silently change semantics. - Whether NUIF should expose a
--strictseverity escalation or require explicit severity policies in capability profiles. - How LIVERPS-style recursion rules translate to NUIF graphs with cycles disallowed but multiple relationship kinds coexisting.
OpenUSD composition, layers, references and variants
Document status:
reviewed. Canonical source.
Summary
OpenUSD composes scene description from ordered layers and composition arcs including references, inherits, variants, payloads and specializes. The key lesson is non-destructive composition: authored opinions remain separate while a stage resolves a composed view.
Evidence
OpenUSD terminology and composition documentation define composition arcs as operators combining layer stacks and prim specifications into resolved values.
NUIF relevance
Adapt layer/reference/variant concepts for design-system libraries, themes, brands, responsive projections and local overrides. Avoid inheriting USD’s 3D-specific namespace and asset assumptions.
Operational transformation (dOPT, GOT, Jupiter, Wave) versus CRDTs (Shapiro et al. 2011) - convergence conditions and hosting either above a canonical document
Document status:
reviewed. Canonical source.
Summary
Operational transformation (Ellis and Gibbs 1989) lets every site apply local operations immediately and transforms remote operations against concurrent ones before applying them; correctness depends on transformation functions satisfying TP1 (the two transformed orders yield the same state) and, for fully decentralised n-way concurrency, TP2 (transforming against equivalent sequences yields the same operation). Sun et al. (1998) state the CCI consistency model (convergence, causality preservation, intention preservation), define inclusion and exclusion transformations with a reversibility requirement, and give the GOT control algorithm with an undo/do/redo scheme. Imine et al. (2003) show with the SPIKE prover that the published transformation functions of Ellis-Gibbs violate TP1 and those of Ressel and Sun violate TP2 on three concurrent string operations. Jupiter (1995) and Google Wave avoid TP2 by a central server that serialises operations and transforms only against a single history, with one client operation in flight at a time. CRDTs (Shapiro et al. 2011) replace transformation with data types whose states form a monotonic join-semilattice (CvRDT) or whose concurrent operations commute under causal delivery (CmRDT); both satisfy strong eventual consistency by construction, and the two forms can emulate each other. A canonical document plus an operation log can host either family as a profile because both reduce, after their respective metadata is stripped, to a totally ordered sequence of semantic operations applied to a snapshot.
Evidence
- Ellis and Gibbs, “Concurrency control in groupware systems”, SIGMOD 1989, pp. 399-407, DOI 10.1145/67544.66963; abstract: users “can operate directly on the data without obtaining locks”, the algorithm “must know some semantics of the operations”, and desired behaviour “is non-serializable” (abstract via Semantic Scholar API, retrieved 2026-08-29; ACM page returned HTTP 403). The dOPT algorithm uses per-site state vectors and a transformation matrix indexed by operation type (Imine et al. 2003 §“Ellis’s Transformation Functions” reproduces
Tii,Tid,Tdiwith priorities). - Sun, Jia, Zhang, Yang, Chen, TOCHI 5(1):63-108, March 1998, DOI 10.1145/274444.274447: Definition 1 causal ordering; Definition 2 dependent and independent operations; Definition 3 intention as “the execution effect which can be achieved by applying O on the document state from which O was generated”; Definition 4 consistency model with convergence, causality preservation and intention preservation; Definition 6 total ordering for convergence (§4); Specification 1
IT(Oa, Ob)with precondition context-equivalence and Specification 2ET(Oa, Ob); Definition 9 reversibilityOa = ET(IT(Oa, Ob), Ob); Functions LIT/LET; Algorithm 2 (GOT control algorithm); §7 integration with undo/do/redo. PDF https://www.cs.cityu.edu.hk/~jia/research/reduce98.pdf (retrieved 2026-08-29). - Ressel, Nitsche-Ruhland, Gunzenhäuser, CSCW 1996, pp. 288-297, DOI 10.1145/240080.240305, introduce the adOPTed algorithm and the two transformation conditions later named TP1/TP2 (metadata via search; paper not retrieved).
- Imine, Molli, Oster, Rusinowitch, ECSCW 2003, DOI 10.1007/978-94-010-0068-0_15: conditions
C1: op1 ∘ T(op2, op1) ≡ op2 ∘ T(op1, op2)andC2: T(op3, op1 ∘ T(op2, op1)) = T(op3, op2 ∘ T(op1, op2)); SPIKE finds a C1 counter-example for Ellis-Gibbs (Fig. 3) and C2 counter-examples for Ressel (Fig. 4) and Sun (Fig. 5) using concurrentIns(2,x),Del(2),Ins(3,y); only Suleiman’s functions survive; counter-examples motivated tombstone transformation functions. PDF https://www.lri.fr/~mbl/ENS/CSCW/2013/papers/Imine-ECSCW03.pdf (retrieved 2026-08-29). - Nichols, Curtis, Dixon, Lamping, “High-latency, low-bandwidth windowing in the Jupiter collaboration system”, UIST 1995, DOI 10.1145/215585.215706 (metadata via Semantic Scholar API, retrieved 2026-08-29; PDF not retrieved).
- Google Wave OT whitepaper: the design is based on Jupiter; the server keeps “a single state space, which is the history of operations it has applied”; clients “wait for acknowledgement from the server before sending more operations” and compose pending operations; a streaming transformer processes two operations linearly; server serialisation removes the need for TP2. https://svn.apache.org/repos/asf/incubator/wave/whitepapers/operational-transform/operational-transform.html (retrieved 2026-08-29).
- Shapiro, Preguiça, Baquero, Zawirski, SSS 2011: Definition 3 strong eventual consistency; Definition 4 monotonic semilattice object; Theorem 1 “any state-based object that satisfies the monotonic semilattice property is SEC”; Definition 6 commutativity of updates; Theorem 2 op-based objects with commuting concurrent updates under causal delivery are SEC; §3.2 Theorems 3 and 4 (CmRDT and CvRDT emulation); §3.3 SEC is incomparable to sequential consistency. PDF https://gsd.di.uminho.pt/members/cbm/members/cbm/ps/sss2011.pdf (retrieved 2026-08-29); Springer landing page redirected to an authorisation endpoint.
- Tree CRDT as CmRDT: Kleppmann et al. prove
apply_ops_commutesand SEC through the Gomes et al. framework (nuif:research:crdt-tree-move-operation, §4.2). - Automerge merge rules (retrieved 2026-08-29, https://automerge.org/docs/reference/under-the-hood/merge-rules/): element IDs instead of indices, deterministic arbitrary ordering of concurrent inserts at one position, deterministic choice among concurrent map writes with conflicts retained.
Mechanism
OT (Ellis-Gibbs shape, Sun et al. terminology):
local op o: apply(o); broadcast(o, state_vector)
remote op o: wait until causally ready (Sun Def. 5)
o' := transform o against every concurrent op already applied (IT), after
excluding operations not in o's context (ET) -- GOT, Algorithm 2
apply(o')
TP1 (C1): apply(o1); apply(T(o2,o1)) == apply(o2); apply(T(o1,o2))
TP2 (C2): T(o3, o1 ∘ T(o2,o1)) == T(o3, o2 ∘ T(o1,o2))
TP1 suffices when a server serialises operations and each client transforms only against the server’s linear history (Jupiter, Wave). TP2 is needed for decentralised n-way concurrency and is where published functions fail (Imine et al.).
CRDT (Shapiro et al.):
CvRDT: states form a join-semilattice (S, ≤, ⊔); local update s' ≥ s; merge = s ⊔ s'
convergence: ⊔ is commutative, associative, idempotent -> SEC (Theorem 1)
CmRDT: op = (prepare at source, effect at all replicas); reliable causal broadcast;
concurrent effects commute (Def. 6) -> SEC (Theorem 2)
SEC: eventual delivery + strong convergence (replicas that delivered the same updates have equal state) + termination
Hosting both above a canonical NUIF document (NUIF interpretation):
canonical snapshot S_0 (hash h_0)
profile log L = [op_1 ... op_n] in a total order chosen by the profile
OT profile: order = server sequence; op_i already transformed against L[1..i-1]; metadata = server seq no, client state vector
CRDT profile: order = any causal linearisation; op_i carries replica id + Lamport/opId; metadata = ids, tombstones, clocks
materialise: S_n = fold(apply, S_0, L) -> canonical hash h_n independent of profile metadata (spec/10 requirement)
Both profiles require that every operation be semantic (identity-addressed, precondition-guarded) so that transformation or commutation is defined per operation type; index-addressed operations force string-style transformation functions with the TP2 hazards above.
NUIF relevance
Borrow
- Sun et al.’s three-part consistency model (convergence, causality preservation, intention preservation) as the stated requirements of spec/10, with intention preservation mapped to preservation of each operation’s preconditions.
- Shapiro et al.’s SEC as the convergence guarantee the profile demands of any engine, with Theorem 2’s causal-delivery assumption made explicit as a transport requirement.
- The Jupiter/Wave server-serialised design as the reference architecture for a centralised NUIF profile, because it needs only TP1.
Adapt
- Transformation functions for NUIF must be defined per semantic operation pair (move/move, move/remove, set/set, insert/insert under one parent) rather than per index arithmetic; the CRDT tree move paper already supplies the move/move and move/cycle cases.
- Causal identifiers and clocks are profile data (spec/10); the canonical patch format keeps only base revision plus ordered operations, and checkpoints strip tombstones.
Reject
- Decentralised OT with n-way concurrency as a NUIF profile, because TP2-correct transformation functions are demonstrably hard to obtain (Imine et al.) and offer no advantage over CmRDTs for identity-addressed trees.
- Mandating one CRDT library (Automerge, Yjs) in the specification, consistent with spec/10.
Open questions
- Whether intention preservation has a precise formulation for tree operations with preconditions, or whether it should be replaced by “preconditions of every applied operation held at application time” plus explicit conflict objects.
- Whether an OT profile and a CRDT profile can share one wire operation schema, or whether OT’s transformed operations (context-dependent) must be re-expressed as identity-addressed operations before they enter the canonical log.
- Verification strategy: the tree move CRDT has Isabelle proofs; a NUIF operation set would need an analogous mechanised commutation proof for property and relation operations.
Patch theory in Darcs and Pijul, and the categorical theory of patches
Document status:
reviewed. Canonical source.
Summary
Darcs treats a repository as a set of patches and defines merge through commutation: two sequential patches A B may be rewritten as B' A' with the same effect, and a patch has an inverse. Conflicts in Darcs 2 are encoded by a special patch type, the conflictor, which stores the conflicting primitive and the closure of what it conflicts with; the Darcs wiki reports that commutation with conflictors is not always invertible and that duplicates are a major source of defects. Mimram and Di Giusto formalise files as objects and patches as morphisms of a category L, define merge as a pushout, observe that conflicting patches have no pushout, and construct the free finite conservative cocompletion P whose objects are finite sets of labelled lines with a transitive relation, so conflicts become ordinary objects and merging always exists. Pijul implements the same idea operationally: a repository is a directed graph of line vertices identified by the hash of the introducing change and an offset, edits are edge relabellings, independent changes commute, and conflicts (unordered alive vertices, cycles, zombie vertices) are stable graph states that later changes resolve. What generalises to semantic operations on identity-bearing trees is stated in the relevance section.
Evidence
- Category of files and patches: a file is
A : [n] → L; a patch is an injective increasing partial functionf : [m] → [n]withB ∘ f = Awhere defined (Definition 1).Lis the free monoidal category generated by insertionη_a : I → aand deletionε_a : a → Iwithε_a ∘ η_a = id_I(Proposition 2). Mimram and Di Giusto, arXiv 1311.3903 PDF, §2 (retrieved 2026-08-29). - Merge as pushout: the merged file of two coinitial patches “should be a pushout of the diagram (2)”; diagram (3) “does not admit a pushout in L. In this case, the two patches f1 and f2 are said to be conflicting.” §3.
- Completion:
Pis the free finite conservative cocompletion ofL(Definition 5); by Theorem 6 it is the subcategory of presheaves preserving finite limits; concretelyPis equivalent to finite sets with a transitive relation and relation-preserving functions (Theorem 15); the pushout inPisB ⊎ C / ~with the transitive closure of the inherited relations (Proposition 16); cycles arise inP(Example 18); a morphismG1 + G1 → G1merges two independent lines, modelling conflict resolution (Example 19); deletions are handled in §6 (Theorem 20). §3-6. - Related-work statement: “The Darcs community has investigated a formalization of patches based on commutation properties [10]” and the residual-as-pushout condition is never stated in Darcs or OT work. §1.
- Darcs theory index: three phases, Darcs 1 “Mergers”, Darcs 2 “Conflictors” described as the current state of the art, Darcs 3 formalisation work; links to Jacobson’s inverse-semigroup formalisation (UCLA CAM report 09-83), Camp, Angiuli et al. ICFP 2014, Mimram and Di Giusto, Pijul. http://darcs.net/Theory (retrieved 2026-08-29). The Darcs manual chapter on patch theory returned HTTP 500 on retrieval; commutation notation below is taken from the wiki pages and the Mimram related-work section, not from the manual.
- Conflictors: “a special patch type that represents a conflict between primitive patches (used for Darcs2 repositories)”; a conflictor stores the original primitive, the transitive closure of the changes it conflicts with, and the tracking “apparently has bugs”; commuting a conflictor with a duplicate “is not guaranteed to be invertible”. http://darcs.net/Theory/Conflictors (retrieved 2026-08-29).
- Pijul commutation and conflicts: “for any two changes A and B, either A and B can be applied in any order, or A depends on B, or B depends on A”; “Conflicting changes always commute in Pijul and never commute in Darcs”; Darcs suffers “the exponential merge problem”; conflicting edits are applied without resolution so no information is lost. https://pijul.org/manual/why_pijul.html (retrieved 2026-08-29).
- Pijul model: a repository is a directed graph
G = (V, E)whose vertices are lines identified by the hash of the introducing change and a position within it; edges carry a status label (alive, deleted) and the introducing change; deletion relabels an edge from alive to dead, so the structure is append-only; an insertion depends on the changes that introduced its context, a deletion on the change that created the edge; three conflict kinds: two alive vertices with no directed path between them, alive vertices with paths in both directions (cycle), and zombie vertices with both alive and dead incoming edges; pseudo-edges connect across deleted regions so alive-subgraph traversal does not become linear in history; files use a name vertex and an inode vertex so renames commute with content edits. https://pijul.org/manual/theory.html (retrieved 2026-08-29). - Pijul conflict handling: insertion conflicts (same position, order undecidable), deletion conflicts (edit inside a deleted block), name and rename conflicts; conflicts persist as repository state, further changes apply on top, resolution is itself a change;
pijul unrecordremoves a change. https://pijul.org/manual/conflicts.html (retrieved 2026-08-29).
Mechanism
Darcs (commutation calculus, notation as used in the Darcs theory pages and the Mimram related-work summary):
sequential composition A ; B (B is written in the context after A)
commutation A ; B <-> B' ; A' when the pair commutes
inverse A ; A⁻¹ = id
merge of parallel A, B = find B' with A ; B' having the effect of both
conflict = commutation fails for a required pair; Darcs 2 records a conflictor
Categorical form (Mimram and Di Giusto):
objects: files A : [n] → L morphisms: patches (injective, increasing, partial, label-preserving)
merge(f1 : A → A1, f2 : A → A2) = pushout A1 → M ← A2 (exists only when f1, f2 compatible)
P = free finite conservative cocompletion of L
objects of P: (S, <) finite labelled set with transitive relation (lines partially ordered; cycles allowed)
pushout in P: B ⊎ C / (f(a) ~ g(a)), relation = transitive closure of <_B ∪ <_C
Pijul (graph of lines):
vertex id = (hash(change), offset)
edge label = (status ∈ {alive, deleted, ...}, change)
insert(line) : add vertex, add alive edges to context vertices; depends on context-introducing changes
delete(line) : relabel incoming edge alive → deleted; depends on edge-introducing change
apply(c1); apply(c0) = apply(c0); apply(c1) when neither depends on the other
conflict states: unordered alive siblings | cycle | zombie (alive + deleted edges)
resolution: a new change that adds ordering edges or kills vertices
Cost model as stated by the sources: Darcs merges can be exponential in the presence of conflicts (Pijul manual); Pijul’s pseudo-edges keep alive-subgraph traversal from scaling with history; Pijul’s apply depends on the change and its dependencies, not on the full history, but the manual gives no formal bound.
NUIF relevance
Borrow
- Conflict as a first-class state (Pijul zombie and unordered states; objects of
P) so a NUIF three-way merge can produce a document containing typed conflict objects rather than failing, matching spec/06’s requirement to surface typed conflicts. - Commutation of independent operations as the definition of independence: two NUIF operations are independent when neither’s preconditions mention the other’s effects, and the merge is their commuted composition.
- Content-derived identity for inserted lines (hash of change plus offset) as the model for assigning entity IDs to elements created inside a patch before the patch has a revision hash.
Adapt
- Pijul’s dependency rule (an edit depends on the changes that introduced its context) maps to NUIF preconditions: a
SetPropertyshould depend on the create of its entity, aMoveon the create of the destination parent; dependencies should be derived from preconditions rather than declared manually. - The categorical merge-as-pushout requirement is a correctness test for a NUIF merge function: for independent patches the merge must equal both compositions, otherwise the result must be a conflict object, never an arbitrary winner.
- Pijul’s dual vertices (name vs. content) correspond to separating
Renamefrom structural operations in nuif-protocol so renames commute with subtree edits.
Reject
- Line-level graph representation for the canonical document: NUIF entities already have stable identity, so a graph of lines is unnecessary; only the conflict states and commutation discipline transfer.
- Darcs-style conflictors that record transitive closures inside the patch stream; the Darcs wiki documents non-invertible commutation and duplicate-related defects.
Open questions
- Whether NUIF’s operation set can be given a category with all pushouts (a completion analogous to
P) or whether some operation pairs (delete versus move-into) must remain conflict objects by design. - What the complexity of merging two long NUIF patches is when conflicts are retained rather than resolved; Pijul provides no formal bound to compare against.
- Whether identity-bearing entities remove the “insertion conflict” class entirely (two inserts under one parent are ordered by order keys) or merely convert it into an ordering conflict.
Penpot workspace UI, plugin API and RPC automation surface
Document status:
reviewed. Canonical source.
Summary
Penpot’s workspace is a single-page ClojureScript/React application over a Clojure backend. The Help Center’s interface tour enumerates twenty regions: a horizontal toolbar and main menu at the top, Pages and Layers on the left, an infinite viewport with rulers in the centre, and Design, Prototype and Inspect tabs plus colour and typography palettes, Assets and Design tokens on the right; view mode, history, comments, zoom, presence and file status sit top-right. Plugins run in iframes with a penpot global that exposes selection, page and root access, shape constructors, events, per-shape plugin data, export and markup generation. External automation uses the backend RPC (POST /api/rpc/command/<name>) with personal access tokens and outbound webhooks. The open frontend source (frontend/src/app/main/ui/workspace/...) is a readable reference for panel decomposition.
NUIF interpretation: Penpot confirms the shared left-structure / right-properties / centre-canvas convention and shows that an open editor can expose the same three surfaces NUIF requires (in-editor API, external RPC, event feed). Its file-level RPC is an internal API rather than a normative contract, which is the gap NUIF’s CLI/API specification addresses.
Evidence
Retrieval date for all locators: 2026-08-29.
- Interface tour regions (numbered 1–20): Viewport (1), Toolbar (2, “tools to quickly and easily create different types of layers”), Main menu (3), Pages (4), Layers (5), Rulers (6), Color palette (7), Typography palette (8), Design properties (9), Prototype mode (10), Inspect mode (11), View mode (12), Share/Invite (13), History (14), Comments (15), Zoom (16), Users (17), Assets (18), Design tokens (19), File status (20). Toolbar and main menu are at the top; pages and layers on the left; design/prototype/inspect and palettes on the right. https://help.penpot.app/user-guide/first-steps/the-interface/ — legend.
- The interface guide lists the toolbar tools as board, rectangle, ellipse, text, graphic, path and free drawing and places zoom controls at the top right. https://help.penpot.app/user-guide/the-interface/.
- Design properties “view and edit the attributes of a selected layer”; size and position always present; stroke, shadow, blur optional. Same page.
- Design panel groups per the layers guide: size and position; layout and constraints; opacity and blend; fill, stroke and border radius; shadow (type, position, blur, spread, colour); blur (layer, background); text; export; interactions. Layer types: boards, rectangles/ellipses, text, curves (freehand), paths (bezier), images. https://help.penpot.app/user-guide/designing/layers/.
- Shortcuts: Board B, Rectangle R, Ellipse E, Text T, Image Shift K, Path P, Comments C, Color picker I; zoom Shift 0 (100%), Shift 1 (fit all), Shift 2 (selected); Layers panel Alt L, Assets Alt I, Color palette Alt P, Text palette Alt T, Rulers Ctrl Shift R, Hide UI
\; Group Ctrl G, Create component Ctrl K, Detach Ctrl Shift K, Duplicate Ctrl D, Flex layout Shift A, Grid layout Ctrl Shift A, Undo Ctrl Z, Redo Ctrl Shift Z; Select all Ctrl A, Select parent Shift Enter. https://help.penpot.app/user-guide/first-steps/shortcuts/. The flexible-layouts guide states Ctrl/Cmd A for flex layout and Ctrl/Cmd Shift A for grid; the discrepancy is recorded as unverified. https://help.penpot.app/user-guide/flexible-layouts/. - Flex layout properties: direction (row, reverse row, column, reverse column), wrap and alignment, align items, justify content, row/column gap, four-side padding, sizing fix/fit per axis; grid layout adds cell positioning modes. Same flexible-layouts page.
- Rulers measure in pixels; pixel-grid snapping is default and can be disabled; nudge distance set in Preferences. https://help.penpot.app/user-guide/workspace-basics/.
- Plugins: installed through the Plugin manager (Ctrl Alt P / Cmd Alt P) by manifest URL; manifest declares permissions
content:read/write,library:read/write,user:read,comment:read/write,allow:downloads,allow:localstorage; “Plugins run separately from the main Penpot app, inside iframes”; onlyplugin.jscan use thepenpotobject; types via@penpot/plugin-types. https://help.penpot.app/plugins/getting-started/. - Plugin API reference is hosted at https://doc.plugins.penpot.app/ (link from https://help.penpot.app/plugins/api/).
- Type definitions:
Penpot extends Contextwithselection: Shape[],currentPage: Page | null,root: Shape | null,currentFile: File | null,viewport,library,theme,createRectangle(),createBoard(),createText(text),createShapeFromSvg(svgString),group(shapes),ungroup(group, ...),on(type, callback, props?),off(listenerId),ui.open/sendMessage/onMessage,generateMarkup(shapes, options?),generateStyle(shapes, options?); eventspagechange,selectionchange,themechange,shapechange,filechange,finish,contentsave; per-shapegetPluginData/setPluginData(key, value)andgetSharedPluginData/setSharedPluginData(namespace, key, value);export(config: Export): Promise<Uint8Array>withtype,scale,suffix,skipChildren. https://raw.githubusercontent.com/penpot/penpot-plugins/main/libs/plugin-types/index.d.ts. - Integration: personal access tokens under “Your account > Access tokens” with expiry options; RPC endpoint
/api/rpc/command/<name>via POST with JSON or Transit; exampleget-profilewithAuthorization: Token …; team-level outbound webhooks mirror RPC calls labelled WEBHOOK. https://help.penpot.app/technical-guide/integration/. - Self-hosted instances must enable
enable-access-tokensinPENPOT_FLAGS;get-fileis a POST to/api/rpc/command/get-filewith the file id (snippet from Penpot blog and community; not retrieved in full). https://penpot.app/blog/how-to-integrate-penpot-with-your-developer-toolchain-apis-and-webhooks-for-workflow-automation/. - Architecture: “a typical SPA” with a ClojureScript/React frontend served statically, a Clojure JVM backend persisting to PostgreSQL, a separate exporter, and shared common code. https://help.penpot.app/technical-guide/developer/architecture/.
- Source layout:
frontend/src/app/main/ui/workspace/containssidebar/,viewport/,shapes/,colorpicker/,tokens/, and files includingtop_toolbar.cljs,left_header.cljs,right_header.cljs,main_menu.cljs,viewport.cljs,viewport_wasm.cljs,palette.cljs,color_palette.cljs,text_palette.cljs,comments.cljs,plugins.cljs,presence.cljs,nudge.cljs. https://github.com/penpot/penpot/tree/develop/frontend/src/app/main/ui/workspace. sidebar/containslayers.cljs,layer_item.cljs,layer_name.cljs,assets.cljs,options.cljs,sitemap.cljs,history.cljs,versions.cljs,shortcuts.cljs,debug.cljs, plusassets/,common/,options/. https://github.com/penpot/penpot/tree/develop/frontend/src/app/main/ui/workspace/sidebar.sidebar/options/menus/holds one module per property group:align,blur,bool,border_radius,color_selection,component,constraints,exports,fill,frame_grid,grid_cell,interactions,layer,layout_container,layout_item,measures,shadow,stroke,svg_attrs,text,typography,variants_help_modal,input_wrapper_tokens,token_typography_row. https://github.com/penpot/penpot/tree/develop/frontend/src/app/main/ui/workspace/sidebar/options/menus.- Unverified: whether the Penpot RPC API is declared stable or internal; the integration guide gives no stability statement.
- Unverified: documentation licence of help.penpot.app.
- Unverified: default sidebar widths and whether sidebars are resizable (not stated in retrieved pages).
- Unverified: exact on-screen order of Design-tab groups; the layers guide lists groups without stating order authoritatively.
Mechanism
Workspace composition (from the interface tour):
┌──────────────────────────────────────────────────────────────────────────┐
│ Main menu │ Toolbar: Board Rect Ellipse Text Image Path Curve │ View History│
│ (top-left)│ (top, horizontal) │ Comments Zoom│
├───────────┬──────────────────────────────────────────┬────────────────────┤
│ Pages │ Viewport (infinite canvas, rulers) │ Design │ Prototype │
│ Layers │ │ Inspect │
│ (Alt L) │ │ palettes, Assets │
│ │ │ (Alt I), tokens │
└───────────┴──────────────────────────────────────────┴────────────────────┘
Hide UI: \ Rulers: Ctrl Shift R
Design tab groups (module names in parentheses from sidebar/options/menus): measures (size, position, rotation, radius), align, constraints, layout_container / layout_item (flex and grid), layer (opacity, blend, visibility), fill, stroke, border_radius, shadow, blur, text / typography, svg_attrs, exports, interactions, component, color_selection (mixed selections), bool, frame_grid.
Automation surfaces:
- In-editor:
penpot.selection,penpot.currentPage,penpot.root,create*,group/ungroup,on('shapechange' | 'selectionchange' | 'filechange' | 'contentsave'),shape.setPluginData / setSharedPluginData,shape.export(),generateMarkup/Style; sandboxed in an iframe; permissions declared in the manifest. - External:
POST /api/rpc/command/<name>with token auth (JSON or Transit), for exampleget-file; outbound webhooks at team level. - Source-level: ClojureScript UI modules per panel, usable as a reference for decomposing a properties panel into per-group components.
NUIF relevance
Borrow
- The per-group module decomposition of the properties panel (
menus/*.cljs) as a template for the Svelte shell’s inspector components inapps/editor/ARCHITECTURE.md. - Manifest-declared permissions (
content:read/write,library:read/write) as a model for scoping automation clients. - The event set (
shapechange,selectionchange,filechange,contentsave) as a minimal change feed for replay capture. - Namespaced shared plugin data per shape as further precedent for opaque extension preservation.
Adapt
- Turn the internal RPC command surface into a documented, versioned contract; NUIF’s
spec/12-cli-api-and-automation.mdmust be normative where Penpot’s is implementation-defined. - Keep the left/centre/right composition but adopt the bottom floating toolbar of UI3 for the test editor, since the toolbar position is the one deliberate divergence between the two references.
- Reuse Penpot’s shortcut overlap with Figma (R, E, T, Shift 0/1/2, Ctrl G/K/D) to define a keymap that is familiar in both ecosystems, while resolving Penpot-specific divergences (E for ellipse versus Figma’s O; B for board versus F).
Reject
- Comments, presence/users indicator, history and versions panels, view mode presentation, share/invite, design-tokens UI beyond what token tests need, Inspect-mode code generation (CSS/HTML/SVG snippets), plugin manager and marketplace, WebGL/WASM viewport variants. Reason: outside the testing/import/export scope or duplicating NUIF’s own token and export machinery.
Open questions
- Does Penpot publish an RPC method catalogue with stability guarantees, or must clients read
backend/src/app/rpc/commands/*? - Which of
Shift AandCtrl Ais the current flex-layout binding, and does it vary by platform or version? - Can
shape.export()output be made deterministic for snapshot diffing, and does the exporter service affect this? - Are Penpot sidebars resizable, and what are their default widths?
Penpot v3 package model and official builder
Document status:
verified. Canonical source.
Summary
Penpot v3 is a ZIP package with JSON metadata and optional binary objects. The manifest, file, page and per-shape members expose stable UUIDs, feature flags and a data-migration list. Penpot’s official JavaScript library can build the same representation outside the editor and is therefore a credential-free foreign producer for an adapter fixture.
The executable NUIF profile covers one legacy per-shape package, one page, one board and direct rectangle, ellipse and literal-text children. It is not a general Penpot importer and does not treat package retention as semantic support for unknown Penpot features.
Evidence
- The technical file-format reference identifies v3 as a ZIP archive with JSON
metadata and binary assets, manifest version 1, per-file JSON, page members,
per-shape members and an
objects/directory. It distinguishes the package format version from the evolving file data version and points tobackend/src/app/binfile/v3.cljas the manifest implementation. https://help.penpot.app/technical-guide/developer/data-model/penpot-file-format/ (retrieved 2026-08-30). - The data-model guide describes pages and components as containers over shape trees. Files can refer to shared libraries, and media objects can be stored separately. https://help.penpot.app/technical-guide/developer/data-model/ (retrieved 2026-08-30).
- The data guide states that many attributes are optional, missing properties
express defaults, and import/export removes
nullproperties. Package projection must therefore compare interpreted values rather than assuming member presence is a distinct state. https://help.penpot.app/technical-guide/developer/data-guide/ (retrieved 2026-08-30). - Penpot’s
library/package exposes a builder and byte export independent of a logged-in editor. The repository source inlibrary/src/lib/builder.cljsandlibrary/src/lib/export.cljsis the primary implementation locator. The committed fixture is generated by exact npm dependency@penpot/library1.1.0 and its SHA-256 is checked after repeat generation. https://github.com/penpot/penpot/tree/develop/library (retrieved 2026-08-30). - The compact v3 representation embeds shapes in page entries. The upstream
change describes it as opt-in through
binfile-v3-compact/format: "compact"while retaining the per-shape representation as the compatibility default. The compact profile remains excluded until upstream stabilization and an independent fixture exist. https://github.com/penpot/penpot/issues/10727 (retrieved 2026-08-30). zip8.6.0 supports bounded archive reading and writing with optional compression features. Its defaults enable more algorithms and crypto than this profile requires, so the workspace disables defaults and enables only Deflate throughdeflate-flate2-zlib-rs. https://docs.rs/zip/8.6.0/zip/ (retrieved 2026-08-30).- CVE-2025-29787 affected the crate’s high-level filesystem extraction routine before 2.3.0 when archive symlinks could redirect later writes. The selected version is beyond the patched boundary; the NUIF adapter additionally never extracts to a filesystem and rejects symlink, directory, encrypted, duplicate and unsafe-path entries before JSON parsing. https://github.com/zip-rs/zip2/security/advisories/GHSA-94vh-gphv-8pm8 (retrieved 2026-08-30).
Mechanism
Import first enforces whole-package, member-count, expanded-byte, per-member, compression-ratio, name, entry-kind and compression-method limits. Every member is then read into bounded memory. The parser validates one manifest/file/page, the root-frame/board relationship and the declared shape subset. Mapped JSON scalars retain member-qualified byte spans; all original member payloads and compression choices remain in a retentive package object.
No-op synchronization returns the original archive bytes. A mapped edit patches only the recorded scalar spans within affected uncompressed payloads and then rebuilds the package deterministically. Unedited member payloads are identical, although ZIP container metadata can change after a rebuild. The rebuilt package is imported again and must equal the requested canonical document before it is returned. Structural and out-of-profile edits fail without partial output.
Native output stores JSON members below 4 KiB and deflates larger members; foreign packages retain each member’s original method. The threshold follows a same-machine profile comparison that reduced small-package writer allocation without changing semantic or retentive laws. It is not part of the Penpot format and can change only with deterministic-output fixtures and a recorded benchmark.
The foreign producer fixture exercises a path independent of NUIF’s writer. The profile runner adds an opaque binary member and unknown JSON object, applies eight mapped changes, checks untouched payload identity and exact canonical re-import, and covers typed traversal and one-over resource failures. A second bridge runs export, import, synchronization and re-import through the public CLI.
NUIF relevance
Borrow the inspectable package boundary, stable UUIDs, explicit feature flags, separate format/data versions and separation of JSON metadata from binary objects.
Adapt one page and a bounded board/rectangle/ellipse/text subset. Package identity maps to NUIF identity, while the page remains structural package data. Unknown fields and members are retentive evidence, not lossless NUIF semantics.
Reject direct mapping of every Penpot shape or library object to a universal NUIF kind. Constraints, grids, variants, interactions, text runs, paths, effects, component libraries, shared-file relations, tokens and media exceed the current model/profile and require their own correspondence and fidelity laws.
Open questions
- When will Penpot declare the compact page representation stable, and will its manifest or feature markers provide an unambiguous dispatch key?
- Does the official library publish a compatibility guarantee for generated packages across Penpot editor releases?
- Which Penpot text fields can carry content-addressed font identity without profile-owned plug-in metadata once variable and embedded fonts are included?
Pix2Struct screenshot parsing as visual-language pretraining
Document status:
reviewed. Canonical source.
Summary
Pix2Struct pretrains a vision-language encoder-decoder by parsing masked webpage screenshots into simplified HTML. It shows that screenshot structure, text recognition and visual language can share a useful pretraining objective, and that variable-resolution inputs matter for visually situated tasks.
The target is simplified HTML, not original DOM/CSS or a lossless authored model. For NUIF it is evidence for structured perception pretraining, not a ready-made converter or an argument that one architecture should be normative.
Evidence
- ICML 2023/PMLR 202 abstract and §2 describe masked webpage screenshot parsing into simplified HTML as the pretraining objective.
- The authors describe the objective as combining signals related to OCR, language modelling and image captioning rather than treating them as entirely isolated tasks.
- The model uses variable-resolution inputs and is evaluated across documents, illustrations, user interfaces and natural images.
- Official implementation: https://github.com/google-research/pix2struct.
Mechanism
A screenshot is divided into variable-resolution patches. The decoder emits a text sequence representing a simplified DOM-like structure. Masking parts of the screenshot forces the model to combine visual layout and text/markup context. Downstream tasks are then fine-tuned from this shared representation.
NUIF relevance
Borrow screenshot parsing as pretraining and variable-resolution or tiled inputs for small text and dense controls.
Adapt the output vocabulary to a versioned typed observation graph or NUIF operation schema. A validator must reject malformed identities, impossible trees, unsupported property kinds and non-finite values before rendering.
Reject simplified HTML as ground truth for original authored semantics and unconstrained text generation of an entire NUIF document as the only interface.
Open questions
- Does structured-operation decoding outperform JSON/document decoding after validity, repair rate and final render are measured?
- How should high-resolution tiling preserve shared coordinates and avoid duplicate elements across overlapping crops?
- Which pretraining targets transfer to design semantics beyond Web markup?
PNG third-edition preservation and deterministic decode inputs
Document status:
reviewed. Canonical source.
Summary
PNG is a lossless encoded raster format, but displaying it is not equivalent to copying an RGBA array. The datastream can declare color through CICP, ICC, sRGB, or gamma/chromaticity chunks; alpha samples are linear and unassociated; Exif and other ancillary data may affect interpretation or preservation. NUIF should retain original bytes as the authoritative resource and treat decoded pixels or GPU textures as derived caches tied to a pinned decoder profile.
Evidence
- PNG Third Edition §4.3 defines four color-signalling routes and their precedence: CICP, ICC, sRGB, then chromaticity plus gamma.
- §6.2 states that PNG color samples are not premultiplied by alpha. Alpha is a linear fraction of full opacity and is not gamma-corrected.
- §11.3 defines ancillary chunks including iCCP, sRGB, cICP, eXIf, physical pixel dimensions and textual data. A decode-only pipeline can therefore lose source information even when its visible pixels are acceptable.
- §13.14 and §13.16 discuss decoder color and alpha handling. The specification discourages unnecessary color conversion during format conversion because gamut and rounding loss can accumulate.
Mechanism
The resource descriptor identifies the original PNG datastream. An image asset
records its intrinsic dimensions and the exact decoder profile used to obtain a
canonical pixel surface. ImagePaint records fit, crop rectangle, transform,
sampling and opacity; none of these values are inferred from the byte path.
PNG bytes (authoritative, digest-pinned)
-> bounded parser + pinned color/orientation policy
-> straight-alpha reference pixels
-> declared conversion/premultiplication at scene lowering
-> optional decoded/GPU cache keyed by source digest + decoder profile
NUIF relevance
Borrow PNG as the first image-resource profile because it has an open, mature specification and supports lossless storage with alpha and explicit color metadata.
Adapt the format into a stricter NUIF decoder profile that fixes accepted color chunks, orientation handling, output color space, alpha conversion, sampling, maximum dimensions, decoded bytes and ancillary-chunk budgets.
Reject replacing the original resource with decoded pixels, silently discarding color metadata, or calling a screenshot crop the original asset. Screenshot crops are derived resources with screenshot digest and crop region in their provenance.
Resolved policy decisions
- Animation remains outside static decoder profiles; accepting APNG requires a separate time/frame/composition contract.
- Profiles zero and one reject conflicting or undeclared colour metadata even where PNG defines precedence. This makes encoded-sRGB interpretation explicit and prevents decoder-specific colour conversion.
- Rust
png0.18.1 is the implementation decoder and independently implementedzune-png0.5.2 is the differential oracle. Both run with explicit allocation bounds and integrity checks.
Executable profiles
nuif-png-rgba8-0 answers the ambiguity question by accepting only
non-interlaced RGBA8 with no colour metadata or one valid pre-image sRGB
chunk. It interprets both cases as encoded sRGB, rejects every other ancillary
chunk and orientation/animation metadata, and caps encoded bytes, dimensions,
pixels, decoded bytes and chunk count. cargo xtask gate-i-image compares
png 0.18.1 with independently implemented zune-png 0.5.2 across every PNG
row filter, then exercises exact package retention, resource-aware lowering,
repeatable CPU rendering and hostile one-over cases.
nuif-png-basic-rgba8-1 is a separate compatible expansion. It admits all
non-interlaced PNG colour/depth combinations that normalize to RGBA8 without
sample-precision loss: 1/2/4/8-bit greyscale and indexed colour, RGB8,
greyscale-alpha8 and RGBA8. Required palettes and valid tRNS transparency are
expanded exactly. Thirteen fixtures span every admitted colour/depth
combination and both colour-key and indexed transparency; both decoders
produce identical normalized RGBA bytes. The original bytes remain the asset
identity, and a profile-one RGB resource passes the same scene/raster path.
The wider profile deliberately rejects 16-bit samples rather than truncating precision, and still rejects interlace, the complete PNG Third Edition colour precedence model, orientation, animation and arbitrary ancillary metadata. A real-world corpus, live host affine equivalence, GPU comparison and hosted cross-platform image-raster reproduction remain open evidence—not implied by decoder agreement on the generated fixtures.
Property-based and model-based testing of stateful systems (QuickCheck, eqc_statem, quickcheck-state-machine, proptest-state-machine)
Document status:
reviewed. Canonical source.
Summary
QuickCheck (Claessen and Hughes, ICFP 2000) introduced properties as executable universally quantified functions checked on random size-bounded inputs. Stateful extensions (Quviq eqc_statem, Hughes 2016) generate command sequences from an abstract model with preconditions, a state transition function and postconditions, run them against the real system, and shrink failing sequences by deleting commands that do not contribute to the failure. quickcheck-state-machine (Haskell) and proptest-state-machine (Rust) implement the same pattern; the Rust crate exposes ReferenceStateMachine and StateMachineTest traits and shrinks by removing unseen transitions, deleting transitions while re-checking preconditions, then simplifying individual transitions and the initial state. Rust proptest represents generated values as ValueTrees with simplify/complicate and persists failing seeds in proptest-regressions files.
For NUIF the pattern is the operations suite of conformance/PLAN.md: generate nuif_protocol::Operation sequences from a simplified reference model of the document tree, apply them to the engine through the CLI/API surface, and compare canonical state, resolved boxes and inverse-replay results after each step.
Evidence
- Properties are Haskell functions such as
prop_RevApp xs ys = reverse (xs++ys) == reverse ys++reverse xs;quickCheckreports “OK: passed 100 tests.” Claessen and Hughes, ICFP 2000, DOI 10.1145/351240.351266, §2.1 (PDF https://www.cs.tufts.edu/~nr/cs257/archive/john-hughes/quick.pdf, retrieved 2026-08-29). - Conditional properties use
==>and stop after a candidate limit (default 1000) with “Arguments exhausted”. Same paper, §2.3. classifyandcollectprint the distribution of generated data so that trivial cases are visible. Same paper, §2.4.- Generators are
Gen awith a size parameter (sized,resize) to bound generated structures;class Arbitrary a where arbitrary :: Gen a. Same paper, §3.1–3.2. - The 2000 paper contains no shrinking and no state-machine framework; the authors observe that errors divide roughly evenly among generators, specification and program. Same paper, §6.6.
- Stateful testing: “We test stateful systems by generating sequences of calls to the API under test”, modelling state abstractly with transitions per operation and postconditions relating results to the model; a test passes if all postconditions hold. Hughes, “Experiences with QuickCheck: Testing the Hard Stuff and Staying Sane”, LNCS 9600, 2016, DOI 10.1007/978-3-319-30936-1_9, §2 and Fig. 2 (PDF https://publications.lib.chalmers.se/records/fulltext/232550/local_232550.pdf, retrieved 2026-08-29).
- Shrinking searches for the smallest similar failing test, removes unnecessary calls and simplifies arguments; lessons: “Errors are often in the model, rather than the code”. Same paper, §2.
- Volvo/AUTOSAR: 20,000 lines of QuickCheck code tested a million lines of C from six suppliers, finding more than 200 problems, over 100 of them ambiguities in the standard. Same paper, §3.
- Parallel testing reuses the sequential model and accepts a run if some interleaving of results matches the model; the dets model is under 100 lines against an implementation of over 6,000 lines. Same paper, §4.
- eqc_statem callbacks:
initial_state(),COMMAND_args(S),COMMAND_pre(S),COMMAND_next(S, V, Args)(“used during both test generation and test execution”),COMMAND_post(S, Args, R); the precondition is “also used when shrinking” so invalid commands do not appear. http://quviq.com/documentation/eqc/eqc_statem.html, version 1.48.3, retrieved 2026-08-29. - quickcheck-state-machine record:
initModel,transition,precondition,postcondition,invariant,generator,shrinker,semantics,mock,cleanup; symbolic references stand for values not yet known at generation time. https://github.com/stevana/quickcheck-state-machine,src/Test/StateMachine/Types.hsand README,master, retrieved 2026-08-29; de Vries, Well-Typed blog 2019-01-23, https://www.well-typed.com/blog/2019/01/qsm-in-depth/. - proptest-state-machine 0.8.0 (
Cargo.tomlonmain, depends on proptest 1.11.0).src/strategy.rsdefinesReferenceStateMachinewithtype State,type Transition,fn init_state() -> BoxedStrategy<Self::State>,fn transitions(state: &Self::State) -> BoxedStrategy<Self::Transition>,fn apply(state: Self::State, transition: &Self::Transition) -> Self::State,fn preconditions(state: &Self::State, transition: &Self::Transition) -> bool(default true), andfn sequential_strategy(size: impl Into<SizeRange>) -> Sequential<...>. https://raw.githubusercontent.com/proptest-rs/proptest/main/proptest-state-machine/src/strategy.rs, retrieved 2026-08-29. - Generation loop: a transition tree is drawn from
transitions(&state); ifpreconditionsholds it is pushed and the model advanced, otherwiserunner.reject_local("Pre-conditions were not satisfied"). Same file,Sequential::new_tree. - Shrinking:
enum Shrink { InitialState, DeleteTransition(usize), Transition(usize) };simplify()first removes transitions never executed before the failure, then deletes transitions from the back re-checking preconditions, then shrinks individual transitions, then the initial state;complicate()undoes the last step. Same file; CHANGELOG 0.3.0 “Remove unseen transitions on a first step of shrinking” (#388), 0.3.1 precondition fix (#482). src/test_runner.rsdefinesStateMachineTestwithtype SystemUnderTest,type Reference: ReferenceStateMachine,fn init_test(ref_state) -> Self::SystemUnderTest,fn apply(state, ref_state, transition) -> Self::SystemUnderTest(ref_state is the state after the transition),fn check_invariants(state, ref_state),fn teardown(state, ref_state), andfn test_sequential(config, ref_state, transitions, seen_counter)which checks invariants before the first and after every transition. https://raw.githubusercontent.com/proptest-rs/proptest/main/proptest-state-machine/src/test_runner.rs, retrieved 2026-08-29.- Macro:
prop_state_machine! { #[test] fn name(sequential 1..20 => MyTest); }optionally with#![proptest_config(...)]; onlysequentialis supported. Same file; book chapter https://proptest-rs.github.io/proptest/proptest/state-machine.html, retrieved 2026-08-29. - proptest 1.11.0:
Strategy { type Tree; type Value; fn new_tree(&self, runner: &mut TestRunner) -> NewTree<Self> };ValueTree { fn current(&self) -> Self::Value; fn simplify(&mut self) -> bool; fn complicate(&mut self) -> bool }where simplify moves current to a halfway point between low and high. https://docs.rs/proptest/latest/proptest/strategy/trait.Strategy.html and trait.ValueTree.html, retrieved 2026-08-29. Config:cases256 (PROPTEST_CASES),max_shrink_itersu32::MAX (PROPTEST_MAX_SHRINK_ITERS),max_shrink_time0,max_global_rejects1024,max_local_rejects65536,rng_seed,failure_persistencedefaultFileFailurePersistence::SourceParallel("proptest-regressions"), which stores seeds, not values. https://docs.rs/proptest/latest/proptest/test_runner/struct.Config.html and https://proptest-rs.github.io/proptest/proptest/failure-persistence.html, retrieved 2026-08-29.- “Shrinking never shrinks a value to something outside the range the strategy describes.” https://proptest-rs.github.io/proptest/proptest/tutorial/shrinking-basics.html, retrieved 2026-08-29.
- Rust quickcheck 1.1.0:
trait Arbitrary: Clone + 'static { fn arbitrary(g: &mut Gen) -> Self; fn shrink(&self) -> Box<dyn Iterator<Item = Self>> }, default empty iterator. https://docs.rs/quickcheck/latest/quickcheck/trait.Arbitrary.html, retrieved 2026-08-29. - Linearizability checkers: Knossos checks a history of invoke/complete pairs against a single-threaded model with “linear” and “wgl” algorithms (https://github.com/jepsen-io/knossos); Porcupine implements P-compositionality with a
Model { Init, Step, Equal, Partition }(https://github.com/anishathalye/porcupine). README level, retrieved 2026-08-29. - File-synchroniser model: Hughes, Pierce, Arts, Norell, “Mysteries of Dropbox”, ICST 2016, DOI 10.1109/ICST.2016.11 uses QuickCheck’s state machine library with a trivial model state and inserts conjectured upload/download events to explain observations; found unexpected behaviour in two of three services. PDF https://www.cis.upenn.edu/~bcpierce/papers/mysteriesofdropbox.pdf, Abstract and §III, retrieved 2026-08-29.
- Editor-adjacent examples: ropey
tests/proptest_tests.rsapplies inserts and removes to aRopeand aStringand asserts equality, with 512 cases and a checked-inproptest-regressionsfile (https://raw.githubusercontent.com/cessen/ropey/master/tests/proptest_tests.rs); Automergerust/automerge/tests/text.rslines 658–712 generatesActionsequences withprop_flat_mapand comparesdoc.text()with an expectedString(https://raw.githubusercontent.com/automerge/automerge/main/rust/automerge/tests/text.rs); yrsrun_scenario(seed, mods, users, iterations)checks pairwise convergence of block stores (https://raw.githubusercontent.com/y-crdt/y-crdt/main/yrs/src/test_utils.rs). Retrieved 2026-08-29.
Mechanism
Model-based state-machine property (eqc_statem, quickcheck-state-machine, proptest-state-machine):
generate(seed, size):
m = init_state(seed) # reference model
ops = []
while len(ops) < size:
t = transitions(m).sample(seed) # model-dependent generator
if preconditions(m, t):
ops.push(t); m = apply_model(m, t)
else: reject_local()
return (m0, ops)
execute(m0, ops):
sut = init_test(m0); m = m0
check_invariants(sut, m)
for t in ops:
m = apply_model(m, t)
sut = apply_sut(sut, m, t) # postcondition compares sut result with m
check_invariants(sut, m)
teardown(sut, m)
shrink(m0, ops, failing):
ops = drop_unseen(ops) # transitions after the failure point
for i in reversed(range(len(ops))): # DeleteTransition
if valid_under_preconditions(m0, ops \ ops[i]) and failing(m0, ops \ ops[i]): ops.remove(i)
for i in range(len(ops)): # Transition: ValueTree::simplify on ops[i]
while ops[i].simplify() and failing(...): pass; ops[i].complicate() as needed
m0.simplify() while failing(...) # InitialState
Invariants of the method:
- The model is simpler than the system; a model of tens of lines is sufficient for an API of thousands (Hughes 2016 §4).
- Preconditions are enforced during generation and during shrinking, so every shrunk sequence is valid (eqc_statem; proptest-state-machine).
- A failure is persisted as a seed and regenerated, not stored as data (proptest
FileFailurePersistence). - Distribution of generated commands is measured (
classify/collect) so that generator bias is visible.
NUIF instantiation (synthesis): Reference::State is a tree of EntityId with parent, index, name and an Extensions map; Transition is nuif_protocol::Operation; apply on the model performs the structural change; preconditions reject moves into descendants, removal of missing entities and duplicate IDs; check_invariants compares nuif_query::roots and children order with the model, checks that opaque extension bytes on untouched entities are unchanged, and checks that inverse replay of the transaction restores the canonical hash.
NUIF relevance
Borrow
- The
ReferenceStateMachine/StateMachineTestsplit of proptest-state-machine maps directly onto a reference document model and thenuif_api::Engineimplementation; the crate is the natural harness for theoperationssuite (proptest-state-machine 0.8.0). - Precondition-guarded generation and shrinking keep operation sequences valid, which is the property the task requires for minimised failing sequences (eqc_statem; proptest-state-machine
Shrink::DeleteTransition). - Seed persistence in
proptest-regressionsfiles gives replayable failures without storing documents (proptestConfig.failure_persistence). - Distribution monitoring (
classify/collectin QuickCheck) should be emitted into the machine-readable report so that swarm or coverage steering can be evaluated.
Adapt
- The reference model must also carry a minimal layout semantics for stack containers so that postconditions on resolved boxes (additivity, containment) are checkable without a second layout engine.
check_invariantsruns after every transition; for NUIF the expensive checks (export-import round trip, render) should be sampled per seed while cheap structural checks run every step.- Parallel or linearizability testing (Hughes 2016 §4; Knossos, Porcupine) applies only to the collaboration profile and is out of scope for the single-writer trial-and-error loop.
Reject
- Symbolic references in the quickcheck-state-machine style are unnecessary because NUIF entity IDs are chosen by the generator, not returned by the system.
- Rust quickcheck’s
shrinkiterator API lacks precondition-aware sequence shrinking and is inferior to proptest-state-machine for this use.
Open questions
- Should the reference model include component instantiation and override semantics, or should instance-related operations be tested only through metamorphic relations?
- How should
prop_flat_map-style state-dependent generation be balanced against swarm-style feature omission to avoid generator bias toward shallow trees? - Can
proptest-state-machine’s sequential strategy be driven from a corpus (coverage-guided) rather than a fresh seed per case without forking the crate?
Content-addressed provider manifests and external AI bills of materials
Document status:
reviewed. Canonical source.
Summary
Reproducible reconstruction needs more than a provider name or mutable model version. The observation must identify the exact operational system that produced it: implementation, weights, processor, task adapter, quantization, prompt/tool configuration and supported wire profiles. That identity must change when any bound artifact changes.
NUIF should not define a competing software or machine-learning bill of materials. SPDX 3.0.1 has AI and Dataset profiles, while CycloneDX 1.7 has a machine-learning-model component and model-card structures. The selected design therefore uses a small canonical NUIF capability wrapper and content-addresses an external SPDX or CycloneDX inventory. The wrapper travels with observation evidence; the potentially larger inventory, model card and artifacts remain separately addressable.
Runtime packaging is a separate concern. MLflow packages model metadata, dependencies, signatures and flavors for loading; ONNX supports tensors stored outside the model protobuf. Those patterns can carry a provider, but neither substitutes for a provider-neutral capability and evidence identity.
Evidence
- SPDX 3.0.1 publishes distinct AI and Dataset model profiles alongside the core software-bill-of-materials model. Source: https://spdx.github.io/spdx-spec/v3.0.1/model/AI/AI/, retrieved 2026-08-31.
- The CycloneDX 1.7 JSON schema admits
machine-learning-modelcomponents and defines a model card with model parameters and quantitative analysis. Its specification overview identifies ECMA-424 as the formal standard. Sources: https://github.com/CycloneDX/specification/blob/1.7/schema/bom-1.7.schema.json and https://cyclonedx.org/specification/overview/, retrieved 2026-08-31. - Ecma’s ECMA-424 page describes the CycloneDX v1.7 bill-of-materials format as a structured inventory. Source: https://ecma-international.org/publications-and-standards/standards/ecma-424/, retrieved 2026-08-31.
- MLflow’s model documentation separates the model package, environment, flavors and input/output signature. This is useful deployment metadata but is tied to the MLflow loading contract. Sources: https://mlflow.org/docs/latest/ml/model/index.html and https://mlflow.org/docs/latest/ml/model/signatures, retrieved 2026-08-31.
- ONNX external data stores tensor content outside the protobuf and requires a relative location; parent-directory components are disallowed. This is a useful large-artifact packaging rule, not a complete lineage or capability manifest. Source: https://onnx.ai/onnx/repo-docs/ExternalData.html, retrieved 2026-08-31.
Mechanism
nuif-reconstruction-provider-manifest-0 is bounded deterministic CBOR. It
declares provider identity and maturity, capabilities, local/remote execution,
input/output profiles and exact SHA-256 identities for one implementation plus
optional model, processor, adapter, quantization, prompt-template and
tool-configuration artifacts. The hash of the canonical bytes is the
ProviderIdentity stored on every observation and proposal.
An observation bundle includes the canonical manifests for every referenced provider. Validation derives every identity again, rejects duplicates and rejects observations or proposals whose identity does not resolve. This makes the evidence locally auditable without embedding model weights or fetching a network resource.
Development-only deterministic providers may omit a supply-chain inventory when they contain no learned artifacts. Released or learned providers require an exact SPDX 3.0.1 or CycloneDX 1.7 inventory digest. Learned artifacts also require a model-card digest. Dataset snapshots remain governed by the separate corpus manifest and dataset card; a future training-run record must bind those inputs and produced provider artifacts.
NUIF relevance
Borrow SPDX/CycloneDX inventory vocabularies, model cards, immutable artifact identity and deployment-package separation.
Adapt them with a small NUIF wrapper that states only the capabilities and wire profiles needed to interpret reconstruction evidence.
Reject mutable provider/version strings, an unresolvable manifest digest,
embedding weights in ordinary .nuif document resources, treating MLflow or
ONNX as the interchange standard, and assuming that a valid inventory proves
completeness, security, performance, rights or safety.
Open questions
- Which signed statement format and transparency log should bind released manifests, inventories, cards and binaries after release provenance exists?
- Should a future remote-provider profile bind endpoint policy and attestation separately from the model/implementation manifest so operational rotation does not rewrite artifact identity?
- What training-run vocabulary can reuse SPDX/CycloneDX relationships while preserving dataset split, evaluator, seed, hardware and accepted-transition evidence without duplicating the corpus contract?
QLoRA memory-efficient quantized fine-tuning
Document status:
reviewed. Canonical source.
Summary
QLoRA backpropagates through a frozen 4-bit quantized base model into LoRA weights. The paper introduces NormalFloat4, double quantization and paged optimizers to reduce memory, demonstrating fine-tuning of a 65B language model on a single 48 GB GPU in its studied setup.
This supports a possible resource-efficient experiment path. It does not show that quantized tuning improves NUIF reconstruction accuracy, applies unchanged to every vision-language architecture, or removes the need to license and distribute a compatible base model.
Evidence
- NeurIPS 2023 abstract defines the frozen 4-bit base plus trainable low-rank adapters and the three memory-saving techniques.
- The reported memory and quality results are for the paper’s model families, instruction datasets and evaluation. They must not be generalized to an untested visual-operation decoder.
- The authors explicitly discuss weaknesses in chatbot benchmarks, reinforcing the need for domain-specific evaluation rather than inherited model rankings.
Mechanism
The base weights are quantized for the forward/backward computation but remain frozen. Gradients update the LoRA parameters. Reproducibility therefore requires the quantization format, compute dtype, module selection, optimizer, base-model revision and adapter configuration in addition to ordinary training metadata.
NUIF relevance
Borrow conditionally QLoRA when a selected open vision-language base fits the method and full-precision adaptation exceeds the experiment budget.
Adapt the comparison to fixed data, seeds and evaluator; report accuracy, calibration, latency, peak RAM/VRAM, energy/time and artifact size against LoRA and untuned inference.
Reject describing QLoRA as an accuracy technique or choosing it before measurement solely because it uses less memory in a language-model study.
Open questions
- Do vision towers, multimodal projectors and structured decoders tolerate the same quantization regime?
- What calibration loss appears after quantization even when aggregate task scores remain stable?
- Is adapter merging compatible with the intended local inference runtimes?
React JSX and DOM element source surface
Document status:
verified. Canonical source.
Summary
JSX is a JavaScript syntax extension whose elements lower to immutable React element objects. DOM components use React’s DOM property vocabulary. JSX expressions, component calls and control flow make arbitrary source a program rather than a declarative document.
Evidence
- The JSX guide requires closed tags, one enclosing returned root and camel-cased property names for many DOM properties. JavaScript expressions are embedded with braces. https://react.dev/learn/writing-markup-with-jsx (retrieved 2026-08-29).
createElement(type, props, ...children)accepts intrinsic tag strings, component types and heterogeneous React children.keyandrefare special fields, and returned elements and props are immutable. https://react.dev/reference/react/createElement (retrieved 2026-08-29).- Common DOM components accept
aria-*anddata-*attributes, event handlers and astyleobject. Thestylekeys use camel-cased CSS property names and numeric values receive property-dependent unit handling. https://react.dev/reference/react-dom/components/common (retrieved 2026-08-30). - React describes inline
styleas a JavaScript object inside the JSX expression, recommends classes for static styles, and shows that quoted JSX attributes are literal strings while braces admit arbitrary expressions. https://react.dev/learn/javascript-in-jsx-with-curly-braces (retrieved 2026-08-30). - Tree-sitter JavaScript 0.25.0 includes JavaScript and JSX in one grammar. Its
Rust
LANGUAGEconstant is compatible with the workspace Tree-sitter API; syntax nodes expose byte offsets through Tree-sitter’s standard interface. https://github.com/tree-sitter/tree-sitter-javascript and https://docs.rs/tree-sitter-javascript/0.25.0/tree_sitter_javascript/ (retrieved 2026-08-30).
NUIF relevance
Borrow intrinsic DOM element semantics, key as a foreign correspondence
hint, literal data-* identity and literal style properties.
Adapt only a statically analyzable JSX subset: intrinsic elements, literal attributes, literal text and profile-owned style objects. The executable first profile intentionally excludes arrays. Retentive edits use syntax-node byte ranges. Formatting, comments, imports and unrelated module source remain unchanged.
Reject evaluation of arbitrary JavaScript during import. Components, hooks,
spreads, conditional expressions, loops, context, event handlers and runtime
style values are preserved source or unsupported until a separately declared
execution profile supplies inputs and a deterministic runtime.
Mechanism
The executable profile parses the entire module with pinned Tree-sitter JavaScript, locates exactly one literal profile marker and verifies that its paired intrinsic JSX root is the direct return value of a synchronous, zero-argument, default-exported function. It converts only exact literal attributes, a fixed style-object vocabulary and raw escaped text. Every mapped scalar keeps its original UTF-8 byte span; synchronization proves those spans are fresh, replaces only changed spans and reimports the result for canonical document equality. Source, syntax-node and mapped-depth limits are checked on the import boundary.
Open questions
- A TSX profile needs a separately pinned grammar and a decision on whether type-only source is merely retained or participates in correspondence.
- Runtime-backed components need an explicit input/state matrix, a sandboxed React renderer and evidence distinct from this non-executing source profile.
- Class names and imported stylesheets need a CSS provenance and cascade model; silently resolving them during JSX import would make results environment dependent.
- The first profile has syntax and round-trip evidence but no browser/runtime equivalence claim. Such a claim requires a separately versioned renderer oracle and layout comparison corpus.
Executable boundary
nuif-react-jsx-0 requires one directly returned marked intrinsic subtree in a
zero-argument default-exported function. It maps fixed flex containers and
literal pinned-font text through 21 scalar correspondences and never invokes a
React, Node or browser runtime. This is deliberately stricter than valid JSX:
the distinction prevents syntax acceptance from being mistaken for program
evaluation or runtime equivalence.
Group-isolated reconstruction corpora and auditable benchmark snapshots
Document status:
reviewed. Canonical source.
Summary
A screenshot reconstruction benchmark needs stronger isolation than a random row split. Several screenshots can originate from one site, template, component library, font set, resource set or synthetic generator; treating those rows as independent lets a system memorize a family while appearing to generalize. Exact duplicate checks alone also miss transformed or cropped members of the same family.
The selected contract combines immutable artifact digests with declared group identities. Every origin, template, component, font, resource, generator and near-duplicate group is confined to one adaptation, calibration, validation or test partition. Inputs and targets retain separate disclosure levels so a public-input/private-target evaluation can be audited without publishing the target. Per-example rights evidence and allowed uses are explicit.
This is an integrity mechanism, not a rights engine or duplicate detector. A validator can prove that declared identities do not cross partitions; it cannot prove that the declarations are complete, that a license applies, that consent is valid or that the resulting sample represents the intended world.
Evidence
- MLPerf Training Rules section 6.5 states that training data may not contain
data appearing in the test set. The same policies require the reference
partitioning in the closed division. Source:
training_policies/training_rules.adoc, section 6.5, retrieved 2026-08-31. - MLPerf Inference Rules section 7 requires a checksum-verification script and an unchanged dataset at the start of each run. It separately identifies accuracy and calibration data. Source: https://github.com/mlcommons/inference_policies/blob/master/inference_rules.adoc, section 7, retrieved 2026-08-31.
- scikit-learn’s
GroupKFoldandStratifiedGroupKFolddocumentation defines folds with non-overlapping groups; stratification is attempted subject to that isolation constraint. Source: https://scikit-learn.org/stable/modules/cross_validation.html, “Cross-validation iterators for grouped data”, retrieved 2026-08-31. - Hugging Face’s Dataset Cards documentation records that a repository
README.mdplus YAML metadata communicates license, composition, use and limitations, while repository revisions provide version history. Source: https://huggingface.co/docs/hub/main/datasets-cards, retrieved 2026-08-31.
Mechanism
The executable nuif-reconstruction-corpus-manifest-0 record pins a snapshot,
dataset card and evaluator by SHA-256. Each example declares its evidence
suite, split, input and target artifacts, disclosure policy, collection class,
rights evidence, permitted uses, sensitivity review and leakage groups.
CorpusManifest::audit rejects:
- duplicate example or per-example artifact identities;
- any non-context artifact digest reused across different partitions;
- any declared family group reused across different partitions;
- adaptation, calibration or evaluation examples without their corresponding permitted use;
- screenshot-only examples carrying exact source/resource inputs;
- source-backed examples without source bytes;
- retained real examples without a withdrawal policy, and private/authenticated examples without explicit authorization;
- missing near-duplicate assignment, invalid digests and bounded-work excess.
The audit is derived data and can be validated against the manifest to detect edited counts. It reports partition/suite/disclosure counts but does not expose artifact bytes.
NUIF relevance
Borrow immutable checksum verification, explicit benchmark rules, grouped partitioning and dataset-card documentation.
Adapt groups to UI reconstruction: origins, templates, components, fonts, resources, synthetic generators and near-duplicate families all matter.
Reject random row splitting, a hash-only contamination claim, treating a public URL as training permission, publishing private targets merely to make a benchmark reproducible, and claiming statistical validity from manifest validation.
Open questions
- Which independently reviewed detector and thresholds should assign transformed/cropped screenshot near-duplicate groups?
- Which strata and minimum group counts are needed for useful uncertainty intervals and distribution-shift reporting?
- What neutral evaluator service can hold restricted targets while publishing reproducible input identities, evaluator versions and signed result records?
Skia, WebRender, Servo pipeline and wgpu rendering architectures
Document status:
reviewed. Canonical source.
Summary
Skia demonstrates a broad mature 2D API across raster/GPU/PDF/SVG targets. Servo explicitly separates DOM/script, layout box/fragment trees, display-list generation and WebRender. wgpu supplies a safe cross-platform Rust API over Vulkan, Metal, D3D12 and WebGPU-class backends.
NUIF relevance
The reference implementation should preserve a renderer-independent display/scene boundary. Interactive GPU rendering and normative conformance rendering can use different backends while sharing the same lowered scene semantics.
Bounded JSON, CBOR and stream ingestion for untrusted NUIF inputs
Document status:
verified. Canonical source.
Summary
Parser recursion limits do not by themselves bound untrusted document work. A complete boundary must cap bytes before an input is read into memory, cap syntax nesting before or during deserialization, and cap the cardinality and retained data of the decoded semantic model before recursive validation, layout or rendering. Output writers need the same byte cap so an in-memory model cannot expand into an encoding the paired decoder refuses.
The profile-0 reference path applies all three layers. CLI and headless-editor readers stop after the first byte beyond the encoded limit. The text preflight scanner ignores quoted strings and JSON5 comments while counting containers; Ciborium uses its caller-selected recursion limit. An iterative semantic walk measures entity, relation, edge, responsive-rule, property-node, property-depth, containment-depth, string and binary totals. Validation retains at most 1,024 ordinary diagnostics plus one truncation issue. Canonical CBOR map keys are encoded once per multi-entry map before sorting so hostile wide maps cannot force repeated key re-encoding in the comparator.
Evidence
serde_json::Deserializer1.0.151 retains a recursion limit by default. Its officialdisable_recursion_limitdocumentation warns that arbitrarily deep input can overflow the stack and that later recursive operations, including destruction, also require protection. Locator:Deserializer::disable_recursion_limit, docs.rs, retrieved 2026-08-29.- Ciborium 0.2.2 exposes
from_reader_with_recursion_limit; its source states that inputs beyond the selected bound returnRecursionLimitExceededand warns that high limits risk stack exhaustion. The defaultfrom_readerpath uses 256. Locator:ciborium/src/de/mod.rs, functionsfrom_reader_with_bufferandfrom_reader_with_recursion_limit, main and 0.2.2 source, retrieved 2026-08-29. - Rust’s
Read::takereturns a reader that yields at most the selected number of bytes. NUIF readslimit + 1, making the first excess byte distinguishable from an exactly-at-limit EOF without buffering the rest. Locator:std::io::Read::take, Rust 1.98 standard-library documentation, retrieved 2026-08-29. stats_alloc0.1.10 instruments allocation, deallocation and reallocation requests and providesRegionsnapshots. NUIF measures each case in a single-threaded release binary after a fixed warmup; report metadata records toolchain, OS, architecture, CPU and available parallelism. Locator:StatsAlloc,Stats,RegionandINSTRUMENTED_SYSTEM, docs.rs source, retrieved 2026-08-29.- Executable regression:
cargo xtask hostile-inputs. It writestarget/hostile-input-report.json, rejects every enumerated one-over byte/depth/cardinality case with the named resource, accepts all semantic boundary classes, and fails when any case exceeds 2 seconds, 64 MiB of allocator traffic or 16 MiB retained at observation time. Core unit tests exercise one-over rejection for every public semantic limit; CI uploads the measured report.
Mechanism
Ingestion uses a limit-plus-one reader so oversized streams cannot allocate beyond the decision point. Text receives a quote/comment-aware structural preflight; CBOR receives the same depth value through Ciborium’s recursion-limit API. Deserialization is followed immediately by iterative semantic accounting. Canonical text and CBOR writers are bounded, and CBOR key encodings are cached only for multi-entry map sorting so both output growth and comparison work remain linear in retained key bytes plus sort comparisons. The isolated release runner creates adversarial inputs before each measured region, warms one fixed fixture, retains the result while sampling allocator counters, and classifies errors without including report construction in the case measurement.
Measured calibration
The 2026-08-29 Apple Silicon release run used rustc 1.98.0 and covered oversized text, over-depth JSON/JSON5 and CBOR, every semantic cardinality class, single and total strings, total binary payload, containment depth and a 16,384-entry hostile CBOR map. Boundary cases included 8,192 entities and tokens, 4,096 roots, 32,768 relations, 16,384 responsive overrides, 8,191 valid child references, 65,536 property values, 128 containment levels, 8 MiB total strings and 8 MiB CBOR binary. The slowest observed case was below 25 ms, maximum allocator traffic was below 39 MiB, and maximum retained data was below 8.5 MiB. The automated ceilings intentionally retain substantial CI/platform margin and are rerun rather than treated as universal hardware performance claims.
NUIF relevance
These measurements replace the earlier unsupported one-million-node and depth-1,024 hypotheses. They establish a reproducible profile-0 safety envelope, not a promise that every conforming implementation must share the reference implementation’s allocator behavior. A foreign implementation may use tighter operational limits, but it must expose them and must accept the normative boundary fixtures if it claims the profile-0 conformance level.
Open questions
- Renderer timeout and memory isolation for future image, font, path and GPU resources remains a Gate D concern; those resource classes do not yet exist in executable profile 0.
- Server deployments should add process-level cancellation and tenant quotas around the synchronous deterministic codec budgets.
Resource-aware NUIF packaging and source-backed capture synthesis
Document status:
reviewed. Canonical source.
Summary
NUIF needs one resource model shared by direct authoring, adapters, browser capture and later screenshot reconstruction. The semantic document names stable assets; immutable descriptors identify exact bytes; package locators explain where those bytes are carried; provenance explains where they came from. No one of these identifiers can substitute for the others.
The recommended alpha path is a deterministic ZIP package with a fixed first
mimetype member, canonical CBOR manifest and document, and SHA-256-addressed
blobs. Bare canonical encodings remain available as .nuif.json and
.nuif.cbor; existing raw .nuif inputs are legacy read-only detection during
the alpha migration. This is a research conclusion pending the package RFC and
executable cross-writer fixtures, not a completed profile claim.
Source-backed browser capture and screenshot-only reconstruction must remain separate products of the same import pipeline. Browser capture can preserve source bytes, downloaded resources and resolved observations. A screenshot can only preserve its own pixels and infer a possible editable structure.
Evidence
- EPUB OCF demonstrates an interoperable ZIP subset, early media-type member, manifest discipline and explicit remote-resource boundary.
- OCI descriptors demonstrate that media type, digest and size should be checked before expensive content interpretation; URLs are retrieval hints.
- PNG Third Edition demonstrates why encoded source bytes, color metadata and straight-alpha semantics must survive independently of decoded caches.
- OpenType
fsTypedemonstrates that technical ability to embed a font is not sufficient; an exporter must preserve and apply redistribution policy. - CDP demonstrates that DOM, layout, style, network resources, fonts, accessibility and screenshots are distinct observations obtainable under a pinned browser execution context.
- Existing NUIF Penpot tests prove that a restricted ZIP reader can reject traversal, duplicates, symlinks, encryption, unsupported compression and expansion-limit attacks without filesystem extraction. They do not prove the proposed NUIF package layout or image/font budgets.
- The executable package profile now proves exact bytes from two ZIP writers and passes a shared-buffer allocation trial: an 8 MiB resource retains the same pointer across package, cloned handle map and session under 1 MiB of allocator traffic and retained bookkeeping.
- Image scene lowering now interns one decoded surface per digest/profile, preflights a 64 MiB decoded total and keeps 1,024 uses of one 512×512 image to one 1 MiB surface under measured release-build ceilings.
Mechanism
The proposed model has three layers:
Semantic asset
AssetId (stable under byte replacement)
kind + intrinsic semantics + policy
|
v
Immutable resource descriptor
media_type + sha256 digest + byte size
|
v
Package/resolver locator
embedded blob path | explicit linked locator + expected digest
Provenance independently records source URL/path/node/range, capture context,
derivation, license evidence and confidence.
Resource roles are source, authoring, derived and cache. Source and
authoring resources can affect fidelity and document semantics. Derived
resources retain their transformation record. Caches never affect the semantic
document hash and may be deleted or regenerated.
The package has two hashes:
- semantic document hash: SHA-256 of canonical
nuif-cbor-0document bytes; - package hash: SHA-256 of the complete deterministic package bytes.
The first remains stable when nonsemantic caches or container metadata change. The second proves the exact delivered artifact. A manifest binds every semantically required resource descriptor and role.
NUIF relevance
This boundary lets every host use one core while choosing a suitable shell: Rust API, C ABI, WASM, CLI, editor or process protocol. Those surfaces do not reimplement resource rules. Browser capture is a new adapter because it owns a pinned runtime and observation protocol; the existing source adapter remains a retentive static compiler path.
Images preserve encoded originals. Fonts preserve exact bytes only where policy allows. Screenshot-derived crops, reconstructed vectors and generated assets are derived approximations with evidence and confidence, never disguised as captured originals. External resolution is opt-in and always digest-checked.
Remaining questions
- Can an externally authored writer reproduce the already exact in-repository two-writer package bytes on every supported host?
- Do the measured package/image allocation ceilings reproduce on hosted Linux, Windows and macOS runners, and what corresponding ceiling is appropriate for the font pipeline?
- Which resource substitutions are allowed by each portability profile, and when must a missing resource make validation fail?
- How should an adapter preserve an inaccessible local font: metrics only, outlines, a linked descriptor, or an unavailable fidelity item?
resvg regression suite, resvg-test-suite reference corpus, usvg lowering and the SVG support table
Document status:
reviewed. Canonical source.
Summary
resvg is a Rust static-SVG renderer built on tiny-skia. Parsing and rendering are split: usvg lowers SVG into a resolved tree (only absolute path segments, resolved use, CSS, text and markers, objectBoundingBox converted to userSpaceOnUse), and resvg rasterises that tree. The regression suite consists of roughly 1,700 single-issue SVG files with a fixed 200 × 200 viewBox, rendered at 300 px width with pinned fonts and compared against PNGs with a per-channel threshold of 1 and zero tolerated differing pixels. A generator script emits one #[test] per SVG. The separately maintained resvg-test-suite holds the same SVGs together with manually verified reference PNGs and publishes a per-feature support table for resvg, browsers and other libraries. The README claims bit-identical output across platforms because no system libraries are used.
Evidence
- Scope: resvg “aims to only support the static SVG subset; i.e. no
a,script,vieworcursorelements, no events and no animations”; SVG Tiny 1.2 “is not supported and support is also not planned”; SVG 2 support is in progress (README, “SVG support”). - Suite size: “a vast test suite that includes around 1600 tests”, described as SVG-to-PNG regression tests that exclude dependency tests (README, lines 23–25). The current tree holds 1,722 SVG files under
crates/resvg/tests/tests/: filters 398, masking 93, paint-servers 151, painting 306, shapes 133, structure 262, text 379 (GitHub tree listing, 2026-08-29);render.rscontains 1,716 generated tests. - Reproducibility claim: “if you render an SVG file on x86 Windows and then render it on ARM macOS - the produced image will be identical. Each pixel would have the same value.” (README, “Reproducibility”).
- Naming: tests are organised as
tests/<category>/<element-or-attribute>/<case>.svgwith a sibling.png, for examplepainting/stroke-linejoin/{arcs,bevel,miter,miter-clip,round}.svg(tree listing). - Authoring rules: fixed 200 × 200 viewBox template with a frame
rect; “Each test must test only a single issue”; every element needs anid; uniquetitleunder 60 characters; line length under 100; UTF-8;check.pyenforces these (crates/resvg/tests/README.md). - Reference generation: render with
--width 300 --skip-system-fonts --use-fonts-dir 'tests/fonts' --font-family 'Noto Sans' --serif-family 'Noto Serif' --sans-serif-family 'Noto Sans' --cursive-family 'Yellowtail' --fantasy-family 'Sedgwick Ave Display' --monospace-family 'Noto Mono', thenoxipng -o 6 -Z; 300 px “to test scaling” (crates/resvg/tests/README.md, “Render PNG”). - Two PNG sets:
resvg-test-suite/pngcontains reference images (“how the SVG files should be rendered”);resvg/tests/pngcontains images rendered by resvg itself “used only for regression testing” (crates/resvg/tests/README.md, “resvg tests vs resvg-test-suite tests”). - Harness:
IMAGE_SIZE: u32 = 300; a globalfontdbloadstests/fontsand sets the five generic families;MAKE_REFregenerates references; the actual image is alpha-demultiplied before comparison;get_diffmarks a pixel different if any of R, G, B, A differs by more thanDIFF_THRESHOLD = 1, treats two fully transparent pixels as equal, counts size mismatches as differences, and writes a three-panel diff PNG totests/diffs/(crates/resvg/tests/integration/main.rs). - Generated tests:
gen-tests.pywalkstests/**/*.svg, derives a function name from the path and emits#[test] fn ... { assert_eq!(render("..."), 0); }; theIGNORElist excludesfilters/feMorphology/huge-radius(CI timeout), invalid-size and non-UTF-8 structure cases, andpaint-servers/radialGradient/focal-point-correctionwith the comment “Produces slightly different output on some hardware. Not a bug, just a SIMD rounding difference.” (crates/resvg/tests/gen-tests.py). - usvg lowering: attributes resolved (inheritance, defaults), CSS applied, basic shapes converted to paths, only absolute MoveTo/LineTo/QuadTo/CurveTo/ClosePath segments,
useand nestedsvgresolved, invalid elements removed, relative units converted, images loaded or decoded, references resolved,switchresolved, text “completely resolved”, markers converted into regular elements, all filters supported, recursive elements removed,objectBoundingBoxreplaced withuserSpaceOnUse(crates/usvg/README.md, “Features”). - Unsupported features are enumerated (font-based SVG elements,
color-profile, externaluse,clip,color-interpolation,direction,unicode-bidi, and others) (docs/unsupported.md). - Support table: rows are SVG elements and attributes grouped by category; columns are resvg, Chrome, Firefox, Safari, Batik, Inkscape, librsvg, SVG.NET, QtSvg; legend “Passed | Failed | Crashed | ? | Undefined behavior” (linebender.org/resvg-test-suite/svg-support-table.html). Results are produced by manual comparison recorded in
results.csv“viatools/vdiff” and charted bystats.py(resvg-test-suite/README.md). - Versions: resvg and usvg 0.48.1 (crates.io, 2026-08-02).
Mechanism
Fixture: tests/<category>/<feature>/<case>.svg (viewBox 0 0 200 200, one issue per file)
Reference: same path with .png, rendered by resvg at width 300 with pinned fonts, oxipng-optimised
Harness (integration/main.rs):
tree = usvg::Tree::from_data(svg, Options { fontdb: pinned, resources_dir: fixture dir })
size = tree.size().scale_to_width(300); pixmap = render(tree, scale transform)
actual = demultiply_alpha(pixmap)
diff = count of pixels where (not both alpha == 0) and any channel |a - b| > 1
assert diff == 0 # generated by gen-tests.py
MAKE_REF=1 -> write/overwrite reference; failures write tests/diffs/<name>.png (expected | mask | actual)
Conformance matrix:
for each implementation impl and fixture f: result[impl][f] in {passed, failed, crashed, unknown, undefined}
aggregate per feature row and per implementation column
NUIF relevance
Borrow
- Adopt the single-issue fixture discipline (fixed canvas, unique title, one feature per file, category/feature/case paths) for the NUIF
rendersuite, because it makes failures attributable and the corpus enumerable as a feature matrix. - Adopt the exact-comparison harness pattern (threshold 1, zero differing pixels, three-panel diff artefact, explicit regenerate flag) for the CPU reference path, because resvg demonstrates cross-platform pixel identity with this policy.
- Adopt the support-table pattern (implementations × features with a five-state legend) as the public conformance matrix for NUIF profiles, because it communicates partial support without collapsing to a single score.
Adapt
- Apply the usvg idea of a lowered, fully resolved tree to NUIF’s resolved-snapshot layer, because conformance fixtures should compare the resolved scene as well as the raster; NUIF must keep authored intent alongside, which usvg discards.
- Keep separate “reference” and “regression” raster sets, because NUIF needs hand-verified references for the draft specification and implementation-rendered snapshots for regression detection.
Reject
- Do not exclude fixtures silently for SIMD rounding as
gen-tests.pydoes; NUIF should instead pin the reference path’s arithmetic (scalar or SIMD with identical rounding) or move the fixture to the tolerance tier with a recorded reason. - Do not adopt manual
vdifftriage as the source of truth for the matrix, because NUIF’s matrix must be produced by the automated suite.
Open questions
- Whether resvg-test-suite’s MIT-licensed SVG corpus can be reused directly as NUIF import fixtures for the SVG adapter, and how to attribute it.
- How NUIF should treat “undefined behavior” cells; resvg marks them but the NUIF spec would need an explicit undefined-behaviour category.
Retentive Lenses
Document status:
reviewed. Canonical source.
Summary
Retentive lenses strengthen ordinary lens laws by requiring unchanged regions of a view to preserve corresponding source regions when other parts change. The work demonstrates tree transformation and resugaring use cases where provenance/correspondence enables minimal retention.
NUIF relevance
This is a direct theoretical foundation for design↔source synchronization. NUIF adapters should maintain correspondence maps and source provenance so an edit to one property does not regenerate unrelated source regions.
Reverse engineering flexible GUI layouts from observations
Document status:
reviewed. Canonical source.
Summary
ReverseORC demonstrates that responsive layout intent can be inferred more reliably by sampling the same UI at multiple sizes and fitting flexible constraint specifications, rather than trying to infer a layout manager from one fixed screenshot. Earlier layout-inference work similarly uses relative-position graphs and graph rewriting to recover higher-level layout structures.
NUIF relevance
Foreign imports should preserve observations and inference confidence. When authored intent is unavailable, adapters should infer from multiple evaluation contexts where possible and label reconstructed constraints as inferred rather than pretending they are lossless source semantics.
Executable boundary
nuif-layout-inference-0 is a deliberately bounded geometric implementation
of that research direction. It ranks five candidate families from 360/768 px
live-browser observations without consulting the 900 px holdout, then reports
the holdout error and all alternatives. On the current fixture its selected
constraint scores 0.0626 versus 0.2918 for fixed freeform. This is useful
falsification evidence for the mechanism, not evidence that the inferred
family is the author’s original program or that the result generalizes to a
corpus. The report therefore retains raw uncalibrated confidence, source
observation identities and the inferred evidence class.
Rust SDK facade and staged C, Swift and Kotlin binding boundary
Document status:
verified. Canonical source.
Summary
The safe common denominator for every integration is not a Rust struct graph or an MCP server. It is a bounded byte-oriented SDK façade over the canonical codecs, deterministic package, semantic operations and diagnostics. Native Rust callers use it directly; WASM, CLI, MCP and later foreign-language bindings translate only their transport and ownership conventions.
Rust’s native ABI is explicitly unstable. extern "C" uses the target’s C
calling convention, but a usable library must still define representations,
allocation ownership, destructors, errors, panic behavior, threads and symbols.
cbindgen generates C/C++ headers from an existing public C API; it does not
design or prove that API. UniFFI generates a shared-library FFI layer and
high-level Swift/Kotlin/Python/Ruby bindings from an object model and is used in
Firefox, but its own guide says shipping platform artifacts remains the user’s
responsibility. UniFFI is production-used yet pre-1.0, and 0.31 changed its
generator command and binding checksums.
NUIF should therefore stabilize in layers. First, make
nuif-api::NuifDocument authoritative for explicit text/CBOR load, validation,
typed operations, canonical hashes, export and verified package/resource
retention. Make WASM delegate to it and compare surfaces. Only after the
semantic API and error classes have a compatibility baseline should a small
separate unsafe C ABI be reviewed. cbindgen is appropriate for C/C++ headers;
UniFFI is the preferred Swift/Kotlin generator. Their generated packages and
versions remain separate from the editor.
Evidence
- The Rust Reference says the Rust ABI offers no stability guarantees and
defines
unsafe extern "C"as matching the dominant C compiler’s ABI for the target. Locator: The Rust Reference, External blocks, “ABI”, Rust 1.98, retrieved 2026-08-31: https://doc.rust-lang.org/reference/items/external-blocks.html#abi. - Rust 1.98 documents exported C symbols with
#[unsafe(no_mangle)] pub extern "C" fnand describes foreign interfaces as inherently unsafe, normally wrapped by safe Rust code. Locator: standard-libraryexternkeyword documentation, retrieved 2026-08-31: https://doc.rust-lang.org/stable/core/keyword.extern.html. - The Embedded Rust Book specifies
cdylib/staticlib, explicitextern "C"and generated or handwritten headers as the normal Rust-to-C/C++ path; C is used because neither Rust nor C++ supplies the needed cross-language stable ABI. Locator: A little Rust with your C, retrieved 2026-08-31: https://doc.rust-lang.org/stable/embedded-book/interoperability/rust-with-c.html. - cbindgen generates C and C++11 headers from Rust crates that already expose a
public C API. Its README says generation reflects Rust layout/ABI guarantees,
while also warning that project support is ad hoc and particular constructs
may be unsupported. Locator: cbindgen README,
master, retrieved 2026-08-31: https://github.com/mozilla/cbindgen. - UniFFI compiles Rust components into shared libraries and generates bindings to load them. Its first-party languages are Kotlin, Swift, Python and Ruby; Mozilla reports extensive Firefox mobile and desktop use. The project calls itself production-ready but far from 1.0. Locator: UniFFI README and user guide overview, retrieved 2026-08-31: https://github.com/mozilla/uniffi-rs and https://mozilla.github.io/uniffi-rs/latest/.
- The UniFFI guide explicitly says it generates bindings but does not help ship the Rust library to target platforms. Swift output includes a C header and module map around the shared library. Locator: guide overview, binding generation and Swift overview, retrieved 2026-08-31: https://mozilla.github.io/uniffi-rs/latest/bindings.html and https://mozilla.github.io/uniffi-rs/latest/swift/overview.html.
- UniFFI 0.31.0 removed prior generator types, changed command usage and changed
method checksums incompatibly with 0.30-generated bindings; 0.31.2 fixed
Kotlin ARM32 return conversion and Swift boundary defects. This supports
pinning generator/runtime pairs and testing generated consumers rather than
treating generated source as timeless. Locator: UniFFI
CHANGELOG.md, 0.31.0–0.31.2, retrieved 2026-08-31: https://github.com/mozilla/uniffi-rs/blob/main/CHANGELOG.md. - Executable NUIF evidence:
nuif-apitests load text and CBOR, apply the same typed transaction, compare canonical hashes, undo/redo and retain package metadata through a byte fixpoint.nuif-wasmdelegates those semantics to the façade, andcargo xtask gate-wasmcross-checks native and generated browser/Node output. Criterion addssdk/direct_documentload/export surfaces. Locator: linked code at revision containing this record.
Mechanism
The direct SDK owns a Session and an optional decoded NuifPackage. Bare
inputs enter through an explicit DocumentEncoding and remain diagnosable even
when structurally invalid. Package inputs enter through the package decoder,
then hand shared digest-verified embedded buffers to the session. Semantic
operations mutate only the session document. Package export clones the retained
package envelope, replaces its document and requested mode, and revalidates the
complete manifest/resource policy before writing bytes.
The WASM object stores this SDK object. Its remaining work is bounded JSON patch decoding and JavaScript error translation. Stateless MCP tools load the same SDK object per request while retaining their protocol framing and JSON Schema boundary. Direct API tests require matching hashes across text and CBOR, replayable operation preconditions, exact undo/redo and a package write/read/write fixpoint after editing. The generated Node/browser packages and live MCP subprocess remain checked against native canonical output.
The future foreign ABI remains one layer farther out:
C / C++ / Swift / Kotlin
│ generated wrapper and owned byte buffers
▼
separately reviewed nuif-ffi
│ NuifDocument byte records and typed error classes
▼
nuif-api
No internal Document, Entity, Rust enum layout, allocator pointer or panic
may cross that ABI by accident.
NUIF relevance
Borrow the compiler-style single façade, explicit C calling convention, cbindgen header generation and UniFFI Swift/Kotlin generation. Adapt them to NUIF’s byte records, typed errors, bounded inputs and independent profile versioning. Reject direct exposure of internal model structs, duplicated business logic in wrappers, a hand-written Swift/Kotlin ownership layer before UniFFI is evaluated, and any claim that generated bindings alone constitute a shippable SDK.
Promotion checklist
- semantic API and stable error-code registry leave
0.0.x; - separate
nuif-ffiunsafe-code review and panic-containment proof; - opaque handles plus allocator-matched bytes and destructors;
- cbindgen header/symbol compatibility diff;
- native C consumer under AddressSanitizer and UndefinedBehaviorSanitizer;
- pinned UniFFI generator/runtime with Swift and Kotlin consumer tests;
- target-specific XCFramework/Swift package and AAR artifacts with manifests, checksums, SBOMs and attestations;
- independent versions from the editor, WASM module and MCP binary.
Open questions
- Which semantic-API milestone is strong enough to register stable foreign error numbers and start ABI compatibility checks?
- Whether the first native package target should be an Apple XCFramework or an Android AAR; demand and a maintained live consumer should decide ordering.
- Whether a plain C ABI plus cbindgen is needed independently from UniFFI’s generated low-level C layer, or whether C/C++ adoption can wait for a named host requirement.
- Whether a measured Node workload ever justifies a native Node-API addon over the already conforming WebAssembly package.
Rust snapshot, property-based, fuzzing, mutation, runner, benchmark and coverage tooling for round-trip trials
Document status:
reviewed. Canonical source.
Summary
The Rust ecosystem provides one maintained tool per testing technique named in conformance/PLAN.md: insta for golden structural snapshots, proptest (with proptest-state-machine) and quickcheck for property-based tests over operation sequences, cargo-fuzz with arbitrary (or bolero as a unified front end) for structure-aware fuzzing of parsers and codecs, kani for bounded model checking of small pure kernels, cargo-mutants for measuring whether tests detect behaviour changes, cargo-nextest as a per-test-process runner with retries and JUnit output, criterion or divan for benchmarks, and cargo-llvm-cov for coverage. loom addresses concurrency interleavings only and is not required by a single-threaded engine. All versions and entry points below were verified against repositories or crates.io on 2026-08-29.
NUIF interpretation: a round-trip trial (encode, decode, canonicalise, apply operations, invert, replay, layout, render, diff) maps onto these tools as generator (proptest/arbitrary), oracle (insta snapshots, reference state machine, canonical-form equality), reducer (proptest shrinking, cargo fuzz tmin), runner (nextest with process isolation and JUnit), and adequacy metrics (cargo-mutants, cargo-llvm-cov).
Evidence
- insta 1.48.0 (2026-06-11): macros
assert_snapshot!,assert_debug_snapshot!, and serde-backedassert_json_snapshot!/yaml/ron/csv/tomlbehind features; featuresredactions,filters,glob;INSTA_UPDATEmodesauto(default,nounder CI),new,always,unseen,no,force;.snap/.snap.newfiles; inline snapshots viacargo-insta;Settingsfor redactions and snapshot paths. Locator:insta/src/lib.rslines 81-190, 206-250;CHANGELOG.md“1.48.0”; crates.io. - insta 1.48.0 added
strip_ansi_escape_codesand lets explicit--acceptoverrideCI=truecheck mode;cargo insta test --profileforwards to nextest as--cargo-profile. Locator:CHANGELOG.mdlines 6-20. - proptest 1.11.0 (2026-03-24): “generation and shrinking is defined on a per-value basis instead of per-type”; README states MSRV 1.86 and a policy of at most
<current stable> - 7; the crate “mainly sees passive maintenance”. Locator:proptest/README.md“Status of this crate”, “MSRV”;proptest/CHANGELOG.md“Unreleased”. - proptest-state-machine 0.8.0 (2026-03-24):
ReferenceStateMachine { type State; type Transition; fn init_state() -> BoxedStrategy<State>; fn transitions(&State) -> BoxedStrategy<Transition>; fn apply(State, &Transition) -> State; fn preconditions(&State, &Transition) -> bool };StateMachineTest { type SystemUnderTest; type Reference: ReferenceStateMachine; fn init_test(&RefState) -> SUT; fn apply(SUT, &RefState, Transition) -> SUT; fn check_invariants(&SUT, &RefState); fn teardown(SUT); fn test_sequential(...) };prop_state_machine!macro; shrinking deletes transitions from the end, then shrinks transitions front to back, then the initial state. 0.8.0 addedSend + Syncbounds tostrategy::Sequential. Locator:proptest-state-machine/src/strategy.rslines 45-81;src/test_runner.rslines 19-177;CHANGELOG.md; proptest book “State Machine testing”. - quickcheck 1.1.0 (2026-02-10): per-type
Arbitrary,quickcheck!macro and#[quickcheck]attribute;QUICKCHECK_TESTS,QUICKCHECK_MAX_TESTS,QUICKCHECK_MIN_TESTS_PASSED; the README states that proptest “improves on the concept of shrinking”. Locator:README.mdlines 27-160, 270-276. - cargo-fuzz 0.13.2 (2026-06-09): subcommands
init,add,run,fmt,tmin,cmin,coverage,list; requires nightly, libFuzzer, x86-64/AArch64 on Unix;fuzzdirectory must be added toworkspace.membersor created as its own workspace (--fuzzing-workspace=true); generatedfuzz/Cargo.tomlcontains[package.metadata] cargo-fuzz = trueand targets atfuzz_targets/<name>.rs; crashes are written underfuzz/artifacts/<target>/crash-<hash>. Locator:README.md;src/templates.rslines 9-49; rust-fuzz bookcargo-fuzz/tutorial.mdlines 21-84. - arbitrary 1.4.2 (2025-08-14):
#[derive(Arbitrary)]with featurederive; per-field attributes such as#[arbitrary(default)];fuzz_target!(|input: T| ...)accepts anyArbitrarytype; the book shows gating the derive behind an optionalarbitraryfeature in the main crate. Locator:arbitrary/README.md; rust-fuzz bookstructure-aware-fuzzing.mdlines 256-311. - bolero 0.13.4 (2025-07-03):
bolero::check!().with_type().cloned().for_each(|v| ...), run undercargo testorcargo bolero test <name>with libFuzzer/AFL/honggfuzz engines; Linux needsbinutils-dev libunwind-dev. Locator:README.md. - kani-verifier 0.67.0 (2026-01-16):
cargo install --locked kani-verifier && cargo kani setup; harness#[kani::proof]withkani::any()andkani::assume(); “bit-precise model checker for Rust” checking panics, overflow, UB and assertions; Linux and macOS. Locator:README.md;docs/src/tutorial-first-steps.mdlines 33-174. - loom 0.7.2 (2024-04-23): permutes concurrent executions under the C11 memory model; enabled via
[target.'cfg(loom)'.dependencies]. Locator:README.md. - cargo-mutants 27.1.0 (2026-06-02):
cargo mutants,-f <file>; works withcargo testorcargo nextest runon “non-flaky tests”; CI guidance: PR-diff mode,--in-place, GitHub annotations (--annotations=github), install viainstall-action. Locator:README.md;book/src/ci.md. - cargo-nextest 0.9.143 (2026-08-04): list phase builds with
cargo test --no-runand lists tests; run phase “executes each individual test in a separate process, in parallel”;--retries Nmarks recovered tests “flaky” (exit code 0 by default; configurable failure since 0.9.131);-jN/--test-threads=N;--no-fail-fast;--run-ignored=only|all; JUnit via[profile.ci.junit] path = "junit.xml"in.config/nextest.toml, written totarget/nextest/ci/junit.xml; custom harnesses must support--list --format terseprinting<name>: test. Locator:site/src/docs/design/how-it-works.md;features/retries.mdlines 8-41;machine-readable/junit.mdlines 9-30;design/custom-test-harnesses.md;running.mdlines 86-132. - criterion 0.8.2 (2026-02-04): bench target with
harness = false,criterion_group!/criterion_main!, featurehtml_reports, gnuplot optional. Locator:README.mdlines 44-76;CHANGELOG.md. - divan 0.1.21 (2025-04-10) requires Rust 1.80.0;
divan::main()in aharness = falsebench and#[divan::bench]attributes. Locator:README.md“Getting Started”. - cargo-llvm-cov 0.9.0 (2026-08-16):
cargo llvm-cov [--lcov|--json|--codecov|--html|--text] [--output-path],cargo llvm-cov nextest,cargo llvm-cov report,--fail-under-lines <MIN>,--branchand--doctests(nightly),clean --workspacerecommended before mixed runs. Locator:README.mdlines 12-15, 57-445. - wasm-bindgen-test coverage requires nightly
-Cinstrument-coverage -Zno-profiler-runtimeandcfg(wasm_bindgen_unstable_test_coverage)and can feedcargo +nightly llvm-cov. Locator: wasm-bindgenguide/src/wasm-bindgen-test/coverage.mdlines 3-46.
Mechanism
Role of each tool in a round-trip trial loop:
| Stage | Tool | Entry point | Output consumed by |
|---|---|---|---|
| generate documents and operation sequences | proptest, arbitrary | Strategy, #[derive(Arbitrary)] on Document/Operation | apply/oracle |
| model-based sequence check | proptest-state-machine | ReferenceStateMachine, StateMachineTest, prop_state_machine! | invariants, shrinker |
| structural golden oracle | insta | assert_snapshot!(canonical_text), assert_json_snapshot! with redactions for volatile IDs | review via cargo insta |
| parser/codec robustness | cargo-fuzz, bolero | `fuzz_target!( | d: &[u8] |
| bounded proof of pure kernels | kani | #[kani::proof] over ID allocation, canonical ordering | CI (Linux) |
| adequacy | cargo-mutants, cargo-llvm-cov | cargo mutants --in-diff, cargo llvm-cov nextest --lcov | thresholds --fail-under-lines |
| execution and reporting | cargo-nextest | cargo nextest run --profile ci --retries 0 | target/nextest/ci/junit.xml |
| timing | criterion or divan | harness = false bench targets | regression tracking |
Sketch of a proptest-state-machine test over a NUIF tree document (interpretation; types from crates/nuif-core and crates/nuif-protocol):
#![allow(unused)]
fn main() {
use proptest::prelude::*;
use proptest_state_machine::{ReferenceStateMachine, StateMachineTest, prop_state_machine};
// Reference model: a minimal ordered forest keyed by EntityId.
#[derive(Clone, Debug)]
struct RefTree { parent: BTreeMap<EntityId, Option<EntityId>>, order: BTreeMap<Option<EntityId>, Vec<EntityId>> }
struct RefMachine;
impl ReferenceStateMachine for RefMachine {
type State = RefTree;
type Transition = Operation; // Insert, Remove, Move, Rename, SetExtension
fn init_state() -> BoxedStrategy<RefTree> { Just(RefTree::single_root()).boxed() }
fn transitions(s: &RefTree) -> BoxedStrategy<Operation> {
let ids = s.parent.keys().copied().collect::<Vec<_>>();
prop_oneof![
(any::<Entity>(), sample(ids.clone()), 0usize..8).prop_map(|(e, p, i)| Operation::Insert { parent: Some(p), index: i, entity: e }),
(sample(ids.clone()), sample(ids.clone()), sample(ids.clone())).prop_map(|(e, p, after)| Operation::Move { entity: e, new_parent: Some(p), anchor: Anchor::After(after) }),
sample(ids.clone()).prop_map(|e| Operation::Remove { entity: e }),
].boxed()
}
fn preconditions(s: &RefTree, t: &Operation) -> bool {
match t { Operation::Move { entity, new_parent, .. } => !s.is_ancestor_or_self(*entity, *new_parent), // no cycles
Operation::Remove { entity } => !s.is_root(*entity), _ => true }
}
fn apply(mut s: RefTree, t: &Operation) -> RefTree { s.apply_reference(t); s }
}
struct EngineTest;
impl StateMachineTest for EngineTest {
type SystemUnderTest = (PrototypeEngine, Document);
type Reference = RefMachine;
fn init_test(_: &RefTree) -> Self::SystemUnderTest { (PrototypeEngine::default(), Document::single_root()) }
fn apply((mut engine, mut doc): Self::SystemUnderTest, _: &RefTree, op: Operation) -> Self::SystemUnderTest {
let patch = Patch { base_revision: None, transactions: vec![Transaction { id: 1, operations: vec![op.clone()] }] };
let before = doc.clone();
engine.apply(&mut doc, &patch).expect("precondition-satisfying op applies");
let inverse = engine.invert(&before, &patch); // hypothetical; QA item 3
let mut replay = before.clone();
engine.apply(&mut replay, &patch).unwrap();
assert_eq!(canonical(&replay), canonical(&doc)); // deterministic replay
let mut undone = doc.clone();
engine.apply(&mut undone, &inverse).unwrap();
assert_eq!(canonical(&undone), canonical(&before)); // inversion
(engine, doc)
}
fn check_invariants((_, doc): &Self::SystemUnderTest, r: &RefTree) {
assert_eq!(doc.children_order(), r.order); // model equivalence
assert!(doc.is_acyclic());
assert_eq!(decode(&encode(doc)), *doc); // codec round trip
assert_eq!(canonical(&decode(&encode(doc))), canonical(doc));
}
}
prop_state_machine! {
#![proptest_config(ProptestConfig { cases: 256, .. ProptestConfig::default() })]
#[test]
fn engine_matches_reference_tree(sequential 1..40 => EngineTest);
}
}
Failure handling: proptest shrinks by dropping trailing transitions, then shrinking individual transitions, then the initial state; the surviving minimal sequence can be serialised as a conformance fixture (QA item 9). For byte-level codecs, cargo fuzz tmin and cmin reduce inputs and corpora. nextest executes each case in its own process, so a panic or abort in one fixture cannot poison others, and --retries 0 in the CI profile makes flakiness a failure rather than a warning.
NUIF relevance
Borrow
- insta with redactions for canonical-text and
RenderScenesnapshots, because theDocument,LayoutSnapshotandRenderScenetypes are plain data and text snapshots are reviewable in pull requests. - proptest-state-machine as the operation-sequence property engine for
conformance/operations, because its reference/SUT split matches the “deterministic operation replay” and “inversion” techniques listed in conformance/PLAN.md. - cargo-fuzz plus
#[derive(Arbitrary)]behind an optionalarbitraryfeature onnuif-core/nuif-codec, because the security suite requires fuzzing parsers and path geometry. - cargo-nextest with a
ciprofile (JUnit path,--retries 0,--no-fail-fast) as the runner, because QA item 10 requires one machine-readable report per run.
Adapt
- Snapshot metadata must include implementation version, capability profile, fixture ID and evaluation context (conformance/PLAN.md), which insta does not model; embed them in the snapshot content or in
Settings::set_info. - kani is limited to small bounded harnesses (unwinding); apply it to ID allocation, canonical ordering comparators and cycle checks, not to layout or rendering.
- cargo-mutants runs should be restricted to
--in-diffon pull requests and full runs on a schedule, because full mutation runs of a layout engine are slow.
Reject
- loom as a default dependency, because the engine is single-threaded by design and loom targets memory-model interleavings.
- Nightly-only tools (cargo-fuzz,
--branchcoverage, cargo-udeps) in the required CI matrix, because the toolchain is pinned to stable 1.85.0; run them in an optional nightly job. - quickcheck for new tests, because its own README defers to proptest for shrinking and the project’s per-type
Arbitraryconflicts with per-value strategies needed for constrained trees.
Open questions
- Whether proptest’s stated MSRV (1.86 on main) is already in effect for 1.11.0, which would exceed the NUIF pin.
- Whether libFuzzer-based fuzzing is acceptable in CI given the nightly requirement, or whether bolero’s
cargo testmode with its built-in generator suffices for the security suite. - Whether insta binary snapshots are appropriate for small PNG references or whether image references should stay outside insta (as egui and Masonry do).
- Whether a
PrototypeEnginewithinvertwill exist innuif-api; the trait currently exposesapply,layout,build_render_sceneonly.
Rust toolchain pin and minimum supported Rust version policy for the NUIF workspace
Document status:
reviewed. Canonical source.
Summary
Rust stable 1.98.0 was released on 2026-08-20; twelve stable minor releases separate it from the 1.85.0 pinned in the NUIF workspace (2025-02-20), at a fixed interval of 42 days. Cargo’s rust-version field declares a minimum supported Rust version (MSRV); raising it “is assumed to be a minor incompatibility” and, under resolver version 3 (the edition 2024 default for packages), the resolver prefers dependency versions whose rust-version is at or below the declaring package’s value. The Cargo book lists N-2, even releases and calendar-year windows as example policies. Comparable projects split into two groups: application-scale projects track the latest stable (rust-analyzer rust-version = "1.98", Zed and Servo pin 1.97.1, Bevy states the MSRV “is generally close to the latest stable release”), while library projects hold an older MSRV and treat bumps as breaking (wgpu 1.87 for the crate, at most stable minus 3 for the repository) or as rolling with a six-month floor (Tokio). Linebender crates state that MSRV increases are not breaking changes. The highest rust-version in the planned dependency graph is 1.96 (Masonry main); the engine dependencies require at most 1.88 (Vello, Parley, icu_properties), 1.87 (wgpu) or 1.85 (AccessKit, HarfRust, proptest, Skrifa).
NUIF interpretation: the workspace pins 1.98.0 in rust-toolchain.toml and declares rust-version = "1.96" for every crate, with the policy “toolchain equals latest stable, bumped in a dedicated commit within one release cycle; MSRV equals the toolchain minus two minor versions or the highest dependency MSRV, whichever is greater, re-evaluated at every toolchain bump”. The workspace manifest switches to resolver = "3" so that MSRV-aware resolution applies, and CI adds an MSRV job at 1.96.0.
Evidence
Rust releases
- Stable releases from 1.85 (RELEASES.md, rust-lang/rust master): 1.85.0 (2025-02-20), 1.85.1 (2025-03-18), 1.86.0 (2025-04-03), 1.87.0 (2025-05-15), 1.88.0 (2025-06-26), 1.89.0 (2025-08-07), 1.90.0 (2025-09-18), 1.91.0 (2025-10-30), 1.91.1 (2025-11-10), 1.92.0 (2025-12-11), 1.93.0 (2026-01-22), 1.93.1 (2026-02-12), 1.94.0 (2026-03-05), 1.94.1 (2026-03-26), 1.95.0 (2026-04-16), 1.96.0 (2026-05-28), 1.96.1 (2026-06-30), 1.97.0 (2026-07-09), 1.97.1 (2026-07-16), 1.98.0 (2026-08-20). Consecutive
.0releases are 42 days apart (for example 2026-07-09 to 2026-08-20). Locator: https://raw.githubusercontent.com/rust-lang/rust/master/RELEASES.md, headings “Version 1.85.0” to “Version 1.98.0”; release blog https://blog.rust-lang.org/2026/08/20/Rust-1.98.0/ (HTTP 200) and https://blog.rust-lang.org/2025/02/20/Rust-1.85.0/ (HTTP 200). - Current stable channel:
channel-rust-stable.tomlhasdate = "2026-08-20"and rustcversion = "1.98.0 (88d9e12ae 2026-08-18)". By the 42-day cadence 1.99.0 falls on 2026-10-01 (computed; the release calendar page did not render and was not verified). Locator: https://static.rust-lang.org/dist/channel-rust-stable.toml. - The machine used for this retrieval has rustup toolchains
stable(1.97.1 at retrieval),1.85.0and1.95(rustup toolchain list).
Cargo semantics
rust-version“must be a bare version number with at least one component; it cannot include semver operators or pre-release identifiers”; “Changingrust-versionis assumed to be a minor incompatibility”; the section “Selecting supported Rust versions” lists “N-2: latest version with a 2 release grace window for updating”, “Every even release with a 2 release grace window” and “Every version from this calendar year with a one year grace window”. Locator: Cargo book “Rust Version” (reference/rust-version.html), sections “Setting and Updating Rust Version” and “Selecting supported Rust versions”; semver reference anchorenv-new-rust.- Resolver: with
resolver.incompatible-rust-versions = "fallback"“the resolver will prefer packages with a Rust version that is less than or equal to your own Rust version”; if no compatible version satisfies the requirement “the resolver won’t error but will instead pick a version”; with mixed workspace MSRVs the resolver “may pick a lower dependency version than necessary” or “too high of a version”. The default isallowfor resolver versions 1 and 2 andfallbackfor resolver version 3, which edition 2024 packages default to. Locator: Cargo book “Dependency Resolution”, section “Rust version”; “Resolver versions”. - Virtual manifests must set
resolverexplicitly (Cargo book “Workspaces”, cited innuif:research:cargo-workspace-xtask-and-ci-layout). The NUIF root manifest setsresolver = "2",edition = "2024",rust-version = "1.85";rust-toolchain.tomlsetschannel = "1.85.0",components = ["clippy", "rustfmt"],profile = "minimal"; CI pinsdtolnay/rust-toolchainto 1.85.0.cargo metadata --lockedreports zero packages outside the workspace. Locator:Cargo.tomllines 1-18;rust-toolchain.toml;.github/workflows/ci.ymllines 44-46;cargo metadata, 2026-08-29.
Dependency MSRVs (crates.io rust_version of the newest non-yanked version, 2026-08-29)
| Crate | Version (date) | MSRV | Licence |
|---|---|---|---|
| masonry (release) | 0.4.0 (2025-10-29) | 1.88 | Apache-2.0 |
| masonry (main b81d8d7) | unreleased (2026-08-28) | 1.96 | Apache-2.0 |
| imaging, imaging_vello_cpu | 0.0.1 (2026-05-21), 0.0.2 (2026-05-30) | 1.92 | Apache-2.0 OR MIT |
| vello | 0.10.0 (2026-08-14) | 1.88 (0.8.0 was 1.92, 0.9.0 1.88) | Apache-2.0 OR MIT |
| vello_cpu, vello_hybrid, vello_common | 0.2.0 (2026-08-07) | 1.88 (0.0.7 was 1.92) | Apache-2.0 OR MIT |
| parley, fontique | 0.11.1 (2026-08-16) | 1.88 | Apache-2.0 OR MIT |
| skrifa | 0.46.2 (2026-08-21) | 1.85 (0.45.x was 1.89) | MIT OR Apache-2.0 |
| peniko, kurbo | 0.6.1 (2026-05-15), 0.13.1 (2026-05-13) | 1.85 | Apache-2.0 OR MIT |
| wgpu | 30.0.1 (2026-08-22) | 1.87.0 | MIT OR Apache-2.0 |
| accesskit, accesskit_consumer, accesskit_winit | 0.25.0, 0.39.0, 0.34.0 (2026-08-29) | 1.85 | MIT OR Apache-2.0; winit adapter Apache-2.0 |
| taffy | 0.14.0 (2026-08-24) | 1.71 | MIT |
| harfrust | 0.13.3 (2026-08-25) | 1.85 | MIT |
| rustybuzz | 0.20.1 (2024-11-12) | none declared | MIT |
| harfbuzz_rs | 2.0.1 (2021-08-28) | none declared | MIT |
| insta | 1.48.0 (2026-06-11) | 1.66.0 | Apache-2.0 |
| proptest | 1.11.0 (2026-03-24) | 1.85 | MIT OR Apache-2.0 |
| libtest-mimic | 0.8.2 (2026-03-16) | 1.65 | MIT/Apache-2.0 |
| ciborium | 0.2.2 (2024-01-24) | 1.58 | Apache-2.0 |
| minicbor | 2.3.0 (2026-07-23) | none declared | BlueOak-1.0.0 |
| winit | 0.30.13 (2026-03-02) | 1.70.0 | Apache-2.0 |
| egui, egui_kittest | 0.36.1 (2026-08-07) | 1.95 | MIT OR Apache-2.0 |
| kittest | 0.4.0 (2026-03-24) | 1.92 | MIT OR Apache-2.0 |
| blitz-dom | 0.3.0-beta.2 (2026-08-24) | 1.89.0 | MIT OR Apache-2.0 |
| floem (main) | 0.2.0 | 1.91 | MIT |
| cargo-deny | 0.20.2 (2026-07-09) | 1.88.0 | MIT OR Apache-2.0 |
| icu_properties, libloading, glifo, tree_arena | 2.3.0, 0.9.0, 0.3.0, 0.2.0 | 1.88 | Unicode-3.0; ISC; Apache-2.0 OR MIT; Apache-2.0 |
| fearless_simd | 0.7.0 | 1.89 | Apache-2.0 OR MIT |
Locator: crates.io API /api/v1/crates/<name> fields versions[].rust_version, versions[].license, versions[].created_at. The resolution probe in nuif:research:masonry-editor-stack-decision confirms 1.96 as the highest rust-version in the combined graph, followed by 1.92.
Masonry’s own MSRV practice (interpretation from the release dates above): 0.3.0 (2025-05-10) required 1.86 while stable was 1.86; 0.4.0 (2025-10-29) required 1.88 while stable was 1.90; main (2026-08-28) requires 1.96 while stable is 1.98. The observed window is therefore between N-0 and N-2 at release time.
Comparable project policies
- rust-analyzer:
Cargo.tomlrust-version = "1.98",edition = "2024"; norust-toolchain.tomlin the repository (HTTP 404). Locator: rust-lang/rust-analyzerCargo.tomllines 7-8, master. - Zed:
rust-toolchain.tomlchannel = "1.97.1", components rustfmt, clippy, rust-analyzer, rust-src, targetswasm32-wasip2,wasm32-unknown-unknown,x86_64-unknown-linux-musl; GPUI README: “You’ll also need to use the latest version of stable Rust”. Locator: zed-industries/zedrust-toolchain.toml;crates/gpui/README.mdline 8, main. - Servo:
rust-toolchain.tomlchannel = "1.97.1"with a comment listing the other files to update at each bump (shell.nix,support/crown/rust-toolchain.toml,.devcontainer/Dockerfile). Locator: servo/servorust-toolchain.toml, main. - Bevy:
Cargo.tomlrust-version = "1.96.0"; README: “Bevy relies heavily on improvements in the Rust language and compiler. As a result, the Minimum Supported Rust Version (MSRV) is generally close to ‘the latest stable release’ of Rust.” Locator: bevyengine/bevyCargo.tomlline 13;README.mdlines 17-18, main. - wgpu: “If you’re using
wgpu, our MSRV is 1.87. If you’re running our tests or examples, our MSRV is 1.93.”; “We will avoid bumping the MSRV ofwgpuwithout good reason, and such a change is considered breaking.”; “This version can only be upgraded in breaking releases, though we release a breaking version every three months.”; “The repository MSRV should never require an MSRV higher thanstable - 3”; thewgpucrate MSRV is bounded by Servo’s andwgpu-coreby Firefox’s. Locator: gfx-rs/wgpuREADME.mdlines 98-125, trunk. - Vello: “This version of Vello has been verified to compile with Rust 1.88 and later. Future versions of Vello might increase the Rust version requirement. It will not be treated as a breaking change and as such can even happen with small patch releases.” Masonry’s README carries the same wording at 1.96. Locator: linebender/vello
README.mdlines 223-228; linebender/xilemmasonry/README.mdlines 209-213. - Tokio: “Tokio will keep a rolling MSRV (minimum supported rust version) policy of at least 6 months. When increasing the MSRV, the new Rust version must have been released at least six months ago. The current MSRV is 1.71.”; “the MSRV is not increased automatically, and only as part of a minor release”. Locator: tokio-rs/tokio
README.mdlines 173-190, master.
Mechanism
Policy expressed as two variables and one invariant:
T := toolchain pinned in rust-toolchain.toml and CI # 1.98.0 on 2026-08-29
D := max(rust-version over the resolved dependency graph) # 1.96 (Masonry main)
M := max(T - 2 minor versions, D) # 1.96
invariant: D <= M <= T; every crate declares rust-version = M; CI checks M and T
Update procedure: a stable release of Rust triggers one commit that raises T (toolchain file, CI matrix, report engine.toolchain field) and recomputes M; a dependency whose MSRV exceeds M is held at its previous version until the next T bump unless the bump is needed for a defect fix, in which case M rises with it. Because Cargo classifies a rust-version change as a minor incompatibility, NUIF crates may raise M in minor releases and record the change in the changelog, following Tokio’s practice of bumping only at a release boundary. resolver = "3" makes the resolver prefer versions compatible with M when the workspace is resolved under T, so Cargo.lock does not drift above M silently; cargo hack check --workspace --rust-version on a 1.96.0 toolchain verifies the invariant.
Why not toolkit-minimum only: D follows Masonry’s N-0 to N-2 practice, so a policy of M := D would be indistinguishable from N-2 in the common case but would leave M undefined when the editor crate is absent from a build. Why not latest stable for M: independent implementers and distribution packagers build the engine crates, whose own requirement is at most 1.88; a two-cycle window costs nothing measurable in language features for those crates and keeps cargo install nuif-cli possible on the previous two stable releases.
NUIF relevance
Borrow
- The Cargo book’s N-2 window as the MSRV rule, because it is the first example policy in the reference and matches Masonry’s observed practice.
- Zed’s and Servo’s pattern of one toolchain file with an exact patch version, because reproducibility of snapshots and reports depends on an exact compiler (
conformance/HARNESS.mdrecordsengine.toolchain). - wgpu’s split between crate MSRV and repository MSRV, transposed as engine crates versus editor crate: both declare
M, but only the editor crate depends on packages atD.
Adapt
rust-toolchain.tomlchanges tochannel = "1.98.0";Cargo.tomlchanges torust-version = "1.96"andresolver = "3";.github/workflows/ci.ymlchanges the pinned toolchain to 1.98.0 and adds anmsrvjob on 1.96.0 runningcargo hack check --workspace --rust-version --locked.- ADR 0006 gating decision 1 records the policy in one sentence so that MSRV bumps occur only in toolchain commits.
Reject
- Keeping 1.85.0, because every GUI candidate and
imagingrequire at least 1.91, andresolver = "2"under 1.85 does not perform MSRV-aware resolution. - Treating MSRV bumps as breaking changes in the wgpu sense, because NUIF is a draft specification with a reference implementation and no downstream crates; Cargo’s minor-incompatibility classification is sufficient.
- Tracking Rust nightly or beta for any job, because reproducible snapshots require a fixed stable compiler.
Open questions
- Whether Masonry’s next release raises its MSRV above 1.96 before the editor crate lands; if so
MfollowsDat that bump. - Whether
cargo hack --rust-versionon a virtual manifest withresolver = "3"reproduces the same lock file as theTtoolchain, or whether a separateCargo.lockfor the MSRV job is needed. - Whether the Rust release calendar confirms 2026-10-01 for 1.99.0 (not verified).
Admission criteria for Protobuf, FlatBuffers and Cap’n Proto as NUIF codecs
Document status:
reviewed. Canonical source.
Summary
A codec benchmark is meaningful only after the candidate represents the full NUIF logical model and survives the format’s correctness obligations. A small generated-schema example can make a schema codec look fast while omitting extensions, version migration, exact integer/real identity, canonical hashes and hostile-input limits. NUIF therefore admits a codec to timing only after it passes exact semantic round trip, canonical encode/decode/encode fixpoint and opaque-data preservation across a neighboring edit.
The implemented nuif-text-0 and nuif-cbor-0 profiles pass that preflight and
are measured at 8, 64, 512 and 4,096 entities. Protobuf and FlatBuffers are not
admitted today. Cap’n Proto is the preferred next schema experiment, not an
accepted NUIF profile: it is the only screened schema codec whose primary
encoding specification defines a schema-agnostic canonical form, but NUIF still
needs a complete mapping, retentive old-reader editing tests, two canonical
writers and calibrated traversal limits before timing it.
Decision criteria
Every candidate must supply one reviewable mapping from every versioned wire field to the semantic model, including explicit unknown kinds and extensions. The mapping must then pass these gates in order:
- exact semantic round trip for the complete responsive-card fixture;
- byte-identical encode/decode/encode and canonicalizer fixpoints;
- exact opaque entity, payload, declarations and document extensions after an unrelated known-property edit;
- rejection or bounded handling of oversized, over-depth and amplification inputs;
- cross-version old-reader/edit/new-reader evidence;
- at least two implementations producing the same canonical bytes;
- only then, size, encode, decode, canonicalize and access measurements over the same corpus and build profile.
Native partial access is reported separately from decode_then_select. The
latter is an honest measurement of today’s full-document decoder followed by a
map lookup; it must not be presented as zero-copy or partial decoding.
Candidate findings
Protocol Buffers
The official Protobuf documentation states that deterministic serialization is not canonical and can vary after schema, build or library changes. It identifies unknown fields as an inherent barrier because a length-delimited unknown value cannot be distinguished as bytes or a nested message without its schema. Proto3 binary messages normally preserve unknown fields, but official guidance also says JSON conversion and field-by-field reconstruction lose them. These properties are useful for message evolution but conflict with NUIF’s stable content hash unless NUIF defines and independently implements a separate canonicalizer. Protobuf is therefore not admitted until a complete schema, canonicalization profile and old-reader retentive edit path exist.
Primary sources:
- https://protobuf.dev/programming-guides/serialization-not-canonical/
- https://protobuf.dev/programming-guides/proto3/#unknown-fields
- https://protobuf.dev/programming-guides/encoding/
FlatBuffers and FlexBuffers
FlatBuffers’ material advantage is direct in-buffer access without constructing an ordinary object graph. Its schema-evolution rules allow an old reader to ignore a newly appended table field, and its verifier and keyed vectors could support bounded lookup. The official internals documentation deliberately leaves table-field and object placement order undefined and explicitly permits different binaries for the same values. Thus the default format is not a canonical hash representation.
Ignoring a future field is sufficient when an old process forwards the original buffer unchanged. It is not evidence that an editor which unpacks, changes a known property and rebuilds the buffer retains that field. This loss statement is an inference from the documented old-reader behavior and must be tested rather than assumed. A NUIF FlatBuffers profile would need a canonical writer and a retentive reconstruction layer, which remove part of the apparent zero-copy simplicity.
FlexBuffers preserves the direct-access property without a schema and sorts map keys for lookup, but its coercing accessors and absence of a specified complete canonical NUIF representation do not improve on deterministic CBOR for the current authoring file. It remains a possible opaque payload or cache encoding, not a primary document candidate.
Primary sources:
- https://flatbuffers.dev/white_paper/
- https://flatbuffers.dev/evolution/
- https://flatbuffers.dev/internals/
- https://flatbuffers.dev/flexbuffers/
Cap’n Proto
Cap’n Proto specifies a canonical, unpacked, single-segment, preorder form with trailing default words removed. The canonicalization algorithm is schema-agnostic. Normal encoders are explicitly not required to emit that form, so a NUIF profile would still require canonical-writer conformance. Its pointer model offers direct traversal, while the same specification requires pointer validation, a traversal limit that accounts for amplification, and a nesting limit. Those security rules align better with the existing NUIF resource model than an undocumented implicit limit would.
The remaining risk is semantic evolution through an editor. A complete NUIF schema must show that an older implementation can edit known data while retaining future fields, opaque extension bytes and integer/real distinctions. Canonical bytes must agree across at least two implementations. Until those tests exist, reporting Cap’n Proto latency alongside complete codecs would be a category error. It is the next experiment because it clears the canonical-form screen, not because its performance is presumed superior.
Primary source: https://capnproto.org/encoding.html, especially “Canonicalization” and “Security Considerations”.
Measured baseline
cargo xtask codec-benchmark records the corpus seed, generator, source
revision, dirty state, Rust toolchain, OS, architecture, CPU, warmups, samples,
latency distribution, allocation counts and exact encoded hashes. The first
local release run on an Apple M5 Pro with Rust 1.98.0 found deterministic CBOR
at 40.83% of canonical-text size for 4,096 entities. At that scale, canonical
text encoded in about 17.9 ms and decoded in 11.1 ms median; deterministic CBOR
encoded in about 18.9 ms and decoded in 24.2 ms. These are one-machine
measurements, not universal rankings. The important current conclusion is that
CBOR materially reduces bytes but is not presently a decode-latency
optimization. The decoder now materializes the typed document directly and
checks canonicality by re-encoding it; the generic value tree is retained only
as an invalid-input fallback needed to classify an over-depth value before a
root-type mismatch. This preserves strictness while avoiding two generic trees
on the accepted path.
The generated target/codec-benchmark-report.json is the evidence source. CI
archives each run; transient measurements are intentionally not committed as
goldens. Catastrophic ceilings detect broken scaling while controlled
before/after runs, not cross-host absolute values, decide optimizations.
NUIF relevance
- Retain canonical text as the review, fixture and Git form.
- Retain deterministic CBOR as the canonical hash and compact package record profile; its size win is real and its current decode cost is visible.
- Continue optimizing CBOR only behind identical conformance fixtures; the direct typed decoder is implemented, while a streaming canonical validator would require a separate hostile-input proof before replacing re-encoding.
- Prototype Cap’n Proto next only as a complete experimental mapping with a retentive edit bridge and bounded reader.
- Do not add Protobuf or FlatBuffers dependencies merely to publish flattering partial-model timings.
- A future zero-copy runtime cache may use a different, explicitly noncanonical compiled-scene profile without replacing the authoring interchange form.
Scholarly citation, archival DOI, preprint and software-paper workflow
Document status:
verified. Canonical source.
Summary
CITATION.cff provides machine-readable software and preferred-publication
metadata on GitHub. Zenodo can archive public GitHub releases and assign a DOI
to each release. Quarto renders one manuscript source to citeable HTML and PDF.
arXiv accepts topical, refereeable scientific contributions. The Journal of
Open Source Software (JOSS) reviews research software only after demonstrated
research use, sustained public development and feature completeness.
Evidence
- GitHub parses
CITATION.cfffrom the repository root and exposes APA and BibTeX citations. Apreferred-citationcan identify a paper or technical report instead of the software. Locator: About CITATION files, lines 24–70, retrieved 2026-08-30. - Zenodo archives a public repository and issues a DOI for each GitHub release. Locator: GitHub, Referencing and citing content, lines 21–35, retrieved 2026-08-30: https://docs.github.com/en/repositories/archiving-a-github-repository/referencing-and-citing-content.
- Quarto can generate citation metadata and citeable HTML from article frontmatter. Locator: Quarto, Creating Citeable Articles, retrieved 2026-08-30: https://quarto.org/docs/authoring/create-citeable-articles.html.
- arXiv submissions must be topical and refereeable; new authors or categories may require endorsement. Locator: arXiv, Submission Guidelines, lines 177–186, retrieved 2026-08-30: https://info.arxiv.org/help/submit/index.html.
- JOSS requires more than six months of active public history, demonstrated research impact, open-source practice, iterative development and feature-complete software. It also requires disclosure of generative-model assistance. Locator: JOSS, Submitting a paper, “Scope and significance” and “Pre-review screening criteria”, retrieved 2026-08-30: https://joss.readthedocs.io/en/latest/submitting.html.
Mechanism
The repository retains one Quarto manuscript and one bibliography. GitHub
Actions renders HTML and PDF without committing either output. CITATION.cff
identifies the software until a reviewed paper supplies the preferred citation.
Zenodo stores immutable release snapshots. arXiv and peer-review venues receive
the manuscript only after their respective evidence gates are met.
NUIF relevance
Borrow CFF and Zenodo for immediate citation and archival identity.
Adapt the paper claim to the bounded profile-zero experiment. The first manuscript reports the model, executable evidence and limitations; it does not claim universal format coverage or standards status.
Reject treating a Pages site, DOI, preprint or software-paper acceptance as equivalent to standards publication.
Open questions
- Authors, order, affiliations and ORCID identifiers require human confirmation before archival deposit or manuscript submission.
- The peer-review venue depends on whether the eventual contribution is an HCI study, a software artifact or an interoperability specification.
ScreenAI element typing, localization and screen annotation
Document status:
reviewed. Canonical source.
Summary
ScreenAI specializes a vision-language model for screens and infographics. Its central screen-annotation task asks for UI element types and locations, with OCR text, icon classes and generated captions contributing to a structured screen description. This supports a modular observation layer before semantic document synthesis.
Screen understanding benchmarks do not prove editable reconstruction. Element boxes, labels and captions are observations that still need hierarchy, layout constraints, exact resources, typography and provenance.
Evidence
- The 2024 publication abstract identifies a novel screen-annotation task in which the model predicts UI element type and location.
- The accompanying research article describes a DETR-based layout annotator, OCR extraction, a 77-class pictogram classifier and captioning for icons or images not covered by the classifier.
- Generated annotations are used to create downstream QA, navigation and summarization data with human quality validation.
- The publication reports a 5B-parameter model and releases three datasets, but the exact terms and suitability of each dataset for derived training artifacts must be reviewed independently.
Mechanism
screenshot
-> region/layout annotator
-> OCR text + icon class/caption + element type/location
-> serialized screen description
-> downstream model task
The intermediate annotation makes perception failures inspectable and can be evaluated separately from higher-level reconstruction.
NUIF relevance
Borrow typed elements, normalized locations and separate OCR/icon/image observations as replaceable ports.
Adapt the screen description into an ObservationGraph with evidence
regions, coordinate system, confidence, detector/version identity and optional
links to accessibility or source-backed evidence.
Reject element annotations as canonical NUIF entities without validation, or icon captions as evidence of behavior. A visually recognized “save” icon does not prove the action implemented by the source application.
Open questions
- Which element taxonomy maps cleanly to NUIF geometry, semantics and behavior without encoding one dataset’s labels into the core?
- How should OCR and detector boxes be reconciled when text is nested inside a control or partially occluded?
- What confidence calibration is needed before observations can seed automatic operations rather than require review?
Sequence-level knowledge distillation
Document status:
reviewed. Canonical source.
Summary
Sequence-level knowledge distillation trains a smaller sequence model on teacher-generated outputs rather than only matching token distributions. In the paper’s neural machine translation setting, the approach simplified the target distribution and enabled a smaller, faster student with limited quality loss.
For NUIF, accepted operation traces are a sequence target, but the teacher must be an evaluated pipeline—not an unversioned model response—and every sequence must pass validation and renderer-based checks before entering training data.
Evidence
- EMNLP 2016 paper defines sequence-level and sequence-level interpolation variants of knowledge distillation for neural machine translation.
- The best studied student ran ten times faster than its teacher with limited task-score loss; pruning further reduced parameters. These figures are specific to translation and are not NUIF projections.
- Distillation transfers teacher behavior, including systematic mistakes. The paper does not supply domain-specific correctness filters for UI semantics.
Mechanism
A teacher decodes target sequences for source examples. The student is trained on those generated targets, optionally mixed with original labels. Applied to NUIF, a “target” is a validated operation sequence plus its execution outcome, not merely a textual document emitted by the teacher.
NUIF relevance
Borrow sequence-level teacher outputs for a smaller student after the teacher pipeline has demonstrably better held-out performance.
Adapt each example into a trace: input hashes/context, observations, proposal, validation diagnostics, accepted operations, intermediate renders, difference maps, final fidelity and exact tool/model versions.
Reject unfiltered self-training, distillation from private inputs without explicit opt-in, and claims that a student is correct because it imitates a larger model.
Open questions
- Should the student learn complete initial transactions, single corrective transactions, or both as distinct tasks?
- How are multiple valid reconstructions represented without collapsing to one arbitrary teacher choice?
- Which teacher errors survive render filtering but damage editability or responsive behavior?
Bounded typed YAML parsing with serde-saphyr
Document status:
verified. Canonical source.
Summary
serde-saphyr 1.1.0 deserializes YAML directly into Serde types. It rejects
duplicate keys by default and exposes configurable resource budgets for input
size, nesting, collections, anchors, aliases and parser lookahead. The crate can
be compiled with only its deserialization feature. These properties fit a
small, read-only metadata boundary better than the deprecated serde_yaml
crate or forks that retain an unsafe LibYAML binding.
Evidence
- The 1.1.0 release added limits for buffered comment events, simple-key
lookahead and flow nesting. Locator: repository release
1.1.0, commitad5c614, retrieved 2026-08-30. - The project documents conservative default budgets and incremental
reader-based parsing for resource-exhaustion control. Locator: repository
README.md, “Pathological inputs & budgets”, retrieved 2026-08-30. - Duplicate keys produce an error by default. First-wins and last-wins policies
require an explicit option. Locator: repository
README.md, “Duplicate keys”, retrieved 2026-08-30. serde_json::Valueis supported for untyped data, while direct typed deserialization rejects values that do not match the destination type. Locator: docs.rs crate page for 1.1.0, “Overview” and “Notable features”, retrieved 2026-08-30.- The package declares Rust edition 2024, dual MIT or Apache-2.0 licensing and
independent
serializeanddeserializefeatures. Locator: docs.rsCargo.toml.origfor 1.1.0, retrieved 2026-08-30.
Mechanism
The documentation compiler caps each source file before parsing, extracts only the initial frontmatter block and deserializes that block into a closed metadata structure. serde-saphyr applies its default syntax budgets and duplicate-key policy. File inclusion, serialization and untyped tag-driven construction are not enabled.
NUIF relevance
Borrow typed, budgeted deserialization with default duplicate-key
rejection. Pin 1.1.0 with default features disabled and only deserialize
enabled.
Reject YAML as a canonical NUIF interchange encoding. It is limited to repository-authored metadata for the documentation and research toolchain.
Reject serde_yaml, which is deprecated, and YAML forks backed by an
unmaintained unsafe LibYAML binding. The documentation compiler does not need
their serialization compatibility.
Open questions
- Parser budget defaults require a regression fixture before metadata is accepted from untrusted pull requests at larger scale.
- YAML 1.1 boolean inference requires quoted strings or strict options if a future metadata field accepts arbitrary scalar values.
Sketch-n-Sketch (PLDI 2016, UIST 2016, UIST 2019) - trace-based program updates and output-directed programming for SVG
Document status:
reviewed. Canonical source.
Summary
Three papers describe successive versions of Sketch-n-Sketch, an editor in which a program in a small functional language (little, later Leo) generates SVG output and direct manipulation of that output is translated into program edits. PLDI 2016 introduces trace-based program synthesis: every numeric literal carries a source location, primitive operations build data-flow traces, a user drag turns the trace of a manipulated attribute into a value-trace equation, and the system solves the equation by changing exactly one program constant per attribute, choosing the constant with a “fair” rotation heuristic that is fixed before the drag begins so that updates apply live. UIST 2016 adds tools that transform program structure rather than constants (Draw, Dig Hole, Fill Hole, Make Equal, Group, Abstract, Duplicate, Merge). UIST 2019 replaces location traces by general value provenance (each value tagged with its producing expression and pointers to the values it was computed from) and uses that provenance to fill “value holes”, expose intermediate values as widgets, and drive refactorings such as Abstract, Repeat and Add Argument; sixteen parametric designs (427 lines) were built with no text editing. The reported limitations are consistent across the papers: only data-flow is traced (control flow is not), only constants are solved for, solvers accept only single-occurrence equations, ambiguity is resolved by heuristics rather than by asking, and the provenance-based tools are hand-coded one by one. Factual content below is separated from NUIF interpretation.
Evidence
PLDI 2016, “Programmatic and Direct Manipulation, Together at Last” (arXiv:1507.02988v3, 18 April 2016; DOI 10.1145/2908080.2908103, PLDI 2016 pp. 341-354, verified via Crossref 2026-08-29):
- Two kinds of traces: locations
ℓannotating every numeric literal, and expression traces built by rule E-Op-Num during primitive operations; traces record data flow but not control flow, a deliberate design choice justified by the observation that visual programs have stable control flow. Source: §2.1 and Figure 2. - Value-trace equations, for example
50 = (+ x0 (* ℓ0 sep)), together with the substitutionρ0from locations to current values, relate program and output. Source: §2.1, equations (1)-(3). - Local updates are substitutions from locations to numbers; only numeric constants change. Source: §2.2 “Local Updates”.
- Frozen constants (
n!) are excluded from updates; all Prelude literals are frozen automatically; range annotationsn{lo-hi}produce sliders. Source: §2.2 “Frozen Constants”, §2.4. - Correctness criteria: a substitution is faithful if re-evaluation produces a structurally similar value context (
V′ ∼ V) and every user-changed value is reproduced (“(c) implies (d)”); plausible if at least one user-changed value is reproduced. Source: §3, Definitions “Faithful Updates” and “Plausible Updates”. - Hard constraints are the j user-changed values, soft constraints the k−j unchanged ones. Source: §3 table “Program / Output / Updates / Constraints”.
- Shape assignments map each shape and zone to a location set; the “fair” heuristic rotates through candidate assignments so each location set is chosen equally often; a “biased” heuristic preferring rarely used locations is described in the appendix. Source: §4.1 “Fair and Other Heuristics”.
- Mouse trigger
τ = λ(dx, dy). ρis computed before the drag;ComputeTrigger(ρ, γ, v)solves one univariate equation per attribute withSolveOne; exactly one location per updated attribute is modified. Source: §4.1 “Computing Triggers” and “Recap: Design Decisions”. - Solutions are only plausible, not faithful, when one location feeds several manipulated attributes; substitutions are then applied in implementation-specific order. Source: §4.1 “Recap”.
SolveOnesupports only single-occurrence equations, inverting primitives top-down; not all primitives have total inverses. Source: §5.1.- Corpus: 68 programs, more than 2,000 lines; 3,772 shapes and 14,106 zones, of which 7% inactive, 34% unambiguous, 59% ambiguous with 3.83 candidates on average. Source: §5.2.1.
- Solvability: 4,574 unique pre-equations; 80% inside the solver fragment; 4% unsolvable for d = 1; 66% solvable for d = 100; failures include bounded functions such as
cos. Source: §5.2.2. - Performance (Chrome 49 / Firefox 45, i7 2.6 GHz): Solve < 1 ms median, Eval 5 ms median (12 ms average), Prepare 13 ms median with 6,789 ms maximum, Parse 53 ms median. Source: §5.2.3 table.
- Limitations: no shapes can be added through the GUI; no abstractions are inferred; no updates introduce new control flow; heuristics sometimes choose unintuitive locations (ferris wheel
numSpokesbecomes 0.3); rotation is poorly served by Cartesian drags. Source: §6.1, §6.2, §5.2.2.
UIST 2016, “Semi-Automated SVG Programming via Direct Manipulation” (arXiv:1608.02829v1, 9 August 2016; DOI 10.1145/2984511.2984575 printed on p. 1, UIST 2016 pp. 379-390):
- Draw inserts a definition
(def y ey)and appendsyto theblobslist when the program has the “simple” structure; otherwise it rewrites to(let y ey (addShapeToCanvas e y)). Source: “Tools for Drawing Shapes”. - Relate workflow: Select Features, Dig Hole, Fill Hole, Clean Up. Dig Hole lifts constants contributing to the selected features into variables in the nearest common scope without changing output; Fill Hole is manual; Clean Up inlines and renames. Source: “Tools for Relating Attributes”.
- Make Equal = Dig Hole, automatic hole fill that eliminates one degree of freedom (one constant replaced by an expression over the others), then Clean Up; the
n?annotation hints which constant to eliminate; otherwise the choice is arbitrary. Source: “Make Equal” and “Fill” paragraphs. - Group rewrites member bounding boxes as percentages of a new group box (
scaleBetween); Abstract turns a definition into a function over non-frozen named constants; Merge compares definitions modulo constants and abstracts over differing leaves. Source: “Tools for Grouping and Abstracting”. - The solver is “a prototype solver” over value-trace equations from PLDI 2016; the live synchronisation “one-equation, one-constant design” is inherited with its limitations. Source: “Related Work / Live Synchronization” paragraph.
- Implementation more than 13,000 lines of Elm and JavaScript; three worked examples; no timing or user study. Source: “Implementation”, “Examples”.
UIST 2019, “Sketch-n-Sketch: Output-Directed Programming for SVG” (arXiv:1907.10699v4, 10 August 2019; DOI 10.1145/3332165.3347925 printed on p. 1, UIST 2019 pp. 281-292):
- Provenance tracing: each value is “tagged with the expression being evaluated as well as pointers to the prior (tagged) values”; list elements carry pointers to their containing lists; pattern-match control flow is discarded. Source: “Provenance Tracing”.
- Value holes: the intended value is inserted as a leaf, then filled “by inspecting the provenance of the value and choosing an expression that evaluates to the value”, usually a variable. Source: Appendix “Value Holes”.
- Tool inventory with per-example usage counts (Draw Shape 16, Snap Drawing 15, Rename in Output 15, Make Equal 12, Abstract 9, Draw Offset 9, Group 8, …). Source: Figure 13.
- Make Equal ranking “prefers changes that rewrite terms near each other and later in the program”; Add Argument enumerates every expression that affected the selected value. Source: “Discussion” paragraphs on Make Equal and Add Argument.
- Repeat by Indexed Merge merges shape expressions into one function of an index
iand fills holes by sketch-based synthesis overi. Source: “Repetition” section and Appendix. - Evaluation: 16 parametric designs, 427 lines total, “built entirely via output-directed manipulations, without any text editing”; 4 of 15 WWID:PBD benchmark tasks fully completed. Source: “Case Study of ODP Examples”, Figure 15.
- Limitations: large numbers of hard-to-distinguish Make Equal candidates; offsets require forethought; sluggish on larger examples due to trace comparison; each program transformation is hand-coded. Source: “Discussion”, “Conclusion and Future Work”.
Mechanism
Trace syntax and update problem (PLDI 2016, Figure 2 and §3):
t ::= ℓ | (op_m t1 ... tm) -- data-flow trace of a number n^t
ρ : location → number -- substitution (local update)
User changes j of k output numbers:
hard: n′_i = t_i (1 ≤ i ≤ j)
soft: n_i = t_i (j < i ≤ k)
Faithful ρ: ρe ⇓ V′(w″) with V′ ∼ V ⟹ w″_i = w′_i for all i ≤ j
Plausible ρ: ... for some i ≤ j
Live synchronisation (PLDI 2016, §4.1):
γ(v)(zone)(attr) = ℓ -- location chosen before the drag ("fair" rotation)
ComputeTrigger(ρ, γ, v) = λ(dx,dy). ρ ⊕ (ℓx ↦ SolveOne(ρ, ℓx, nx+dx = tx))
⊕ (ℓy ↦ SolveOne(ρ, ℓy, ny+dy = ty))
SolveOne: univariate, single-occurrence equations, inverted top-down
Structural tools (UIST 2016) operate on the syntax tree with provenance only used to find the constants behind a selected feature: Dig Hole (lift constants to variables), Fill Hole (eliminate one degree of freedom), Group (re-parameterise by bounding box), Abstract (definition to function), Merge (anti-unification over constants).
Provenance-directed tools (UIST 2019): values carry (expression, [parent values]) tags; a UI gesture produces a target value; the tool inserts a value hole and searches the provenance graph for an expression or variable to fill it, or enumerates all contributing expressions for the user to pick (Add Argument).
NUIF relevance
- Borrow: Location-level provenance on numeric literals (PLDI 2016 §2.1) is the minimal provenance record that lets a resolved property be traced to an authored literal; NUIF correspondence records for source literals should carry the same information (file, span, literal value) so that a design-side geometry edit can be lowered to a literal replacement.
- Borrow: The hard/soft constraint split (§3) matches NUIF patch preconditions: user-edited properties are hard, all other resolved values are soft and may change only with an explicit fidelity report.
- Borrow: Freeze (
!) and prefer-to-eliminate (?) annotations are per-literal editing policies; NUIF should support the same policies in correspondence metadata rather than in source syntax. - Adapt: Dig Hole / Fill Hole / Clean Up (UIST 2016) is a three-phase refactoring that keeps output unchanged until the fill; NUIF’s “relate” operations (token binding, constraint creation) should be specified the same way, with a no-op precondition check that lowering the intermediate state reproduces the current resolved layout.
- Adapt: UIST 2019 provenance (expression plus parent-value pointers) is richer than NUIF needs for declarative sources; for template languages (Svelte, JSX) NUIF adapters can restrict provenance to static literals and expression spans obtained from a syntax tree (nuif:research:tree-sitter) instead of an instrumented evaluator.
- Reject: Silent heuristic disambiguation (“fair” rotation, arbitrary constant elimination) is incompatible with NUIF’s requirement that patches be deterministic and conflicts typed; NUIF must return ranked alternatives or a semantic-lowering conflict.
- Reject: The one-equation-one-constant solver as the only update mechanism; NUIF constraint layouts require simultaneous solving (nuif:research:cassowary) and stack/flex edits should lower to layout-property operations, not to numeric literal changes.
Open questions
- The papers give no correctness result comparable to lens laws; which of the UIST 2019 transformations preserve output exactly (Group, Abstract, Merge claim to) and can that be checked by re-evaluation as a conformance oracle?
- Ambiguity statistics (59% of zones ambiguous, 3.83 candidates) were measured on hand-written
littleprograms; comparable statistics for real component code are unknown. - Provenance size and comparison cost caused sluggishness (UIST 2019); NUIF resolved snapshots need a bound on provenance payload per property.
- Whether Repeat by Indexed Merge (synthesising loops from repeated shapes) has an analogue for NUIF component instances with overrides is unexplored.
Skia GM tests, DM, Skia Gold triage and fuzzy matching; browser reftests (WPT, Chromium, Firefox)
Document status:
reviewed. Canonical source.
Summary
Skia tests rendering with GM (“golden master”) programs executed by the DM driver across configurations (“sinks”) such as the software raster backend and GPU backends. Images are not compared against files in the repository; they are uploaded to Skia Gold, a service that stores expectations per test and per key set (OS, architecture, backend) and lets humans triage each new digest as positive, negative or untriaged. Gold supports several non-exact matchers configured through optional keys (image_matching_algorithm = fuzzy, sobel, sample_area, positive_if_only_image). Fuzzing is separate: libFuzzer targets in the fuzz binary are run by OSS-Fuzz. Browser engines use reftests instead of stored images: a test page and a reference page must render identically under the same engine, with an optional fuzzy allowance expressed as a maximum per-channel difference and a maximum number of differing pixels.
Evidence
- Gold purpose: “Gold is a web application that compares the images produced by our bots against known baseline images” (skia.org, Skia Gold page). The page lists positive (“the diff is considered acceptable”), negative (“requires a fix”) and untriaged states, and states that Gold processes more than 500,000 images per commit across OS, architecture and backends including CPU, OpenGL and Vulkan.
- Multiple positives: the client manual lists “Multiple correct (or ‘positive’) images for a single test” and pre-submit pass/fail plus post-submit triage modes (skia-buildbot
golden/docs/README.md, “What is Gold?”). - Keys:
goldctl imgtest init --keys-file ./keys.jsoncarries key-value pairs “describing how these inputs got drawn”, such as OS and GPU;goldctl imgtest add --test-name ... --png-file ...uploads;--passfailenables presubmit gating (golden/docs/README.md, “Using Gold”). - Matching algorithms:
image_matching_algorithmselectsexact,fuzzy,positive_if_only_image,sample_areaorsobel; parameters arefuzzy_max_different_pixels,fuzzy_pixel_delta_threshold,fuzzy_pixel_per_channel_delta_threshold,fuzzy_ignored_border_thickness,sobel_edge_threshold,sample_area_width,sample_area_max_different_pixels_per_area,sample_area_channel_delta_threshold(gold-client/go/imgmatching/constants.go). - Fuzzy semantics: images must be equal in size; the number of differing pixels must not exceed
MaxDifferentPixels; ifPixelDeltaThreshold > 0no pixel may have dR + dG + dB + dA above it (range 0–1020), else no pixel may have max(dR, dG, dB, dA) abovePixelPerChannelDeltaThreshold(0–255); a border ofIgnoredBorderThicknessrows/columns is skipped;MaxDifferentPixels = 0degenerates to exact matching (gold-client/go/imgmatching/fuzzy/fuzzy.go, type comment andMatch). - Parameters are parsed from optional keys with validation and range checks (
gold-client/go/imgmatching/factory.go,MakeMatcher,getAndValidateIntParameter). - GM API:
DrawResult { kOk, kFail, kSkip };getGoldKeys()returnsnameandsource_type = "gm";DEF_SIMPLE_GM(NAME, CANVAS, W, H)(Skiagm/gm.h). - DM usage:
--srcacceptstests gm image skp;--config 8888draws “using the software backend into a 32-bit RGBA bitmap” andgluses the Ganesh OpenGL backend;-wwrites results,-rreads a baseline directory,--matchfilters,--nogpu/--nocpurestrict work; DM emitsdm.jsonwith checksums of raw pixels (skia.org, Testing page). A GM is added undergm/, registered ingn/gm.gni, built withninja -C out/Debug dmand run without/Debug/dm --match newgmtest(skia.org, Writing Skia Tests). - Fuzzing: fuzzers use the libFuzzer entry point
LLVMFuzzerTestOneInput; reproduction isout/ASAN/fuzz -t api -n RasterN32Canvas -b testcase; OSS-Fuzz “rebuilds Skia and certain fuzzers and then runs said fuzzers” with configuration inoss-fuzz/projects/skia(skia.org, Fuzzing page). - WPT reftests: reftests are “made up of the test and one or more other pages (‘references’)” with assertions on whether they render identically;
<link rel=match href=...>passes if the pages render “pixel-for-pixel identically within an 800x600 window”,rel=mismatchpasses if they differ; with several references, at least one match must match and all mismatches must mismatch (web-platform-tests.org, Writing reftests). - WPT fuzzy syntax:
<meta name=fuzzy content="maxDifference=15;totalPixels=300">, shorthand<meta name=fuzzy content="15;300">, rangesmaxDifference=10-15;totalPixels=200-300, per-reference prefixoption1-ref.html:10-15;200-300;maxDifferenceis “a maximum difference in the per-channel color value for any pixel”,totalPixels“a number of total pixels that may be different”; unprefixed values apply to references without a specific value (same page, “Fuzzy Matching”). - Chromium policy: pixel tests are “less robust” because rendering “is influenced by many factors such as the host computer’s graphics card and driver, the platform’s text rendering system”; reference pages are named
foo-expected.htmlorfoo-expected-mismatch.*; “You should only write a pixel test if you cannot use a reference test” (chromium/srcdocs/testing/writing_web_tests.md). - Firefox manifest:
==passes if renderings are the same,!=if different;fuzzy(minDiff-maxDiff,minPixelCount-maxPixelCount)passes if per-pixel value differences and the count of differing pixels fall in the given inclusive ranges;fuzzy-if(condition,...),fails-if,skip-if,random,pref()andasserts(count)annotate conditions (firefox-source-docs, Reftest manifest).
Mechanism
Gold fuzzy matcher (gold-client/go/imgmatching/fuzzy/fuzzy.go)
require size(expected) == size(actual)
n_diff = 0; max_delta = 0
for each pixel outside the ignored border:
if p1 != p2: n_diff += 1
delta = per_channel ? max(|dR|,|dG|,|dB|,|dA|) : |dR|+|dG|+|dB|+|dA|
max_delta = max(max_delta, delta)
pass iff n_diff <= MaxDifferentPixels and max_delta <= threshold
Gold data model
digest = hash(png bytes); trace = (test name, keys...) ; expectation[trace][digest] in {positive, negative, untriaged}
new digest -> untriaged -> human triage (or matcher auto-approval against the most recent positive)
Reftest (WPT / Firefox)
render(test), render(ref) with the same engine, same window (800x600)
match: pass iff max per-channel |test - ref| <= maxDifference and count(differing pixels) <= totalPixels
mismatch: pass iff images differ
Baselines in Gold are keyed by configuration rather than shared, so a single test may have distinct positives for 8888, gl and vk. This is the source’s design; the consequence for NUIF is interpretation.
NUIF relevance
Borrow
- Adopt the reftest form for layout and paint semantics wherever a fixture can be expressed as two NUIF documents that must resolve to the same raster, because it removes stored images and platform baselines from the normative suite.
- Adopt the three-state triage vocabulary (positive, negative, untriaged) and configuration keys (OS, backend, adapter, font stack) for the non-normative GPU tier, because it is proven at browser scale and separates “different” from “wrong”.
Adapt
- Encode WPT-style
maxDifference;totalPixelsas fixture metadata in the NUIF conformance manifest, with values required to be justified per fixture, because the browser suites show that ad hoc tolerances accumulate without rationale. - Replace Gold’s remote service with in-repository digests plus a small triage file for the reference implementation, because NUIF’s suite must run offline and be vendor-neutral.
Reject
- Do not use per-platform positive baselines for the normative CPU reference path, because Chromium’s own guidance treats platform-specific expected images as a maintenance burden to be avoided.
- Do not adopt
positive_if_only_image(auto-approve when a test has a single image), because it converts an untriaged result into a passing baseline without human or metric review.
Open questions
- Whether NUIF should require a reference-document form for every render fixture, or permit stored rasters only for the CPU path.
- How to express
fuzzy-if(condition)-style conditional tolerance in a vendor-neutral manifest without encoding browser-specific platform names. - Which fuzz targets (codec, path geometry, layout) map onto NUIF’s
securitysuite; Skia’s fuzz taxonomy (api, image decoders, skp, path ops) is a starting list.
SSIM, MS-SSIM, PSNR and exact-pixel comparison policies for rendering tests
Document status:
reviewed. Canonical source.
Summary
SSIM (Wang, Bovik, Sheikh, Simoncelli, IEEE Trans. Image Processing 13(4):600–612, 2004) replaces error-visibility models with a structural comparison of local luminance, contrast and correlation, computed in a sliding Gaussian window and averaged into a mean SSIM (MSSIM). MS-SSIM (Wang, Simoncelli, Bovik, Asilomar 2003) applies the contrast and structure terms at five dyadic scales with calibrated exponents. PSNR is a monotone transform of MSE and shares its limitations. These indices are cheap and widely implemented, but they carry no display or viewing-distance model, can return negative values, and were calibrated on compression distortions rather than rendering artefacts. Exact per-pixel comparison remains the appropriate policy when the renderer is deterministic by construction (single CPU reference path, pinned fonts, pinned Unicode data); tolerance policies are needed only where the implementation is permitted to vary.
Evidence
- Identity: IEEE Transactions on Image Processing, vol. 13, no. 4, pp. 600–612, April 2004, DOI 10.1109/TIP.2003.819861 (Crossref record).
- MSE critique: Section I states that MSE “objectively quantifies the strength of the error signal” but that images with the same MSE “may have very different types of errors” (p. 600–601). Figure 2 shows “Boat” distortions all with MSE = 210 but MSSIM ranging from 0.9900 (mean shift) to 0.6949 (JPEG) (p. 603).
- Luminance term: l(x, y) = (2 μ_x μ_y + C1) / (μ_x² + μ_y² + C1), Equation 6; C1 = (K1 L)², Equation 7, with L the dynamic range (255 for 8-bit) (Section III.B).
- Contrast term: c(x, y) = (2 σ_x σ_y + C2) / (σ_x² + σ_y² + C2), Equation 9; C2 = (K2 L)² (Section III.B).
- Structure term: s(x, y) = (σ_xy + C3) / (σ_x σ_y + C3), Equation 10; the paper notes “s(x, y) can take on negative values” (Section III.B).
- Combined index: SSIM = l^α c^β s^γ (Equation 12); with α = β = γ = 1 and C3 = C2/2 the closed form is SSIM(x, y) = (2 μ_x μ_y + C1)(2 σ_xy + C2) / ((μ_x² + μ_y² + C1)(σ_x² + σ_y² + C2)), Equation 13 (Section III.B).
- Windowing: an 11 × 11 circular-symmetric Gaussian window with standard deviation 1.5 samples, normalised to unit sum, defines μ_x, σ_x, σ_xy (Equations 14–16, Section III.C). Constants K1 = 0.01, K2 = 0.03 are described as “somewhat arbitrary” with performance “fairly insensitive” to them (Section III.C, p. 607).
- Pooling: MSSIM(X, Y) = (1/M) Σ_j SSIM(x_j, y_j), Equation 17 (Section III.C).
- Scale dependence: the authors’ project page recommends downsampling by F = max(1, round(N/256)) before SSIM for typical viewing distances and states that the right scale “depends on both the image resolution and the viewing distance” (ece.uwaterloo.ca/~z70wang/research/ssim/).
- MS-SSIM identity: Wang, Simoncelli, Bovik, “Multiscale structural similarity for image quality assessment”, 37th Asilomar Conference on Signals, Systems and Computers, 2003, pp. 1398–1402, DOI 10.1109/ACSSC.2003.1292216 (Crossref).
- MS-SSIM form: the system iteratively low-pass filters and downsamples by 2; contrast and structure are compared at every scale j, luminance only at the coarsest scale M; SSIM = l_M^α_M Π_j c_j^β_j s_j^γ_j, Equation 7 (Section 3). Calibrated exponents for M = 5: β1 = γ1 = 0.0448, β2 = γ2 = 0.2856, β3 = γ3 = 0.3001, β4 = γ4 = 0.2363, α5 = β5 = γ5 = 0.1333 (Section 3.2); the calibration study fixed viewing distance at 32 pixels per degree (Section 3.1).
- Rendering-specific critique: the FLIP paper reports that SSIM “does not consider viewing distance and pixel size” and produces uninterpretable negative values, and that SSIM and Butteraugli “spread errors too widely, particularly near fireflies” (FLIP paper, Section 6.1).
- Exact-comparison practice: resvg counts a pixel as different when any channel differs by more than 1 and asserts zero differing pixels (
crates/resvg/tests/integration/main.rs,DIFF_THRESHOLD: u8 = 1,is_pix_diff); Vello’s sparse-strips tests use a per-component tolerance of 0 for the f32 CPU pipeline and 2 for the u8 pipeline (sparse_strips/vello_dev_macros/src/lib.rs, lines 12–23); WPT fuzzy matching usesmaxDifference(per-channel) andtotalPixels(web-platform-tests reftest documentation, “Fuzzy Matching”).
Mechanism
PSNR(x, y) = 10 * log10( L^2 / MSE(x, y) ), MSE = (1/N) * sum_i (x_i - y_i)^2
SSIM(x, y) = (2*mu_x*mu_y + C1) * (2*sigma_xy + C2)
--------------------------------------------- # Eq. 13
(mu_x^2 + mu_y^2 + C1) * (sigma_x^2 + sigma_y^2 + C2)
C1 = (0.01 * L)^2, C2 = (0.03 * L)^2, L = 255 for 8-bit
mu, sigma, sigma_xy computed under an 11x11 Gaussian window, sigma_w = 1.5 # Eqs. 14-16
MSSIM = mean over windows # Eq. 17
MS-SSIM (M = 5):
x_1 = x; x_{j+1} = downsample2(lowpass(x_j))
MS-SSIM = l_M^{0.1333} * prod_{j=1..5} (c_j * s_j)^{beta_j}
beta = [0.0448, 0.2856, 0.3001, 0.2363, 0.1333] # Eq. 7
Exact policy (deterministic path):
pass iff for all pixels, all channels: |a - b| <= t, with t = 0 (bit-exact) or t = 1 (rounding slack),
and count(different pixels) == 0
Count-and-delta policy (WPT / Gold / WebRender):
pass iff max_channel_delta <= maxDifference and count(pixels with any delta > 0) <= totalPixels
Perceptual hashes (block-mean, DCT or gradient hashes) reduce an image to a short bit string and compare Hamming distance; they are designed for near-duplicate retrieval and are insensitive to exactly the localised, low-amplitude artefacts (one-pixel clipping offsets, anti-aliasing changes on thin strokes) that rendering conformance must detect. This statement is NUIF interpretation; no primary source was retrieved for it.
NUIF relevance
Borrow
- Adopt the count-and-delta policy (
maxDifference,totalPixels) as the intermediate tolerance tier for platform-pinned baselines, because it is auditable, has three independent industrial implementations (WPT, Gold, WebRender) and needs no perceptual model. - Keep exact comparison (t = 0) as the normative policy for the CPU reference path, because resvg and Vello’s f32 CPU pipeline demonstrate that a Rust rasteriser without system dependencies can be bit-identical across platforms.
Adapt
- If SSIM is reported at all, report it alongside FLIP and only after the downsampling rule tied to the fixture’s PPD, because the index is scale-dependent and its calibration assumed 32 PPD.
- Map any single-channel tolerance t = 1 to an explicit rationale (rounding of premultiplied alpha, SIMD reassociation) recorded in the fixture, because resvg had to exclude a gradient fixture for a SIMD rounding difference.
Reject
- Do not use PSNR or MSSIM as conformance gates, because both lack a display model and can score semantically wrong images higher than perceptually acceptable ones (Figure 2 of the SSIM paper).
- Do not use perceptual hashes for conformance, because their design goal is retrieval robustness, which is the opposite of artefact sensitivity.
Open questions
- Whether a luminance-only SSIM map is still useful as a cheap diagnostic overlay in conformance reports, given that FLIP maps exist.
- What per-channel slack, if any, the normative CPU path should allow for premultiplied-alpha round trips without weakening the exactness claim.
Standards-development venue requirements for an interface interchange specification
Document status:
verified. Canonical source.
Summary
W3C, Khronos, Ecma, OASIS and the Joint Development Foundation provide different entry conditions and intellectual-property boundaries. W3C Community Groups permit no-fee public incubation but do not produce W3C Standards. Khronos accepts non-member initiative proposals but reserves detailed Working Group design for members and couples adoption claims to a conformance test suite. Ecma Technical Committees require General Assembly formation and member support. OASIS Open Projects combine public code and specification work with a sponsor-governed path to an OASIS Standard. The Community Specification process supplies a repository-based contributor, scope and patent framework without asserting formal standards-body status.
Evidence
- A W3C Community Group proposal needs a W3C account and four additional supporters. Participation has no fee, and Community Group reports are not W3C Standards. Locator: W3C Community Groups, lines 79–116, retrieved 2026-08-30: https://www.w3.org/community/.
- Khronos permits member and non-member initiative proposals. An Exploratory Group develops use cases, requirements and a statement of work without detailed design contributions. Detailed Working Group participation requires Khronos membership. Locator: Khronos “New Initiative Process Overview”, sections “New Initiative Process Overview” and “How To Propose”, retrieved 2026-08-30: https://www.khronos.org/exploratory/new-initiative-process/.
- Khronos requires an official Conformance Test Suite pass before a product can use a specification’s trademarked name or make conformant-product claims. Locator: Khronos “About”, “Open Standards for 3D, the Metaverse and More”, retrieved 2026-08-30: https://www.khronos.org/about/.
- Ecma Technical Committees are formed by General Assembly decision. New work items require support from at least three Ecma members, of which at most one is a not-for-profit member. Royalty-Free Technical Committee operation requires General Assembly approval. Locator: Ecma Rules, Article 7.1, retrieved 2026-08-30: https://ecma-international.org/policies/rules/.
- OASIS Open Projects support code, APIs, prose specifications and protocols under contributor agreements. Their formal path includes public review, Statements of Use, project governance approval and an OASIS membership ballot. Locator: OASIS Open Projects Handbook, sections 1 and 9; Open Project Rules, sections 13–14, retrieved 2026-08-30: https://www.oasis-open.org/oasis-open-projects-handbook/ and https://www.oasis-open.org/policies-guidelines/open-projects-process/.
- Community Specification 1.0 uses a contributor agreement, bounded scope,
notices and separate specification and source-code licenses. Locator:
Community Specification
getting-started.md, lines 195–245, retrieved 2026-08-30: https://github.com/CommunitySpecification/Community_Specification/blob/main/getting-started.md.
Mechanism
An incubation venue supplies participation and contribution terms before it supplies formal publication status. Formal advancement adds a chartered scope, intellectual-property commitments, public review, consensus and implementation evidence. Conformance claims require a versioned specification, tests and independent implementation results that use the same feature profile.
NUIF relevance
Borrow the Community Specification scope and contributor terms when the first external organization contributes normative text.
Adapt W3C Community Group incubation if browser, design-tool and design-token stakeholders support a Web-facing scope. DTCG provides the closest existing liaison surface.
Adapt Khronos if the center of adoption becomes cross-platform graphics and content-tool interoperability and multiple member companies will fund a Working Group and conformance suite.
Reject selecting a formal venue before there are independent implementers and organizational sponsors. Each formal path depends on participation and intellectual-property commitments that a single repository owner cannot substitute.
Open questions
- The primary scope may resolve toward Web authoring, graphics content tools or a general document protocol; each direction changes the suitable venue.
- No legal entity currently owns the specification trademark, contributor agreement administration or patent-notice process.
- Independent implementation and statement-of-use thresholds have not been met.
ISO 10303 STEP product-data exchange architecture
Document status:
reviewed. Canonical source.
Summary
STEP is a family of product-information representation and exchange standards designed to survive exchange among heterogeneous systems across a product lifecycle, with explicit data specification methods and modular parts.
NUIF relevance
The lesson is organizational: a durable interchange specification needs modular normative parts, conformance classes and schema discipline. Avoid STEP’s complexity explosion by aggressively constraining the NUIF core and using profiles/extensions.
Structured differencing and three-way merge
Document status:
reviewed. Canonical source.
Summary
Structured merge research shows AST-aware mappings and top-down/bottom-up strategies can reduce false conflicts compared with line merge, particularly when elements move. Recent 2026 work also argues for explicit correctness properties for structural merge.
NUIF relevance
Stable entity identity gives NUIF an advantage over inferred AST matching. Three-way merge should operate over semantic operations and graph relationships while preserving a textual fallback for human review.
Svelte component AST, retentive syntax and compiler-oracle boundary
Document status:
verified. Canonical source.
Summary
The official Svelte compiler is the semantic oracle for .svelte syntax. It
exposes component and CSS parsers with source-offset AST nodes and separately
models markup, instance/module scripts and component CSS. A NUIF production
adapter still needs a concrete syntax tree because the official print API is
explicitly allowed to change whitespace and quoting. The selected split is the
official compiler for pinned foreign conformance and
tree-sitter-svelte-next 0.1.1 for bounded UTF-8 byte spans in Rust.
The first executable profile is deliberately smaller than Svelte’s static
surface: regular div/span elements, double-quoted literal identity and name
attributes, one literal inline-style declaration list and one literal text run.
Scripts, component CSS, expressions and all directives or blocks are rejected
inside the mapped component. Top-level comments and whitespace may be retained,
but the profile does not infer their runtime relationship to the marked root;
other top-level nodes are rejected.
Evidence
svelte/compilerexposescompile,parse,parseCss,preprocessandprint. Modern AST nodes containstartandendoffsets. The root separates markup, CSS, instance script and module script. https://svelte.dev/docs/svelte/svelte-compiler#parse (retrieved 2026-08-29).printemits valid Svelte plus a source map but may change whitespace and quoting. It is therefore not an edit-locality mechanism for retained source. https://svelte.dev/docs/svelte/svelte-compiler#print (retrieved 2026-08-29).- The AST distinguishes static text, expression tags, HTML tags, regular elements, components, blocks and directives. This supplies a structural boundary between literals and executable expressions. https://svelte.dev/docs/svelte/svelte-compiler#AST (retrieved 2026-08-29).
- Component CSS is scoped by default through a hash-derived class added to affected elements and selectors. Scoped keyframe names are also rewritten. https://svelte.dev/docs/svelte/scoped-styles (retrieved 2026-08-29).
tree-sitter-svelte-next0.1.1 is a dual MIT/Apache-2.0 grammar compatible with Tree-sitter 0.25 and later. Its crate is generated from commitbdea454a8ae7272498b8fe9d4b6b24fbd3dfe7b6; its Rust API exposes the language and node types without a Svelte runtime. The grammar includes elements, literal and expression attributes, blocks, scripts and styles, so the adapter can reject executable nodes structurally. Locators: crate manifest,src/lib.rs,src/node-types.json,.cargo_vcs_info.json, retrieved 2026-08-30: https://github.com/PRRPCHT/tree-sitter-svelte-next and https://docs.rs/tree-sitter-svelte-next/0.1.1/.- The npm registry reported official
svelte5.57.0 as latest on 2026-08-30. A foreign fixture must pin that exact package and invoke bothparse(source, { modern: true })andcompile(source, ...); floating latest is not evidence because the compiler and modern-AST defaults evolve. - The independent Rust
svelte-compiler0.1.4 declares Rust 1.94 and depends on a broad compiler stack. Its ownAUDIT.mdreports about 11,300 lines in the API module, duplicated modern/legacy paths and manual recovery for grammar gaps. This is useful comparative work but is neither the official compiler nor a smaller retentive boundary. Locators: crate manifest and project audit, retrieved 2026-08-30: https://docs.rs/svelte-compiler/0.1.4/svelte_compiler/ and https://github.com/themixednuts/svelte/blob/main/AUDIT.md.
Alternatives and decision
| Candidate | Strength | Blocking mismatch for this profile | Decision |
|---|---|---|---|
official svelte/compiler | authoritative syntax, diagnostics and generated output | JavaScript runtime boundary; print does not retain formatting | pinned foreign oracle |
tree-sitter-svelte-next 0.1.1 | small CST, byte offsets, compatible Tree-sitter line | community grammar, not semantic authority | production span parser, checked against oracle |
Rust svelte-compiler 0.1.4 | typed Rust compiler project with broad Svelte ambition | unofficial, much larger graph and documented recovery debt | reject as production dependency |
| HTML parser alone | mature markup parsing | cannot classify Svelte blocks, directives or embedded regions safely | reject |
| regular expressions | minimal code | cannot prove nesting, quoting or executable-syntax exclusion | reject |
Inline literal style is selected over a component <style> block for profile
zero. It binds every mapped scalar to the element that owns it, avoids cascade,
specificity and Svelte scope-hash semantics, and provides one exact replaceable
span per property. A later class/CSS profile needs separate selector, cascade,
scope and unused-selector conformance; it is not an implicit expansion of this
profile.
Mechanism
The Rust parser first enforces encoded-size, syntax-node and mapped-depth limits. It locates exactly one marked regular-element root, verifies the closed attribute and inline-style vocabularies, decodes only canonical entity escapes, and records every mapped UTF-8 span. Export self-imports. Synchronization renders the before and after profile forms to obtain canonical replacement values, checks that each retained span is still current, applies replacements from the end of the source, and self-imports the result. The foreign gate separately parses and compiles generated and synchronized sources with the exact official compiler package.
NUIF relevance
Borrow compiler AST offsets and literal-node categories for correspondence.
Adapt a static subset of regular elements, literal attributes, literal text and profile-owned inline CSS declarations. Edits replace original spans rather than printing the AST. Grammar revision, compiler version and modern-AST mode are part of provenance.
Reject automatic semantic lifting of runes, scripts, expressions, snippets, blocks, directives, actions, transitions, dynamic components or preprocessors. They are programs whose behavior depends on inputs and runtime state.
Falsification and update triggers
The adapter decision fails if the Tree-sitter grammar accepts a generated fixture that official Svelte rejects, assigns unusable byte spans, or cannot structurally distinguish one of the excluded executable constructs. The gate therefore compiles every exported and synchronized fixture with the pinned official package and maintains negative fixtures for scripts, expressions, directives, blocks, components and component CSS. A Svelte or grammar update is one isolated dependency commit that regenerates the foreign lockfile, runs the complete adapter corpus, and records any AST or diagnostic change before the pin moves.
Open questions
- Whether a future component-CSS profile should own one style block per mapped root or preserve user CSS through a selector-aware correspondence layer.
- Whether official compiler warnings should become hard failures or a separately versioned diagnostic baseline once the profile accepts more than generated source.
- Whether the community grammar will publish a stable compatibility and release policy; until then, its exact version and source commit remain provenance.
SVG 2 vector graphics model
Document status:
reviewed. Canonical source.
Summary
SVG 2 defines an XML vocabulary for nested graphics elements, coordinate
systems, geometry, paint, text, reuse and accessibility metadata. SVG element
identity uses the XML id attribute. The g element groups descendants without
introducing NUIF component or layout semantics.
Evidence
- SVG 2 §5.1–5.2 defines an SVG document fragment and the
svgelement. TheviewBoxand viewport establish coordinate-system mappings; they are not equivalent to an authored responsive layout rule. https://www.w3.org/TR/SVG2/struct.html#NewDocument and https://www.w3.org/TR/SVG2/coords.html#ViewBoxAttribute (retrieved 2026-08-29). - SVG 2 §10 defines
rect,circle,ellipse,line,polylineandpolygonas basic shapes. Arectis axis-aligned in the current user coordinate system; anellipseis defined bycx,cy,rxandry. https://www.w3.org/TR/SVG2/shapes.html (retrieved 2026-08-29). - SVG 2 §12 defines text layout through
text,tspanand text positioning attributes. SVG text permits per-character positioning, text paths and shaping behavior beyond NUIF profile zero. https://www.w3.org/TR/SVG2/text.html (retrieved 2026-08-29). - SVG 2 §13 makes fill and stroke presentation properties available to shape and text elements. CSS cascading and inheritance apply, so a computed paint cannot be attributed to one source span without cascade analysis. https://www.w3.org/TR/SVG2/painting.html and https://www.w3.org/TR/SVG2/styling.html (retrieved 2026-08-29).
- SVG 2 §16 permits WAI-ARIA attributes and the
roleattribute on SVG elements. This surface can carry NUIF role and accessible-name correspondences, subject to the SVG Accessibility API Mappings. https://www.w3.org/TR/SVG2/struct.html#WAIARIAAttributes (retrieved 2026-08-29).
NUIF relevance
Borrow the basic-shape geometry, sRGB presentation attributes, containment order, XML identity and accessibility attributes for a bounded vector profile.
Adapt svg, g, rect, ellipse and text into NUIF surface, container,
shape and text entities. A generated profile requires explicit data-nuif-*
metadata for document identity, stable entity identity, pinned font identity
and authored sizing intent. Static numeric attributes can retain byte-span
correspondence.
Reject a claim that arbitrary SVG is a lossless semantic import. Paths,
transforms, CSS cascade, paint servers, clipping, masks, filters, animation,
scripts, external resources, per-character text positioning and <use>
instancing require separate profiles. Unknown XML can be retained as source but
cannot be classified as lossless NUIF semantics without a declared extension.
SwiftUI proposal-response layout model
Document status:
reviewed. Canonical source.
Summary
SwiftUI custom layout uses proposal–response measurement followed by subview
placement. A Layout implementation receives proxy values rather than direct
subviews and can query dimensions, spacing, priority and custom layout values.
Evidence
- The
Layoutprotocol requiressizeThatFits(proposal:subviews:cache:)andplaceSubviews(in:proposal:subviews:cache:). Optional methods provide alignment, spacing, axis properties and caches. https://developer.apple.com/documentation/swiftui/layout (retrieved 2026-08-29). LayoutSubview.sizeThatFits(_:)accepts a proposed size. SwiftUI views choose a size while considering the parent proposal; the result is not a CSS-style fixed-width declaration. https://developer.apple.com/documentation/swiftui/layoutsubview/sizethatfits(_:) (retrieved 2026-08-29).ViewThatFitscan choose the first child that fits the proposal. Custom layout examples share measurements between sizing and placement through a cache. https://developer.apple.com/documentation/swiftui/composing-custom-layouts-with-swiftui (retrieved 2026-08-29).
NUIF relevance
Borrow proposal–response sizing, intrinsic probes, stack alignment, spacing and explicit separation between sizing and placement.
Adapt profile-zero stacks and fixed/intrinsic/fill sizing into a generated, profile-owned Swift subset. Compiler, SDK, operating-system version, dynamic type, locale and font registry are evaluation-context provenance.
Reject arbitrary SwiftUI import as a lossless document operation. Result
builders, modifiers, state, environment values, custom Layout types and
platform views are executable semantics.
Symmetric Lenses
Document status:
reviewed. Canonical source.
Summary
Symmetric lenses model synchronization where both sides may contain information absent from the other, avoiding a permanently privileged source/view direction.
NUIF relevance
Design files and source frameworks are peers with asymmetric capabilities. Synchronization should therefore be symmetric at the system level even when individual adapters implement directional lowering/lifting passes.
Browser-derived layout fixtures in Taffy and Yoga (Chrome as reference oracle)
Document status:
verified. Canonical source.
Summary
Both Taffy and Yoga treat Chrome as the reference implementation of CSS Flexbox (and, for Taffy, Block and Grid). Each repository keeps HTML fixtures in which the tree structure and inline style attributes are the test input; a generator loads each fixture in headless Chrome through WebDriver, reads back the DOM geometry with getBoundingClientRect(), and emits unit tests whose expectations are the browser’s numbers. Taffy (Rust, scripts/gentest, fantoccini WebDriver client) downloads a Chrome for Testing build and matching ChromeDriver, measures each fixture under four variants (border-box/content-box × ltr/rtl), records unrounded, naively rounded and “smart” rounded layouts, and writes XML test descriptions that a Rust harness replays with a tolerance of 0.1 px. Yoga (TypeScript, gentest/src/cli.ts, selenium-webdriver) injects each fixture into a template that sets Yoga’s defaults as CSS, measures ltr and rtl trees with edges rounded to integers, and emits C++, Java and JavaScript tests asserting exact equality; generated files are signed and CI regenerates them to detect drift. Neither project uses a numeric tolerance against the browser beyond rounding; known divergences are handled by excluding or flagging fixtures rather than by relaxing assertions. Facts below were verified against the repositories at the commits stated.
Evidence
Taffy at commit b3b387132be1dda0e9d08d5044692236532c166d (2026-08-26, crate version 0.14.0), retrieved 2026-08-29:
- Generator dependencies:
fantoccini = "0.22.0"(WebDriver client), local crategetchrome,tokio,walkdir,xmlwriter,serde_json. Source:scripts/gentest/Cargo.toml. getchromedownloads the latest Stable “Chrome for Testing” browser and matching ChromeDriver fromgooglechromelabs.github.io/chrome-for-testing/last-known-good-versions-with-downloads.jsoninto.chrome-for-testing/<version>; the Chrome version is not pinned in the repository (CHANNEL = "Stable"). Source:scripts/getchrome/src/lib.rs, lines 1-30 andscripts/getchrome/Cargo.toml.- Fixture discovery walks
test_fixtures/, skips_scratchdirectories and any file whose name starts withx. Source:scripts/gentest/src/main.rs, lines 37-49. At this commit there are 1533 HTML fixtures (block 235, blockflex 11, blockgrid 14, contain 8, flex 674, float 27, grid 543, gridflex 7, leaf 14) of which 17 arex-prefixed and excluded; the excluded files carry no explanatory comment (titles are the placeholder “Test description”). - Chrome is launched headless with
--headless --no-sandbox --disable-gpuand a per-run profile directory; ChromeDriver is started on a free port with 10 s timeouts. Source:main.rs, lines 197-284. - Before measuring, the generator asserts that scrollbars occupy space (15 px) and aborts otherwise; the stylesheet forces
::-webkit-scrollbar { width: 15px; height: 15px }and the comment states the width must match the test runner. Source:main.rs, lines 484-505;scripts/gentest/test_base_style.css. - Each fixture is loaded from a
file://URL, the load event is awaited, andgetTestData()is executed; it togglesbody.classNamethroughborder-box ltr,content-box ltr,border-box rtl,content-box rtland describes#test-rootunder each. Source:main.rs, lines 507-554;scripts/gentest/test_helper.js,getTestData, lines 1308-1321 of the concatenated listing (function at the end of the file). describeElementreads input styles from the inlinestyleobject (e.style.*), exceptboxSizinganddirection, which are read fromgetComputedStyle; grid template strings are passed through verbatim so that line names survive. Source:test_helper.js,describeElement.- Three geometry records are captured per element:
unroundedLayoutfromgetBoundingClientRect()withx/yrelative to the parent rectangle;naivelyRoundedLayoutfromoffsetWidth/offsetHeight/offsetLeft + parent.clientLeft;smartRoundedLayoutcomputed asMath.round(right) - Math.round(left)andMath.round(x - parent.x). The comment states that Chrome uses a smarter rounding algorithm but does not expose its output, so the script emulates Taffy’s algorithm. Source:test_helper.js,describeElement. - Text measurement: the Ahem font is embedded as a WOFF2 data URI; the
Xglyph is 10 × 10 px; zero-width spaces are used to control min-content and max-content;#test-rootsetsfont-family: ahem; line-height: 1; font-size: 10px. LeaftextContentis captured to drive measure functions. Source:test_base_style.css;test_helper.js. - Opt-out attributes:
data-test-rounding="false"disables rounding for a fixture (5 fixtures);data-test-resolved-track-lists="false"suppresses comparison of resolved grid track lists (26 fixtures), documented in the script as intended for “overlarge grids, where Taffy’s MAX_GRID_TRACKS clamp intentionally differs from Chrome’s track limit”. Source:test_helper.js,describeElement; counts fromgrepovertest_fixtures/. - Expectations use
smartRoundedLayoutwhen rounding is enabled andunroundedLayoutotherwise; scroll sizes are recorded only for scroll containers asscrollWidth - naive clientWidthfloored at zero; resolved grid rows/columns are recorded from computed style. Source:main.rs,generate_assertions, lines 586-628. - Output: one XML file per variant under
tests/xml/<family>/<name>__{border_box,content_box}_{ltr,rtl}.xml(6064 files at this commit, equal to (1533 − 17) × 4) plus a generatedtests/xml/mod.rswith one#[test]per file, gated by#[cfg(feature = "grid")]for grid names. Source:main.rs, lines 95-152. - Comparison: the harness constructs a
TaffyTree, enables or disables rounding from theuse-roundingattribute, sets available space from<viewport>, and comparesx,y,width,heightwithabs() < 0.1; scroll sizes likewise; resolved track lists compare line names exactly and track sizes with< 0.1, with the comment that Chrome and Taffy “format/round subpixel used sizes slightly differently”. Source:tests/xml.rs,impl PartialEq for OutputNodeandtrack_lists_match, lines 60-100 and 178-193. - Freshness gate: CI job “Generated Test Freshness” runs
cargo run -p gentestand fails ifgit status --porcelain -- tests/xmlis non-empty. Source:.github/workflows/ci.yml, lines 287-307. - Policy statements: “Flexbox layouts are tested by validating that layouts written in this crate perform the same as in Chrome”; generated tests must not be edited by hand. Source:
CONTRIBUTING.md, lines 26-31 (the text still namestests/generated, while the current output directory istests/xml). - Changelog entries record behaviour changes made to track Chrome: content alignment “updated to match the latest spec (and Chrome 123+)” and rounding “fixed … to follow latest Chrome”. Source:
CHANGELOG.md, lines 449 and 1137-1140.
Yoga at commit bd8fe0d6d243cc7e0334d4cc68864a994f63beae (2026-08-27), retrieved 2026-08-29:
- Dependencies:
selenium-webdriver ^4.16.0,signedsource ^2.0.0,minimist; scriptsgentest(runssrc/cli.ts) andgentest-validate. Source:gentest/package.json. A serial predecessor,gentest/gentest-driver.ts(single driver, LTR/RTL by textualstart/endsubstitution, expectations read from console logs), is still present but is not referenced bypackage.json;gentest/gentest.jsandgentest/gentest.rbreturn 404 on themainbranch. - Documentation: “Many of Yoga’s tests are automatically generated, using HTML fixtures … rendered in Chrome to generate an expected layout result”. Source:
README.md, lines 19-31. - Fixtures: 25 HTML files in
gentest/fixtures(for exampleYGAlignItemsTest.html,YGRoundingTest.html,YGIntrinsicSizeTest.html,YGBoxSizingTest.html,YGStaticPositionTest.html). Each top-level<div id="...">becomes one test named by its id. - Template:
gentest/test-template.htmlloads Ahem fromgentest/fonts/Ahem.ttf, setsbody { font: 10px/1 Ahem }, and gives everydiv, spanYoga’s defaults as CSS:box-sizing: border-box; position: relative; display: flex; flex-direction: column; align-items: stretch; align-content: flex-start; justify-content: flex-start; flex-shrink: 0; test roots are absolutely positioned. Source:gentest/test-template.html. - Browser: a pool (default 8) of headless Chrome sessions with
--force-device-scale-factor=1 --window-position=0,0 --hide-scrollbars --headless; the generator waits fordocument.fonts.readybefore measuring. Source:gentest/src/ChromePool.ts;gentest/src/cli.ts, lines 93-104. - Measurement:
buildLayoutTreesetsstyle.directiontoltrthenrtlon each test root, walks the DOM, and recordswidth = Math.round(rect.right) - Math.round(rect.left),heightlikewise,left = Math.round(rect.left - parentLeft),toplikewise, the originalstyleattribute string,data-experiments(space separated) anddata-disabled === 'true', andinnerTextfor leaves. Source:gentest/src/buildLayoutTree.ts, lines 40-77. - Style mapping:
CssToYoga.tsparses the inline style string (not computed style), expandsflex: Ntoflex-grow: N; flex-shrink: 1; flex-basis: 0%, and emits setter calls only for values that differ from Yoga defaults. Source:gentest/src/CssToYoga.ts,parseStyleAttribute,expandShorthand,applyStyles. - Emission: each fixture yields
tests/generated/<Name>.cpp(GoogleTest,ASSERT_FLOAT_EQ),java/tests/generated/com/facebook/yoga/<Name>.java(assertEquals(expected, actual, 0.0f)), andjavascript/tests/generated/<Name>.test.ts(expect(...).toBe(...)); each test computes layout withYGNodeCalculateLayout(root, YGUndefined, YGUndefined, YGDirectionLTR)and asserts left/top/width/height for every node, then repeats for RTL. Source:gentest/src/emitters/Emitter.ts,generateFixture, lines 126-160;tests/generated/YGAlignItemsTest.cpp, lines 16-54. - Known-bug handling:
data-disabled="true"emitsGTEST_SKIP();in C++ andtest.skipin JavaScript;data-experiments="Foo"emitsYGConfigSetExperimentalFeatureEnabled(config, YGExperimentalFeatureFoo, true). At this commit no fixture uses either attribute. Source:gentest/src/emitters/CppEmitter.ts, lines 91-112;JavascriptEmitter.ts, lines 136-150;grepovergentest/fixtures. - Rounding: Yoga rounds every node’s absolute left/top/right/bottom to the pixel grid with
pointScaleFactor(default 1.0) after layout, so integer expectations derived from rounded browser rectangle edges are comparable. Source:yoga/algorithm/PixelGrid.cpp,roundLayoutResultsToPixelGrid, lines 65-136;yoga/config/Config.h, line 80;yoga/algorithm/CalculateLayout.cpp, line 2938. - Integrity: generated files carry a
@generated SignedSource<<hash>>header;gentest-validate.tsverifies signatures; CI workflowvalidate-tests.ymlrunsyarn gentest-validateandyarn gentest -hand fails when regeneration modifies any test. Source:gentest/scripts/gentest-validate.ts;.github/workflows/validate-tests.yml, lines 23-33.
Mechanism
Common pipeline:
fixture.html (structure + inline styles)
→ headless Chrome via WebDriver (Ahem font, fixed defaults, scale factor 1)
→ per-element {input styles, getBoundingClientRect geometry}
→ emitter → unit tests in the engine's language
→ engine computes layout → assert per-node x, y, width, height
Rounding models differ:
Taffy smart rounding (per element, relative to parent rect):
width = round(right) − round(left)
x = round(x_abs − parent.x_abs)
Taffy comparison: |expected − actual| < 0.1 on x, y, w, h (and scroll sizes, track sizes)
Yoga (browser side): identical edge-rounding formula
Yoga (engine side): roundLayoutResultsToPixelGrid with pointScaleFactor = 1
Yoga comparison: exact equality (ASSERT_FLOAT_EQ / toBe / assertEquals delta 0)
Taffy captures both unrounded and rounded layouts so that a fixture can opt out of rounding (data-test-rounding="false") and be compared against unrounded floating-point Chrome values; Yoga has no unrounded mode.
Divergence handling is structural, not numeric: Taffy excludes fixtures by file-name prefix and suppresses particular assertions by attribute; Yoga skips tests or enables experimental features by attribute. Neither project stores the Chrome version used for generation in the generated artefacts; Taffy downloads the current Stable channel at generation time, and Yoga uses whatever Chrome is installed, so regenerating after a Chrome release can change expectations. Both projects rely on CI regeneration to detect such drift.
NUIF relevance
- Borrow: The fixture format (HTML tree with inline styles plus browser-measured expectations under a controlled font and scale factor) is directly reusable for nuif:experiment:layout-differential; NUIF flex/grid fixtures can be lowered to the same HTML and compared against both Chrome and Taffy without new tooling.
- Adapt: Taffy’s four-variant measurement (box-sizing × direction) is a tested baseline. Its 0.1 px value is only a safety ceiling in NUIF; the executable report derives and stores a smaller value independently for each fixture from the measured Taffy/browser delta.
- Borrow: The signed-artefact plus CI-regeneration pattern (Yoga) and the porcelain-status freshness gate (Taffy) are appropriate for NUIF conformance fixtures that are derived rather than authored.
- Adapt: NUIF resolved-layout snapshots must record the evaluation context fingerprint, including the reference browser version, since neither project pins it and expectations are known to move with Chrome releases (Taffy changelog, Chrome 123+ alignment change).
- Adapt: Rounding must be an explicit part of the NUIF layout family contract; Taffy’s separation of unrounded and edge-rounded geometry should be mirrored so that fixtures can assert either.
- Adapt: Divergence flags (
xprefix,data-test-*,data-disabled) are untyped; NUIF should replace them with the typed fidelity classes of spec/09 (evaluator bug, target semantic difference, schema loss) as required by the layout-differential experiment. - Reject: Treating Chrome as the sole ground truth is not acceptable for a vendor-neutral draft specification; NUIF fixtures should record specification citations and, where browsers disagree (see nuif:research:css-flexbox-grid-algorithm-specs), state which behaviour is normative for the NUIF family.
- Reject: Reading input styles from the inline
styleobject couples the fixtures to CSS syntax; NUIF fixtures should be authored in the NUIF layout vocabulary and lowered to CSS, not the reverse.
NUIF executable verification
cargo xtask gate-c now applies this method without hand-edited generated expectations. The lock file selects Chrome for Testing 152.0.7977.64 revision 1669021 and Taffy is exactly pinned to 0.14.0. The generator revision is the NUIF source revision recorded in the JSON report. One deterministic seed produces the v0 card at 360, 768 and 1,440 px plus 24 stack/flex/Grid cases; the harness retains all three box maps and compares NUIF/Taffy, NUIF/browser and Taffy/browser.
The 2026-08-30 strict run evaluated 27 cases, 81 engine pairs and 1,536 box components. Eight Grid cases cover positive fixed and zero-minimum fr tracks, sparse row/column flow, explicit placement and spanning items. The complete v0, stack, flex and bounded-Grid sets passed with zero classified, blocking or unexplained divergence. Twenty-six fixtures had exact Taffy/browser agreement; one fractional Grid fixture measured 0.015594482421875 px and therefore received the fixture-local 0.02 px assertion bound. Across every case, the maximum NUIF/Taffy delta was 0.00003051757818184342 px and the maximum NUIF/browser delta was 0.015625000000056843 px. The run first exposed fill lowering as auto under non-stretch Grid alignment; explicit justify-self/align-self: stretch fixed the foreign lowering without changing the normative evaluator.
This closes the bounded explicit-Grid implementation criterion. It does not claim the broader CSS Grid surface: intrinsic, percentage, named, repeated, implicit, subgrid and masonry tracks remain capability-reported exclusions. Text-dependent Grid layout is also outside this experiment; Gate D separately pins font and shaping inputs.
Open questions
- What is the empirical divergence between Chrome, Firefox and WebKit on the Taffy fixture corpus? The corpus is Chrome-only; running it through another browser would quantify how much “CSS-compatible” behaviour is Chrome-specific.
- Should NUIF pin a Chrome for Testing version per fixture generation and store it in the resolved snapshot’s context fingerprint?
- Text fixtures rely on Ahem; NUIF text pinning (nuif:experiment:text-pinning) needs an equivalent deterministic font for layout fixtures that include shaped text.
- Taffy’s 17 excluded
xfixtures are undocumented; their content (margins withstart/end, aspect-ratio stretch fills, grid fr spans) indicates areas where Taffy and Chrome disagree and could seed NUIF’s divergence catalogue.
Taffy Rust CSS layout engine
Document status:
reviewed. Canonical source.
Summary
Taffy is a Rust layout library implementing CSS Block, Flexbox and Grid algorithms and is embedded by Servo, Bevy, Slint and other systems. Its traitified style boundary is useful for integrating a separate authored model with a standards-derived evaluator.
NUIF relevance
Recommended initial evaluator for CSS-compatible layout families, but not the canonical NUIF layout model. NUIF needs a superset vocabulary and explicit lowering/loss reports for unsupported CSS features and non-CSS layout families.
Text rendering reproducibility (shaping determinism, HarfBuzz test format, FreeType hinting and anti-aliasing modes, browser differences, pinning strategy)
Document status:
verified. Canonical source.
Summary
Text is the least reproducible part of a rendering pipeline because three independently versioned stages contribute: Unicode data and shaping (character-to-glyph mapping, positioning), glyph outline processing (hinting, interpreter version, stem darkening) and rasterisation (grayscale vs. LCD anti-aliasing, subpixel positioning, gamma and blending). HarfBuzz shows that shaping is deterministic and testable at the glyph-string level when the font bytes are pinned by hash and the shaper, font-functions backend and options are fixed; its suite stores expected output as a compact serialisation of glyph names, clusters, offsets and advances. FreeType documents that the same outline yields different bitmaps under FT_LOAD_TARGET_* modes, interpreter versions 35/38/40, native versus auto-hinting and stem darkening, and that correct gamma-aware blending is generally absent in desktop stacks. Browser suites respond by avoiding pixel comparisons for text or pinning them to one platform. A reproducible NUIF text path therefore pins font hashes, Unicode and shaper versions, disables hinting, uses grayscale area coverage with a defined subpixel-position quantum, and compares at three levels: glyph string, outline, raster.
Evidence
- HarfBuzz test recording:
record-test.shsubsets the font to the tested code points, compareshb-shapeoutput of original and subset fonts, then moves the subset intodata/in-house/fontsand names it “after its hash”; test cases go todata/in-house/testsand must be registered indata/in-house/meson.build; only open-source fonts are accepted (test/shape/README.md). - Test line format:
fontfile;options;unicodes;glyphs_expected, split on;(test/shape/run-tests.py,fontfile, options, unicodes, glyphs_expected = line.split(";")); directives such as@font-funcs=ot,ftselect backends for a file; absolute font paths may carry@<sha1>and are skipped when the on-disk SHA-1 differs (“Different version of %s found; Expected hash %s, got %s; skipping.”);*as expected output accepts any result; when glyph names are unavailable the comparison is redone with--no-glyph-names(run-tests.py). By default only theotshaper is tested (HB_SHAPER_LIST) while all supported font-funcs (ot,ft) are exercised, so expectations must hold under both FreeType and OpenType font functions (run-tests.py, “Right now we only test the ‘ot’ shaper”). - Example test line:
../fonts/df768b9c257e0c9c35786c47cae15c46571d56be.ttf;;U+0633,U+064F,...;[uni06CC.fina=10+1655|uni062A.medi=9+868|...|uni0650=2@148,0+0|...]and../fonts/SimpArabicTest.ttf;--no-positions;U+0628,...;[daggerdbl=31|c142=30|...](test/shape/data/in-house/tests/arabic-fallback-shaping.tests); cluster-level variants such as--cluster-level=2(cluster.tests). - Serialisation format: glyphs delimited by
[], separated by|; each glyph is name or index,=clusterunlessNO_CLUSTERS,@x_offset,y_offsetwhen either offset is non-zero,+x_advanceand,y_advancewhen non-zero,<x_bearing,y_bearing,width,height>withGLYPH_EXTENTS; example[uni0651=0@518,0+0|uni0628=0+1897](src/hb-buffer-serialize.cc,hb_buffer_serialize_glyphsdocumentation; harfbuzz.github.io hb-buffer reference). - Corpus: 87 in-house test files, 94 files mirrored from Unicode’s text-rendering-tests, 128 AOTS files (counts of
.testsentries in the threemeson.buildfiles). - Unicode data version: HarfBuzz’s generated UCD table is built from “Unicode 17.0.0” (
src/hb-ucd-table.hhheader). - Cluster coordinates are client data, not an implicit universal byte offset: the HarfBuzz cluster manual says each input code point receives a cluster value and that clients commonly use its code-point index;
hb_glyph_info_t.clusterreturns that value after any shaping merges. HarfRust 0.13.3 exposes the equivalentUnicodeBuffer::add(char, u32)and documents buffer length in Unicode code points (HarfBuzz manualworking-with-harfbuzz-clusters.html; docs.rsharfrust/0.13.3/harfrust/struct.UnicodeBuffer.html). - Version drift: HarfBuzz 14.4.0 (2026-08-26) changed outputs in ways that affect expectations: “Glyph positions and extents now saturate instead of overflowing” and “Arabic Windows-1256 fallback shaping is now enabled on all platforms” (
NEWS). The second item removed a platform dependency in shaping output. - Unicode text-rendering-tests: test cases are HTML snippets with rendering parameters and expected SVG; engines (FreeType+HarfBuzz+FriBidi+Raqm “FreeStack”, CoreText, Allsorts, Swash, fontkit, OpenType.js, others) emit SVG; matching “is implemented by iterating over SVG paths, allowing for maximally 1 font design unit of difference” (
unicode-org/text-rendering-tests/README.md). - FreeType load targets:
FT_LOAD_TARGET_NORMALis the default gray-level hinting;FT_LOAD_TARGET_LIGHTsnaps only vertically, keeping horizontal spacing, and “Advance widths are rounded to integer values”;FT_LOAD_TARGET_MONOis for monochrome;LCD/LCD_Vtarget decimated displays;FT_LOAD_NO_HINTING“generally generates ‘blurrier’ bitmap glyphs”;FT_LOAD_FORCE_AUTOHINT/FT_LOAD_NO_AUTOHINTchoose the engine; “A font’s native hinters may ignore the hinting algorithm you have specified (e.g., the TrueType bytecode interpreter)”; render modes NORMAL, LIGHT, MONO, LCD, LCD_V, SDF (freetype.org, Glyph Retrieval reference). - FreeType interpreter versions:
interpreter-versionaccepts 35, 38, 40; “Version 40 corresponds to MS rasterizer v.2.1; it is roughly equivalent to the hinting provided by DirectWrite ClearType”; the v40 interpreter’s approach is to “ignore all horizontal hinting instructions”, whereas v35 followed the 1990s TrueType specification (freetype.org, Driver properties; Subpixel hinting article).hinting-enginedefaults toadobeforcff,type1andt1cid; auto-hinter stem darkening is off by default (no-stem-darkeningTRUE) (Driver properties). - FreeType gamma and darkening: the correct approach is to “alpha blend it onto the surface in linear space and then apply gamma correction”; “No library supports linear alpha blending and gamma correction out of the box on X11”; gamma correction lightens text and stem darkening counteracts thinning; the Adobe CFF engine has darkened stems since 2013 (freetype.org, “Text rendering: general” LCD/gamma article).
- Browser policy: Chromium states page rendering “is influenced by many factors such as the host computer’s graphics card and driver, the platform’s text rendering system” and prefers reference tests (chromium/src
docs/testing/writing_web_tests.md); WebRender pins text reftests withplatform(linux) == isolated-text.yaml isolated-text.png, usesoptions(disable-subpixel)andoptions(disable-aa), and applies large tolerances such asfuzzy(1,3692)withfuzzy-if(platform(win),2,5585)for decoration suites (gfx/wr/wrench/reftests/text/reftest.list); WebRender’s README notes residual “differences depending on font libraries on your system” (gfx/wr/README.md). - Rust renderer practice: resvg renders text fixtures with
--skip-system-fonts --use-fonts-dir tests/fontsand fixed generic families (crates/resvg/tests/README.md); Vello keepshinting.rsandemoji.rssnapshot groups (vello_tests/tests/), andvello_cpumarks glyph caching “experimental” (sparse_strips/vello_cpu/README.md). - Fontations exposes unhinted glyph drawing through
OutlineGlyph::draw(DrawSettings::unhinted(...)); Skrifa’sPathStyle::HarfBuzzselects HarfBuzz-compatible point-stream interpretation. Zeno 0.3.3 documents 256-level anti-aliased rasterization into 8-bit alpha masks and explicit nonzero/even-odd fills (docs.rs/skrifa/0.46.2,docs.rs/zeno/0.3.3). - NUIF spec baseline: resolved text “MAY store shaped glyph runs keyed by font hashes, shaping configuration and Unicode data version” (spec/05-geometry-paint-text.md, line 9).
Mechanism
Stage 1 - shaping (deterministic given pins)
input: (font bytes -> sha256), text (Unicode scalars), script, language, direction, features, cluster level,
shaper id+version (harfbuzz 14.4.0 / harfrust x.y), unicode data version (17.0.0)
output: [glyph=cluster@dx,dy+adv|...] # HarfBuzz serialisation; compare as strings
test: fontfile@sha1;options;U+....;[expected] # one line per case; skip if hash mismatch
Stage 2 - outlines (deterministic given pins)
hinting = none (FT_LOAD_NO_HINTING / no bytecode, no autohint), no stem darkening, no synthetic emboldening
outline units: font units -> device space via exact affine; compare paths with <= 1 font-unit tolerance
(Unicode text-rendering-tests criterion)
Stage 3 - rasterisation (deterministic only on the CPU reference path)
anti-aliasing: grayscale area coverage (no LCD filtering, no MSAA)
subpixel positioning: quantise glyph origin to 1/q px in x (q declared, e.g. 4) and integer y
blending: linear-light or sRGB-space declared explicitly; gamma exponent recorded
compare: exact per-channel on the CPU path; count-and-delta or FLIP on GPU tiers
Result record: font hashes, unicode version, shaper version, hinting=off, aa=grayscale, q, blend space, pixel ratio
Executable verification
The profile-0 shaping layer pins the 22,572-byte Ahem 1.50 font at SHA-256 f0a92cd0cc45735591c9b5b1fa8aecd5194e8dc518895ca22af94a46c23550dc, HarfRust 0.13.3 and its Unicode 17.0.0 data. It assigns explicit Unicode-scalar indices through UnicodeBuffer::add, matching HarfBuzz’s documented client-defined cluster contract instead of inheriting UTF-8 byte offsets from a convenience input method. The independent fixture was captured with HarfBuzz 14.4.0 hb-shape --no-glyph-names; eight ASCII, Unicode, LTR and RTL glyph strings match exactly.
cargo xtask gate-d-text repeats each shaping and outline call, rejects missing context fonts and malformed font hashes, and repeats scene/raw-RGBA/PNG generation at 360×640, 768×768 and 1440×900. It matches five independently captured hb-vector paths after a declared normalization that removes the redundant explicit line-to-start before contour close. The render candidate uses unhinted Skrifa 0.46.2 outlines quantized to signed 26.6 font units, Zeno 0.3.3 8-bit grayscale nonzero coverage, a fixed Ahem baseline and encoded-sRGB alpha composition.
The machine report separately classifies exact shaping, exact normalized outlines, bounded hard-line semantics and raster equality on its recorded platform matrix. Profile 0 treats CRLF as one hard break and CR, LF, NEL, LINE SEPARATOR and PARAGRAPH SEPARATOR as individual hard breaks; it shapes each line independently, uses shaped advances for intrinsic width, positions by line_height, aligns to the context’s inline-start edge and clips without automatic soft wrapping. UAX #14 revision 55 for Unicode 17.0.0 defines break opportunities but permits disclosed tailoring (https://www.unicode.org/reports/tr14/tr14-55.html, retrieved 2026-08-29); ICU4X 2.2.0’s general line segmenter still documents UAX #14 version 15.1 compatibility (https://docs.rs/icu_segmenter/2.2.0/icu_segmenter/struct.LineSegmenter.html, retrieved 2026-08-29), so importing it would conflict with the pinned Unicode 17 shaping claim. The narrower no-soft-wrap contract avoids that version mismatch and does not invent an unauthored wrapping property.
The three committed text scene and raw-RGBA hashes reproduce on macOS/aarch64, Linux/aarch64 and Linux/x86_64, so cross_platform_raster_verified is true for that matrix. This is not a claim about untested systems. A separate render-profile report fixes encoded-sRGB rectangle/ellipse coverage and integer composition, and requires property-attributed fidelity for unsupported and extension-defined visuals.
The raw-RGBA boundary is intentional. Enabling the Penpot ZIP adapter’s flate2/zlib-rs feature changed all five compressed PNG hashes while the scene hashes and raw-RGBA hashes remained byte-identical to revision 79b6d96. The comparison used cargo run --locked -p nuif-testing --bin {text-pinning,render-profile} in detached old and current worktrees on macOS/aarch64; the three text RGBA hashes were 005ee1…22900, 6f2a8b…3a1f0 and 8f0cae…faf29, and the two paint hashes were 213988…b81e8 and 88ecc7…2cadd. cargo tree -e features -i flate2 identifies the ZIP feature edge. PNG repeatability remains reported, but a lossless compressor change cannot redefine pixel conformance.
NUIF relevance
Borrow
- Adopt HarfBuzz’s test-line format (font hash, options, code points, expected glyph string) for a NUIF
text-shapingfixture class, because it makes shaping conformance font-pinned, textual and diffable without any raster. - Adopt the Unicode text-rendering-tests criterion (outline paths within 1 font design unit) as the middle tier for glyph geometry, because it isolates outline correctness from rasterisation policy.
Adapt
- Pin fonts by SHA-256 of the full font file rather than HarfBuzz’s SHA-1 of a subset, because NUIF documents reference whole assets and the codec already hashes them.
- Define the normative raster policy as unhinted, grayscale area coverage with a declared subpixel quantum and blend space, because FreeType documents that every one of these choices changes the bitmap and desktop stacks disagree on defaults.
Reject
- Do not compare text rasters across platforms with system font stacks (CoreText, DirectWrite, FreeType with native hinting), because WebRender and Chromium both show this requires per-platform references or large tolerance allowances.
- Do not allow the resolved snapshot to omit the shaper and Unicode versions, because HarfBuzz 14.4.0 changed positioning outputs and NUIF’s portability reports must attribute such diffs to version drift rather than document loss.
Open questions
- Whether a future profile should keep HarfRust 0.13.3 as its normative shaper or treat it only as a reference implementation once an independent implementation reproduces the declared glyph fixtures.
- Which future profile should introduce automatic soft wrapping, and whether it should pin full UAX #9/#14 implementations or a narrower disclosed tailoring. Profile 0 intentionally avoids this unresolved dependency.
- What subpixel quantum and blend space the CPU reference path should fix; the surveyed sources document the variance but do not prescribe a value.
Tree-sitter incremental concrete syntax trees
Document status:
verified. Canonical source.
Summary
Tree-sitter maintains concrete syntax trees incrementally and can reuse unchanged structure after precisely described text edits. It preserves source ranges and exposes changed ranges between trees.
Evidence
- The official parser guide defines Tree-sitter as an incremental parser that builds concrete syntax trees and distinguishes named from anonymous nodes. Node ranges use byte offsets as well as row/column points (
https://tree-sitter.github.io/tree-sitter/using-parsers/,2-basic-parsing.html, retrieved 2026-08-29). - The official editing contract requires an
InputEditwith start, old-end and new-end byte/point positions before reparsing with the old tree. Previously retained node objects must receive the same edit or be fetched again from the edited tree (https://tree-sitter.github.io/tree-sitter/using-parsers/3-advanced-parsing.html, retrieved 2026-08-29). - Multi-language documents are supported through included ranges, and injection queries identify content that should be parsed with another language. NUIF’s bounded adapter instead takes the simpler deterministic route of parsing the HTML tree and then parsing the mapped style element’s exact raw-text range as CSS (
3-advanced-parsing.html;https://tree-sitter.github.io/tree-sitter/3-syntax-highlighting.html, retrieved 2026-08-29). - Tree-sitter grammar design explicitly targets an intuitive concrete tree whose nodes correspond to recognizable source constructs rather than a normalized abstract tree. This is the property a retentive adapter needs to retain attribute, text and declaration byte spans (
https://tree-sitter.github.io/tree-sitter/creating-parsers/3-writing-the-grammar.html, retrieved 2026-08-29).
Executable verification
nuif-html-css-0 pins Tree-sitter 0.26.10, tree-sitter-html 0.23.2 and tree-sitter-css 0.25.0. Import rejects recovery/error trees, extracts the raw style-element range from the HTML CST, validates that range with the CSS grammar and records the exact scalar byte spans used by correspondence records. Synchronization does not rely on a formatter or AST regeneration: it validates stale spans, applies replacements in descending byte order, reparses both languages and requires exact edited-document equality.
cargo xtask gate-f independently checks the complement of the edited ranges: after a token, four padding edges and escaped text change, every byte outside the six recorded spans is identical. HTML/CSS comments and an unmapped element inserted before import survive. The repeated synchronization has the same source and edit list. This verifies the Tree-sitter-based mechanism for the declared profile, not arbitrary source languages or arbitrary HTML.
cargo xtask gate-f-v0 applies the same mechanism to the complete responsive-card model. It retains 181 scalar correspondences, requires exact re-import after eight token/padding/text/responsive span edits, detects inconsistent marked media CSS and separately drives an editor-authored name/width change through CLI synchronization and CLI import. This verifies NUIF model preservation for nuif-html-css-v0; it still does not establish arbitrary HTML/CSS semantics or browser rendering for preserved path, instance and unknown kinds.
Mechanism
The adapter parses HTML into a concrete tree, locates identity-bearing elements and the mapped style raw-text node, then parses that raw range as CSS. Each semantic scalar is paired with an absolute half-open byte span. A source update first regenerates only the profile encodings of the before/after scalar values, checks that the retained span still contains the before encoding, and replaces changed values from the end of the file toward the start. A complete reparse and semantic import is the postcondition; no edited source is returned unless it equals the requested document.
NUIF relevance
Source adapters need concrete-syntax-aware minimal editing rather than AST regeneration. Tree-sitter is a strong parser substrate, but formatting/comment-preserving patch generation remains adapter-specific and may require language-native tooling for some frameworks.
Open questions
- Whether broader CSS shorthand, cascade and media-query mappings should use Tree-sitter queries or a CSS semantic parser while retaining Tree-sitter spans.
- How correspondence spans should be rebased after independent source edits that change formatting but leave mapped values equivalent; profile 0 deliberately returns
StaleSpan. - Whether a future framework adapter can preserve embedded JavaScript/TypeScript with included ranges alone or needs language-native refactoring APIs.
ttf-parser retirement after RUSTSEC-2026-0192
Document status:
verified. Canonical source.
Summary
NUIF previously pinned ttf-parser 0.25.1 for the static package-font metadata
path. RustSec advisory RUSTSEC-2026-0192 classifies the crate as unmaintained,
lists no patched versions and recommends Skrifa as an alternative. NUIF removed
the dependency instead of suppressing the advisory or maintaining an
unreviewed parser fork.
The useful parts of the former design remain: NUIF still owns sfnt directory,
range, packing, checksum, size and embedding-policy checks. Skrifa 0.46.2 now
provides names, character maps and metrics after those checks. A committed
hb-info 14.4.0 metadata capture supplies independent evidence for the exact
Ahem fixture without requiring a foreign executable in every test run.
Evidence
- RUSTSEC-2026-0192 marks all
ttf-parserversions unmaintained, lists no patched versions and names Skrifa as the alternative. Locator: RustSec advisory, issued 2026-06-29, retrieved 2026-08-30: https://rustsec.org/advisories/RUSTSEC-2026-0192.html. - The project’s security-reporting discussion remained unresolved and directed private reports to HarfBuzz infrastructure that did not cover the standalone Rust crate. Locator: issue 217, retrieved 2026-08-30: https://github.com/harfbuzz/ttf-parser/issues/217.
- The maintenance discussion records the original author’s limited time and identifies a community fork, but does not restore an upstream release and review boundary. Locator: issue 230, retrieved 2026-08-30: https://github.com/harfbuzz/ttf-parser/issues/230.
Mechanism
The retired implementation constructed ttf_parser::Face only after NUIF’s
own bounded sfnt validation and used it for names, Unicode coverage, metrics and
embedding flags. The replacement keeps the exact surrounding policy and reads
required head, maxp and OS/2 fields independently. It requires Skrifa’s
units-per-em and glyph count to agree with the direct fields and derives
embedding restrictions from the profile’s explicit bit policy.
The conformance gate no longer compares two in-process Rust parsers. It binds a pinned HarfBuzz capture to the exact font digest and compares units, glyph count, family, table inventory and a normalized Unicode-scalar hash. This is a stronger maintenance boundary but remains only one external fixture.
Alternatives and decision
Ignoring the advisory would make a known maintenance failure an undocumented
release exception. Adopting xberg-ttf-parser would transfer trust to a smaller
fork without improving NUIF’s declared static profile. FreeType would add a C
ABI and native deployment surface for a metadata-only path. Continuing a local
fork would require ongoing parser security ownership that the project has not
claimed.
NUIF therefore retires ttf-parser, adopts the already pinned Skrifa
stack for production metadata, and retains external HarfBuzz evidence as a
versioned golden. Cargo Deny must pass with no advisory exception.
NUIF relevance
Borrow the former small immutable-parser boundary and fail-closed profile.
Adapt the production implementation to Skrifa behind NUIF-owned validation, with direct required-table checks and a separately produced HarfBuzz golden.
Reject advisory suppression, an unreviewed maintenance fork, parser acceptance as a license decision, and claims of broad OpenType conformance.
Open questions
- Extend the external corpus beyond Ahem before adding TTC, CFF/CFF2, variable, color, bitmap or WOFF2 profiles.
- Add a reproducible pinned HarfBuzz capture job without making native HarfBuzz a release-build dependency.
- Consider FreeType or a browser font stack as a third oracle only with a measured sandbox and exact version provenance.
Screenshot-to-code and interaction-inference limits
Document status:
reviewed. Canonical source.
Summary
DCGen divides a screenshot into smaller regions, describes and generates those regions, then reassembles the result. Its reported improvement supports hierarchical crops as an ablation, but it still generates a possible frontend from pixels rather than recovering original source. Design2Code independently shows persistent element-recall and layout errors on real webpages. Visually reconstructing a static state also does not establish its interaction or state model.
Evidence
- Wan et al., DOI 10.1145/3729364 / arXiv:2406.16386, identifies omission, distortion and arrangement failures and reports up to a 14% visual-similarity improvement from divide-and-conquer generation on its studied models/data.
- DCGen segments the screenshot, generates descriptions and code for manageable regions and then integrates them; the reported percentage is not a NUIF benchmark or a guarantee on unseen models.
- Design2Code (ACL Anthology 2025.naacl-long.199) uses 484 real pages and also identifies element recall and layout as material weaknesses.
Mechanism
Hierarchical segmentation preserves more local detail than a single resized full-screen image and lets a model focus on smaller visual relationships. The integration step must still reconcile coordinate spaces, shared styles, hierarchy and responsive constraints. NUIF can make those joins explicit in an observation graph and typed operations.
NUIF relevance
Pixels are evidence, not authored truth. Hierarchical crops should be tested against a deterministic/OCR baseline and one-shot proposal under identical budgets. Inferred semantics, layout and behavior carry confidence/provenance and cannot be classified as lossless without stronger source evidence.
Open questions
- Which deterministic segmentation is stable enough across viewports to share proposed identities?
- Does local detail improve final typed structure or only pixel similarity?
- How should overlapping region proposals be merged without duplicate entities?
W3C UI Specification Schema Community Group
Document status:
reviewed. Canonical source.
Summary
The group proposed a common implementation-agnostic meta-model for UI design, layout, behavior, constraints, accessibility and QA with a JSON/JSON-Schema deliverable and coordination with Open UI and DTCG. W3C records show the group closed on 21 May 2026.
NUIF relevance
This is direct prior art and evidence that the problem is recognized. NUIF should study why a schema-only standards effort failed to sustain momentum and differentiate through executable semantics, renderer/layout reference implementations, conformance fixtures, bidirectional synchronization and neutral editor proof rather than a field catalog alone.
Unity prefab overrides, YAML identity and UnityYAMLMerge
Document status:
reviewed. Canonical source.
Summary
Unity serializes scenes and prefabs as a sequence of YAML documents, one per engine object, each addressed by a signed 64-bit local identifier (fileID) and a class identifier tag. Cross-file references combine the target file’s .meta GUID with a fileID. Prefab instances are not expanded in the containing file; a PrefabInstance document stores a source reference and an override set (m_Modifications, added/removed components and GameObjects), and placeholder documents marked stripped stand in for referenced nested objects. Prefab variants reuse the same mechanism with a parentless root instance. UnityYAMLMerge performs a three-way merge of these files by treating specified arrays as identity-keyed sets, excluding volatile paths, and comparing floats with tolerances; it falls back to a user-specified textual tool for unresolved conflicts. Text serialization must be enabled for any of this to apply. Known failure classes are order instability of override lists, local-only identity that differs between prefab assets, and a YAML dialect that is not meant to be externally produced.
Evidence
- Unity writes each object of a scene as a separate YAML document introduced by
---; the tag!u!<n>encodes the class ID and&<n>the object’s file-local ID. Unity Manual, “Format description” (Manual/FormatDescription.html), retrieved 2026-08-29. - Header lines are
%YAML 1.1and%TAG !u! tag:unity3d.com,2011:; references to other objects are{fileID: n}; asset references are{fileID: n, guid: <32 hex>, type: t}; the scene ends with aSceneRootsdocument listing ordered root transforms. Unity Manual, “YAML scene example” (Manual/YAMLSceneExample.html), retrieved 2026-08-29. - A prefab instance is a document of class ID
1001and typePrefabInstancewithm_SourcePrefab: {fileID: 100100000, guid: ..., type: 3};100100000is the prefab asset handle created at import. Unity Manual 6000.6, “YAML serialization of prefabs” (Manual/yaml-prefab-serialization.html), retrieved 2026-08-29. - The
m_Modificationblock containsm_TransformParent,m_Modifications,m_RemovedComponents,m_RemovedGameObjects,m_AddedGameObjects,m_AddedComponents. Eachm_Modificationsentry hastarget,propertyPath,value,objectReference. Same page. - Referenced nested objects appear as placeholder documents tagged
stripped, carrying onlym_CorrespondingSourceObject,m_PrefabInstanceandm_PrefabAsset. Same page; also Unity blog “Understanding Unity’s serialization language, YAML” (N. A. Borromeo), section “Prefab instances, Nested Prefabs, and Variants”, retrieved 2026-08-29. - A variant is identified by
m_Modification.m_TransformParentequal to{fileID: 0}on the rootPrefabInstance. Same manual page; same blog section. fileIDis local to a file and “can be repeated in different files”; cross-file identity is (GUID from.meta, fileID). Unity blog, section on cross-file references, retrieved 2026-08-29.- Replacing a nested prefab’s GUID by hand loses overrides because object
fileIDs in the replacement prefab “will differ” from those referenced bym_CorrespondingSourceObject. Unity blog, same section (NUIF reading: identity is asset-scoped, not semantic). - Local file IDs “are signed 64-bit values and can be negative”;
GlobalObjectIdcasts them toulong, so the sign is lost; the docs advise not relying ontargetObjectIdto find an object. Unity Scripting API,GlobalObjectId, retrieved 2026-08-29. - Overrides on a prefab instance are property values, added/removed components, and added/removed child GameObjects; an overridden value “always takes precedence” over the asset value; root instance position and rotation are not explicit overrides. Unity Manual, “Prefab instance overrides” (
Manual/PrefabInstanceOverrides.html), retrieved 2026-08-29. - A variant “inherits properties from a base prefab”; overrides take precedence; variants can be based on variants; “Apply all to Prefab Variant parent” pushes overrides one level up. Unity Manual, “Prefab variants” (
Manual/PrefabVariants.html), retrieved 2026-08-29. - Nested prefabs “keep their links to their own prefab assets” while forming part of another prefab; adding one from the Hierarchy is itself recorded as an override. Unity Manual, “Nested prefabs” (
Manual/NestedPrefabs.html), retrieved 2026-08-29. - Asset Serialization Mode defaults to Force Text; the setting exists “to help with version control merges”; a separate option writes references on one line “which reduces version control noise”. Unity Manual, “Editor settings” (
Manual/class-EditorManager.html), retrieved 2026-08-29. UnityYAMLMergeis shipped inEditor/Data/Tools(Windows) andUnity.app/Contents/Helpers(macOS); it can be run from the command line and configured as a merge driver for P4V, Git, Mercurial, SVN, TortoiseGit, UVCS and SourceTree;mergespecfile.txtdeclares fallback tools for unresolved conflicts. Unity Manual, “Smart merge” (Manual/SmartMerge.html), retrieved 2026-08-29.mergerules.txthas four sections. Arrays:set *.GameObject.m_Component *.fileID,set *.Prefab.m_Modification.m_Modifications target.fileID target.guid propertyPath,plain *.MeshRenderer.m_Materials,plain *.Renderer.m_Materials; the default for unlisted arrays is a hybrid heuristic match. Exclusions: paths such as*.SpriteRenderer.m_ColorandexcludeIfContains *.MonoBehaviour.* x y z; excluded paths modified on both sides become conflicts. Comparisons: relative/absolute epsilons such asfloat *.Transform.m_LocalPosition.x 0.0000005andfloat *.Transform.m_LocalRotation.x 0.00005 0.001. Unity Manual 6000.4, “Smart merge”, retrieved 2026-08-29.- UnityYAML “does not support the full YAML specification”; the manual states that users “cannot externally produce or edit UnityYAML files”; unsupported features include comments, multiple documents in the YAML sense, tags and complex keys. Unity Manual, “UnityYAML” (
Manual/UnityYAML.html), retrieved 2026-08-29. - Unity staff (MirceaI) state
m_Modificationsis sorted bytargetthenpropertyPath, but the internal representation oftargetdepends on load order, so entries with different GUIDs are “not … stable in different Editor sessions”, producing spurious diffs; reported on 2022.3.20 long-term support (LTS) with references back to 2019. Unity Discussions thread 943063, retrieved 2026-08-29.
Mechanism
Data model. A file is an ordered list of documents (classID, fileID, body). fileID is an int64 unique within the file. A reference is either intra-file {fileID} or inter-file {fileID, guid, type} where guid is the 128-bit identifier stored in the referenced asset’s .meta file and type distinguishes built-in, importer-generated and native assets. A GameObject owns an ordered m_Component array of references; a Transform owns m_Father and an ordered m_Children array; containment is therefore encoded twice (parent pointer and child list) and both must agree.
Prefab instances. Instead of copying the source hierarchy, the file stores one PrefabInstance document. Its override set is a list of (target, propertyPath, value, objectReference) tuples where target is an inter-file reference to an object inside the source prefab, and propertyPath is a dotted path into that object’s serialized property tree (m_LocalPosition.x, m_Name, array indices). Structural overrides are separate lists: removed components, removed GameObjects, added GameObjects, added components. Any object in the instance that must be referenced from the containing file (as a parent transform or as a reference target) is materialized as a stripped placeholder document with its own fileID; the placeholder records m_CorrespondingSourceObject (identity in the source asset) and m_PrefabInstance (the owning instance). Resolution reconstructs the full object graph by instantiating the source prefab, applying m_Modifications in order, applying structural add/remove lists and binding placeholders to the instantiated objects. A prefab variant is a prefab file whose root is a PrefabInstance with m_TransformParent = {fileID: 0}; nesting and variants are therefore the same mechanism composed recursively (variant of variant, instance inside variant).
Authored versus resolved. The saved file contains only authored opinions: the source reference and the sparse override set. The Editor materializes the resolved GameObject graph in memory; Apply moves overrides down into the asset and Revert deletes them. Because the file omits the resolved graph, any change in the source asset is reflected on load. Root position/rotation are treated as always-instance-local and are not counted as overrides.
Merge. UnityYAMLMerge parses base, ours and theirs into document trees keyed by (classID, fileID). Within a document it merges mappings key-wise. Arrays declared set are matched by the listed key paths (for m_Component, by fileID; for m_Modifications, by (target.fileID, target.guid, propertyPath)), so insertions and removals on both sides merge without positional conflicts. Arrays declared plain merge positionally. Unlisted arrays use a heuristic hybrid. Excluded paths are never auto-merged; if both sides changed them the result is a conflict. Float comparison uses per-path epsilons so that re-serialization noise is not reported as change. Unresolved conflicts are delegated according to mergespecfile.txt, typically to an interactive textual tool over the partially merged file. The tool is a structural three-way merge over identity-keyed trees rather than a semantic merge: it does not know that m_Father and m_Children must agree, nor that an override’s target must exist in the referenced prefab.
Failure classes (source-documented unless marked interpretation).
- Identity scope:
fileIDis file-local; identity of the “same” object in two prefab assets is unrelated, so replacing a source prefab invalidatesm_CorrespondingSourceObjecttargets and overrides are lost (Unity blog). - Sign loss: negative
fileIDs are reinterpreted inGlobalObjectId(Scripting API). - Ordering instability:
m_Modificationsorder depends on an internaltargetrepresentation that varies with load order, producing spurious diffs and merge noise (Discussions 943063). - Dual encoding of containment: parent pointer and child arrays can be merged independently and disagree (interpretation from the format description; the
setrule keyed onfileIDform_Componentdoes not coverm_Children). - Dialect closure: the YAML subset is declared not externally producible, so third-party tooling has no conformance target (UnityYAML manual page).
- Prerequisite: none of this works unless Force Text is enabled and the merge driver is installed per version control system (Editor settings; “Smart merge” manual page).
NUIF relevance
Borrow
- Sparse override sets addressed by
(target identity, property path)with separate structural add/remove lists; this is the minimal information needed to keep an instance non-destructive and matches NUIF’sapply instance overrideoperation. - Identity-keyed set merge for child and override lists, with explicit exclusion and tolerance rules declared in a rules file; NUIF’s three-way merge can express the same as typed merge policies per relation kind.
- Force-text plus one-line references as a canonicalization concern: NUIF’s
nuif-text-0profile should define a deterministic serialization precisely so that merge tools see only semantic change.
Adapt
- Replace file-local
int64identity with NUIF’s stable semantic entity IDs so that the same entity is addressable across component definitions, variants and documents; Unity’s failure class 1 disappears when override targets are semantic IDs rather than (asset GUID, local ID) pairs. - Encode containment once (ordered relation or fractional index) and derive parent pointers, so that merge cannot produce disagreeing parent/child encodings.
- Make override ordering canonical by a total order over
(target id, property key)defined in the spec, eliminating failure class 3. - Represent placeholder (“stripped”) objects as explicit correspondence records in the provenance layer instead of pseudo-entities in the containment tree.
Reject
- A merge tool that is separate from the document model and driven by path-pattern rules; NUIF merge must be defined over typed operations and relations (spec/06) so that structural invariants (acyclic containment, target existence) are checked during merge.
- Treating root transform properties as implicitly instance-local; NUIF should make every instance-level deviation an explicit override with fidelity accounting.
- A closed serialization dialect that third parties may not produce; NUIF serialization profiles are normative and externally implementable.
Open questions
- How
UnityYAMLMergematches objects when the samefileIDis created independently on both branches (collision on newly added objects); no primary source retrieved describes the generation algorithm for local IDs. - Whether the hybrid heuristic for unlisted arrays is stable across Unity versions; the manual does not specify it.
- Whether structural overrides (
m_RemovedGameObjects, introduced later than property overrides) interact correctly withsetmatching inmergerules.txt, whose default only listsm_Modifications.
Preservation of unknown entity kinds and extension payloads; OTIO UnknownSchema, Godot MissingNode, USD typeName, glTF extensions, Protocol Buffers unknown fields and CBOR tag 24 compared
Document status:
reviewed. Canonical source.
Summary
Six systems preserve data they do not understand, with three distinct preservation units. OpenTimelineIO (OTIO) and Godot preserve whole objects of unknown type: OTIO instantiates UnknownSchema holding the original schema name, version and dictionary, and Godot instantiates MissingNode/MissingResource recording the original class and every assigned property, writing both back under the original name on save. OpenUSD preserves unknown type names as plain metadata on a prim whose properties compose normally and lets a writer declare fallbackPrimTypes for older readers. glTF preserves extension objects and extras per property with a document-level declaration split into extensionsUsed (all) and extensionsRequired (subset); the validator grades an undeclared extension as Error and an unsupported one as Information. Protocol Buffers retain unknown fields and unknown enum values as raw wire data since 3.5 and re-emit them, but lose them on JSON conversion and field-by-field copying. RFC 8949 tag 24 wraps an embedded CBOR item as a byte string that is not decoded with its container, and §5.4 recommends that decoders pass unknown tags through with a marker rather than fail. WebAssembly custom sections and HTML’s HTMLUnknownElement show the same pattern at the container and DOM levels: unknown content is kept, is inert for the core semantics and must not invalidate the container.
Preservation is structural (typed values re-encoded by the writer) in OTIO, Godot and USD, and byte-level only in Protocol Buffers and CBOR tag 24. No retrieved system defines both, and none states a canonicalization rule for preserved payloads. Source statements are reported in ## Evidence; the NUIF interpretation follows in ## NUIF relevance.
Evidence
- OTIO
UnknownSchemaderives fromSerializableObjectand storesoriginal_schema_name,original_schema_versionand the raw dictionary;read_fromtakes the whole dictionary minusOTIO_SCHEMA;write_tore-emits every stored key under the original label; nested known objects inside the dictionary are decoded.src/opentimelineio/unknownSchema.hlines 12–40,unknownSchema.cpplines 8–45,tests/test_unknown_schema.pylines 9–100 (nuif:research:opentimelineio, retrieved 2026-08-29). - OTIO typed slots:
Reader::read(key, Retainer<T>*)performsdynamic_cast<T*>and reportsErrorStatus::TYPE_MISMATCH“Expected object of type …; read type … instead” when the object is not aT;Compositionchildren arestd::vector<Composable*>.src/opentimelineio/serializableObject.hlines 162–192;composition.cpplines 55–77, main, retrieved 2026-08-29. AnUnknownSchemaobject in aComposable-typed slot therefore fails to load; this is inferred from the code and was not executed. - OTIO version handling: an object whose version is newer than registered is rejected with
SCHEMA_VERSION_UNSUPPORTED; older versions are upgraded through registered dictionary transforms.typeRegistry.cpplines 360–420 (nuif:research:opentimelineio). - Godot: when
ClassDB::instantiatefails,SceneState::instantiatecreatesMissingNode, setsoriginal_classandrecording_properties = true;MissingNode::_setrecords any property while recording; on pack_parse_nodewritesoriginal_classas the node type; signals were recorded only from PR #105449 (merged 2025-10-10).scene/resources/packed_scene.cpp,scene/main/missing_node.cpp, master; PR #60597 (merged 2022-05-05): “missing types no longer cause data loss” (nuif:research:godot-tscn-scene-format, retrieved 2026-08-29).MissingNodeis aNode, so it participates in tree operations; the class reference calls it “an internal editor class intended for keeping the data of unrecognized nodes” and emits the configuration warning “This node was saved as class type ‘%s’, which was no longer available when this scene was loaded.” (Godot docs, classMissingNode;missing_node.cpp). - Godot placeholder values are typed by the runtime
Variantinferred from the value, and property order and formatting are regenerated by the writer; byte-identical round trips are not claimed (nuif:research:godot-tscn-scene-format,## Mechanismand## Open questions). - OpenUSD:
UsdPrim::GetTypeName“returns the composed type name as authored”; unknown type names compose and round-trip;fallbackPrimTypeslets “prims with the unrecognized type name … be treated as having the effective schema type of the first recognized type in the list”; properties outside any schema arecustom, “the same function as Alembic’s ‘userProperties’”.pxr/usd/usd/prim.hlines 192–204,property.hlines 179–185, https://openusd.org/release/api/_usd__page__object_model.html “Fallback Prim Types” (nuif:research:openusd-composition-and-crate, retrieved 2026-08-29).UsdValidationErrorType { None, Error, Warn, Info };usdchecker --strictescalatesWarnto failure (pxr/usdValidation/usdValidation/error.hlines 37–42;usdchecker.cpplines 217–218). - glTF 2.0: “Any glTF object MAY have an optional
extensionsproperty”; “All extensions used in a glTF asset MUST be listed in the top-levelextensionsUsedarray”; “All glTF extensions required to load and/or render an asset MUST be listed in the top-levelextensionsRequiredarray”; “extensionsRequiredis a subset ofextensionsUsed”.specification/2.0/Specification.adoclines 2639–2689, main, retrieved 2026-08-29.extrasis “Application-specific data” that “SHOULD be a JSON object rather than a primitive value for best portability” (schema/extras.schema.json;schema/glTFProperty.schema.jsonattachesextensionsandextrasto every property object). Extension registry rule: “If lack of extension support prevents proper geometry loading, extension specification must state that (and such extension must be mentioned inextensionsRequired)”.extensions/README.mdline 182. - glTF-Validator
ISSUES.md:UNDECLARED_EXTENSIONError “Extension is not declared in extensionsUsed.”;UNSUPPORTED_EXTENSIONInformation “Cannot validate an extension as it is not supported by the validator”;UNUSED_EXTENSION_REQUIREDError;UNEXPECTED_EXTENSION_OBJECTError “Unexpected location for this extension.”;NON_REQUIRED_EXTENSIONError “Extension ‘%1’ cannot be optional.”;UNEXPECTED_PROPERTYWarning;EXTRA_PROPERTYInformation;UNKNOWN_ASSET_MAJOR_VERSIONError;UNKNOWN_ASSET_MINOR_VERSIONWarning. https://github.com/KhronosGroup/glTF-Validator/blob/main/ISSUES.md, retrieved 2026-08-29 (line numbers in nuif:research:gltf-validator-and-sample-assets). - Protocol Buffers: “Proto3 messages preserve unknown fields and include them during parsing and in the serialized output, which matches proto2 behavior”; before 3.5 proto3 dropped unknown fields; “unrecognized enum values will be preserved in the message” and “will still be serialized with the message”; unknown fields are lost on JSON serialization and field-by-field copying, so “message-oriented APIs, such as CopyFrom() and MergeFrom()” are recommended. https://protobuf.dev/programming-guides/proto3/ “Unknown Fields” and enum sections, retrieved 2026-08-29.
- RFC 8949 §3.4.5.1: “Tag number 24 (CBOR data item) can be used to tag the embedded byte string as a single data item encoded in CBOR format. Contained items that aren’t byte strings are invalid. A contained byte string is valid if it encodes a well-formed CBOR data item; validity checking of the decoded CBOR item is not required for tag validity”. §5.4: for an unrecognised tag or simple value a decoder “can report an error (and not return data). Note that treating this case as an error can cause ossification and is thus not encouraged” or “can emit the unknown item … and then give the application an indication that the decoder did not recognize that tag number”; the latter “provides forward compatibility”. §7.1: implementations “can choose to process just the enclosed tag content or, preferably, to process the tag as an unknown tag number wrapping the tag content”. §4.2.1 lists the core deterministic encoding requirements (preferred serialization, minimal argument lengths, no indefinite lengths, bytewise-lexicographic map key order). https://www.rfc-editor.org/rfc/rfc8949.txt lines 1189–1199, 1726–1760, 2130–2134, 1382–1443, retrieved 2026-08-29.
- WebAssembly: custom sections “are intended to be used for debugging information or third-party extensions, and are ignored by the WebAssembly semantics”; they consist of “a name further identifying the custom section, followed by an uninterpreted sequence of bytes”; “If an implementation interprets the data of a custom section, then errors in that data, or the placement of the section, must not invalidate the module.” https://webassembly.github.io/spec/core/binary/modules.html “Custom Section”, retrieved 2026-08-29.
- HTML: element interface lookup ends “If name is a valid custom element name, then return HTMLElement. Return HTMLUnknownElement.”;
HTMLUnknownElementis anHTMLElementwithout[HTMLConstructor]. https://html.spec.whatwg.org/multipage/dom.html, retrieved 2026-08-29. - NUIF current state:
Extensions(pub BTreeMap<String, Vec<u8>>)onDocumentandEntity;EntityKindhas no unknown variant;Fidelity::PreservedUnrenderable { extension };Operation::SetExtension { entity, namespace, payload: Vec<u8> }.crates/nuif-core/src/lib.rs,crates/nuif-protocol/src/lib.rsat commit af8d5cb. spec/07 requires preservation “byte/value-for-byte at their attachment point”; RFC 0002 requires survival across “load/save/edit cycles unless the owner is deleted” and states that unsupported required extensions “block claims of faithful rendering but do not necessarily block structural editing”.
Mechanism
Whole-object placeholders (OTIO, Godot). The reader dispatches on a type label; when the registry lookup fails it constructs a placeholder that implements the same interface as a known object (OTIO SerializableObject, Godot Node), records the label and every field, and answers the writer’s field enumeration with the recorded pairs so that the writer emits the original label. The placeholder is inert: no migrations run, no behaviour executes, and the editor warns. Two limits follow from the design. First, the placeholder is accepted only where the containing slot’s type admits the placeholder’s base class; OTIO’s typed Retainer<T> slots reject it with TYPE_MISMATCH, so unknown objects survive only in untyped maps such as metadata. Second, values are re-typed through the host value model (OTIO AnyDictionary, Godot Variant) and re-serialized by the host writer, so the round trip is value-preserving, not byte-preserving.
Type as metadata (USD). The type name is a field like any other; every property is stored regardless of schema membership; unknown types therefore need no placeholder because nothing is dispatched on the type at load time. Behaviour attached to the type (schema fallbacks, validators) is absent for unknown names, and fallbackPrimTypes is an author-declared substitution list evaluated by older readers. Preservation is complete for anything expressible in USD’s value model and undefined for foreign encodings.
Attachment-point blobs with declarations (glTF, Protocol Buffers, CBOR, WebAssembly). Unknown content is attached to the object it describes (extensions.<name>, unknown field numbers, tag 24 byte strings, named custom sections) and the container’s core semantics ignore it. glTF adds a document-level contract: every extension present must be declared, and the required subset gates loading; the validator makes the declaration, not the understanding, the pass/fail criterion. Protocol Buffers shows the failure mode of codec-only retention: any path that reconstructs the message field by field (JSON, per-field copy) drops the unknowns, which is the gap RFC 0002 names when it says preservation “goes beyond codec unknown fields”.
Canonicalization of preserved data. RFC 8949 deterministic encoding constrains the encoder of a data item; a payload the implementation does not decode cannot be re-encoded and must be treated as a byte string, which the deterministic rules cover (minimal length argument, no indefinite length). Tag 24 is the standard marker that a byte string is itself CBOR without requiring the container decoder to decode it. Hashing the byte string yields a stable canonical hash regardless of whether the embedded item is itself deterministically encoded; the embedded item’s own canonical form is the responsibility of the implementation that declares the namespace.
Severity and negotiation. glTF distinguishes undeclared (Error, the document is malformed), declared and unsupported (Information, the document is valid and the feature is degraded) and required and unsupported (loading fails by specification). USD offers --strict to promote warnings. Both make severity a property of the declaration state rather than of the implementation’s coverage.
NUIF relevance
Borrow
- The OTIO/Godot placeholder shape: an
EntityKind::Unknownvariant carrying namespace, kind name, schema version and the kind-specific payload, implementing the same containment interface as known kinds so that move, rename, delete and reparent apply unchanged (GodotMissingNodeis aNode). - USD’s separation of core fields from type: NUIF core properties (
nuifnamespace), name, children, relations and extensions are typed and editable on an unknown entity; only the kind-specific payload is opaque. - glTF’s declaration contract and severity mapping: unknown namespace declared in
extensions_usedis Information; present but undeclared is Error; declared inextensions_requiredand unsupported is a fidelity block, not a load failure (RFC 0002). - RFC 8949 tag 24 as the encoding of opaque payloads in
nuif-cbor-0, and the §5.4 rule that decoders pass unknown items through with a marker instead of failing. - WebAssembly’s rule that errors inside uninterpreted custom data must not invalidate the container: a malformed opaque payload is a diagnostic on the owning entity, not a document rejection.
- Protocol Buffers’ warning that field-by-field reconstruction loses unknowns: NUIF lowering, flattening and codec conversion passes must carry
UnknownandExtensionsthrough explicitly, and a conformance fixture must assert this.
Adapt
- OTIO’s typed-slot limitation must not be replicated: every containment slot in NUIF admits
EntityKind::Unknown; relation endpoints typed to a specific kind treat an unknown target asFidelity::PreservedUnrenderable, not as a load error. - OTIO rejects newer schema versions; NUIF should instead demote a known kind with a newer
schema_versionthan the implementation supports toUnknownwith the same payload, so forward-compatible editing remains possible (the open question in nuif:research:opentimelineio). - Godot’s value-preserving round trip becomes byte-preserving in NUIF for uninterpreted payloads, and value-preserving (re-encoded deterministically) for payloads whose namespace the implementation declares; RFC 0002’s “byte/value-for-byte” should be split into these two normative cases.
fallbackPrimTypesbecomes a per-namespacefallback_kinddeclaration in the extension registry (for example, a vendor chart kind falls back toContainer), so layout treats the unknown entity as its fallback kind with authored size intents; without a declaration the fallback isContainerwithSizeIntent::Auto.- glTF
extras(schema-less, per object) maps to a reservednuif.extrasextension namespace rather than a separate field, keeping one attachment mechanism.
Reject
- Silent drop of unknown kinds or of payloads that fail the implementation’s own decoder (pre-3.5 proto3 behaviour; OTIO
TYPE_MISMATCHon typed slots). - Re-typing opaque payloads through the host value model (Godot
Variantinference), because the canonical hash would then depend on the host’s re-encoding. - Hard failure on a newer schema version (OTIO
SCHEMA_VERSION_UNSUPPORTED) for kinds whose core fields still parse. - Environment-variable or ambient selection of fallback behaviour; capability declarations are explicit inputs to load and validate.
Open questions
- Whether
nuif-text-0should render an opaque CBOR payload as a byte string only, or additionally as non-normative CBOR diagnostic notation in a comment; the second form aids review but must not participate in hashing or parsing. - Whether an implementation that declares a namespace may rewrite that namespace’s payload on every save (canonical re-encoding) or only when the value changed; the first churns diffs for documents authored by a different implementation of the same namespace.
- How relation operations typed to a kind (for example,
Instance { component }) behave when the target isUnknown; the fidelity mapping above is a proposal without a fixture. - The OTIO typed-slot behaviour was inferred from
serializableObject.handcomposition.cppand not confirmed by executing a fixture with an unknown schema inside a track. - Godot’s byte-level round-trip stability for
MissingNoderemains unverified (nuif:research:godot-tscn-scene-format).
Unreal Engine asset versioning, transactions, automation framework and asset diffing
Document status:
reviewed. Canonical source.
Summary
Unreal Engine serializes assets through FArchive with three independent version streams: an Epic engine object version, a licensee object version, and any number of custom versions keyed by FGuid and registered at startup. Each Serialize implementation declares which custom versions it uses and branches on the stored value, which yields backward compatibility by construction; forward compatibility is refused (newer assets are hidden and references become null). Editor undo is a command-pattern transaction system: FScopedTransaction brackets a transaction, UObject::Modify() snapshots an object into the transaction buffer before mutation, and UTransBuffer keeps an undo/redo stack of FTransaction records bounded by memory. The Automation Framework runs C++-registered tests from the Session Frontend or headlessly through -ExecCmds="Automation RunTests ...;Quit" with -nullrhi -unattended, exports JSON/HTML reports, and includes screenshot comparison with per-channel tolerance, local and global error budgets and anti-aliasing tolerance. Commandlets provide a raw headless execution environment. Asset diffing exports a text form for generic tools and uses a Blueprint-specific graph diff for Blueprints.
Evidence
FEngineVersioncarries major, minor, patch (uint16), changelist (uint32) and branch name; assets saved in a newer engine “will simply not show up in the Content Browser, and any references to them will be treated as null”, with a data-loss risk on re-save. Epic docs, “Versioning of Assets and Packages in Unreal Engine”, sections “Engine Version” and loading behaviour, retrieved 2026-08-29.- Object-level versions:
EUnrealEngineObjectUE5Version(Epic) andEUnrealEngineObjectLicenseeUEVersion(licensee). Custom versions: aconst FGuidplus a globalFCustomVersionRegistrationobject, e.g.FCustomVersionRegistration GRegisterAnimationCustomVersion(FAnimationCustomVersion::GUID, FAnimationCustomVersion::LatestVersion, TEXT("AnimGraphVer"));. Same page, section “Custom versions”. FArchiveaccessors:UEVer(),LicenseeUEVer(),UsingCustomVersion(FGuid),CustomVer(FGuid); example override callsAr.UsingCustomVersion(FFrameworkObjectVersion::GUID)then branches onAr.CustomVer(...) < FFrameworkObjectVersion::WheelOffsetIsFromWheel. Same page, “Serialize override” example.- Rule: the version associated with a registered
FGuid“is assumed never to decrease”, which is what lets the engine refuse newer assets while loading older ones. Same page. UsingCustomVersion“registers the custom version to the archive” and has no effect on loading archives;CustomVerqueries a custom version and, when writing, requires prior registration. Epic API docsFArchive::UsingCustomVersionandFArchive::CustomVer(via search summary of the 5.6 API pages), retrieved 2026-08-29.FScopedTransaction: “Delineates a transactable block; [Begin()]s a transaction when entering scope, and [End()]s a transaction when leaving scope”; header/Engine/Source/Editor/UnrealEd/Public/ScopedTransaction.h; constructors take a sessionFTextandbShouldActuallyTransact;Cancel()is reentrant;Indexstores the transaction index. Epic API docs 5.8,FScopedTransaction, retrieved 2026-08-29.UTransBufferis the “Transaction tracking system, manages the undo and redo buffer”; membersUndoBuffer: TArray<TSharedRef<FTransaction>>,UndoCount,MaxMemory(“Maximum number of bytes the transaction buffer is allowed to occupy”),ActiveCount,ActiveRecordCounts;End()succeeds only when the action counter is 1. Epic API docs 5.8,UTransBuffer, retrieved 2026-08-29.FTransactionis “A single transaction, representing a set of serialized, undo-able changes to a set of objects”, header/Engine/Source/Editor/UnrealEd/Classes/Editor/Transactor.h; innerFObjectRecord; methodsSaveObject,SaveArray,StoreUndo,Apply(“Enacts the transaction”),Finalize(“try and work out what’s changed”),BeginOperation/EndOperation. Epic API docs 5.8,FTransaction, retrieved 2026-08-29.UObject::Modify(bool bAlwaysMarkDirty): if the engine is recording into the transaction buffer, saves a copy of the object into the buffer and marks the package dirty; returns whether the object was saved;SaveToTransactionBufferis the underlying function. Epic API docs 5.1UObject::ModifyandSaveToTransactionBuffer(via search summary; the 5.8 API page is client-rendered and returned no body), retrieved 2026-08-29.- Automation test categories: unit, feature, smoke (“complete within 1 second”, run at every start), content stress, screenshot comparison; built in C++ in core modules, independent of the
UObjectenvironment; run from Window > Test Automation in the Session Frontend. Epic docs, “Automation Test Framework in Unreal Engine”, retrieved 2026-08-29. - Command-line forms:
-ExecCmds="Automation RunTest Test1+Test2;Quit",... RunTest MySet.MySubSet;Quit,... RunTest Group:MyGroup;Quit;-ReportExportPath="<path>"writes JSON plus HTML;-ResumeRunTestresumes an interrupted run. Epic docs 5.8, “Run Automation Tests in Unreal Engine”, retrieved 2026-08-29. -ExecCmds“Execute the specified console commands”;-unattendeddisables dialogs for unmonitored runs;-nullrhi“Use null rendering hardware interface to run UE headless”;-stdout,-BUILDMACHINE,-NoShaderCompileare documented. Epic docs 5.8, “Unreal Engine Command-Line Arguments Reference”, retrieved 2026-08-29.- Commandlets “are executed in a ‘raw’ environment, in which the game isn’t loaded … no levels are loaded, and no actors exist”; entry point
virtual int32 Main(const FString& Params); flagsIsClient,IsEditor,IsServer,LogToConsole; the name suffixCommandletis appended automatically. Epic API docs 5.8,UCommandlet, retrieved 2026-08-29. - Screenshot comparison stores results under
Saved/Automation/Comparisons; the first run requires approving a ground-truth image through the Screenshot Browser, which creates a source-control changelist. Epic docs, “Screenshot Comparison Tool in Unreal Engine”, retrieved 2026-08-29. AutomationScreenshotOptionsproperties:resolution,delay,frame_delay,override_time_to(“Sets Delta Time to 0”),disable_noisy_rendering_features(disables anti-aliasing, motion blur, screen-space reflections, eye adaptation, tonemapper, contact shadows),disable_tonemapping,visualize_buffer,tolerance(quick defaults, “we default to low”),tolerance_amount(per channel and brightness),maximum_local_error,maximum_global_error,ignore_anti_aliasing(search neighbouring pixels),ignore_colors(compare luminance only). Epic Python API 5.4,unreal.AutomationScreenshotOptions, retrieved 2026-08-29.- Asset diffing: any asset can be exported to a readable text format and diffed with a user-configured external tool (
Diff Against Depot, history-pair diff); Blueprints use a built-in graph/defaults diff. M. Noland, “Diffing Unreal Assets” (originally on the Unreal Engine blog, 2014-03-28), retrieved 2026-08-29. - UE Diff Tool supports “Blueprints, Blueprint adjacent types”; unchanged nodes appear grey; red = present on left only, green = right only, cyan = changed, grey = moved nodes/comments; entry points
Diff > DepotandDiff Selected. Epic docs 5.8, “UE Diff Tool in Unreal Engine”, retrieved 2026-08-29. DiffAssetsis exposed as an editor scripting function that “tries to diff two assets using class-specific tool”, doing nothing if classes differ. Epic Blueprint API,AssetTools/DiffAssets(via search summary), retrieved 2026-08-29.
Mechanism
Versioning. A package header records the engine version and a container of (FGuid, int32) custom versions that the writer touched. Registration is static: a global FCustomVersionRegistration inserts (GUID, LatestVersion, FriendlyName) into a process-wide registry at module load. During save, Ar.UsingCustomVersion(GUID) adds the registry’s latest value to the archive’s container; during load the container is populated from the file and Ar.CustomVer(GUID) returns the stored value, or a sentinel “before any version” when the file predates the GUID. Migration logic lives inline in Serialize as monotone if (CustomVer < X) branches; the invariant “never decreases” makes the branches a total order and allows the loader to refuse files whose stored value exceeds the registry’s latest. There is no embedded schema; a reader that lacks the class or the GUID cannot interpret the bytes, hence the documented null-reference behaviour and re-save data loss.
Transactions. The transaction system is a memento-based command pattern. FScopedTransaction is a resource-acquisition-is-initialization (RAII) wrapper over Begin/End on the global transactor. Mutating editor code calls Object->Modify() before changing state; Modify serializes the object into the active FTransaction as an FObjectRecord (a serialized before-image) and marks the package dirty. Finalize diffs recorded objects to determine what changed; Apply re-serializes the saved state back into the objects (undo) and re-records the current state (redo), so the same record supports both directions. UTransBuffer holds UndoBuffer with UndoCount marking the redo frontier and trims by MaxMemory. Undo is therefore state-based (object snapshots), not operation-based; the granularity is the object, and correctness depends on every mutation path calling Modify first.
Automation. Tests are C++ classes registered by macros into a registry with flags (filter, priority, application context). A controller executes them in the editor, game, or commandlet process. The headless path is a normal executable invocation with -nullrhi -unattended plus -ExecCmds="Automation RunTests <filter>;Quit"; results are written by -ReportExportPath as JSON with HTML. Screenshot tests capture with deterministic settings (disable_noisy_rendering_features, fixed delta time), then compare against an approved ground-truth image using a two-level tolerance: per-pixel channel/brightness tolerance decides whether a pixel differs; maximum_local_error bounds the fraction of differing pixels inside sub-regions; maximum_global_error bounds the fraction over the whole image; ignore_anti_aliasing accepts a match in neighbouring pixels. Commandlets (-run=<Name>) provide the same process without world or client code loaded.
Diffing. Generic assets are diffed by exporting a text projection and delegating to an external tool; the projection is not the storage format. Blueprints are diffed structurally per graph, per node and per pin, with node matching sufficient to classify moved nodes separately from changed ones; the docs do not state the matching key (NUIF reading: node GUIDs in the graph model, unverified from retrieved sources).
NUIF relevance
Borrow
- GUID-keyed, monotone, per-extension version numbers declared per serialized entity; NUIF
extensions_used/extensions_requiredshould carry a version integer per namespace with the same monotonicity invariant. - The headless execution contract: one executable, no GPU (
-nullrhi), no dialogs (-unattended), a filter expression, an exit-on-completion command and a machine-readable report path, which maps directly onto the QA contract inapps/editor/QA.mdand spec/12. - Two-level screenshot tolerance (per-pixel channel tolerance, local error budget, global error budget, anti-aliasing neighbourhood) as the model for NUIF deterministic snapshot comparison in conformance.
- Deterministic capture preconditions (fixed delta time, disabled temporal effects) as an explicit evaluation context for renders.
Adapt
- Replace before-image snapshots with inverse semantic operations; NUIF undo is operation-based (spec/06) so that undo logs double as patches and replay fixtures, which
FTransactionrecords cannot. - Keep
Modify()-style pre-mutation hooks as an internal invariant of the editor’s operation layer, but make the failure mode (mutation without a transaction) a conformance error surfaced by the CLI rather than silent. - Export-to-text-then-diff should become diff-over-canonical-form: NUIF’s
nuif-text-0is the canonical form, not a projection.
Reject
- Refusing forward compatibility by hiding newer assets and nulling references; NUIF requires unknown data to be preserved as opaque extensions with fidelity records (RFC 0002), not dropped on re-save.
- Migration logic embedded inside per-class
Serializefunctions without an embedded schema; NUIF migrations must be declared operations (migratecommand) that are testable independently of the loader. - Class-specific diff tools that only exist for one asset family; NUIF diff must be generic over the typed document model.
Open questions
- Whether newer-version assets are still hidden in UE 5.8 or now load with a warning; the retrieved page states the hide-and-null behaviour without a version qualifier.
- The exact node-matching key used by the Blueprint diff tool and how it handles node re-creation; no retrieved primary source specifies it.
- Whether
UTransBufferrecords dependent-object changes transitively or relies on each caller invokingModifyon every affected object.
Vello test infrastructure (vello_tests, vello_sparse_tests) and the vello_cpu renderer as a deterministic reference path
Document status:
reviewed. Canonical source.
Summary
The Vello repository contains two test systems. vello_tests targets the compute-shader renderer (vello): property tests, snapshot tests that treat GPU shaders as the source of truth while also executing the CPU shader fallbacks, and GPU-versus-CPU comparison tests; all image comparisons pool a FLIP error map and assert on its mean. vello_sparse_tests targets the sparse-strips renderers (vello_cpu, vello_hybrid, WebGL): a proc macro expands each test into per-backend and per-SIMD-level variants compared against one reference PNG with an integer per-component tolerance (0 for the f32 CPU pipeline, 2 for the u8 pipeline, 1 for SIMD and hybrid) and an optional count of pixels allowed to deviate fully. As of August 2026, vello_cpu 0.2.0 is described by its README as a CPU-only renderer with broad feature support, SIMD paths for all major architectures, an optional f32 pipeline intended for test snapshots, and remaining gaps (complex filter graphs panic, experimental glyph caching); the enclosing sparse_strips README still marks the directory as not production-ready.
Evidence
- Test kinds: property tests run on GPU and CPU; snapshot tests “use the GPU shaders as a source of truth, but the CPU shaders are also ran”; they have “a non-exact comparison metric, because of small differences between rendering on different platforms”, including “fast math” on Apple platforms; comparison tests check that “the GPU renderer matches the reference CPU renderer” and are expected to be phased out (
vello_tests/README.md). - Storage: smoke snapshots live in-repository under
smoke_snapshotsand “are always required to pass”; other snapshots use git LFS as “an experiment”, and tests pass on CI if LFS files fail to download because of bandwidth or storage limits (vello_tests/README.md, “LFS”). - Metric:
nv_flip::flip(expected, rendered, nv_flip::DEFAULT_PIXELS_PER_DEGREE)builds aFlipPool; images are converted to RGB8 (alpha dropped); a size mismatch is a failure (vello_tests/src/snapshot.rs, lines 300–345).DEFAULT_PIXELS_PER_DEGREE = 67.0(nv-flip/src/lib.rs, line 15; crate 0.1.2). - Thresholds:
assert_mean_less_than(0.01)in all four smoke snapshots (vello_tests/tests/smoke_snapshots.rs, lines 29, 47, 75, 119) and0.001in a known-issue reproduction (vello_tests/tests/known_issues.rs, line 55); the helper assertsvalue < 0.1as a sanity bound and documents that a non-zero mean may arise “due to fast math on the GPU or different precisions” (vello_tests/src/compare.rs, lines 37–48). - Controls:
VELLO_TEST_UPDATE,VELLO_TEST_CREATE,VELLO_TEST_GENERATE_ALL,VELLO_SKIP_LFS_SNAPSHOTS, each acceptingall,cpu,gpuor a test name (vello_tests/src/snapshot.rs, lines 90–290;src/lib.rs,env_var_relates_to);VELLO_CI_GPU_SUPPORT=nosetscfg(skip_gpu_tests)(vello_tests/build.rs);VELLO_DEBUG_TESTdumps intermediate images (src/lib.rs, line 88). Default anti-aliasing for tests isAaConfig::Area(src/lib.rs, line 71). - Sparse-strips tolerance semantics: a tolerance of 0 “means that it must be an exact match”; 1 means each component may differ by at most 1;
DEFAULT_CPU_U8_TOLERANCE = 2,DEFAULT_SIMD_TOLERANCE = 1,DEFAULT_CPU_F32_TOLERANCE = 0,DEFAULT_HYBRID_TOLERANCE = 1; the u8 value of 2 avoids per-test overrides for bilinear image cases (sparse_strips/vello_dev_macros/src/lib.rs, lines 12–23). - Macro attributes:
cpu_u8_tolerance,hybrid_tolerance(added to the defaults),diff_pixels(“maximum number of pixels that are allowed to completely deviate”, motivated by gradient colour-stop boundaries under floating-point inaccuracy),transparent,skip_cpu,skip_multithreaded,skip_hybrid,hybrid_only,hybrid_no_depth,no_ref,glyph,ignore_reason; generated variants are_cpu_u8_scalar,_cpu_u8_neon,_cpu_u8_sse42,_cpu_u8_avx2,_cpu_u8_wasm, f32 counterparts,_hybrid,_hybrid_webgl,_hybrid_no_depth; one instance is flaggedis_referenceand writes the reference PNG (sparse_strips/vello_dev_macros/src/test.rs, lines 13–70, 80–140, 218–225, 486–580). - Sparse-strips comparison:
check_refrenders, encodes PNG, loadssnapshots/<test>.png, computesget_diff(ref, actual, threshold, diff_pixels);is_pix_diffcompares R, G, B only (alpha ignored) withabs_diff > threshold, treats two alpha-0 pixels as equal; a test fails when the count of differing pixels exceedsdiff_pixels;REPLACE=1rewrites the reference from the reference instance; references are oxipng-optimised; on wasm the snapshot bytes are inlined withinclude_bytes!(sparse_strips/vello_sparse_tests/tests/util.rs, lines 360–460, 549–666;vello_dev_macros/src/test.rs, lines 197–215). - Sparse-strips targets: the crate tests “CPU, WGPU, WASM32 WebGL”; WebGL runs via
wasm-pack test --headless --chrome --features webgl --release(sparse_strips/vello_sparse_tests/README.md). - Architecture: sparse strips aim to run “on GPUs without compute shader support, using only fragment and vertex shaders”, mitigate performance cliffs and handle low-memory conditions; crates
vello_common,vello_cpu(“CPU-based renderer optimized for multithreading and SIMD”),vello_hybrid,vello_sparse_shaders(WGSL→GLSL for WebGL); the directory is “not yet suitable for production use” (sparse_strips/README.md). - vello_cpu status: “a solid CPU-only 2D renderer with broad, reliable feature support” with “optimized SIMD implementations for all major architectures”; limitations: complex filter graphs panic, multi-threaded filters unsupported, glyph caching experimental, API lifecycle rough; features
u8_pipeline(OptimizeSpeed) andf32_pipeline(OptimizeQuality, “espectially useful for rendering test snapshots”),std/libm,multithreading,text; MSRV 1.88 (sparse_strips/vello_cpu/README.md). Design is documented in a 2025 ETH master’s thesis linked from the README. - Versions: vello 0.10.0 (2026-08-14), vello_cpu/vello_hybrid/vello_common 0.2.0 (2026-08-07) (crates.io;
CHANGELOG.md; GitHub releasessparse-strips-v0.2.0). - Text:
vello_tests/tests/hinting.rsandemoji.rsexist as snapshot groups (directory listing);vello_sparse_tests/tests/glyph.rsgenerates cached and uncached glyph variants (vello_dev_macros/src/test.rs,glyph).
Mechanism
vello_tests snapshot:
img = render(scene, params{use_cpu, aa=Area}) # GPU via wgpu or CPU shader fallback
ref = decode(smoke_snapshots/<name>.png | lfs snapshots)
require size(img) == size(ref)
map = FLIP(rgb(ref), rgb(img), ppd = 67)
pass iff mean(map) < threshold # 0.01 typical, 0.001 strict
vello_tests compare_gpu_cpu:
pass iff mean(FLIP(cpu_render, gpu_render)) < threshold # threshold < 0.1 enforced
vello_sparse_tests (#[vello_test(width, height, ...)]):
for variant in {cpu_u8_{scalar,neon,sse42,avx2,wasm}, cpu_f32_..., hybrid, hybrid_webgl, hybrid_no_depth}:
tol = base_tol(variant) + user_tol
n = count(pixels p: not both alpha 0 and any c in RGB |ref_c - img_c| > tol)
pass iff n <= diff_pixels # diff_pixels default 0
reference PNG written once by the designated reference variant; REPLACE=1 regenerates
The CPU f32 pipeline with tolerance 0 is the only configuration in either harness that asserts bit-exact equality against a stored image; this observation is NUIF’s, derived from the constants above.
NUIF relevance
Borrow
- Use
vello_cpuwithRenderMode::OptimizeQuality(f32 pipeline, scalar or a pinned SIMD level) behind the NUIF renderer trait as the deterministic conformance path, because the crate’s own harness already holds that configuration to tolerance 0. - Reuse the tiered tolerance model (exact for CPU f32, ±1 for SIMD/hybrid, ±2 for u8, FLIP mean for wgpu) as the template for NUIF’s determinism tiers, because it is derived from measured behaviour of a Rust renderer rather than assumed.
Adapt
- Replace
diff_pixelsescape hatches with fixture-level tier assignment and recorded reasons, because gradient boundary flips are a property of the fixture class and should be visible in the conformance report. - Record the vello_cpu version, pipeline, SIMD level and thread count in every
renderresult, because the harness shows that each of these changes tolerance.
Reject
- Do not treat wgpu output as a source of truth for conformance, as
vello_testsdoes for its snapshots, because NUIF’s ADR 0003 requires the CPU path to be normative and the GPU path to be an experiment. - Do not depend on git LFS for reference rasters; NUIF fixtures must be small, in-repository and always required to pass, because Vello’s harness deliberately passes when LFS is unavailable.
Open questions
- Whether
vello_cpumultithreaded rendering is bit-identical to single-threaded output in the f32 pipeline; the harness has askip_multithreadedattribute but the tolerance tables do not distinguish thread counts. - Whether the
vello_hybrid±1 tolerance is stable across wgpu backends (Vulkan, Metal, D3D12, WebGL), since the variants share one reference image. - Filter support gaps in
vello_cpu(complex filter graphs) versus the effects vocabulary NUIF intends to specify in spec/05.
Vello Rust 2D renderer
Document status:
reviewed. Canonical source.
Summary
Vello is a Rust GPU-compute-centric 2D renderer using wgpu. Its scene abstraction covers vector shapes, images, gradients and text-oriented drawing, while current releases still document evolving APIs and some incomplete effect/glyph areas.
NUIF relevance
Use Vello as an implementation experiment, not a normative dependency. Maintain a renderer trait and conformance raster path so the draft specification’s visual semantics remain independent of Vello’s evolution.
VisRefiner difference-aligned supervision and render-feedback refinement
Document status:
reviewed. Canonical source.
Summary
VisRefiner proposes training screenshot-to-code models on visual differences between a target and the rendered result, then applying a self-refinement stage. Its central insight—supervise corrections with the actual renderer outcome—is well aligned with a deterministic NUIF operation loop.
This is a February 2026 arXiv preprint. It is current evidence for an experiment design, not mature proof, a required dependency or a benchmark result that NUIF can inherit without reproduction. The record intentionally avoids stronger claims until source, code and data are independently reviewed.
Evidence
- arXiv:2602.05998 abstract defines “difference-aligned supervision” that links rendered visual discrepancies to code edits.
- The abstract describes a reinforcement-learning stage in which the model observes the target and current render, identifies differences and updates code.
- Reported improvements concern screenshot-to-frontend-code generation. No reviewed evidence here establishes NUIF operations, editable design structure, resource recovery or cross-implementation reproducibility.
Mechanism
target screenshot + current rendered output + difference evidence
-> proposed code edit
-> execute/render
-> outcome-derived supervision or reward
-> next correction
NUIF can make the edit target safer and more measurable by using a bounded typed operation grammar and validator instead of arbitrary source-code edits.
NUIF relevance
Borrow experimentally difference-aligned correction traces and render/edit iteration.
Adapt from code patches to validated NUIF transactions; split visual, structural, text, resource and provenance rewards; cap iterations and retain every proposal, diagnostic, render and accepted correction.
Reject pixel-only reinforcement, execution of arbitrary generated programs, or adoption before a frozen baseline reproduces an improvement on held-out NUIF fixtures.
Open questions
- Does difference-aligned supervision outperform ordinary accepted-edit traces after controlling for data and compute?
- Which difference representation best predicts typed corrective operations?
- How often does a visually beneficial edit make hierarchy, accessibility or responsive behavior worse?
W3C Community Group incubation and Recommendation-track requirements
Document status:
verified. Canonical source.
Summary
W3C Community Groups provide a no-fee incubation forum for specifications, test suites and stakeholder discussion. Anyone with a W3C account can propose a group; four additional supporters are required before launch. Participants accept contribution and licensing terms. Community Group reports are not W3C Standards. Recommendation-track work requires a chartered Working Group, consensus, wide review and implementation experience.
Evidence
- Anyone with a W3C account can propose a Community Group and obtain four additional supporters. Membership is not required and Community Group participation has no fee. Locator: W3C Community Groups, lines 79–94, retrieved 2026-08-30.
- Community Group reports carry royalty-free patent commitments and permissive copyright terms; a final report can request the stronger Final Specification Agreement. Locator: same page, lines 98–105, retrieved 2026-08-30.
- Community and Business Group reports are not W3C Standards. Recommendation Track work adds security, privacy, accessibility and internationalization review plus broader interoperability work. Locator: same page, lines 108–116, retrieved 2026-08-30.
- The W3C Process states that standards quality depends on consensus, public and member review, implementation and interoperability experience. Locator: W3C Process Document dated 2025-08-18, lines 216–226, retrieved 2026-08-30: https://www.w3.org/policies/process/.
Mechanism
Incubation develops a problem statement, scope, use cases, draft text, tests and an implementer community. Recommendation-track transition occurs only when W3C members support a charter and resource the work. Candidate Recommendation and Recommendation advancement then use formal review and implementation evidence.
NUIF relevance
Borrow Community Group participation for coordination with DTCG and Web stakeholders after an external implementation exists.
Reject creating a NUIF Community Group before the project has external participants and a bounded implementer draft. A one-project group would add process without demonstrating stakeholder demand.
Open questions
- W3C is appropriate only if the interoperable scope is primarily Web-facing and browser or design-tool stakeholders commit implementation resources.
- A general cross-platform package format may fit a foundation specification process better than the W3C Recommendation Track.
WebAssembly Component Model and WIT
Document status:
reviewed. Canonical source.
Summary
WIT defines language-neutral interfaces and worlds for WebAssembly components and drives generated bindings across Rust, C/C++, Go, C# and other languages. The ecosystem also demonstrates text/binary interface round-trips and introspection.
NUIF relevance
Potential future plugin/adapter ABI, especially for sandboxed importers/exporters. Do not couple the initial core API to Component Model stability; keep an adapter boundary that can later expose WIT.
Headless execution of the Rust engine in browsers, Node and WASI runtimes for differential tests
Document status:
reviewed. Canonical source.
Summary
Three headless execution paths exist for a Rust engine compiled to WebAssembly. wasm-bindgen-test compiles #[wasm_bindgen_test] functions for wasm32-unknown-unknown and runs them under Node by default or, with wasm_bindgen_test_configure!(run_in_browser) or WASM_BINDGEN_USE_BROWSER=1, in headless Chrome, Firefox or Safari through WebDriver; wasm-pack test --headless --chrome --firefox wraps the runner and driver management. wasmtime executes wasm32-wasip1 modules and wasm32-wasip2 components (both Tier 2 targets shipped by rustup) with capability-scoped filesystem access (--dir) and direct function invocation (--invoke), which suits CLI-equivalent conformance runs without a browser. Playwright launches Chromium, Firefox and WebKit headlessly and evaluates JavaScript in the page (page.evaluate), which is the mechanism for browser differential layout tests: a page renders the CSS-equivalent of a NUIF fixture, the test reads getBoundingClientRect results, and compares them against the WASM engine’s LayoutSnapshot loaded in the same page or in Node.
NUIF interpretation: the engine’s differential layout suite should run in three layers: wasmtime for fast, GPU-free CLI parity; wasm-bindgen-test under Node for binding-level round trips; Playwright for browser-oracle comparisons where the oracle is the browser’s own layout, not WebGPU rendering.
Evidence
- wasm-bindgen-test: “an experimental test harness for Rust programs compiled to Wasm using
wasm-bindgenand thewasm32-unknown-unknowntarget”; tests are written with#[wasm_bindgen_test]and run withcargo test --target wasm32-unknown-unknown;#[wasm_bindgen_test(unsupported = test)]falls back to#[test]on native targets; tests “must be in the root of the crate, or within apub mod”. Locator: wasm-bindgenguide/src/wasm-bindgen-test/{index.md,usage.md}, main (2026-08). - Versions: wasm-bindgen 0.2.127 and wasm-bindgen-test 0.3.77 published 2026-08-08; wasm-pack 0.15.0 published 2026-05-15. Locator: crates.io API; wasm-bindgen
CHANGELOG.md“[0.2.127]”. - Browser configuration: default is Node;
WASM_BINDGEN_USE_BROWSER=1,WASM_BINDGEN_USE_DEDICATED_WORKER,..._SHARED_WORKER,..._SERVICE_WORKER,WASM_BINDGEN_USE_DENO,WASM_BINDGEN_USE_NODE_EXPERIMENTAL; forced per crate viawasm_bindgen_test_configure!(run_in_browser | run_in_dedicated_worker | run_in_shared_worker | run_in_service_worker | run_in_node_experimental). Locator:guide/src/wasm-bindgen-test/browsers.mdlines 1-35. - Headless drivers:
wasm-pack test --headless --chrome --firefox --safari; without wasm-pack setCHROMEDRIVER,GECKODRIVERorSAFARIDRIVER(orCHROMEDRIVER_REMOTE) and runcargo test --target wasm32-unknown-unknown;webdriver.jsonorWASM_BINDGEN_TEST_WEBDRIVER_JSONsupplies capabilities;NO_HEADLESS=1serves the tests for a visible browser. Locator:browsers.mdlines 58-165. - CI examples run
cargo testnatively, thenwasm-pack test --headless --chromeand--firefox(GitHub Actions). Locator:guide/src/wasm-bindgen-test/continuous-integration.md. - wasm-pack test: wraps
wasm-bindgen-test-runner; accepts a crate path,--release, environment flags--node --firefox --chrome --safari --headless,--panic-unwind(nightly,-Z build-std), and passes extra arguments tocargo test. Locator: wasm-packdocs/src/commands/test.md. - Coverage on wasm requires nightly
-Cinstrument-coverage -Zno-profiler-runtimeand--cfg=wasm_bindgen_unstable_test_coverage, thencargo +nightly llvm-cov test --target wasm32-unknown-unknown. Locator:guide/src/wasm-bindgen-test/coverage.mdlines 12-46. - wasmtime 48.0.1 (2026-08-24): README shows
rustup target add wasm32-wasip2,rustc hello.rs --target wasm32-wasip2and running the resulting component; built on Cranelift; supports WASI. Locator: wasmtimeREADME.mdlines 63-131. - wasmtime CLI:
wasmtime --dir=. --dir=/tmp demo.wasm args...grants directory capabilities (--dir=host::guestmapping);wasmtime run --invoke 'add(1, 2)' add.wasmcalls exported functions of modules or components and skipswasi:cli/run;-W/--wasmconfigures proposals. Locator:docs/WASI-tutorial.mdlines 157-243;docs/cli-options.mdlines 22-142, 336. - rustc targets:
wasm32-wasip1is Tier 2, cross-compiled, shipsstdwith a self-contained sysroot, installed byrustup target add wasm32-wasip1, and “will generate core WebAssembly modules”;wasm32-wasip2is Tier 2 and “outputs a component” built on the component model. Locator: rustc bookplatform-support/wasm32-wasip1.md,wasm32-wasip2.md. - Playwright: latest release v1.62.1 (2026-07-30);
npx playwright install chromium|firefox|webkit,install --with-deps; projects run the same test underchromium,firefox,webkit;page.evaluate(() => document.location.href)returns serialisable values from page scripts, includingasyncfunctions. Locator:docs/src/browsers.md;docs/src/evaluating.mdlines 15-44;gh api repos/microsoft/playwright/releases/latest. - Taffy generates layout fixtures by driving Chrome for Testing through ChromeDriver with
fantoccini, downloading a matching Chrome/driver pair once per version. Locator: Taffyscripts/gentest/src/main.rslines 11-67;CONTRIBUTING.mdlines 26-35. - egui_kittest removes
Backends::BROWSER_WEBGPUfrom its test setup because it relies on blocking screenshots. Locator: eguicrates/egui_kittest/src/wgpu.rslines 20-27. - The NUIF whitepaper and ADR 0001 assign the WASM boundary to Rust and the editor shell to a web stack; ARCHITECTURE.md draws the Rust core behind a WASM boundary. Locator:
docs/whitepaper/06-language-and-runtime-choice.md;adrs/0001-rust-reference-core.md;apps/editor/ARCHITECTURE.md. - Figma documents that a plug-in UI iframe can use browser APIs including WebAssembly, while host-document access remains in the plug-in API. This supports a capability-free WASM core in the iframe and a thin, separately tested host adapter. Locator: https://developers.figma.com/docs/plugins/, retrieved 2026-08-30.
Mechanism
Three execution layers and their contracts:
1. wasm32-wasip1 / wasip2 ──▶ wasmtime run --dir=fixtures::/fixtures nuif-cli.wasm layout /fixtures/x.nuif --context ...
oracle: identical stdout/JSON to the native CLI (parity test); no browser, no GPU; capability-scoped FS.
2. wasm32-unknown-unknown ──▶ cargo test --target wasm32-unknown-unknown (Node by default)
#[wasm_bindgen_test] fn layout_roundtrip() { let snap = engine.layout(doc, ctx); assert_eq!(canonical(snap), expected) }
oracle: expectations embedded or fetched; exercises wasm-bindgen glue and JS-facing API.
3. Playwright ──▶ chromium/firefox/webkit headless
page.setContent(html_from_fixture); const boxes = await page.evaluate(() => [...document.querySelectorAll('[data-nuif-id]')]
.map(e => { const r = e.getBoundingClientRect(); return [e.dataset.nuifId, r.x, r.y, r.width, r.height]; }));
const ours = await page.evaluate(() => nuif.layout(doc, ctx)); // WASM engine loaded in the same page, or run in Node
compare(boxes, ours, tolerance_from_context);
oracle: browser layout; only for semantics where NUIF declares CSS equivalence (conformance/PLAN.md).
Invariants and constraints from the sources: wasm-bindgen tests must be in the crate root or a pub mod; browser runs need a driver binary and a browser on the host (wasm-pack manages drivers); Safari runs only on macOS; wasmtime exposes files only under --dir mappings; wasm32-wasip2 produces components, so a CLI targeting wasip2 must use wasi:cli/run or --invoke; blocking GPU readback is unavailable in browsers, so in-browser tests should assert on scene data or layout boxes, not on rendered pixels.
Reproducibility: pin browser versions (Playwright pins browser builds per release; Taffy pins Chrome for Testing), pin wasmtime (cargo install --locked wasmtime-cli at a fixed version), and record the browser name and version in the differential report as part of the evaluation context.
The implemented first layer is nuif-wasm-api-0: a
wasm32-unknown-unknown module generated with wasm-bindgen 0.2.127. It accepts
explicit canonical-text/CBOR, deterministic packages, capability-set and patch
byte arrays; exposes validation, hashing, encoding, bounded atomic application
and exact undo/redo; and declares no host authority. Structural package load
retains inert verified resources. Evaluation requires a separate exact
manifest-capability check. cargo xtask gate-wasm generates Node and
direct-browser packages, initializes the web target in pinned headless Chrome,
drives the Node package, and requires edited bare and package bytes to equal the
native CLI. A behavior-bearing package proves exact resource preservation plus
typed missing/exact capability outcomes without executing behavior. This closes
binding and browser-package initialization only; it does not close
browser-layout, WASI CLI or host-adapter behavior described above.
NUIF relevance
Borrow
- wasmtime with
--diras the sandboxed executor for awasm32-wasip1build ofnuif-cli, because it yields CLI parity tests without a browser and demonstrates the capability boundary that the security suite needs. - The wasm-bindgen-test/wasm-pack headless flow for binding-level tests of the engine’s JS API, because ARCHITECTURE.md requires the editor’s in-process API to mirror CLI semantics and this tests the boundary itself.
- Taffy’s pinned Chrome for Testing approach for browser oracles, because differential layout results are only reproducible with a pinned browser build.
Adapt
- Playwright rather than raw WebDriver for the browser differential suite, because one runner covers Chromium, Firefox and WebKit and
page.evaluatereturns structured results; the runner should live underxtaskor atests/browserpackage and emit the NUIF report format. - Browser-differential fixtures should be generated from NUIF fixtures into HTML with
data-nuif-idattributes so that boxes are keyed byEntityId, mirroring Taffy’s generated fixtures.
Reject
- In-browser WebGPU rendering as a conformance oracle, because blocking readback is unavailable (egui_kittest removes
BROWSER_WEBGPU) and GPU results vary by implementation; render conformance stays on the CPU reference path. - Node-only testing as the sole WASM layer, because browser layout is the oracle for CSS-equivalent semantics and only a browser provides it.
Open questions
- Whether
wasm32-wasip2component output should be the CLI’s WASM form now, or whetherwasip1modules are preferable until the component-model boundary (nuif:research:wasm-component-model) is adopted. - Whether wasm-bindgen’s browser runner or Playwright should own the browser differential tests; running both duplicates driver management.
- Whether font availability in headless browsers can be pinned tightly enough for text-dependent layout fixtures, or whether differential tests must exclude text metrics.
- Whether
wasm32-unknown-unknownbuilds ofnuif-renderwithwgpuare needed at all for tests, given that in-browser pixel assertions are rejected above. - Whether to acquire wasm-bindgen’s immutable prebuilt CLI archives with per-platform checksums instead of compiling its full optional test-runner tool graph. The latter currently warns about future-incompatible HTTP-server dependencies, but those dependencies are absent from the NUIF module and workspace runtime graph.
Finite browser lowering for portable behavior effects
Document status:
verified. Canonical source.
Summary
A browser adapter can lower the first NUIF behavior profile without accepting
authored JavaScript. The narrow mapping is enabled native-button activation,
the HTML hidden property for visibility, and an ARIA status live region for
advisory announcements. The state machine remains validated data interpreted by
one fixed generated runtime. An exact CSP hash grants that runtime—and no other
inline script—authority inside the self-contained output.
Native HTML invoker commands are not a replacement for this profile. Their built-in vocabulary covers popover and dialog actions. A custom command still requires script and does not carry NUIF state, ordered guards, typed variables or abstract effects. The smallest faithful result is therefore a finite interpreter, not a general code generator or a misuse of popover state.
Evidence
- The HTML Living Standard defines
hiddenfor all HTML elements; the hidden state is not rendered. This is a direct Boolean host operation rather than a CSS-cascade approximation. Locator: HTML §6.1, lines 92–96 of the retrieved multipage interaction document, https://html.spec.whatwg.org/multipage/interaction.html#the-hidden-attribute, retrieved 2026-08-31. - The HTML button
commandfor/commandsurface has built-ins for showing, hiding and toggling popovers and closing or showing dialogs. The standard’s custom-command example installs a JavaScriptcommandevent listener. This supports using native commands when their component semantics are exact, but not claiming that they encode a general guarded state machine. Locator: HTML button element,commandforand examples, retrieved lines 98–175, https://html.spec.whatwg.org/dev/form-elements.html#the-button-element, retrieved 2026-08-31. - WAI-ARIA 1.2 defines
statusas advisory live-region information, says it should not receive focus as a result of status change, and gives it implicitaria-live=politeandaria-atomic=true. This is the correct first target for non-urgent NUIF announcement effects. Locator: WAI-ARIA 1.2statusrole, lines 4471–4480, https://www.w3.org/TR/wai-aria-1.2/#status, retrieved 2026-08-31. - CSP Level 3 permits policy delivery through an early
metaelement, while preferring the response header for served resources. Hash sources use SHA-256/384/512 plus base64, and a matching hash can allow an inline script withoutunsafe-inline. The inline body is hashed after UTF-8 encoding. Locators: CSP3 §§2.3.1, 3.1, 3.3, 4.2 and 8.4, https://www.w3.org/TR/CSP3/, retrieved 2026-08-31. - Playwright documents locators as its auto-waiting/retry unit and recommends
user-facing roles or explicit test contracts over DOM-structure chains. It
exposes ARIA snapshots computed from browser accessibility trees. Stable
data-nuif-idis the adapter’s explicit test contract; snapshots provide a foreign host observation rather than a source-string assertion. Locators: https://playwright.dev/docs/locators#introduction and https://playwright.dev/docs/aria-snapshots#aria-snapshots, retrieved 2026-08-31. - The HTML activation model says non-click manual activation, including
keyboard or voice input, fires a
clickevent at an element with activation behavior. Playwright’slocator.press()focuses the element and produces a key sequence, so separate Enter/Space runs test the native route without adding key handlers to the adapter. Locators: https://html.spec.whatwg.org/multipage/interaction.html#activation-behavior-of-elements and https://playwright.dev/docs/input#keys-and-shortcuts, retrieved 2026-08-31.
Mechanism
nuif-web-behavior-0 composes the behavior and accessibility profiles. It
rejects checkbox/radio transition sources because a native click also mutates
checked state that the source behavior vocabulary cannot currently express. It
rejects disabled transition sources because native disabled controls do not
produce the promised activation. All enabled button/switch entities receive a
listener so a valid activation with no transition remains an observable no-op.
It also rejects a visibility effect that can hide one of those controls through
the control itself or an ancestor; otherwise later source events could exist in
the abstract trace but not in the native host.
The program JSON is embedded only after <, >, &, U+2028 and U+2029 are
escaped. The static runtime contains no evaluation, dynamic import, request,
timer or handler-attribute surface. Its meta CSP denies all resource classes and
allows the one exact script body. A serving host should deliver and merge its
own response-header policy; the self-contained policy is not assumed to survive
embedding in another document.
One transition may emit at most one non-empty announcement, and may not repeat the same effect/target pair. A browser can otherwise coalesce multiple run-to-completion effects into one terminal DOM observation, producing a false equivalence claim.
NUIF relevance
The Rust stage validates unsafe embedding strings and every web-specific
refusal, computes the exact script/CSP digests and produces the reference run.
The Playwright stage performs separate five-event pointer and alternating
Enter/Space keyboard sequences in each pinned engine and checks state, selected
transition, retained hidden values, live-region text, target attribution,
runtime errors and ARIA snapshots after every event. The first
macOS/arm64 run passes Chromium 151.0.7922.34, Firefox 153.0 and WebKit 26.5.
This proves one browser-host mapping. It does not prove screen-reader speech, focus order, native UI behavior, checkbox/radio semantics, authored-script round trips, broader events/effects or a wire-format decision.
Open questions
- Add a portable checked/pressed-state effect before admitting checkbox/radio activation, then test keyboard and pointer traces separately.
- Run assistive-technology announcement timing experiments before claiming more than DOM and browser accessibility-tree exposure.
- Define focus effects and focus-restoration laws before adding dialog or popover lowering.
- Compare a second host family against the same abstract effects before moving the behavior sidecar into a canonical schema proposal.
WebGPU security and robustness model
Document status:
reviewed. Canonical source.
Summary
WebGPU explicitly treats malicious use, uninitialized/out-of-bounds data, driver bugs, timing channels and GPU robustness as first-class concerns. Validation and zero-initialization guarantees are central to safe exposure of GPU resources.
NUIF relevance
The reference renderer must assume documents are hostile, enforce allocation/complexity budgets before issuing GPU work, validate shader/effect extensions and preserve a sandbox boundary around any programmable rendering extension.
WebRender wrench reftest harness, reftest.list syntax and RON scene capture/replay
Document status:
reviewed. Canonical source.
Summary
wrench is WebRender’s standalone driver. It loads YAML scene descriptions or RON captures, renders them through WebRender (GL, ANGLE or the SWGL software rasteriser), and runs a reftest suite declared in reftests/reftest.list. Each manifest line names an operator (==, !=, and the tile-accuracy operators **, !*), optional tolerance functions (fuzzy(max_diff,num_diff), fuzzy-range(...), conditional -if variants), platform predicates (platform(...), skip_on(...)), render options and extra draw-statistics checks. Comparison is per-pixel on 8-bit channels: the difference of a pixel is the maximum channel delta, and the tolerance bounds how many pixels may exceed each delta bucket. Captures from Firefox (ctrl-shift-3) serialise the scene, frame and resources as RON files under ~/wr-capture, which wrench show replays; this is the mechanism for deterministic reproduction of a browser frame outside the browser.
Evidence
- Tool purpose: “
wrenchis a tool for debugging webrender outside of a browser engine”; headless mode “for use in continuous integration” is invoked via./headless.py args; reftests run withscript/headless.py reftest [path]; failures are examined with the Firefox reftest analyzer; new tests add a scene and a reference toreftests/plus a line inreftests/reftest.list(gfx/wr/wrench/README.md). - Capture: enable WebRender, “Hit ctrl-shift-3 to capture the frame. The data will be put in
~/wr-capture”, thenwrench show ~/wr-capture(gfx/wr/wrench/README.md, “show”). - Capture format:
CaptureConfig { root, bits: CaptureBits, scene_id, frame_id, resource_id }with aron::ser::PrettyConfigusingenumerate_arrays(true)and single-space indentation; scenes, frames and resources are written underscenes/{:05},frames/{:05}andresources/{:05}with.ronextensions; external images are described byExternalCaptureImage { short_path, descriptor, external }andPlainExternalImage { data, uv }; PNG dumps of RGBA8/R8/RG8 targets are available behind thepngfeature (gfx/wr/webrender/src/capture.rs). - CI rendering: “Tests run using OSMesa to get consistent rendering across platforms. Still there may be differences depending on font libraries on your system” (
gfx/wr/README.md, “Testing”). - Operators:
ReftestOp::Equal→"==",NotEqual→"!=",Accurate→"**"(rendering at different tile sizes must be pixel-exact),Inaccurate→"!*"(gfx/wr/wrench/src/reftest.rs, lines 60–67). - Fuzzy structures:
RefTestFuzzy { max_difference: usize, num_differences: usize }(lines 94–95);fuzzy(andfuzzy-if(parse two integers and assert that only one plainfuzzyis present, recommendingfuzzy-rangeotherwise (lines 396–408);fuzzy-range(andfuzzy-range-if(accept a list of<=max,*numbucket pairs (lines 372–393). - Comparison: pixel values are asserted to be 8-bit; the per-pixel difference is the maximum over channels (
pixel_max); a 256-bin histogram of differences is built and a prefix sum checks that the number of pixels whose difference lies in (previous max, bucket max] does not exceed the bucket’snum_differences, with a final check that no pixel exceeds the largest allowed difference (lines 118–205). - Platform predicates:
platform()yields"swgl"when the window is software-rendered, else"win","linux","mac","android"by target OS (lines 594–606);skip_on(...)and nested conditions such asenv(android,device)are evaluated on the manifest (lines 649–660);includelines splice other manifests (lines 446–451). - Options and extra checks:
options(...)recognisesdisable-subpixel,disable-aa,allow-mipmaps;force_subpixel_aa_where_possible(bool)andmax_surface_size(usize)are line-level settings; extra checks aredraw_calls(n),alpha_targets(n),color_targets(n)(ExtraCheckenum; lines 414–440). - Manifest examples (
gfx/wr/wrench/reftests/text/reftest.list):skip_on(android,device) fuzzy(1,3692) fuzzy-if(platform(win),2,5585) fuzzy-if(platform(swgl),3,13540) == decorations-suite.yaml decorations-suite.png;options(disable-aa) == ahem.yaml ahem-ref.yaml;platform(linux) == isolated-text.yaml isolated-text.png;fuzzy(1,774) platform(linux) draw_calls(3) == colors.yaml colors-subpx.png;platform(mac) fuzzy(195,30) == color-bitmap-shadow.yaml color-bitmap-shadow-ref.yaml. - Anti-aliasing manifest (
gfx/wr/wrench/reftests/aa/reftest.list):skip_on(android) fuzzy(1,1) fuzzy-if(platform(swgl),4,27) == rounded-rects.yaml rounded-rects-ref.png;fuzzy-if(env(android,device),6,792) == fractional-radii.yaml fractional-radii-ref.yaml. - Directory layout: reftest groups include
aa,backface,blend,border,boxshadow,clip,compositor,filters,gradient,image,mask,scrolling,snap,split,text,tiles,transforms(gfx/wr/wrench/reftests/). - Canonical home: the GitHub
servo/webrenderrepository is “a downstream mirror” ofgfx/wrin mozilla-central (gfx/wr/README.md).
Mechanism
reftest.list grammar (as implemented in wrench/src/reftest.rs)
line := [predicate | fuzzy | option | check]* op test reference [# comment]
op := "==" | "!=" | "**" | "!*"
fuzzy := "fuzzy(" max_diff "," num_diff ")" | "fuzzy-if(" cond "," max_diff "," num_diff ")"
| "fuzzy-range(" ("<=" max "," "*" num)+ ")" | "fuzzy-range-if(" cond "," ... ")"
predicate := "platform(" name ")" | "skip_on(" cond, ... ")"
option := "options(" ("disable-subpixel" | "disable-aa" | "allow-mipmaps")* ")"
| "force_subpixel_aa_where_possible(" bool ")" | "max_surface_size(" n ")"
check := "draw_calls(" n ")" | "alpha_targets(" n ")" | "color_targets(" n ")"
include := "include" path
compare(test_img, ref_img, fuzziness):
hist[0..=255] = 0
for each pixel: d = max over channels |a_c - b_c|; hist[d] += 1
prefix[k] = sum(hist[0..=k]); prev_max = 0; prev_fail = prefix[0]... (pixels with d = 0 are always allowed)
for (max_diff, num_diff) in fuzziness sorted by max_diff:
n = prefix[min(255,max_diff)] - prev_fail
fail if n > num_diff
prev_fail = prefix[max_diff]; prev_max = max_diff
fail if prefix[255] - prev_fail > 0 # pixels above the largest allowed difference
capture layout (~/wr-capture)
scenes/00001/*.ron frames/00001/*.ron resources/00001/*.ron (+ externals as texel .ron / png)
NUIF relevance
Borrow
- Adopt a manifest-driven reftest list with
==/!=operators,fuzzy(max_diff,num_diff)and explicit per-line rationale comments for the GPU tier, because the syntax is compact, machine-parseable and in production use on a GPU renderer. - Adopt the
**/!*idea (rendering with different internal tiling must be pixel-identical) as a metamorphic test for NUIF’s renderer trait, because it catches tiling-dependent nondeterminism without a reference image.
Adapt
- Replace
platform(win|linux|mac|android|swgl)with capability keys (backend, adapter class, font stack, pixel ratio) in NUIF’s evaluation context, because platform names are not vendor-neutral and do not capture the actual sources of variance. - Use NUIF’s own canonical serialisation for scene captures instead of RON, because the capture must round-trip through the codec suite and remain a NUIF document rather than a renderer-internal dump.
Reject
- Do not accept platform-specific PNG references for the normative path (as
platform(linux) == x.yaml x.pngdoes), because the NUIFrendersuite requires one deterministic CPU reference output per fixture and context. - Do not rely on OSMesa or any system GL for consistency in CI, because the WebRender README itself notes residual differences from system font libraries.
Open questions
- Whether NUIF’s conformance manifest should allow
draw_calls-style structural checks against the renderer trait (scene statistics) as a non-image assertion. - How captured interactive scenes (scroll, animation frames) map to NUIF fixtures, since wrench captures multiple frames per scene under
frames/{:05}.
WebSight synthetic screenshot and HTML pairs
Document status:
reviewed. Canonical source.
Summary
WebSight provides large synthetic HTML/screenshot pairs for screenshot-to-code training. Version 0.1 contained 823,000 synthetic pairs; version 0.2 expanded to roughly two million and changed generation toward real images and Tailwind CSS. Its scale makes it useful for pretraining or controlled ablations, but the synthetic distribution and version-specific generation choices prevent it from being the sole training or evaluation corpus for NUIF reconstruction.
Evidence
- The official March 2024 project article describes v0.1 as 823,000 synthetic HTML/screenshot pairs and v0.2 as two million examples.
- The article states that v0.2 introduced real images in screenshots and switched the generated frontend style to Tailwind CSS.
- The paper record is arXiv:2403.09029. The authoritative dataset namespace is
HuggingFaceM4/WebSight; mirrors are not equivalent provenance. - The dataset is intended for screenshot-to-HTML fine-tuning, not recovery of original production sites or binary assets.
Mechanism
Synthetic prompts and generated HTML/CSS are rendered by a browser to create paired targets. This provides exact generated structure for each synthetic screenshot and makes controlled perturbations possible. It also inherits the generator’s design patterns, framework choices, text distribution and resource simplifications.
NUIF relevance
Borrow the scale and exact synthetic pairing for perception pretraining and render-loop ablations.
Adapt by generating a larger share of data directly from canonical NUIF so every entity, operation, resource, layout context and provenance label is known. Hold out entire templates, component families, fonts and visual themes.
Reject treating synthetic HTML as proof of real-world generalization, training on unversioned mutable dataset snapshots, or assuming image pixels identify the original asset bytes.
Open questions
- What are the exact redistribution, image-source and generated-output terms of the pinned WebSight revision selected for any training run?
- Which visual/layout distributions are underrepresented compared with the project’s target corpus?
- Does WebSight pretraining improve typed NUIF operations after controlling for model size and total training tokens?
Yoga embeddable Flexbox layout engine
Document status:
reviewed. Canonical source.
Summary
Yoga is an embeddable C++20 Flexbox-focused layout engine with broad language/platform bindings. It demonstrates the value of a small portable layout runtime used outside browsers.
NUIF relevance
Yoga is a differential/reference target for Flexbox semantics and a reminder that adapter/runtime portability matters. Taffy remains preferable for the Rust reference implementation because NUIF also needs Grid and native Rust integration.
NUIF Editor 0.1.0-alpha.1
This prerelease distributes the profile-zero reference editor for implementation review and conformance work. It does not declare the draft specification or the editor stable.
Included capability
- Native application archives for Linux x86-64, Linux Arm64, Windows x86-64, macOS Apple Silicon, and macOS Intel.
- Canonical
.nuifdocument open and save operations. - Profile-zero page, containment-tree, canvas, and properties-panel editing.
- Rectangle, ellipse, path, text, frame, stack, and flex authoring supported by the current semantic operation surface.
- PNG snapshot export, fidelity-confirmed SVG/HTML/CSS/DTCG profile import, reported profile export, and the headless JSONL automation interface.
- SHA-256 checksums, package manifests, a CycloneDX software bill of materials, a combined release manifest, and GitHub artifact attestations.
The executable surface and evidence are specified in apps/editor/README.md,
apps/editor/UI-SPEC.md, and apps/editor/QA.md at the tagged revision.
Limitations
- The archives are unsigned. macOS Gatekeeper and Windows SmartScreen may show warnings, and managed systems may reject execution.
- The release does not include an installer, automatic updater, collaboration service, or vendor-host plug-in.
- The native editor exposes the declared SVG, HTML/CSS and DTCG profile import/export paths. Retentive synchronization remains a CLI and library operation, and arbitrary files outside those profiles are rejected.
- The reference editor implements profile zero rather than every section of the draft UI specification.
Verification
Verify an archive against SHA256SUMS and its GitHub attestation before
execution:
gh attestation verify <archive> --repo refpath/nuif
The package-specific manifest records the source revision, binary digest,
archive digest, platform, architecture, smoke-test result, and unsigned status.
The release mechanism is defined in ADR 0007 and docs/VERSIONING.md.
NUIF Editor 0.1.0-alpha.2
This editor research preview makes verified, source-built, user-scoped
installation the canonical way to use the reference editor as a persistent
development and conformance tool. The application prerelease does not assign a
maturity level to the pre-draft specification. It retains all profile-zero
editor, adapter, benchmark and release evidence from 0.1.0-alpha.1.
Developer installation
Clone this exact tag and keep the checkout as the lifecycle control plane:
git clone --branch v0.1.0-alpha.2 --depth 1 https://github.com/refpath/nuif.git
cd nuif
git rev-parse HEAD
cargo xtask editor-install --user --channel alpha
The install builds with the checked-in Rust toolchain and Cargo.lock, creates
an immutable version directory, activates the host integration, writes a
source/build receipt and runs editor-doctor. macOS applies and verifies a
local ad-hoc signature. Windows and Linux remain user scoped.
After this release, explicit updates can resolve and install the highest published alpha:
cargo xtask editor-update --user --channel alpha --check
cargo xtask editor-update --user --channel alpha
cargo xtask editor-doctor --user
cargo xtask editor-rollback --user
cargo xtask editor-uninstall --user
The updater verifies the release-manifest attestation against the repository, release workflow, tag, source revision and GitHub-hosted runner before fetching the exact tag with Git hooks disabled. The checkout must match the attested revision and remain clean before it is built. Updates are never silent.
Users of 0.1.0-alpha.1 must perform the exact-tag clone above once because
that release did not yet contain the source lifecycle. Subsequent alpha
updates use editor-update.
Safety and evidence
- Installer receipts bind the editor version, commit, deterministic working
tree,
Cargo.lock, Rust toolchain, platform, architecture and installed binary digest. - Active and previous immutable versions are retained for offline rollback.
- Install, doctor and uninstall run in sandbox roots on Linux x86-64 and Arm64, Windows x86-64, macOS Apple Silicon and macOS Intel release hosts.
- Existing unrelated paths are rejected. Removal requires a NUIF product marker and never targets a filesystem root.
- No command disables Gatekeeper, System Integrity Protection, Defender, SmartScreen or Smart App Control, changes a certificate store, or performs a system-wide installation.
The release still includes five native CI archives, five package manifests,
SHA-256 checksums, a CycloneDX SBOM, release-manifest.json, and GitHub
artifact attestations. These downloads are reproducibility evidence and an
expert opt-in path rather than the primary developer installation.
Remaining external boundary
Release archives are not Developer ID-notarized or Windows publisher-signed. A managed machine may require an organization-approved signing identity or device policy. The source lifecycle does not work around administrator-owned controls and does not require Apple or Microsoft marketplace publication.
NUIF Editor 0.1.0-alpha.3
This research preview adds a bounded Penpot package path, a more focused native workspace and a single-source publication pipeline. It remains a development and conformance tool. The application version does not assign maturity to the pre-draft specification or claim general interchange with any vendor product.
Developer installation
Clone the exact tag and retain the checkout as the lifecycle control plane:
git clone --branch v0.1.0-alpha.3 --depth 1 https://github.com/refpath/nuif.git
cd nuif
git rev-parse HEAD
cargo xtask editor-install --user --channel alpha
Existing alpha.2 installations can request the update explicitly:
cargo xtask editor-update --user --channel alpha --check
cargo xtask editor-update --user --channel alpha
cargo xtask editor-doctor --user
The source updater verifies the release-manifest attestation, exact tag and source revision before building with the pinned toolchain and lockfile. It does not perform silent updates or modify operating-system trust policy.
Changes since alpha.2
- Added
nuif-penpot-v3-0, a bounded Penpot v3 ZIP package adapter covering one file, page and board with direct rectangle, ellipse and text children. The foreign fixture is generated by the official@penpot/librarypackage. - Added Penpot import/export to the CLI and native File menu. Unchanged imported packages synchronize byte-for-byte; mapped edits retain unknown members and unmapped JSON, and unsupported structure fails atomically.
- Reduced small native package writer allocation by about 93% using a measured 4 KiB compression threshold while retaining foreign compression methods.
- Focused the native editor chrome on document actions and moved secondary controls to the command surface. Pixel units, the background grid and rulers remain enabled by default; the File menu exposes all seventeen supported native and adapter routes.
- Added captured canvas movement for freeform children with a local outline preview, one semantic transaction on release, default whole-pixel snapping and Control-suspended snapping. Managed-layout children are rejected instead of receiving ineffective position edits, and the native trial drives the real pointer path before replaying undo and redo.
- Added bounded eight-handle resize for freeform children and the three semantically effective trailing handles for managed-layout children. Leading freeform handles preserve the opposite edges through an atomic position and fixed-size transaction; Shift preserves corner aspect ratio and Control suspends snapping. The native trial locates the north-west handle from resolved layout, drives its real pointer path, and proves deterministic undo, redo and operation replay.
- Added same-parent Stack/Flex canvas reorder using resolved sibling geometry,
so responsive direction overrides follow the displayed axis. Reorder emits
one protocol
Move, avoids no-op history, asserts the resulting child order in the native trial, and fails closed for Grid, Constraint, cross-parent and instance-child semantics. - Added allocation-aware smoke coverage and Criterion surfaces for every integrated adapter, including official-foreign Penpot import, exact no-op synchronization and edited package synchronization.
- Corrected raster conformance to gate scene and raw-RGBA bytes instead of lossless PNG-compressor output. Detached old/current trials proved identical pixels while Cargo feature unification changed only compressed PNG bytes.
- Added a frontmatter-validated documentation compiler, GitHub Pages workflow and PDF manuscript composer. Repository Markdown remains the only editable documentation source.
- Added explicit maturity, security-reporting and standards-development gates; the project remains pre-standard and has not published a conformance profile.
- Added the byte-oriented browser WebAssembly developer binding and a static React JSX retentive adapter, each checked against canonical native output.
- Added
nuif-svelte-static-0to the core, CLI and native File menu, with byte-local synchronization, hostile-input limits, performance coverage and exact officialsvelte/compiler5.57.0 parse/compile evidence. - Added a strict workflow metadata audit to the complete harness. It rejects duplicate YAML keys, mutable external action references and repeated paths inside one artifact upload before GitHub interprets the workflow.
- Removed the unmaintained
ttf-parserdependency after RUSTSEC-2026-0192. Static package-font metadata now uses the already pinned Skrifa stack behind NUIF-owned sfnt/checksum/OS/2 checks, with a digest-bound HarfBuzz 14.4.0 metadata capture as the external conformance oracle and no advisory waiver. - Added
nuif-mcp-tools-0, a stateless MCP 2026-07-28 stdio adapter with four pure tools, bounded frames and no filesystem, network or host-document authority. The live subprocess gate checks metadata, schemas, annotations, native byte parity and hostile frames.
Verification and distribution
The complete harness covers 10,000 seeded protocol iterations, hostile-input and allocation budgets, browser layout differential trials, text/paint pins, headless and real native GUI authoring, sandboxed install/update lifecycle, eight executable adapter profiles, an independent Python reproduction and 5,040 collaboration delivery orders.
The release workflow builds editor and MCP binaries on Linux x86-64 and Arm64, Windows x86-64, macOS Apple Silicon and macOS Intel, plus a direct-browser WASM package. It publishes per-artifact manifests, SHA-256 checksums, separate editor/MCP CycloneDX SBOMs, a combined release manifest and GitHub artifact attestations.
Remaining external boundaries
- Native archives are not Developer ID-notarized or Windows publisher-signed. Source-built user installation remains the primary path.
- Penpot coverage is the declared bounded profile, not arbitrary
.penpotcompatibility. At publication, Figma and Adobe remained researched host- specific adapters rather than implemented vendor plug-ins. ADR 0012 later replaced Adobe in the active delivery queue with an Affinity interchange profile and a Canva Apps SDK adoption profile; this historical release does not claim either live integration. - Structural collaboration, Grid schema, general vector paint, automatic text wrapping, an externally maintained implementation and external interoperability review remain future gates.
Schemas and IDLs
This directory will contain machine definitions for stable interchange surfaces. NUIF’s logical specification is serialization-independent; no schema technology is allowed to become the semantics accidentally.
Encoding research will compare canonical text representation, compact binary transport/cache forms, schema evolution, unknown-field preservation, streaming/random access, deterministic hashing, and package security before a normative codec is selected.
NUIF command-line tool
nuif is the explicit-filesystem developer interface to the reference engine.
It validates, inspects, canonicalizes, patches, lays out, renders, packages and
converts declared adapter profiles without requiring the native editor or an
MCP host.
nuif capabilities
nuif validate document.nuif
nuif inspect document.nuif
nuif snapshot document.nuif snapshot 1440 900
nuif export document.nuif svg-0 output.svg fidelity.json
Run nuif --help for the exact command forms in this package. Machine-facing
commands emit JSON. Inputs are bounded by the profile limits reported by
nuif capabilities; package and adapter operations fail closed outside their
declared subsets.
The CLI declares no support for package behavior or other extension
capabilities. It can structurally validate, inspect, hash, extract and
byte-preservingly copy a package that declares them. Commands that evaluate or
rewrite that package—layout, render, snapshot, external adapter export, a
changed .nuif save or package-mode conversion—fail atomically with
PACKAGE_CAPABILITIES_REQUIRED. Native .nuif import/export retains verified
resources and requirements instead of silently rebuilding a document-only
archive.
The archive is an unsigned research-preview developer package. Verify its SHA-256 entry and GitHub artifact attestation before use. The binary has no background service or implicit network authority. It reads or writes only the paths/stdin/stdout selected by the caller.
Build the same tool from a reviewed checkout with:
cargo install --path crates/nuif-cli --locked
NUIF MCP adapter
nuif-mcp-tools-0 is a stateless, stdio-only Model Context Protocol adapter
over the same nuif-api, codec and semantic-operation crates used by the CLI,
WebAssembly binding and reference editor. It is an experimental developer tool,
not a network service or a canonical NUIF protocol.
Build it from a reviewed checkout without an app store or payment gateway:
cargo install --path crates/nuif-mcp --locked
Point an MCP host’s local stdio configuration at the resulting nuif-mcp
executable. The exact host configuration shape belongs to that host; the server
itself accepts MCP 2026-07-28 only and writes no non-protocol data to stdout.
The profile exposes four pure tools:
nuif_validatenuif_inspectnuif_canonicalizenuif_apply_patch
Every call supplies canonical NUIF text inline and returns a value. Even
nuif_apply_patch mutates only a temporary in-memory session and returns a new
canonical document. The process has no filesystem, network, package-resource,
host-document, credential, roots, sampling, task or hidden document-session
authority.
Limits are 4 MiB per newline-delimited MCP message, 1 MiB per inline document,
1 MiB per patch, 1,024 transactions and 16,384 operations. Larger documents and
.nuif packages should use the direct library, CLI or WASM surfaces under
explicit host control.
Run the independent subprocess and native-core oracle:
cargo xtask gate-mcp
cargo xtask mcp-package
The gate opens with server/discover and no legacy initialization handshake,
requires complete metadata on every request, checks generated schemas and
annotations, compares canonicalization and patch bytes with the native CLI,
classifies malformed and stale inputs, sends one frame above the transport
limit, and records a small wire-latency sample in
target/mcp-conformance-report.json.
mcp-package repeats the live gate against an optimized binary, then creates a
host archive and sibling manifest under target/dist/. Tagged GitHub
prereleases build and attest those archives on Linux x86-64/Arm64, Windows
x86-64 and macOS Arm64/x86-64. The binary is independently versioned at
0.0.1; the editor’s alpha version does not imply MCP protocol maturity.
NUIF WebAssembly binding
nuif-wasm-api-0 is a byte-oriented browser and JavaScript binding over the
same nuif-api, codec and semantic-operation crates used by the CLI and native
editor. It is an experimental developer package, not a stable npm release.
Bare and package loading, validation, capability negotiation, hashing,
canonical export and history delegate to nuif-api::NuifDocument; this crate
owns only the JavaScript byte boundary and its transport limits.
The generated package has no filesystem, network, host-document or rendering authority. A Figma, Canva or browser integration owns those capabilities and passes only selected NUIF or patch bytes into this module. Affinity currently uses the SVG interchange path outside the host because no stable public document API is claimed.
import init, { NuifDocument, capabilities } from "./nuif.js";
await init();
const contract = JSON.parse(new TextDecoder().decode(capabilities()));
const document = new NuifDocument(bytes, "nuif-text-0");
const validation = JSON.parse(
new TextDecoder().decode(document.validationReport()),
);
const revision = document.canonicalHash();
const nextRevision = document.applyPatch(patchJsonBytes);
const output = document.exportBytes("nuif-cbor-0");
document.free();
Portable .nuif packages retain digest-verified embedded images, fonts and
other inert resources across authorized edits and deterministic export.
Structural load is suitable for inspection, bare extraction and exact
same-mode copying. If the manifest has requirements, applyPatch, undo/redo
and mode-changing package export fail with
NUIF_PACKAGE_CAPABILITIES_REQUIRED until the complete set is authorized. A
plug-in that evaluates or changes such a package must declare its supported
capability identifiers as a bounded JSON string array first:
const text = new TextEncoder();
const structural = NuifDocument.fromPackage(packageBytes);
const report = JSON.parse(
new TextDecoder().decode(
structural.packageCapabilityReport(text.encode(JSON.stringify(hostCapabilities))),
),
);
structural.requirePackageCapabilities(
text.encode(JSON.stringify(hostCapabilities)),
);
const authorizedOutput = structural.exportPackage("portable");
structural.free();
const document = NuifDocument.fromPackageWithCapabilities(
packageBytes,
text.encode(JSON.stringify(hostCapabilities)),
);
const output = document.exportPackage("portable");
document.free();
The capability transport is limited to 64 KiB, 256 unique identifiers and 128
bytes per identifier. fromPackage does not execute a capability, fetch a
linked resource or imply that the host supports the manifest. Package inputs
are limited to the nuif-package-0 80 MiB archive budget.
Inputs and outputs remain canonical byte records instead of a JavaScript copy
of the NUIF data model. applyPatch is atomic and enforces 4 MiB, 1,024
transaction and 16,384 operation limits in addition to the core document
limits. Errors begin with a stable code such as
NUIF_PATCH_LIMIT_EXCEEDED followed by a human-readable message.
Build and test both the Node conformance package and direct-browser package:
cargo xtask gate-wasm
The command pins wasm-bindgen 0.2.127, initializes the web target in pinned
headless Chrome, and runs the generated Node binding. It requires byte-identical
bare and package output from the native CLI, exact preservation of a packaged
behavior resource, read-only structural mutation rejection and typed
missing-capability negotiation failure. The browser package is left under
target/nuif-wasm-web/.
NUIF fuzz harness
This standalone cargo-fuzz package keeps sanitizer-only dependencies out of the release workspace while calling the same production crates. The toolchain is pinned because libFuzzer instrumentation requires nightly Rust.
Generate valid seed inputs and run the bounded smoke campaign:
rustup toolchain install nightly-2026-08-28 --profile minimal --component rust-src
cargo +nightly-2026-08-28 install cargo-fuzz --version 0.13.2 --locked
cargo xtask fuzz-smoke
An exact driver can instead be selected with NUIF_CARGO_FUZZ; the command
rejects every version except 0.13.2. NUIF_FUZZ_RUNS changes the per-target
run count within the enforced 1–1,000,000 range.
The nested dependency graph has its own lock file and policy because the bundled LLVM libFuzzer runtime adds the OSI-approved NCSA license to the non-shipping test graph. Audit it with:
cargo deny --manifest-path fuzz/Cargo.toml --config fuzz/deny.toml check
For a longer local campaign:
cargo +nightly-2026-08-28 fuzz run codec_roundtrip \
target/fuzz-corpus/codec_roundtrip -- \
-max_len=1048576 -timeout=10 -rss_limit_mb=2048 -use_value_profile=1
The five targets have deliberately separate contracts:
codec_roundtrip: arbitrary text/CBOR bytes; accepted documents must reach canonical encode/decode fixpoints.package_decode: NUIF and Penpot archive parsers; accepted packages must deterministically re-encode and re-import.resource_decoders: bounded PNG and static-font inspection/decoding.adapter_import: UTF-8 HTML, SVG, DTCG, React and Svelte profile source import followed by export/import equivalence.operation_sequence: a byte choice stream becomes valid typed scalar operations and must preserve replay, inverse, codec and optional render relations.
Crash artifacts are generated under fuzz/artifacts/ by cargo-fuzz and are not
committed until reviewed and converted into a named regression fixture.
Independent Python profile 0
This directory contains a second, mechanically independent implementation of the bounded NUIF v0 conformance path. It uses only Python’s standard library and neither imports nor invokes a Rust workspace package.
The implementation reads and structurally validates canonical nuif-text-0, writes the canonical bytes, preserves an opaque unknown payload through an unrelated edit, evaluates the declared profile-0 stack/freeform/responsive layout, lowers explicit fidelity, and rasterizes the v0 solid rectangles and pinned Ahem text. The differential harness supplies reference artifacts; the implementation computes its own results before comparing boxes, decoded RGBA and fidelity.
Its scope is deliberately narrower than the full draft model. It supports the semantics exercised by the responsive-card fixture and returns a failure for visual operations outside that independent render subset. It is evidence for Gate G’s v0 reproduction criterion, not a second general-purpose NUIF product.
Run its local unit tests with:
python3 -m unittest discover -s implementations/python/tests -p 'test_*.py'
The complete cross-implementation run is exposed through cargo xtask gate-g.
Refpath research ingestion contract
This directory will contain tooling that projects repository research/spec/code metadata into Refpath’s research graph.
Node classes
source, paper, standard, repository, claim, question, experiment, rfc, adr, spec_section, fixture, crate, adapter, commit.
Edge classes
supports, contradicts, extends, implements, inspired_by, compares_to, supersedes, depends_on, tests, specified_by, decided_by, evidenced_by.
Determinism
Stable IDs are supplied by front matter/spec identifiers rather than generated from prose. Importers should hash normalized source records to detect changes and maintain supersedes history instead of destructive replacement.
Code indexing is separate from research ingestion but joins through explicit links.code references and repository commit identity.
NUIF bounded PNG decoder profiles
Status: executable experimental profile (nuif-png-rgba8-0). It is a narrow,
fail-closed image path, not a claim of general PNG support.
Accepted datastream
The resource is an exact PNG datastream with:
- one
IHDR, non-zero dimensions at most 8,192 by 8,192, and at most 16,777,216 pixels; - bit depth 8, colour type 6 (truecolour with alpha), standard compression and filtering, and no interlace;
- optionally one valid
sRGBchunk before image data; - one or more contiguous
IDATchunks followed by one emptyIEND; - no other chunks and no bytes after
IEND.
Absence of sRGB means encoded RGBA samples are interpreted as sRGB by this
profile. It does not mean the source asserted an sRGB chunk. The profile
rejects palette, grayscale, RGB-only, 16-bit, CICP, ICC, gamma/chromaticity,
Exif, animation, textual and arbitrary ancillary metadata instead of applying
host-dependent precedence or conversion.
Encoded input is limited to 32 MiB and 4,096 chunks. The decoder verifies PNG CRC and DEFLATE integrity and allocates exactly four decoded bytes per accepted pixel. Original encoded bytes remain the content-addressed authoritative resource; RGBA output is a deletable cache.
A render scene retains at most 64 MiB of unique decoded image surfaces. The builder inspects each new resource’s decoded size before inflation, rejects a total one-over atomically, and stores one surface per unique digest/profile. Image commands carry deterministic numeric handles, so repeated use does not duplicate pixels or descriptor strings in memory or serialized scenes.
Image-paint lowering
The reference scene supports fill, contain and cover, normalized crop,
nearest or fixed 16-bit-weight bilinear sampling, finite opacity from zero
through one, and color_conversion = "srgb". Source alpha is straight. Paint
opacity multiplies alpha, and the CPU raster applies the same encoded-sRGB
integer source-over rule as profile-zero solid paint.
The reference renderer executes the bounded normalized affine contract in
spec/05-geometry-paint-text.md. Singular or numerically unbounded transforms,
unresolved resources, dimension mismatch, unsupported decoder/profile values
and invalid crop/opacity values produce item-level
fidelity or typed errors; they never substitute a bounds rectangle or fetch a
resource implicitly.
Profile-zero evidence
cargo xtask gate-i-image:
- generates all five PNG row filters plus the encoder’s adaptive selection,
with and without an explicit
sRGBchunk; - requires identical dimensions and RGBA bytes from
png0.18.1 and independently implementedzune-png0.5.2 with unsafe paths disabled and CRC/Adler checks enabled; - preserves encoded bytes through package fixpoint and an unrelated semantic edit;
- repeats scene lowering and CPU rasterization exactly;
- checks identity, horizontal flip, clockwise rotation and translation through forward affine matrices and rejects a singular matrix;
- checks hostile/unsupported colour types, metadata, corruption, trailing bytes, and dimension, pixel, chunk and encoded-byte one-over cases.
- requires 1,024 uses of one 512×512 resource to retain one 1 MiB surface; the warmed release trial allocates under 8 MiB and retains under 4 MiB;
- rejects a declared decoded-surface total of 64 MiB plus 16 bytes before attempting the second image decode.
Basic RGBA8 profile one
Status: executable experimental profile (nuif-png-basic-rgba8-1). This is a
new profile, not a silent expansion of profile zero.
Profile one accepts the same bounds, compression/filter methods, contiguous
image data, optional sRGB declaration and encoded-sRGB interpretation as
profile zero. It additionally accepts every non-interlaced PNG colour/depth
combination that can be normalized to RGBA8 without discarding sample
precision:
- greyscale at 1, 2, 4 or 8 bits;
- indexed colour at 1, 2, 4 or 8 bits, with its required
PLTE; - 8-bit RGB, greyscale-alpha and RGBA;
- one valid pre-image
tRNScolour key or indexed alpha table where the PNG colour type permits it.
Sub-byte greyscale samples use PNG’s exact full-range expansion. Indexed
samples use their exact 8-bit palette entries. Missing alpha becomes 255 and
tRNS becomes an explicit 8-bit alpha channel. These are lossless
normalizations into RGBA8; original encoded bytes remain authoritative.
The profile rejects 16-bit samples rather than silently dropping precision. It also continues to reject Adam7 interlace, CICP, ICC, gamma/chromaticity, Exif, animation, textual data, suggested palettes on non-indexed images and arbitrary ancillary chunks. Those features need explicit colour/orientation/animation contracts and independent fixtures.
cargo xtask gate-i-image adds thirteen profile-one fixtures spanning every admitted
colour/depth combination and both colour-key and palette
transparency. png 0.18.1 and independently implemented zune-png 0.5.2 must
produce the same normalized RGBA bytes. Seven profile-one negatives cover
16-bit precision, rejected metadata, suggested palettes, a missing required
palette and the not-yet-profiled interlace path. A profile-one RGB resource
also passes renderer lowering and CPU rasterization.
Evidence boundary
The gate does not establish GPU or hosted cross-platform image-raster equivalence, host-specific affine interoperability, a broad real-world corpus, 16-bit/interlaced/colour-managed PNG, or any non-PNG image format. Those require distinct profiles and fixtures.
Static OpenType resource profile 0
Status: experimental, implemented, and not a general OpenType conformance claim.
Identifier: nuif-opentype-static-single-0
This profile gives a NUIF package one deterministic, bounded baseline for an exact authoring font resource. It accepts a single-face, statically instanced TrueType-outline sfnt and rejects other font source categories instead of silently interpreting them differently across hosts.
Accepted input
- sfnt signature
0x00010000and face index0; - at most 32 MiB and 256 strictly sorted, unique table records with consistent sfnt search fields;
- required
OS/2,cmap,glyf,head,hhea,hmtx,loca,maxp, andnametables; - aligned, in-range, contiguously packed table data with exact zero padding and no trailing data;
- valid per-table checksums and complete-font checksum;
OS/2version 0 through 5 with one unambiguousfsTypeusage permission;- Unicode coverage derived from mappings that resolve to glyphs;
- at most 256 family names, 65,536 coverage ranges, and 64 declared feature settings.
Collections, CFF/CFF2 outlines, variable fonts, color or bitmap glyph tables,
SVG glyphs, WOFF/WOFF2 containers, and unknown OS/2 versions are outside this
profile. They require separately named profiles and conformance evidence.
Asset binding
The font asset must exactly match the parsed face index, family names, static
axis state, and Unicode coverage. Its resource descriptor must use font/ttf.
The asset records:
font.decoder_profile = nuif-opentype-static-single-0;opentype.fs_type = 0xNNNN, matching the exact bytes;- a non-empty
license.expressionchosen by the publisher; license.embedding_review = approved, recording an explicit human or organizational decision.
The parser rejects restricted or bitmap-only embedding evidence for this
portable outline profile. The review field does not grant rights, interpret a
license, or make OS/2.fsType authoritative over the font’s actual license.
Publishers remain responsible for redistribution and embedding permission.
Package and resolver behavior
Embedded fonts are validated during manifest construction, package encoding, and package decoding. Digest-pinned linked fonts in an authoring package remain unresolved; a caller-provided resolver must return the exact bounded bytes, after which the same profile validation runs. Package parsing never performs a network request.
Security limits and non-claims
The implementation contains no unsafe code and uses pinned Skrifa 0.46.2
only after NUIF-owned sfnt directory, range, packing and checksum checks. NUIF
also reads the required head, maxp and OS/2 fields directly, requires the
first two to agree with Skrifa metrics, and applies its own conservative
embedding-bit policy.
Resource limits are validation policy, not proof that an accepted font is safe
for every downstream native rasterizer. A renderer must preserve its own
sandbox and work budgets.
The release gate measures each of the four accepted fixtures after one parser
warmup. A single inspection and a packaged-font validation must each remain at
or below 4 MiB total allocator traffic and 2 MiB retained memory. These are
reference-implementation regression ceilings measured with stats_alloc
0.1.10, not portable format limits or a downstream rasterizer budget.
This baseline does not yet prove shaping equivalence, glyph-outline equivalence, subsetting, variable-axis behavior, color-font behavior, browser font decoding, layout fidelity, or licensing compliance.
Evidence boundary
cargo xtask gate-i-font accepts four static TrueType fixtures from
font-test-data 0.9.1 and compares the exact pinned Ahem metrics, family,
tables and Unicode coverage with a committed hb-info 14.4.0 capture. It
rejects 20
synthetic and real cases spanning malformed/checksum-invalid sfnt data, TTC,
CFF, variable, COLR, embedded bitmap, CBDT and sbix categories. Ten metadata and
embedding-policy mutations plus six portable/private/linked/substituted/
unavailable package outcomes are blocking. The real rejected fixtures prove
that those categories fail closed; they do not specify how a future profile
will accept them. Four warmed inspection-allocation trials and one warmed
packaged-validation allocation trial are also blocking. Six item-level trials
prove substituted/unavailable text binding, layout fidelity and render-command
behavior through a package round trip.
NUIF collaboration profiles
nuif-collab-registers-0 is a bounded operation-set collaboration profile above canonical NUIF. It proves convergence for register-like semantic operations without adding replica IDs, version vectors, histories or conflicts to Document.
Change model
A change has a dot (replica, counter), a version-vector context and one NUIF semantic operation. Replica counters are contiguous. Contexts must name received changes and transitively include their contexts; incomplete history fails closed. Replica identifiers and collection sizes have declared limits.
The profile maps these operations to multi-value registers:
- rename, horizontal/vertical size and layout;
- set/remove token;
- document extension declarations;
- set/remove authored property value;
- set/remove entity extension;
- set unknown payload.
Insert, remove, move and restore-subtree are rejected before ingestion. They require a tree/list CRDT with explicit cycle, deletion and sibling-order semantics; total-ordering them as ordinary registers would overstate correctness.
For each property key, causally superseded changes leave the frontier. Concurrent identical values coalesce without a conflict. Concurrent different values create a SemanticConflict containing every frontier candidate and a deterministic selected dot. The selected values materialize a canonical checkpoint in causal order; cross-register model invariant failures return a typed apply error rather than a partial checkpoint.
Two materializers
OperationSetEngine joins a BTreeMap<ChangeId, Change> and computes maximal changes pairwise. ReplicaLogEngine joins per-replica logs and maintains each register’s maximal causal frontier incrementally. Their merge methods are atomic on error. Both materialize the same public Checkpoint, but their frontier algorithms and storage representations are distinct.
This is algorithmic independence inside one repository, not an externally authored CRDT implementation and not an Automerge/Yjs interoperability claim.
Automated evidence
cargo xtask gate-h runs a seven-change, three-replica responsive-card history with:
- a causal overwrite;
- concurrent card-name and variant edits producing two explicit property conflicts;
- all 5,040 delivery permutations through both materializers;
- different three-way merge orders and duplicate delivery;
- opaque unknown-payload preservation and canonical-text inspection for leaked collaboration metadata;
- negative cases for missing history, duplicate dots, invalid local context, structural operations and semantic apply failure.
The release-mode report is target/collaboration-report.json and is part of cargo xtask all and CI artifact upload.
Structural tree profile 0
nuif-collab-tree-0 is a separate bounded profile for moves, reorders and
deletion of identities already present in the canonical base. It does not
weaken the register profile’s rejection of structural operations or pretend a
move is an ordinary last-writer-wins property.
Each move has a unique Lamport-ordered dot, target parent and stable sibling
origin. Base positions are identified by entity ID; later positions are
identified by the change dot. Position identifiers, inactive origins and the
synthetic trash parent are collaboration metadata and never enter canonical
NUIF. Within one sibling list, entries sharing an origin are traversed in
descending identifier order and retain inactive origins, following the core
RGA rule. The public checkpoint resolves canonical Anchor values to stable
positions so a later operation cannot accidentally bind to a different move of
the same entity. Both materializers are bound to one canonical base hash;
different-base joins fail. A change-position anchor must exist and occur in the
author’s transitive causal history.
Changes are replayed in ascending unique timestamp order. A move that would
make its destination a descendant of itself is retained but has no tree effect
and produces CycleRejected. Deletion moves an entity under profile trash;
its descendants remain available so a concurrent or later move can rescue
them. Canonical checkpoints contain only the forest reachable outside trash.
Concurrent move/move, delete/move, deleted-parent and delete/descendant-move
intent remains in typed conflicts even though a deterministic checkpoint is
available.
StructuralOperationSetEngine replays a sorted operation set.
StructuralUndoRedoEngine applies monotonic local changes directly and rolls
back/replays when a lower timestamp arrives. Gate H exhausts all 5,040 deliveries
of a seven-replica move/delete/cycle/stable-anchor fixture, checks join and idempotence, and
compares both paths. A 4,096-change/4,097-entity release trial guards the linear
checkpoint path.
Pinned @automerge/automerge 3.4.1 independently merges immutable structural
change records forward, reverse and in a different partition order, then
checks duplicate merge and save/load. Automerge is the foreign convergent
transport oracle only: it does not implement NUIF’s tree move, cycle, trash or
semantic-conflict rules. Concurrent creation, causally stable garbage
collection, combined property/structure transactions and an independently
authored tree materializer remain outside this profile.
NUIF behavior state-machine profile 0
Status: executable research sidecar. nuif-behavior-state-machine-0 is not
part of the canonical semantic Document and does not establish the final
behavior schema. Its first experimental transport is the separately profiled
content-addressed package resource below.
Model and execution
The profile is a flat deterministic state machine keyed by stable NUIF entity
identifiers. External activate events are accepted only from entities whose
semantic role is button, checkbox, radio or switch. In the active
state, transitions are examined in authored order. The first matching
event/guard executes its actions sequentially and changes the active state; an
unmatched event is a no-op. One external event always runs to completion before
the next starts.
State values are bounded Booleans and strings. Actions can set a value, toggle a Boolean, or emit one of two abstract effects:
visibility, carrying a Boolean for a stable target entity;announcement, carrying a bounded string for a stable target entity.
The runtime emits effects as data and does not directly mutate the document or
call a host API. The separate nuif-web-behavior-0 adapter now maps the first
admitted subset to native browser activation, DOM visibility and a status live
region. Native and presentation adapters still require their own fidelity
contracts.
Capabilities and limits
Each used effect capability is declared required or optional_noop. Missing
required capabilities reject runtime construction before any action executes.
An unavailable optional capability follows the profile’s declared no-op
fallback and is recorded in the trace. Silent fallback is not permitted.
The static envelope admits at most 128 states, 1,024 transitions, 4,096 total actions, 64 actions per transition, 128 variables and 64 capabilities. One run accepts at most 4,096 external events. Identifiers and strings have explicit byte limits. Unknown fields, entities, states, variables, capabilities, value types, unreachable states and incompatible activation sources fail closed.
Timers, internal event queues, parallel states, floating-point or integer arithmetic, navigation, animation, document mutation, filesystem/network effects and arbitrary scripts are outside profile 0.
Differential oracle
cargo xtask gate-behavior executes the same fixture through the Rust
reference runtime and a separately written JavaScript interpreter under pinned
Node in CI. It compares complete event, transition, state, variable, emitted
effect and skipped-optional traces for both full and required-only capability
sets. It separately requires a missing required capability to fail before
execution.
Artifacts:
target/behavior-portability-fixture.json;target/behavior-portability-static-report.json;target/behavior-portability-report.json.
The JavaScript oracle is a second implementation of this profile. It is not a
browser DOM adapter or native UI runtime. Browser host mapping is tested
separately by cargo xtask gate-web-behavior; neither gate is evidence for
excluded behavior.
Package attachment
nuif-behavior-package-resource-0 stores one program as canonical CBOR in one
embedded source resource with provisional media type
application/nuif-behavior+cbor. The normal package manifest records its size,
SHA-256 digest and digest-derived blob path and declares
nuif-behavior-state-machine-0 as required. No new Document field or ZIP
member family is introduced. The API is behind the opt-in Cargo feature
package, keeping state-machine-only consumers independent of the package and
codec dependency stack.
#![allow(unused)]
fn main() {
let digest = nuif_behavior::attach_behavior(&mut package, &program)?;
let bytes = package.encode()?;
let package = nuif_package::NuifPackage::decode(&bytes)?;
package.require_capabilities(&host_capabilities)?;
let attachment = nuif_behavior::attached_behavior(&package)?;
}
Generic package decode verifies and preserves the resource without executing
it. attached_behavior is the explicit opt-in that checks exact cardinality,
descriptor policy, canonical CBOR and every entity reference against the
package document. Runtime construction remains a later operation requiring a
caller-supplied set of effect capabilities. The behavior digest identifies the
program bytes; the complete package hash binds those bytes to the delivered
document.
cargo xtask gate-behavior-package records the Rust attachment checks and an
independent Python standard-library ZIP inspection in:
target/behavior-package-fixture.nuif;target/behavior-package-expected.json;target/behavior-package-static-report.json;target/behavior-package-report.json.
cargo xtask gate-behavior runs this attachment gate before the independent
Rust/Node trace gate.