Download the generated PDF manuscript

NUIF authored-interface interchange research manuscript

Architecture, executable profiles, resource packaging, reconstruction research and open risks

Refpath contributors
Version working-2026-09-06 · 2026-09-06

Manuscript status: working technical manuscript; not peer reviewed. This generated document does not change the status of any included specification module.

Abstract

This manuscript evaluates a layered authored-interface interchange model through prior-art synthesis, bounded adapter profiles and executable conformance experiments. It also separates proposed resource packaging, source-backed capture and screenshot reconstruction from the narrower implemented alpha evidence. It records the architectural hypothesis, implemented evidence, unresolved risks and governance conditions without claiming specification stability, reconstruction accuracy or universal format coverage.

Reproducibility

The body is compiled from 13 canonical whitepaper modules at source digest sha256:618d9a6296ca8725a948dddac1890674f41a62bd0105c6752977e409df94f6f1. Editorial changes belong in those source modules.

NUIF foundation

NUIF investigates authored user-interface interchange through a draft document model, a Rust reference implementation and bounded source-adapter experiments. The research question is whether explicit identity, correspondence and fidelity records permit useful edits across representations without discarding unrelated source or unknown extension data. The current evidence concerns named profiles; it does not establish arbitrary application interchange.

Architectural hypothesis

The model separates containment, typed relationships, authored properties and resolved snapshots. A snapshot records an evaluation context rather than replacing the authored document. Editable entities and assets have stable identities; immutable resource bytes have content digests. These distinctions are specified in identity, layout and serialization.

The proposed architecture coordinates components, layout, visual properties, tokens, behavior, provenance and extensions. These areas have different implementation maturity. The adapter inventory and research coverage contract delimit the implemented projections and unresolved questions. The architecture is tested through the following hypotheses:

• For a declared scalar-edit profile, re-import of synchronized source equals the requested document, and bytes outside the reported edits remain identical.
• Unknown extension payloads survive supported neighboring edits and canonical serialization without requiring the editor to interpret those payloads.
• Pinned evaluation contexts allow implementations to compare resolved output under explicit exact or tolerance-based criteria.

Each hypothesis requires evidence for its own domain. Success for scalar HTML edits does not establish structural source reconciliation, behavior equivalence or native rendering portability.

Research method

The repository combines targeted primary-source review, executable profile construction, generated tests, differential comparisons and failure analysis. Research records preserve source locators, retrieval dates, interpretation and open questions. The search process is iterative and architecture-directed; it is not a systematic literature review with a preregistered search strategy or an exhaustive publication census. A reviewed record is not an independently verified result; the audit policy defines the stronger verification conditions.

Evidence is reported at the level of the tested assertion:

The distinction between local repetition and another team’s reproduction follows ACM’s artifact-review terminology, version 1.1, retrieved 2026-09-06. No ACM evaluation or badge is claimed. Profile-specific scope and conformance clauses follow the approach described in the W3C QA Framework, sections 2–3, retrieved 2026-09-06; this is a methodological reference, not W3C endorsement.

The verification manifest records revision, environment and artifact hashes. Performance measurements require a workload, build mode, hardware, sampling procedure and reference implementation. Timing on one host does not establish a cross-platform performance bound. A valid research registry establishes traceability, not resolution of its open experiments.

Fidelity model

The fidelity specification defines five classes: lossless, representable, approximated, preserved_unrenderable and unsupported. A class applies to a stated mapping and evidence boundary. Preserving an unknown payload does not establish that a target renders or edits its meaning. Retaining foreign CSS bytes does not establish visual equivalence under the browser cascade.

Screenshot reconstruction produces hypotheses about a document. Multiple programs can render the same pixels, so screenshot-only evidence cannot identify unique authored source. Reconstruction accuracy, confidence calibration and edit usefulness require separate held-out evaluation, as described in the resource and reconstruction chapter.

Threats to validity

The implementation and many fixtures share authorship. Generated inputs cover selected structural families and boundary values, with no claim of sampling from a deployment population. Dependency pins constrain variation but can mask host-specific behavior. Small oracle matrices leave untested combinations of fonts, layout and browser state. The native editor has no completed external usability study. These limitations prevent extrapolation from passing local gates to general beta readiness or broad interoperability.

The developer beta contract defines the narrower workflow and the evidence required for promotion. The risk register identifies observations that would require narrowing or revising the design.

Canonical module.

NUIF architecture thesis

NUIF defines authored-interface semantics in draft specification modules and tests selected projections through a reference implementation. The model, adapter profiles and implementation choices have separate acceptance criteria.

Core thesis

The architectural hypothesis is that explicit authored properties, typed relationships and context-scoped evaluated results permit source edits with declared fidelity. Flattening is treated as a lowering with a loss report when it removes information required by subsequent edits.

The recommended architecture is a layered hybrid:

Document containment tree
        │ stable IDs
        ├── component / instance graph
        ├── token / theme graph
        ├── layout constraint graph
        ├── interaction / state graph
        ├── provenance / correspondence graph
        └── asset dependency graph

Authored model ──evaluate/lower──► resolved model ──► render scene
      ▲                               │
      └──────── reconcile / lift ◄────┘

Prior-art foundations

• MLIR: dialects, explicit lowering, partial legality and multiple abstraction levels.
• OpenUSD: non-destructive composition, references, layers and variants.
• glTF: small core, extension registry, used/required capabilities.
• DTCG: token interchange.
• SVG/Unicode/OpenType: geometry and text foundations.
• Retentive/symmetric lenses: synchronization with preserved source regions.

The prior-art comparison and cross-industry synthesis identify the source records and adaptation decisions.

Integration work

The unresolved integration work concerns correspondence between authored properties, resolved observations and source syntax. The adapter inventory distinguishes implemented scalar synchronization from unsupported structural changes.

Canonical layers

This decomposition includes proposed semantics beyond the implemented profiles. Research coverage records their evidence status.

No lower layer is permitted to silently erase a higher-level authored construct. Lowerings that cannot represent a construct must emit fidelity records.

Stable identity

Identity is semantic and independent of path, order and display name. Moving an entity does not change its ID. Content hashes identify immutable resources and canonical snapshots, not editable semantic entities.

Compiler and reconstruction ports

Deterministic source adapters and probabilistic screenshot reconstruction meet at the operation boundary:

retained source + resolved host observations ─┐
                                              ├─> typed operations -> core
pixels + OCR/CV/model hypotheses ─────────────┘                    -> render/evaluate

Source-backed and screenshot-only inputs retain distinct evidence classes. A model/provider is replaceable and cannot redefine the operation grammar, validator, layout semantics, resource identity or fidelity ceilings.

Falsifiability

The v0 HTML experiment tests retention of component identity, token bindings, layout intent and an opaque extension through an editor/source round trip. Its named fixture passes. The broader hypothesis remains subject to counterexamples from other documents and edits.

The resource/reconstruction extension fails if independent package writers cannot reproduce the proposed bytes, if browser capture cannot be pinned without secret leakage, if visual objectives reward flat screenshot copies, or if adaptation fails to beat the untuned tool-assisted baseline on a frozen holdout.

Canonical module.

Layout and rendering research synthesis

Authored and resolved layout

NUIF separates authored constraints from resolved boxes. Fixed x/y/width/height are valid authored values for freeform content, but they are not the universal layout representation.

The proposed layout vocabulary contains several families. Implemented subsets and refusal behavior are defined in layout profile 0:

• freeform — transforms/anchors and explicit geometry.
• stack — one-dimensional flow with intrinsic sizing, distribution, alignment and gaps.
• flex — web-compatible flexible layout semantics.
• grid — bounded explicit fixed/fr tracks, spans and deterministic no-implicit-track placement in profile 0; broader CSS Grid features remain capability-reported adapter input.
• constraint — relational linear constraints for editor/native-layout cases.
• custom — extension/dialect-defined evaluator with declared fallback/resolved geometry.

Common sizing primitives are normalized across families: fixed, intrinsic-min, intrinsic-max, fit-content, fill/available, percentage, min/max clamps, aspect ratio and content measurement.

Taffy is the reference evaluator selected by ADR 0002. Its types remain behind the NUIF layout interface. The proposed vocabulary also accounts for proposal–response layout and linear constraints; these do not acquire Taffy compatibility by being represented in the schema.

Evaluation context

Resolved layout is keyed by an explicit context including viewport/container size, pixel ratio, locale, writing direction, font set, token/theme selection and feature/dialect capabilities. Multiple resolved snapshots may coexist as caches or conformance fixtures.

Rendering semantics

The draft specification defines the visual meaning of paths, fills, strokes, transforms, clipping, masks, gradients, compositing, images, text and supported effects. It does not specify GPU command buffers or a renderer implementation.

The reference renderer separates scene evaluation from rasterization. The rendering decision defines the backend boundary; render conformance specifies exact and tolerance criteria. Interactive GPU output does not define normative pixel identity.

Text

The proposed text model retains Unicode content, style runs, semantic annotations and font references. Shaping produces resolved glyph IDs, clusters, advances and offsets using pinned font data and a declared Unicode/shaping version. A glyph cache never replaces semantic text.

Portability reports must distinguish font substitution, missing glyphs, line-break differences and rasterization differences from document-model loss.

Canonical module.

Protocol, portability and synchronization

NUIF models editing through semantic operations and source correspondence. Operations and patches define the protocol; the adapter inventory limits the currently implemented source mappings.

Operations

The protocol operates on stable entities and semantic properties. The draft operation vocabulary includes creation, deletion, movement, property changes, relation edits, overrides, token bindings and extension edits within transactions. Editor gestures lower to these operations.

A drag inside a stack should usually become a reorder or layout-property edit; a drag in freeform space may become a transform edit. GUI coordinates are input data, not the protocol abstraction.

Patch model

A patch is a deterministic ordered set of operations with base snapshot identity, optional preconditions, transaction metadata and provenance. Patches can be replayed headlessly.

Three-way merge uses stable identity first and structural matching only when identity is absent. Conflicts are typed: property, delete/edit, ordering, relationship, extension and semantic-lowering conflicts.

Correspondence

Adapters maintain correspondence records between NUIF entities/properties and foreign constructs such as DOM nodes, CSS declarations, static component attributes or design-tool node IDs. Correspondence is separable from the canonical design so source-specific metadata can be detached when unnecessary.

Fidelity classes

Every adapter/evaluator may report:

• lossless — semantics preserved and reconstructable.
• representable — equivalent semantics represented through different constructs.
• approximated — visible/behavioral approximation with known semantic loss.
• preserved_unrenderable — data retained opaquely but not understood/rendered.
• unsupported — data could not be safely preserved.

Silent degradation is a conformance failure.

Canonical module.

Serialization, collaboration and governance

Logical model before encoding

NUIF defines one logical model with multiple conforming encodings.

Text form

A canonical, reviewable representation is implemented for examples, fixtures, diffs and Git workflows. nuif-text-0 fixes number formatting, UTF-8 key order, layout and strict decode/canonicalize behavior; later text profiles may evolve only through explicit versioning.

Binary form

Deterministic CBOR is the profile-0 binary form because the NUIF profile closes the choices left by RFC 8949 without coupling the logical model to generated code. The executable codec gate finds it near 41% of canonical-text size at 4,096 entities on an Apple M5 Pro run, while its typed decode path is slower than text. That result supports CBOR as a compact canonical form, not as a universal latency advantage. The workload and artifact contract are defined in serialization research and crates/nuif-testing/src/bin/codec-benchmark.rs.

A candidate is timed only after complete-model round trip, canonical fixpoint and unknown-data preservation through a neighboring edit. Protobuf does not specify canonical binary output. FlatBuffers deliberately permits different byte layouts and old readers ignore new fields, so a rebuilding editor needs a separate retention strategy. Cap’n Proto specifies a schema-agnostic canonical form and is the preferred next experiment, but it still needs a complete NUIF mapping, bounded old-reader edit trial and two agreeing canonical writers. Compiled zero-copy runtime caches remain separate, explicitly noncanonical profiles rather than replacements for authoring interchange.

The experimental package form separates manifest/document records from content-addressed resources. RFC 0010 selects a candidate deterministic ZIP profile with fixed mimetype, canonical manifest/document records and SHA-256-addressed blobs. Bare encodings use explicit .nuif.json and .nuif.cbor names. Exact ZIP header fixtures, two independent local writers and bounded image/font segments now exist. Cross-platform and externally authored writer evidence remains required before package-profile acceptance.

Semantic document, resource and package hashes have different scopes. Stable asset identity is not content addressing. Unknown extension payloads remain explicit typed bytes/values and must not depend on accidental codec unknown- field behavior.

Collaboration

Canonical documents do not require CRDT tombstones, clocks or replica metadata. A collaboration profile maps NUIF operations to an append-only/change structure and can use Automerge, Yjs or another convergent transport. Checkpoints serialize back to canonical NUIF.

Canonical files omit collaboration transport metadata; each collaboration profile must define its checkpoint mapping.

The executable register profile uses causal multi-value registers. The separate existing-tree profile replays uniquely ordered moves, rejects cycles, models deletion as profile trash and orders siblings through stable RGA-style origins. Semantic move and deletion conflicts remain visible even when the profile can choose a deterministic checkpoint. Automerge is presently tested as an operation-set transport, not claimed as an implementation of the tree algorithm.

Governance

Early development occurs in refpath/nuif, but the architecture assumes eventual neutral stewardship. The standards roadmap proposes the following progression:

1. OSS research/reference implementation under Refpath.
2. RFC process and implementer registry.
3. independent community/working group once two independent implementations exist.
4. investigate W3C Community Group for UI/document semantics and/or Khronos-style governance if renderer/asset vendors become primary stakeholders.

Specification text, schemas and conformance tests need clear royalty-free contribution/IP terms before claiming standards-track stability.

Canonical module.

Prior-art comparison

The surveyed systems provide precedents for individual parts of authored interface interchange. This comparison records their relevance to NUIF’s proposed scope. It is not a product ranking or an exhaustive novelty claim. Each linked research record identifies the reviewed source and retrieval date.

Repository interpretation

The architecture adopts established geometry, token and text specifications where their semantics match a declared profile. Composition, lowering, capability declarations and source correspondence require adaptation to UI editing. The cross-industry synthesis records those decisions individually.

The proposed contribution is the integration of authored and resolved state, stable semantic identity, opaque preservation, fidelity reporting and retentive source edits. Evidence for that integration comes from bounded experiments. Neither the number of surveyed systems nor the absence of an identical system in this comparison proves novelty, usefulness or adoption potential.

Canonical module.

Implementation language and runtime choice

Rust reference core

ADR 0001 selects Rust for document validation, operations, codecs, evaluation and language bindings. The existing workspace demonstrates those interfaces through native, WebAssembly and C consumers. This is implementation evidence for the selected stack, not a comparative benchmark of programming languages.

Rust’s ownership and type system support the implementation’s memory-management and typed-operation requirements. They do not prove semantic correctness or eliminate unsafe behavior in dependencies. Bounded parsing, coverage-guided fuzzing and independent oracle comparisons remain necessary. The toolchain policy records the pinned compiler and minimum supported Rust version.

Alternatives and replacement criteria

No equivalent NUIF core has been implemented in C++, Zig, Go or TypeScript for a controlled comparison. Claims that these languages are categorically unsuitable are therefore unwarranted. A replacement proposal requires evidence for the actual workloads: hostile-input handling, canonical serialization, operation replay, native/browser embedding and maintenance cost.

Target-native adapter code remains appropriate where a host exposes its API through another language. The binding decision separates those host interfaces from the document model and does not make Rust a conformance requirement.

Dependency boundaries

The dependency register records each direct library’s role, alternatives and rationale. Taffy supplies layout evaluation; Harfrust and Skrifa supply shaping and font interpretation; Masonry and AccessKit supply the native shell and its semantic test surface. Source adapters use Tree-sitter where retained syntax is required. Each dependency remains behind NUIF-owned types or an explicitly declared host boundary.

The editor fork review records the retained patches and a CPU-rendered egui alternative. A small alternative-backend probe does not establish replacement parity for the editor. Migration requires the same canvas, semantic-action, text, rendering and host integration acceptance criteria as the current implementation.

Canonical module.

Research limitations and implementation risks

Representational limits

Rendered pixels do not determine a unique layout program. Source-backed observations and inferred properties therefore require distinct provenance. The reconstruction experiment can test whether an inferred document supports specified edits or held-out viewports; it cannot prove recovery of unique original source. The reconstruction synthesis provides the relevant source records and evaluation contract.

Behavior portability is limited to declared operations and capabilities. Arbitrary JavaScript, Swift or Dart application behavior is outside the current model. Native controls, font substitution and rendering environments also require separate fidelity judgments. These limits are recorded in the adapter inventory and text conformance research.

Resource preservation has two independent requirements: exact bytes and authorized use. A valid digest or accepted font header establishes neither redistribution permission nor decoder safety. Capture can expose private data; the collection, export, retention and training policies are separate inputs. The applicable implementation limits are defined in security and the resource and reconstruction chapter.

Evidence and adoption risks

Generated round trips can preserve a shared exporter/importer mistake. Independent expected values, separate implementations and externally authored fixtures address different parts of this risk. None is replaced by increasing the generated case count. Research methods describe the distinction.

Visual similarity can conceal loss of structure, text, accessibility or editability. Reconstruction evaluation therefore reports those properties separately. Raw model probabilities remain uncalibrated until measured against held-out outcomes; they cannot promote inferred values to source evidence. See confidence calibration.

A larger semantic core requires more adapter mappings and conformance cases. The project has not measured how this cost affects adoption. The response is to delimit profiles and measure implementation effort, rather than infer an adoption outcome from the complexity of IFC or STEP. External implementer and workflow evidence is required by the standards roadmap.

Implementation boundaries

The fork evidence and alternatives are documented in the editor fork review. File-output documentation states permission, symlink, hard-link and durability limits. Resource and capture continuations are enumerated in the experiment registry.

Architecture revision criteria

The source-edit design requires revision if common edits within a declared profile repeatedly require regeneration of unrelated source. Opaque preservation requires revision if supported neighboring edits discard unknown payloads. The specification requires revision if an independent implementation cannot derive profile behavior from its requirements and fixtures without consulting Rust internals. Each failure should produce a reduced case and a scoped decision, rather than a compatibility exception without a semantic contract.

Reconstruction scope should narrow if gains in pixel similarity reduce structural or edit-task quality, if confidence does not support useful risk/coverage tradeoffs, or if adaptation fails to outperform the untuned baseline under the same held-out data and budget. These are proposed decision criteria; current synthetic results do not resolve them.

Canonical module.

Governance and standardization strategy

NUIF is maintained under Refpath. The governance policy defines current decisions; the standards roadmap separates them from proposed neutral stewardship.

Repository governance now

• RFCs for semantic changes.
• ADRs for reference-implementation choices.
• Research evidence is distinct from normative requirements.
• Extension registry changes require examples and conformance fixtures.
• No vendor adapter can redefine core semantics.

Standardization path

A W3C Community Group is a plausible early venue for document/component semantics and coordination with Open UI/DTCG, but NUIF should not enter formal standards work before an executable v0 and at least one external implementer exist. Khronos/ASWF-style governance offers useful precedent for graphics/rendering and extension registries. A neutral foundation can become appropriate once multiple vendors/projects depend on the format.

IP/licensing goals

• reference code: permissive dual MIT/Apache-2.0;
• specification/schema/conformance text: permissive terms compatible with standards adoption;
• contributions: explicit patent/IP policy before standards-track claims;
• trademarks/conformance branding: separate from implementation copyright.

The project must not call itself an industry standard merely because the repository is public. Conformance, multiple implementations and neutral governance are prerequisites.

Canonical module.

Naming and project identity

NUIF is the current working name and repository slug, not yet a cleared standards trademark.

The naming review recorded prior acronym use in an old Nexus User Input Framework and in a 2024 computer-vision paper, plus unrelated uses. This reconnaissance is not trademark clearance.

The architecture therefore separates human branding from protocol identity. Stable schema namespaces, extension IDs and version identifiers must not depend on a product trademark remaining unchanged.

Before v0.1 branding is promoted outside the research project:

1. perform repository/package/domain/trademark clearance in key jurisdictions;
2. decide whether NUIF is an acronym or a proper project name;
3. reserve crate/package/extension namespaces;
4. define conformance-mark governance separately from the open specification;
5. make any rename before third-party persisted documents become common.

Canonical module.

Research coverage and evidence status

research/coverage.yaml maps architectural topics to source records, questions, specification modules, decisions and experiments. The validator checks those links and the required continuation fields. Registry validity establishes that the research inventory is internally traceable. It does not establish that each question is resolved or each cited result has been independently verified.

Interpretation of status

The audit policy defines record verification and profile gates. covered identifies a topic with the required evidence and decision links. experiment-required identifies an unresolved empirical question. ongoing identifies work requiring continued source review or measurement. The experiment registry records the continuation class, prerequisites and next action for open work.

A count of research records is an inventory measure. It is not a measure of literature-search recall, evidence quality or beta readiness. The methods section describes how local tests, independent oracles and external evaluation differ.

Implemented evidence and remaining limits

The source workflow is specified in the beta contract. Accessibility and behavior boundaries are listed in the adapter status document. Package, font, capture and reconstruction methods and source records are consolidated in the resource chapter. The implementation roadmap identifies the corresponding commands and artifact contracts.

Continuation policy

New source evidence can supersede a decision without promoting its dependent experiments. Passing one profile cannot close a broader question unless that profile meets the question’s stated acceptance conditions. A failed experiment retains its counterexample and provenance. Scope changes require an updated contract and explicit exclusions rather than retroactive reinterpretation of a passing result.

Canonical module.

Cross-industry patterns: evidence, adoption and rejection

This document synthesizes 52 research records added on 2026-08-29 from visual-effects interchange, game-engine asset systems, programming-language research on bidirectional transformation and layout verification, distributed-systems testing, and 2D-rendering conformance practice. Each pattern below is classified as borrowed (adopted as is), adapted (adopted with a stated change) or rejected (ruled out with the reason). Record identifiers (nuif:research:*) carry the locators; this document does not repeat them. The decisions classify design patterns. They do not establish implementation completion; current executable evidence is listed in research coverage.

Method

The 52 records synthesized here were reviewed from primary sources (specifications, source code at a named commit and papers with DOI). reviewed does not mean every material claim has completed locator-level verification; research/AUDIT.md defines the stricter verified state. Claims resting on one record inherit its evidence status as well as its confidence. Source conflicts remain explicit until an RFC or experiment resolves them.

Patterns

Document model and composition

Identity and ordering

Unknown data preservation and schema evolution

Operations, undo and merge

Canonical encoding

Layout

Rendering determinism

Text

Testing methodology

Editor automation

Ruled out

The following alternatives were excluded under the cited evidence. New source or implementation results can justify reassessment.

• A self-describing binary struct layout (Blender DNA): solves version drift for one implementation but does not preserve data it cannot re-save and does not compose with a schema-versioned interchange model.
• Memento undo as the canonical history: does not commute, cannot be merged, and bloats logs; inverse operations are required by spec/06.
• Integer child indices in Move and Insert as the only order representation: non-commutative under concurrency; a list identifier is required for the collaboration profile and harmless for the canonical form.
• GPU rendering as a normative oracle: implementation-defined by the WebGPU and WGSL specifications.
• Perceptual UI screenshot tests as the primary editor test: platform-dependent text rendering makes them a shell-wiring check only.
• Heuristic tree matching for NUIF-native merges: unnecessary with stable identity and a source of spurious moves.
• Whole-project regeneration as the synchronization model: contradicted by the lens and delta-lens laws that NUIF’s patch model must satisfy (lenses-foster-boomerang, bidirectional-evaluation-direct-manipulation).

Consequences for the specification

The records imply the following changes. Items 1–3 were decided by follow-up research on 2026-08-29 and are recorded as accepted RFCs; items 4–6 remain proposals.

1. Sibling order is a canonical array without keys; Insert and Move use anchors (Start, After(id)); the collaboration profile maps anchors onto a Fugue-family list CRDT (RFC 0006).
2. nuif-cbor-0 follows deterministic preferred serialization while preserving integer/real data-model identity; real zero is positive floating zero, text and CBOR key orders are distinct, strict decoders reject non-canonical input, text hashes through CBOR and strings remain verbatim (RFCs 0005 and 0008).
3. Entities of unknown kind load as Unknown with typed core fields and an opaque payload; ignorant implementations preserve bytes, knowing ones may re-encode; validation severities follow the glTF pattern (RFC 0007).
4. Every serialized record kind carries a schema version; migrations are registered pure functions; newer-than-known versions load as Unknown for entities (RFC 0007) and are diagnosed for other records.
5. Validation, import and export reports follow one schema with stable codes, severities and pointers.
6. Layout conformance declares tolerance tiers per case and classifies every divergence from a browser reference as schema loss, evaluator defect or implementation-defined behavior.

Open questions raised by this synthesis

Recorded in research/questions.yaml: cbor-float-zero (decided, RFC 0005), sibling-order-identifier (decided, RFC 0006), unknown-kind-preservation (decided, RFC 0007), editor-toolchain-msrv (ADR 0006), layout-assertion-vocabulary, render-tolerance-tiers.

Canonical module.

Resources, capture and model-neutral reconstruction

NUIF separates portable resource bytes, source-backed observations and inferred reconstruction. Images and fonts retain resource identities; browser capture records observations under an evaluation context; screenshot reconstruction proposes editable hypotheses. Their mutation boundary is the shared operation, validation and fidelity contract. This chapter distinguishes the implemented profiles from the proposed evaluation and training work.

Decision summary

The recommended direction is:

1. define stable assets separately from immutable byte resources;
2. make .nuif a deterministic portable package after cross-writer proof;
3. grow the executable narrow PNG and static TrueType resource baselines only through named profiles and measured hostile-input budgets;
4. add a pinned browser-capture adapter separate from static source sync;
5. build screenshot reconstruction as a replaceable observation/proposal loop;
6. freeze a structural and visual evaluation suite before training;
7. consider adaptation or distillation only after the untuned loop exposes a repeatable learnable error distribution.

RFC 0010, RFC 0011 and the variable-TrueType proposal in RFC 0013 remain proposed contracts. Their bounded package, narrow PNG/font and capture/reconstruction experiments are implementation evidence only for the named subsets; RFC 0013 now has capability-gated direct package-to-raster evidence and local binding/process snapshot agreement through Rust, CLI, Node/browser WASM, MCP and C. Broader hosted reproduction and external implementation remain required before profile acceptance.

Source-backed and screenshot-only inputs

Source-backed lane
HTML/CSS + browser execution + resource responses
      -> retained source + resolved observations
      -> deterministic adapter/lowering + explicit inference where needed
                              |
                              v
                    typed NUIF operations
                              |
                              v
                     core validation/apply
                              |
                              v
                      resource-aware NUIF
                              ^
                              |
Screenshot-only lane          |
pixels + context              |
      -> OCR/CV/grounding observations
      -> hierarchy/layout/resource hypotheses
      -> typed operations -> render/diff/correct

Both input classes use the same typed operations and retain their distinct evidence classes. Neither provider can write core structs directly. A browser observation may support an exact resolved value under one pinned context, but it does not automatically reveal authored intent. A screenshot cannot establish source equivalence regardless of visual score.

Portable resource model

Four identities must remain distinct:

An asset points to a content descriptor containing media type, SHA-256 and byte length. A package path or external URL only locates candidate bytes; size and digest are checked before decoding. External resolution is opt-in. Opening a document never triggers network access.

Resource roles clarify hash and retention behavior:

• source: original encoded bytes;
• authoring: exact bytes needed to evaluate/edit semantics;
• derived: crop, trace, selected frame, conversion or generated result with input digests and transformation identity;
• cache: decoded pixels, GPU textures or acceleration state that can be deleted without changing the semantic document.

This avoids a common failure: storing only a decoded bitmap and calling the source image preserved, or hashing an editable asset by its current bytes and therefore breaking every reference after replacement.

Candidate .nuif package

The proposed first package is a deliberately small deterministic ZIP profile:

mimetype
manifest.cbor
document.cbor
blobs/sha256/<digest>

mimetype is first and stored. Manifest and document are deterministic CBOR. Every embedded blob is addressed by exact SHA-256 bytes. The first profile uses stored members only, fixed metadata and sorted ASCII paths to make independent writer byte equality attainable and to avoid compression-version variability.

There are three hashes:

• semantic document hash over canonical document.cbor;
• resource digest over each exact blob;
• package hash over the exact ZIP artifact.

Cache or report changes may change the package hash while leaving the semantic document hash untouched. Bare canonical forms remain .nuif.json and .nuif.cbor. Historical alpha .nuif raw files need read-only detection during migration; new .nuif output becomes the package only after RFC acceptance.

The package reader rejects duplicate or unsafe paths, symlinks, directories, encryption, split archives, unsupported compression, inconsistent headers, undeclared/missing blobs and digest mismatch. It does not extract to a filesystem. Exact byte fixtures, member/resource limits and two independent writers are acceptance gates.

Package-to-session handoff uses shared immutable buffers. The release gate passes an 8 MiB resource through package, handle map and session with the same allocation pointer while keeping handoff allocator traffic and retained bookkeeping below 1 MiB. This prevents a host from paying one full resource copy merely to enter the core.

Images

The original encoded image is authoritative. A semantic image asset records its resource digest and intrinsic interpretation; each image paint records fit, crop, transform, sampling, opacity and color conversion. Derived decoded pixels and GPU textures are caches.

PNG is the selected initial format. Its W3C specification covers lossless encoded pixels, alpha and explicit colour metadata. The executable nuif-png-rgba8-0 baseline chooses an intentionally smaller contract: non-interlaced RGBA8, no ancillary metadata or one valid sRGB chunk, encoded samples interpreted as sRGB, straight decoded alpha, identity encoded orientation, declared fit/crop/sampling/opacity and bounded integer CPU composition. png 0.18.1 and zune-png 0.5.2 must emit identical RGBA bytes for the accepted fixtures; encoded resources remain digest-identical through package edits.

Agreement on this subset does not establish conformance to all of PNG Third Edition. The compatible nuif-png-basic-rgba8-1 profile now admits the lossless-to-RGBA8 subset: 1/2/4/8-bit greyscale and indexed images, RGB8, greyscale-alpha8, RGBA8 and valid palette/colour-key transparency. It preserves encoded bytes and requires exact normalized RGBA agreement between both decoders. It is separately named so profile zero never changes meaning.

Image-paint affine semantics are orthogonal to decoder choice. The executable matrix [a c tx; b d ty; 0 0 1] maps crop-local source coordinates forward into the fitted rectangle. The CPU reference inverse-maps destination pixel centers, clips to the entity, and rejects singular or numerically unbounded matrices. Flip, rotation and translation fixtures make composition order observable; live host trials are still required for vendor interoperability.

Decoded pixels are interned once per digest/profile in the renderer-independent scene and commands carry compact deterministic handles. A 64 MiB total is preflighted before each new inflation. The release gate retains one 1 MiB surface for 1,024 image instances under 8 MiB allocated and 4 MiB retained, and rejects a 64 MiB plus 16 byte declared total before the second decode.

Any broader profile still has to pin:

• accepted chunks and metadata conflicts;
• color-space precedence and output space;
• Exif orientation;
• conversion and premultiplication points for every accepted colour signal;
• sampling and compositing;
• encoded, pixel, decoded-byte, chunk and metadata limits;
• independent decoder and malformed-input fixtures.

16-bit/interlaced PNG, CICP/ICC/gamma/chromaticity, Exif, animation, perspective/tiling and host-specific affine equivalence are not claimed. A Linux/Windows/macOS CI matrix runs the profile, but the cross-platform claim remains withheld until its hosted artifacts pass. JPEG, WebP, AVIF, video and SVG follow as separate profiles. Freezing a frame or tracing a screenshot crop is a derived approximation, not recovery of the original asset. Generative upscaling/inpainting requires an explicit user policy and cannot silently become canonical source evidence.

Fonts

Exact typography depends on exact font bytes, face/collection index, variation axes, features, coverage, shaping inputs and renderer parameters. Packaging also depends on redistribution policy.

The proposed policy states are portable, private_authoring, linked, substituted and unavailable. OpenType fsType is preserved as machine- readable evidence, including restricted/preview/editable/no-subsetting/bitmap flags, but is not treated as a complete legal license decision.

The executable nuif-opentype-static-single-0 baseline accepts only one canonically packed, checksummed TrueType-outline sfnt face at index zero. Skrifa 0.46.2 supplies package-facing metadata after NUIF validates the directory, ranges, packing and checksums and directly checks required sfnt and OS/2 fields. A committed hb-info 14.4.0 capture independently checks Ahem metrics, family, tables and Unicode coverage. Exact bytes, family names, coverage, fsType, license expression and explicit embedding review must agree. Package encode/decode and caller-resolved linked bytes run the same validation. Four static TrueType fixtures are accepted, while six package trials distinguish portable, private-authoring, linked, substituted and unavailable outcomes. Each accepted inspection and packaged-font validation is also measured after warmup against a 4 MiB allocator-traffic and 2 MiB retained reference ceiling; these are implementation regressions, not format semantics. Six additional trials retain requested identity separately from a stable font asset, render with an available declared replacement as approximated, and emit no text command with item-level unsupported fidelity when replacement bytes or the font are unavailable.

The shared session now also evaluates one non-Ahem exact package end to end. It registers the embedded digest without host setup, applies declared global features in HarfRust, uses the font’s shaped advances and ascent for layout, extracts unhinted Skrifa outlines and produces deterministic CPU pixels. Six blocking runtime trials cover those facts plus lossless fidelity and repeated snapshot equality.

The first RFC 0013 milestone separately checks variable TrueType fvar/avar metadata and five final coordinate vectors against a pinned HarfBuzz capture. Its follow-on matches seven HarfBuzz shapes, including FeatureVariations, while reusing that vector for Skrifa metrics/outlines that exactly match HarfBuzz advance and draw captures. The later direct runtime gate requires explicit package capability authorization, records the same normalized vector in shaped runs, and matches default/interior HarfBuzz advances and paths through layout and CPU rasterization. The cross-surface follow-on requires the interior package’s complete snapshot report to agree through direct Rust, CLI, generated Node/browser WASM, live stdio MCP and a linked C release-library consumer on POSIX.

The implemented profiles cover selected OpenType features. TTC, CFF/CFF2, color, bitmap, SVG and WOFF/WOFF2 sources, historic ambiguous permission combinations, subsetting, cluster-level fallback and cross-platform raster behavior remain separate fixtures and profiles. Collections, variable-TrueType, CFF/CFF2, COLR-vector, bitmap/SVG-glyph and WOFF2 transport are deliberately separate capability tracks. The configured three-OS parser/package matrix does not establish cross-platform raster behavior. Parser acceptance and fsType do not grant redistribution rights.

Browser capture can identify platform fonts used for a node and capture downloaded web-font response bodies. It generally cannot retrieve arbitrary local font bytes. A family/PostScript name is therefore never exact resource identity. Missing bytes produce a link, substitution or unavailable fidelity record instead of a false portable-font claim.

Source-backed browser capture

Static Tree-sitter source synchronization and live browser capture solve different problems. The former preserves source spans for a bounded authored subset. The latter runs a pinned browser to observe actual cascade, layout, fonts, resource responses, accessibility and pixels. They should correlate through provenance, not become one oversized adapter.

The first browser-capture profile records browser/protocol build, OS, viewport, DPR, page scale, locale, timezone, color/reduced-motion preferences, font environment, scroll/pseudo state, navigation identity, readiness/network policy and animation freeze. It collects:

• original HTML/CSS and stylesheet text where accessible;
• DOM snapshots including available frame/template/shadow content;
• boxes, inline text boxes, paint order and a bounded style set;
• downloaded image/font/style response bodies and hashes;
• platform-font usage and font readiness;
• accessibility tree;
• reference screenshots with exact parameters.

Canvas, WebGL, video and worklets are bounded observation surfaces; a frame can be preserved without pretending its generating program was reconstructed. Cookies, authorization headers, credentials, storage and secret form values are not exported. Scripts remain inert.

Multiple viewports and states are more valuable than one oversized capture: they constrain layout hypotheses and permit held-out responsive evaluation.

The automated nuif-cdp-live-0 segment now implements that boundary for one loopback fixture and exact Chrome for Testing 152.0.7977.82. It starts fresh temporary profiles, retains a structured runtime context, waits for the exact navigation loader and declared freeze/readiness point, captures bounded DOM/layout/background/font/accessibility/resource/PNG evidence, and replaces opaque browser node IDs with deterministic preorder identities. Four runs at 360, 768, held-out 900 and repeated 360 px retain exactly the expected five response bodies and repeat the narrow capture bytes. Five exercised query/cookie/storage/authorization/header canaries are absent from serializable capture, observations, proposals and package bytes. The two fitted viewports beat the one-viewport freeform baseline on the held-out fixture.

The separate bounded nuif-layout-inference-0 artifact ranks row stack, column stack, Grid, linear constraint and fixed freeform candidates using only the 360/768 px observations. It then evaluates the untouched 900 px holdout, where the selected constraint records 0.0626 normalized error versus 0.2918 for freeform. All alternatives and exact geometry observation identities are retained; confidence is raw and uncalibrated, and the result remains inferred. The trial tests a mechanism on one fixture, not general accuracy or recovery of original authored intent.

This local baseline covers one fixture within the proposed capture profile. Cross-browser/OS reproduction, opaque frames and response bodies, full matched-style/source correlation, canvas/video frame capture, authenticated sites and licensed real-page evaluation remain open. WebDriver BiDi is the standards-track transport to revisit as its implemented evidence surface grows; Playwright is the higher-level candidate when NUIF owns a real multi-engine matrix and can make one tool the browser-version authority.

Screenshot reconstruction

A screenshot supplies visible samples but not the unique scene graph or layout program. The recommended pipeline is:

screenshots + contexts
  -> OCR and baselines
  -> deterministic regions, colors, edges, repetitions and asset candidates
  -> optional replaceable UI grounding
  -> typed observation graph with confidence and evidence regions
  -> replaceable reasoner proposes hierarchy/layout and NUIF operations
  -> core validates and applies atomically
  -> deterministic layout/render
  -> text/structure/geometry/resource/visual differences
  -> bounded corrective operations

The model emits typed operations, not an unconstrained full document or code to execute. Invalid and stale transactions fail without partial state. High- resolution full views, overlapping tiles and semantic crops share explicit coordinate transforms; duplicate or conflicting observations remain visible.

The result contains a valid document or no-result, accepted operation log, observations, derived resources, item fidelity, alternatives/abstentions, evaluation report and exact pipeline artifact identities.

Evaluation before training

The proposed benchmark separates synthetic-exact, licensed real screenshot and source-backed suites. Current evaluation fixtures are synthetic; licensed real-image accuracy remains unmeasured. Synthetic NUIF rendering provides exact entities, properties, operations and resources. Real images need human-reviewed visible targets and must preserve ambiguity. Source-backed cases evaluate retained bytes and observations unavailable to the screenshot-only route.

Required metrics include:

• valid operation/document rate;
• OCR region recall, character/word error and baselines;
• element precision/recall and hierarchy error;
• property/geometry accuracy;
• held-out viewport behavior;
• exact resource digest only when bytes exist;
• provenance/fidelity honesty;
• accessibility evidence where justified;
• raw pixels, FLIP, SSIM and pinned LPIPS diagnostics;
• calibrated confidence, abstention and risk/coverage;
• latency, peak RAM/VRAM, iterations and cost.

A page-sized image can match reference pixels while discarding text, structure and editability. Separate structural, text, resource and edit-task metrics are required to detect that failure. Dataset splits group by origin, template, component, font, resource and generator; near duplicates cannot cross splits.

The executable nuif-reconstruction-corpus-manifest-0 turns that rule into a bounded audit. It pins the data snapshot, dataset card, evaluator and every input/target by digest; records public/restricted/withheld disclosure and evaluation/calibration/adaptation/redistribution permission independently; and rejects exact artifact or declared family reuse across adaptation, calibration, validation and test. Private/authenticated records also require explicit authorization and a withdrawal-policy artifact. This is declaration integrity, not automated legal review or duplicate discovery; real records and their group assignments still need independent human/tool review.

The ablation ladder is deterministic OCR/CV, one-shot reasoner, observation- assisted reasoner, hierarchical crops, multi-viewport ranking, correction loop, then any tuned or distilled student. Every addition uses the same frozen holdout and budget.

Adaptation and distillation

Training is justified only after the untuned loop and error taxonomy are reproducible, rights-cleared traces exist, and the remaining errors appear learnable. Training examples contain input hashes, observation versions, proposals, diagnostics, accepted operations, intermediate renders/differences and final package/fidelity reports. Positive sequence targets are validated accepted transitions, not raw model transcripts.

Compare prompt/schema/tool improvements and retrieval before fine-tuning. If adaptation remains justified, compare ordinary supervised tuning, low-rank adaptation and quantized low-rank adaptation under equal data and evaluation. Quantized adaptation is a memory technique, not an accuracy claim.

Sequence-level distillation may train a smaller student from a teacher selected under the declared evaluation metrics and budget. The teacher is a measured system of tools plus a model, not a provider name. Distillation transfers errors too, so render validation and held-out evaluation remain mandatory.

Models, processors, adapters and datasets are separately versioned optional artifacts with digests, model cards, dataset datasheets, license lineage and training manifests. They never redefine nuif-core or travel as ordinary document resources.

The executable provider boundary uses a canonical wrapper with artifact-inventory references. It binds NUIF capabilities, execution modes and observation/proposal profiles to exact implementation, model, processor, adapter, quantization, prompt and tool artifacts. Observation bundles carry the complete manifest registry, so a digest cannot dangle and a proposal cannot substitute an unpublished provider before mutation. Released or learned providers point to content-addressed SPDX 3.0.1 or CycloneDX 1.7 inventory; learned providers also point to a model card. This complements runtime packaging such as MLflow or ONNX external data without making either a required NUIF dependency.

Private/authenticated captures default to local processing, no retention and no training. Remote transfer, telemetry, retention and training are independent consent/policy decisions.

Maturity boundary

The current 0.1.0-alpha.3 label belongs to the developer editor application. It provides no evidence that the broad image/font resource, portable browser capture or screenshot reconstruction accuracy profiles are complete. A deterministic package, narrow PNG/static-font segments, fixed provider-input contracts and one pinned local live-browser segment are implemented, but their deliberately narrow evidence does not promote the broader profiles.

Promotion requires the package/resource cross-writer fixtures, pinned capture reproduction, baseline/closed-loop/calibration harness, leak-resistant licensed evaluation data, independent result reproduction and at least one real edit workflow that benefits from the inferred semantics. Until then the work is research and proposed specification text, not a standard or production reconstruction promise.

Primary research records

• nuif:research:resource-packaging-and-source-capture-synthesis
• nuif:research:model-agnostic-screenshot-reconstruction-and-training
• nuif:research:provider-artifact-manifests-and-ai-boms
• nuif:research:epub-ocf-package-container
• nuif:research:oci-resource-descriptors
• nuif:research:opentype-font-embedding-and-portability
• nuif:research:ttf-parser
• nuif:research:fontations
• nuif:research:chromium-source-backed-ui-capture
• nuif:research:live-chromium-cdp-capture
• nuif:research:design2code-real-world-benchmark
• nuif:research:pix2struct-screenshot-parsing-pretraining
• nuif:research:screenai-ui-annotation
• nuif:research:confidence-calibration-and-selective-prediction
• nuif:research:lora-low-rank-adaptation
• nuif:research:qlora-quantized-adaptation
• nuif:research:sequence-level-knowledge-distillation

Canonical module.