operators, memfile undo and headless regression testing

Document status: reviewed. Canonical source.

Summary

A .blend file is a sequence of typed blocks, each headed by a BHead carrying a block code, byte length, the writer’s old memory address, an index into the writer’s struct table and an element count. The struct table (SDNA) is generated at build time by makesdna from the DNA headers and is embedded in every file in a DNA1 block, so a reader compares the file’s SDNA with its own and reconstructs each struct member-by-member by name, dropping members that no longer exist and zero-initialising new ones; explicit versioning_*.cc code then applies semantic migrations gated on file version and on DNA_struct_member_exists. RNA is a reflection layer generated by makesrna from explicit rna_*.cc definitions; it adds metadata, callbacks and update hooks and drives the UI, animation, library overrides and the bpy Python API. Every user action is an operator with typed properties, registered under a bl_idname and callable as bpy.ops.<id>(); undo steps are pushed by the operator system, and global undo (memfile) is a chunked in-memory .blend write whose chunks are compared with the previous step and shared when identical. blender --background --python script.py executes the same operator and data API without a window, and the regression suite renders scenes headlessly and compares them with reference images using oiiotool thresholds tuned per category and platform.

Evidence

DNA “describes the data structures stored in .blend files in a reproducible, forward- and backward-compatible manner”; makesdna parses headers under source/blender/makesdna/ into SDNA; “When saving a .blend file, Blender embeds this SDNA block alongside the actual binary data”; on load Blender “compares the file’s SDNA to the current binary’s SDNA and performs any necessary compatibility conversions”; DNA is sensitive to “structure padding, pointer size, and endianness”; structs prefixed with #\n# are excluded from SDNA. Blender developer docs, “DNA”, retrieved 2026-08-29.
Compatibility policy: backward compatibility expected for any previous version (conversion code may be removed two years after deprecation); forward compatibility “though with some loss of data”; when reading, “Unknown data is ignored. Missing data is initialized with default values.” and versioning code runs incrementally; unknown data “typically also cannot re-save”; partial re-save of unknown data exists only for ID properties and “should not be expected or relied on”; critical breakages only at major cycles, with the previous long-term support (LTS) release acting as converter. Blender developer docs, “Blend File Compatibility”, retrieved 2026-08-29.
enum eSDNA_StructCompare { SDNA_CMP_REMOVED = 0, SDNA_CMP_EQUAL = 1, SDNA_CMP_NOT_EQUAL = 2, SDNA_CMP_UNKNOWN = 3 } with comments: removed structs “will not be loaded by the current Blender”, equal structs load “with straight memory copy”, not-equal structs are “copied/converted field by field”; DNA_sdna_from_data decodes an SDNA block; DNA_struct_get_compareflags returns one flag per old struct. source/blender/makesdna/DNA_genfile.h, main, retrieved 2026-08-29.
SDNA layout: magic SDNA, then NAME (count, strings), TYPE (count, strings), TLEN (shorts), STRC (count, then per struct <typenr><nr_of_elems> followed by <typenr><namenr> pairs); “Remember to read/write integer and short aligned!”; DNA_struct_reconstruct matches members by elem_streq, which compares names “excluding any array-size suffix”; members absent in the new struct are dropped, new members get RECONSTRUCT_STEP_INIT_ZERO; cast_primitive_type converts among the SDNA primitive types; 32/64-bit pointer casts warn that “pointers may lose uniqueness on truncation”. source/blender/makesdna/intern/dna_genfile.cc, main, retrieved 2026-08-29.
Block header: struct BHead { int code; int SDNAnr; const void *old; int64_t len; int64_t nr; } with old documented as “the pointer that the memory had when it was written”, used “to remap memory blocks on load”; on-disk variants BHead4 (32-bit old), SmallBHead8 (64-bit old, 32-bit len/nr) and LargeBHead8 (64-bit len/nr); codes include BLO_CODE_DNA1 and BLO_CODE_ENDB. source/blender/blenloader_core/BLO_core_bhead.hh, main, retrieved 2026-08-29.
File header: “the first 12-17 bytes”, magic BLENDER, file format version “Currently always 0 or 1”, pointer size 4 or 8, endianness, and writer version; LargeBHead8 is the format-version-1 layout. scripts/modules/_blendfile_header.py, main, retrieved 2026-08-29.
Versioning code example: if (!MAIN_VERSION_FILE_ATLEAST(bmain, 400, 9)) { ... } and if (!DNA_struct_member_exists(fd->filesdna, "bPoseChannel", "BoneColor", "color")) { ... } inside blo_do_versions_400(FileData *fd, Library *, Main *bmain) and do_versions_after_linking_400. source/blender/blenloader/intern/versioning_400.cc, main, retrieved 2026-08-29.
RNA “provides a high-level description of Blender’s data structures and related functions, which is used to drive the UI, animation system, library overrides, and the Python API (bpy)”; definitions in rna_*.cc processed by makesrna; metadata includes ranges, enum items, get/set callbacks, “Update flags and functions to notify other systems”, override metadata (RNA_def_property_override_flag(prop, PROPOVERRIDE_IGNORE)); RNA “has since evolved into a more general-purpose runtime data definition system” with structures independent of DNA. Blender developer docs, “RNA”, retrieved 2026-08-29.
Operators: a class deriving from bpy.types.Operator with bl_idname = "object.simple_operator" and execute(self, context); registration via bpy.utils.register_class; invocation bpy.ops.object.simple_operator(); registering validates signatures (“expected Operator, SimpleOperator class ‘execute’ function to have 2 args”); add-on modules require register()/unregister(). Blender Python API, “API Overview” (api/current/info_overview.html), retrieved 2026-08-29.
Undo: “Undo is organized as an ‘undo stack’ storing a list of ‘undo steps’”; the stack is “fully relative”; steps are stateful or differential; “The main part of step creation (‘undo push’) is controlled by the operator management system”; layers ed_undo.cc, undo_system.cc, and per-type implementations such as memfile_undo.cc using BLO read/write code. Blender developer docs, “Undo System”, retrieved 2026-08-29.
Memfile: chunks are compared with the reference step (memcmp(compchunk->buf, buf, size) == 0 sets is_identical); is_identical_future handles the redo direction; BLO_memfile_write_init builds a mapping from ID session_uid to previous-step storage so reuse survives reordering of Main. source/blender/blenloader/intern/undofile.cc, main, retrieved 2026-08-29.
UndoType callbacks step_encode/step_decode (is_final flag), poll; memfile steps “must be read before loading other undo steps” when active. source/blender/blenkernel/BKE_undo_system.hh, main, retrieved 2026-08-29.
Command line: -b, --background “Run in background (often used for UI-less rendering)”; -P, --python <filepath> “Run the given Python script file”; --python-expr; --python-exit-code <code> sets the exit code on uncaught Python exception “(only for scripts executed from the command line)”; --factory-startup skips the user startup.blend; -f, -a, -o, -F, -E; “Arguments are executed in the order they are given” with worked examples of misordering. Blender Manual (latest), “Command Line Arguments”, retrieved 2026-08-29.
Render tests: “Each blend file generates a test in its category. It always renders frame number 1 to a PNG image”; results in build/tests/report.html; references updated with BLENDER_TEST_UPDATE=1 ctest -R <name>; EEVEE reference is “the Nvidia OpenGL result”; threshold policy: strict on the reference platform, bump fail_percent first, then fail_threshold when > 0.5% pixels fail, per platform if large; non-deterministic tests are blocklisted; tests are batched per Blender launch (WITH_TESTS_BATCHED). Blender developer docs, “Render Tests”, retrieved 2026-08-29.
Comparison implementation: oiiotool ref out --fail <fail_threshold> --failpercent <fail_percent> --diff; defaults fail_threshold = 0.016, fail_percent = 1; set_fail_threshold/set_fail_percent; on BLENDER_TEST_UPDATE the new image overwrites the reference; HTML report with new/reference/diff images. tests/python/modules/render_report.py, main, retrieved 2026-08-29.
EEVEE tuning: report.set_fail_percent(0.08), report.set_fail_threshold(4.0 / 255.0), tightened to 0.049 and 2.0/255 on NVIDIA OpenGL, loosened per category (e.g. transparency 0.22 and 10.0/255); Blender is invoked with --background --factory-startup --enable-autoexec --debug-memory --console-crash-handler --debug-exit-on-error, optional --gpu-backend <backend> --debug-gpu-backend-no-fallback, then <file> -E BLENDER_EEVEE -P <script> -o <out> -F PNG -f 1; AMD Vulkan blocklist is ".*". tests/python/eevee_render_tests.py, main, retrieved 2026-08-29.

Mechanism

File model. A .blend is header || BHead+data ... || DNA1 || ENDB. Each BHead says which struct (index into the writer’s SDNA), how many, how many bytes, and the writer’s address of the block. Pointers inside data are the writer’s addresses; the reader builds an address map from old to the newly allocated block and patches pointers, which is why identity in a .blend is address-based rather than semantic. The SDNA is a schema: names (with array suffixes and pointer prefixes), type names and sizes, and struct member lists. The reader decodes the file SDNA, computes per-struct compare flags against its own SDNA, and for NOT_EQUAL structs builds reconstruction steps: per new member, find a same-named member of the old struct ignoring array size, and either memcpy, cast primitive, or zero-init; old members with no counterpart are dropped. Removed structs are skipped. This yields structural forward compatibility (an older reader ignores a new field) and backward compatibility (old files gain zeroed fields). Semantic migrations are then applied by version-gated code; the use of DNA_struct_member_exists on the file SDNA lets a migration run exactly when a file predates a field, independent of the version number bump discipline.

Preservation limits. Unknown structs and unknown members are discarded on load and therefore absent on re-save; the compatibility document states this and limits round-trip preservation to ID properties. Forward compatibility is defined as “open with some loss”, and critical breakages are scheduled at major version boundaries with the previous LTS as a converter.

Reflection and operations. RNA definitions are the single description used by the UI layout code, the animation system (which animates RNA paths), library overrides (per-property override flags) and Python. Operators are the command pattern: a registered type with an ID, typed properties, poll, invoke, execute, and flags such as REGISTER/UNDO; the window manager records executed operators with their property values (the redo panel re-executes them) and pushes an undo step after each undoable operator. Undo is a stack of relative steps; the global step type serialises Main into memory as a .blend-format MemFile of chunks and marks chunks identical to the previous step so that consecutive steps share memory; ID-level session_uid mapping keeps the chunk correspondence stable under reordering. Mode-specific step types (edit-mesh, sculpt, text) store their own state. Undo is therefore state-based (memento) at the file level, while the operator log provides re-executable commands.

Headless surface. The same binary runs with --background; arguments are executed in order, so --python after a file argument sees the loaded file. Scripts use bpy.ops (operators) and bpy.data (RNA data access) identically to the GUI; --python-exit-code maps uncaught exceptions to a process exit code for continuous integration (CI); --factory-startup removes user-state dependence. Regression tests are Python scripts run through this surface, batched, producing PNGs compared by oiiotool with an absolute per-pixel threshold and a maximum failing-pixel percentage, both tunable per test category and per GPU backend, with a reference-update mode and blocklists for non-deterministic cases.

NUIF relevance

Borrow

Embedding the writer’s schema with the document so readers can reconstruct records by member name; NUIF’s CBOR/text profiles should carry a schema identifier and member names (or a schema registry hash) rather than positional layouts.
Migration gated on “does the file’s schema contain member X” as well as on version numbers; NUIF migrate should be able to inspect declared schema features, not only a version integer.
Operator-as-command with typed properties, poll preconditions, registration-time validation and a re-executable log; this is the same shape as NUIF semantic operations and the QA replay requirement.
The headless invocation contract (--background --factory-startup --python-exit-code) and threshold policy (absolute per-pixel error plus failing-pixel percentage, tuned on a reference platform, with blocklists and a reference-update mode) for NUIF render conformance.

Adapt

Memfile undo shares identical chunks between steps; NUIF can use content-addressed canonical snapshots for checkpointing while keeping inverse operations as the primary undo representation (spec/06).
Address-based block identity must become stable semantic IDs in NUIF; the old-pointer remap is a useful reminder that identity derived from memory or byte offset is not portable (spec/02).
RNA’s override flags per property suggest that NUIF property definitions should declare override and merge policy in the schema.

Reject

Dropping unknown structs and members on load; NUIF requires opaque preservation of unknown data and a fidelity report (RFC 0002, spec/07), which Blender explicitly does not promise.
Coupling the on-disk layout to compiler padding, pointer size and endianness; NUIF encodings are logical, deterministic and platform-independent (spec/08).
Undo semantics that vary by mode (global versus edit-mode step types); NUIF has one operation model for all editing contexts.

Open questions

Whether recent work on ID-property preservation (the documented partial re-save of unknown data) has a stable specification; the compatibility document says not to rely on it.
Whether the session_uid mapping in memfile undo can be reused as a semantic identity for diffing two .blend states, or is strictly session-local.
How --python-exit-code interacts with batched test execution when one test in a batch fails.