zagg store specification (1.0)

This page is the normative record of zagg's byte-level store conventions: the ragged vlen-bytes layout, the t-digest payload bytes, the packed composition word, the pyramid/overview declarations, the O11 content-hash recipe, the temporal and located declarations on the word-typed coordinate and companion arrays, and the temporal section of the store-root coverage sidecar. It exists so an independent reader (moczarr) can decode a zagg store from this page and the committed conformance fixtures alone — no zagg import, no reverse-engineering of grids/base.py (issue #340, the reader-migration gate).

The precedent is mortie's docs/specification.md, which governs everything below this page: the packed morton word, the decimal path grammar, the morton-hive tree layout and leaf naming, the coverage-MOC serializations, and the rank-space deinterleave. This page owns the array-level contracts inside a leaf; it cites mortie's page for path and word semantics and never restates them.

Latitude convention of the words (informative — see Normative language below; issue #549): the packed words are produced by mortie's geodetic ingress (geo2mort) under mortie's default latitude convention — zagg passes no latitude= override anywhere (zagg.grids.healpix.HealpixGrid.assign). That default is authalic-wgs84 (geodetic latitude converted to authalic before the spherical HEALPix mapping, mortie spec §9) from mortie 0.9.8 on (espg/mortie#186, on PyPI 2026-08-16); earlier mortie encoded geodetic-spherical. The convention a store carries is therefore mortie's default at write time, not a property zagg declares: zagg ≥ 0.45.0 floors mortie at ≥ 0.9.8 (issue #438), so any environment that resolved those declared dependencies encodes authalic — but on the fleet mortie comes from the separately built Lambda layer, not from the function zip, so a worker whose layer predates the bump would write geodetic-spherical, and nothing in the store records which. The published o9 stores are authalic-wgs84 on stronger, artifact-level evidence: they were written by zagg 0.52.0, whose temporal encode path runs from mortie import TOC_MAX_NS (a name mortie exports only from 0.9.10 on), so a worker on a stale layer raises there instead of writing. The two conventions are non-corresponding partitions of the sphere (mortie spec §9): the offset peaks near 45° latitude at 0.12830° (~14.26 km), so a cover or store built under one convention does not describe cells under the other, and composing them is meaningless (mortie spec §9 is where the prohibition binds). The dggs attrs block (zagg.grids.healpix.HealpixGrid._dggs_attrs) stamps mortie's latitude token — "latitude": "authalic-wgs84", mandatory for a writer at the current mortie spec version, since #580 (folding #549's resolution; zagg.grids.morton.LATITUDE_CONVENTION) — beside ellipsoid: {name: WGS84, semi_major_axis: 6378137.0, inverse_flattening: 298.257223563} and no sphere radius: that entry is the ingress datum (the geodetic coordinates fed to geo2mort, equal-area on that ellipsoid by construction), not an instruction to compute cell geometry on the ellipsoid — the words themselves live on the R = 6371.0088 km authalic sphere the conversion maps onto. A store whose dggs block carries no latitude key predates the token; this paragraph is its record, and it is authalic by the version evidence above. The absence rule is per artifact, not per store: a partially rewritten store legitimately carries both vintages — a sweep, stage or column backfill re-templates the overviews and columns it writes, so those pick up the token while the leaves beside them, never rewritten, do not — and each artifact's own dggs block is authoritative for that artifact. A reader reproducing cell geometry (e.g. a viewer's boundary golden test) needs the geodetic ↔ authalic conversion at every geodetic seam, exactly as mortie spec §9 prescribes.

The ellipsoid key spelling follows the same per-artifact vintage rule (issue #616): artifacts written by zagg through 0.58.0 spell it semimajor_axis, which the dggs v1 schema rejects (its ellipsoidObject is additionalProperties: false and spells semi_major_axis); artifacts written or re-templated by zagg after 0.58.0 carry semi_major_axis. No spec marker records the rename — the key belongs to the convention block the schema governs, and zagg readers never consult the ellipsoid.

Design rationale — why each decision was made, with trade studies and ratification records — lives in design/sparse_coverage.md (the D/O-numbered decisions registry) and in the narrative companions ragged_layout.md and signal_strata.md. Those documents cite this page; this page is the spec. Byte layouts, attrs grammars, and constants are normative here only — duplicated normative text drifts.

Normative language

The key words MUST, MUST NOT, SHOULD, and MAY are used as in RFC 2119. Text marked Contract is frozen for the revision that carries it; text marked informative explains or motivates and binds nothing.

Conformance

• Every versioned convention on this page is signaled in store metadata — by a spec marker ("zagg-ragged/1", "zagg-composition/1", …), the coverage-envelope discipline, or, where a section names it, by the array's registered zarr data type. A conforming reader MUST strict-check whichever signal the owning section names and fail loudly on an unknown or future revision, never half-parse under a guessed layout.
• Marker-absence is legal only where a section names the data type that replaces the marker. Today there is exactly one such carve-out: the typed vlen-ndarray dtype is the zagg-ragged/2 signal, and the ragged attrs marker is deliberately retired (not written) on those arrays (§1.6/§6.1) — so a reader MUST NOT treat a missing marker there as an unknown revision. Everywhere else absence is a hard failure: a variable_length_bytes array with no ragged block is not signaled at all and MUST be refused (§1.2).
• A revision, once published here, is frozen: its text never changes semantics, and stores written under it remain valid indefinitely. New behavior is a new revision (/2, /3, …) with its own section and an explicit succession clause; readers add revisions, they never drop them.
• The committed conformance fixtures (§7) are part of the contract: a reader implementation that reproduces the fixtures' expected decoded values and content hashes conforms to §1–§3, §5, §8, §9 and §10. zagg's own test suite asserts the same expectations (tests/test_spec_conformance.py), so the spec, the fixtures, and the shipping reader cannot drift apart silently.

Contents:

1. zagg-ragged/1 — the vlen-bytes ragged layout
2. Digest payload semantics
3. zagg-composition/1 — the packed composition word
4. Pyramid / overview declarations
5. O11 content hashes
6. zagg-ragged/2 — the typed vlen-ndarray revision
7. Conformance fixtures
8. zagg-toc/1 — the temporal declaration
9. zagg-located/1 — the located declaration
10. zagg-coverage-toc/1 — the root coverage temporal section
11. Icechunk companion repo — the virtual-ref index

---

1. zagg-ragged/1

Status: contract — pinned as the 1.0 wire contract (#340 amendment). This is the shipping format, not a placeholder: /1 stores remain valid indefinitely, existing stores never require rewriting, and every conforming reader MUST support /1 unconditionally. Rationale and history: ragged_layout.md, issue #209.

A kind: ragged field (per-cell variable-length data — e.g. a t-digest) is stored as one variable_length_bytes zarr v3 array per field on the cells axis. Source of truth in code: zagg.grids.base.ragged_array_spec / RAGGED_ELEMENT_ATTR / RAGGED_SPEC.

1.1 Layout

A ragged field {field} under a product group is up to four sibling arrays:

{group}/{field}             <- vlen payload array; populated cell i holds the raw
                               little-endian bytes of its (n, *inner_shape) payload
{group}/{field}_locations   <- LOCATED fields only (issue #87): per-row uint64
                               location words, row-aligned with {field}
{group}/{field}_times       <- TEMPORAL fields only (§8.3, issue #410): per-row
                               uint64 toc words, row-aligned with {field}
{group}/morton              <- per-cell uint64 morton coordinate (zagg's standard
                               HEALPix coordinate array; the chunk-identity source).
                               STORED on an unwindowed leaf, DERIVED on a windowed
                               one, which stores no such array (§1.5 "The cell
                               coordinate")

• Each populated cell's value MUST be the raw little-endian bytes of an (n, *inner_shape) array (n varies per cell — e.g. (k_centroids, 2) for a t-digest whose inner_shape is (2,)), C-order, in the declared element dtype, independent of the producing machine.
• Empty cells keep the b"" fill (fill_value: ""); an inner chunk with no ragged data MUST be absent from the store — omitted as an object in the per-inner-chunk geometry, and marked absent with the §1.5 sentinel inside the shard object when sharded (the default). Either way object size scales with populated chunks only — the same sub-shard sparsity the dense arrays get.
• A located field's sibling {field}_locations array is itself a zagg-ragged/1 vlen array (element dtype uint64, empty inner_shape) with the same shape and chunk geometry as the payload array, and MUST be row-aligned: cell i of the sibling holds exactly one uint64 word per payload row of cell i. Readers MUST bind the sibling by the payload array's locations attrs declaration (§1.2), never by reconstructing the {field}_locations naming convention.
• A temporal field's sibling {field}_times array (§8.3) is the same shape of thing under the same row-alignment rule, bound by the payload array's times attrs key. A field may carry either sibling, both, or neither.

1.2 The ragged attrs block

The element interpretation is self-describing in the payload array's attrs under the ragged key:

"ragged": {
  "spec": "zagg-ragged/1",
  "element": {"dtype": "float32", "shape": [-1, 2]},
  "locations": "h_tdigest_locations"
}

• spec — the convention revision. Readers MUST strict-check it: an unknown/future spec raises, never half-parses.
• element — {"dtype": "<numpy dtype>", "shape": [-1, *inner_shape]}. The -1 marks the per-cell varying count; a reader reconstructs cell i as np.frombuffer(a[i], dtype).reshape(-1, *inner_shape), with the dtype read little-endian.
• locations — present only on a located field's payload array; its value is the name of the sibling uint64 array carrying the per-row location words. An unlocated field records nothing here.

A vlen array without a well-formed element declaration is not a zagg-ragged/1 array; a reader MUST refuse it with a pointed error rather than decode under a guessed layout (pre-issue-209 CSR stores are a hard break). The ragged attrs key is reserved: config-declared field attrs MUST NOT shadow it (enforced at config validation). The §2.0 weights key and §8.3's times binding are likewise spec-owned on a ragged payload array — writer-stamped from the field's declaration, never author-transcribed — as are the §8/§9 temporal and located declaration blocks on the companion arrays that carry them. A located field's provenance attrs (e.g. stratum, signal_threshold — §3.3) land on the payload array only; a sibling array carries no user attrs, only the spec-owned declaration its own section defines.

1.3 Codec chain

The per-chunk codec chain MUST be [vlen-bytes, zstd(level=3, checksum=false)]. The zstd deviates from the dense arrays' bytes-only/uncompressed policy deliberately: a vlen payload has no fixed-width raw layout to preserve, and the level is fixed so identical payloads produce identical objects across workers. (zarr-python names the dtype variable_length_bytes in array metadata while the v3 registry name is bytes — zarr-python#3517, accepted both ways on read; readers MUST accept the variable_length_bytes spelling.)

1.4 Wire framing

Contract (golden-pinned). Within one inner chunk the vlen-bytes codec frames the cells before compression as (little-endian throughout):

u32  cell_count
per cell:  u32 payload_length  ||  payload_bytes

i.e. numcodecs' VLenBytes/VLenArray framing — a u32 count of cells, then for each cell a u32 byte length followed by that many payload bytes (0 for an empty cell). The payload_bytes are np.ascontiguousarray(value).tobytes() of the cell's (n, *inner_shape) array in the declared dtype. This exact byte vector is frozen by a golden test (tests/test_processing.py::TestRaggedVlenLayout::test_golden_inner_chunk_framing) and exercised by the §7 fixtures; it is what makes the §6 typed-dtype revision a metadata-only migration.

1.5 Storage geometries

Both geometries hold the same logical data and are self-describing in the array's own zarr metadata (its chunk_grid, and whether a sharding_indexed codec wraps the chain), so a single reader code path MUST read either — deriving the stored span from the array's shard shape when sharded, else its chunk shape:

geometry	on disk	single-cell read
sharded (ShardingCodec; every hive leaf, the sharded flat path, and a leaf column group of 4^5 cells or more — §4.6)	ONE object per shard; the shard's K inner chunks live inside it with an internal index	2 ranged GETs (index suffix + one ranged inner chunk)
per-inner-chunk (regular array; the unsharded streaming path)	one object per inner chunk	1 GET (the object)

When sharded, the §1.3 chain rides INSIDE a sharding_indexed codec whose outer chunk spans the shard, with index codecs [bytes(endian=little), crc32c] and index_location: end: the shard object's suffix is K (offset, nbytes) u64 pairs plus a crc32c, and an inner chunk with no ragged data is marked absent with the 2^64 - 1 sentinel in both fields (zarr v3 sharding spec) — object size scales with populated chunks only. The 2-GET random-access recipe follows: fetch the 16*K + 4-byte index suffix, then the one ranged inner chunk holding the cell.

Subtree spans. The cells axis MUST be in canonical nested order — the per-cell morton coordinate (stored or derived, "The cell coordinate" below) ascending, every aligned power-of-four span sharing its ancestor cell (the ordering every §1 identity derivation and the rank-space deinterleave already presuppose; a zagg writer has never produced anything else, this sentence makes it citable). Consequently the order-k subtree below an ancestor at nested rank r on an order-c cells axis occupies exactly the contiguous index span [r·4^(c−k), (r+1)·4^(c−k)), and a reader MAY serve "everything below one morton node" as a contiguous-slice read: on the sharded geometry the index suffix plus only the covering inner chunks (the 2-GET recipe generalized to a span), on the per-inner-chunk geometry only the covering chunk objects — never a whole-array sweep. The span property is normative; a dedicated subtree reader is implementation (zagg: issue #351).

The cell coordinate. The per-cell morton coordinate is a pure function of the leaf and the rank, and this is its law (issue #586, espg ruling 2026-10-01): the cells axis of a leaf is the shard's children at the cell order, in canonical nested order, and cell j carries the packed word of the j-th child. In mortie's terms that is generate_morton_children(shard, cell_order)[j]: the shard's own base cell and order-p tuples, then the c − p tuples that spell j in base 4 (most significant first), under the suffix of order c (mortie spec §1). The shard is the leaf's id — the {id} of its name, {id}.zarr or {id}_{window}.zarr (§4.2), the D3 address every reader computes. For a cell order c ≤ 27, where mortie's suffix is the order itself and every tuple is two bits, the law reduces to one arithmetic progression across the whole shard, in both hemispheres:

word[j] = word[0] + j · 2^(60 − 2c)        word[0] = shard word + (c − p)

— 4,194,304 per cell at cell order 19, the instance the §7 windowed/ fixture pins on an order-9 shard in each hemisphere (a southern word sets bit 63; the progression is unsigned). At cell orders 28 and 29 mortie's suffix packs the last tuples (mortie §1), the stride is not uniform, and only the children law applies — so the law is mortie's children, and the stride is an instance of it, never the definition.

Whether the coordinate is stored depends on the leaf:

• An unwindowed leaf ({id}.zarr, a morton-hive/1 stamp) MUST store it as {cell_order}/morton, exactly as at every earlier revision. Its written inner chunks hold the derived words and its unwritten ones the 0 fill (§7), so the stored array doubles as a chunk-occupancy record.
• A windowed leaf ({id}_{window}.zarr, a morton-hive/2 stamp naming window) written from this revision on MUST NOT store it. N windows of one shard would each carry the same 8 bytes per cell — 8.4 MB per full order-9 shard at cell order 19, of an array that carries no information. This binds point and raster leaves alike (for a raster leaf the derived word simply is the coordinate of its (time, cells) arrays), and a versioned leaf's version subgroups with them. The group's dggs attrs still read "coordinate": "morton": that names the coordinate, it does not promise an array of that name.
• Windowed leaves written before this revision store the array, and remain valid as they stand — no leaf requires rewriting. So readers carry one rule: use {cell_order}/morton where the leaf has it, and derive it where a windowed leaf does not. A derived word is never 0: a reader of a derived coordinate MUST take occupancy from the payload arrays (or the stamp's coverage), never from the coordinate. Absence on an unwindowed leaf is corruption, not a licence to derive, and MUST be refused.
• The §4 artifacts — overview zarrs and leaf columns — are not leaves: each of their resolution groups stores its morton array at every revision, under a windowed store too (§4.4, §4.6).

(Informative.) A reader holding only a leaf-rooted store, not the leaf's path, can recover the id from the leaf's own stamp: every member of the tier-0 coverage box is the shard or a descendant of it, and the cells axis is exactly one shard subtree, so the ancestor of any box member at order cell_order − log4(n_cells) is the shard (zagg.hive.derived_leaf_words; zagg.grids.morton.cell_words is the derivation every zagg reader goes through).

The containment check. A derived coordinate has no stored twin to be compared with, so what verifies it is the data. For a located field (§2.2/§9), every location word stored in cell j MUST lie inside cell j: the word's ancestor at the cell order equals the derived word word[j]. Location words are heterogeneous in order (§2.2), so a checker MUST decode each word's order from the word and MUST NOT assume order 29 — two order-29 point words in one order-19 cell merge to an area word (order 20, say) whose order-19 ancestor is still the cell, and a spill-block close or an overview fold coarsens words but never past the cell they were folded in. The one word that can be coarser than its cell is a §9 area word ingested from an input resolved only above the cell order (no shipped config does this); there the check is that the cell lies inside the word — the two on one ancestor line. A leaf with no located field carries no stored morton word at all and rests on the derivation alone: a checker MUST report that as derivation-only, never as a pass (zagg: the coordinates check of python -m zagg.pyramid_check, which also compares the stored array with the derivation wherever a leaf still stores one).

Leaf immutability. A committed leaf's data objects are write-once: once the commit stamp lands, a writer MUST NOT modify bytes at an existing leaf key in place — no partial rewrite, no append, no re-encode of one array. The only legal change to a stamped leaf is wholesale replacement under the D4 retry discipline: the leaf template clears the prefix, rewrites it, and lands the stamp last. This is what lets a byte-range index into a leaf — the §11 Icechunk companion refs, moczarr's 2-GET reads — trust (key, offset, length) for as long as the stamp it was taken under stands.

Versioned leaves. On a legacy leaf (a stamp without current) a replacement is not a change of keys: clear-then-template rewrites the same key layout, and on an unversioned bucket the new object lands at exactly the key an old reference names — so a reference taken under a superseded stamp does not 404: it reads the new object's bytes at a stale offset and would silently mis-decode. Every recorded reference therefore carries the referenced object's ETag — or, on a local store, its last_modified — as its Icechunk checksum (§11.3), which Icechunk verifies on read, so a stale reference fails loudly instead of returning wrong bytes (issue #580).

A versioned leaf retires the case (issue #582, espg ruling 2026-09-26). Its stable prefix {id}.zarr/ (§4.2; the D3 address every reader computes) is a pointer root: the root zarr.json carries the commit stamp as before plus one key, current, naming the version subgroup that holds the arrays — {id}.zarr/{current}/{cell_order}/…, a complete, self-describing zarr leaf with its own stamp, whose objects are never rewritten once stamped. The version name is per attempt: run-{run_id}-{attempt}, where run_id is the run's id (uuid4().hex) and attempt is a per-invocation nonce — 8 hex characters of a fresh uuid4, drawn by the writer for each unit it writes — so two writers of one unit (a duplicate-invoke retry, a redundant fleet worker) never share a prefix. The run's tag run-{run_id} in the §11 repo still groups all of a run's versions by prefix. A replacement writes a new version subgroup and moves the pointer; the superseded version's objects stay at their keys until the §11 garbage collector finds no retained snapshot naming them. Consequently a reference taken under any stamp stays valid for as long as its version exists — a run tag reads every base leaf exactly as that run left it — and a same-key rewrite of a base leaf's committed bytes never happens. Only base leaves are versioned: the §4.6 leaf column and every overview level, which §11.3 also indexes, are still rewritten at their keys, so a run tag reads them only as the latest run left them and a superseded ref into one fails checksum-loud (the legacy semantics) until they are native Icechunk overviews (issue #584).

The write order is normative: (1) write the version subgroup's arrays; (2) stamp the version (its own root zarr.json, whose spec is /1 or /2 exactly as the legacy rule assigns it — windowed ⇒ /2); (3) record the refs against the version's objects (§11.3) — a commit on main under commit: "leaf", the ladder ref sidecar under commit: "ladder" (§11.4); (4) swap the pointer — one PUT of the stable root zarr.json, mirroring the version's stamp and naming it as current. What readers see: a path reader (the D3 address, the stamp, the pointer) sees the previous state until (4) lands; an Icechunk reader of main sees the new version at its commit — under commit: "leaf" that is step (3), before the swap; under the ladder it is the staged sweep's node commit, which may follow (4). Two racing attempts of one unit under commit: "leaf" may leave main and the pointer naming different complete versions; both are retained (one referenced, one current), and the collector reclaims neither. A retry always writes a new version — same run or not, it draws a fresh attempt — so no writer ever opens, clears or resumes a prefix another writer may hold. An attempt that died before its pointer swap leaves a version that is not current: unreferenced (under commit: "ladder", where only the sidecar named it) or referenced by main (under commit: "leaf", refs already committed); the §11.4 collector reclaims it under its reference rule.

The lifecycle touch never reaches a version. The #388 skip-path touch of a versioned unit (a write path's objects are fresh and need none) refreshes exactly the stable root zarr.json — the pointer stamp, one object, never {root}/ as a tree — and the unit's sibling objects: the stats sidecar, the granule-id sibling, the sub-map, the Icechunk ref sidecar, and the column tree plus its sidecar when declared. Those siblings stay unversioned, keyed off the leaf stem and rewritten per run. A version subgroup's objects — current or superseded — are never touched, so no touch re-mints an ETag or moves an mtime that a recorded reference carries (§11.3). Their lifetime is governed by the §11.4 collector, not by bucket expiration: an expiration rule on a store holding versioned leaves MUST NOT cover version subgroups — the same posture the repo's own icechunk/ prefix takes — or the untouched current version ages out under a live pointer.

Readers carry one rule: open the stable root; if its stamp names current, the arrays are under {root}/{current}/, else under {root}/ itself. Absent current is a legacy leaf — every store written before this revision — so no store requires migration and a store legitimately mixes legacy and versioned leaves after its first post-revision write. The manifest's own spec (§4.2's window-naming dialect) does not move, and no stamp spec value marks versioning either: a leaf is versioned exactly when its root stamp names current — the one marker, orthogonal to the naming dialect the spec token carries.

The root's states, for readers and for a writer's skip gate: no root zarr.json — debris (D4), as today; the (4) PUT is what creates the root group of a fresh leaf. A root stamp without current — a legacy leaf, arrays at the root. A root stamp naming current — arrays at the version. A root stamp naming current over a root that also holds {cell_order}/… arrays — a converted legacy leaf; the root arrays are the last legacy write, read only by readers that ignore current. A pointer naming a missing or unstamped version is a corrupted leaf that a reader, and a writer's skip gate, treats as debris: the pointer is only ever written after the version is stamped, so the state arises only from out-of-band deletion. The root's mirrored §5.3 content_hashes are computed over the version store, so their keys are relative to the version root ({cell_order}/morton, not {current}/{cell_order}/morton) — identical in form to a legacy leaf's; a verifier opening the pointer resolves current first. A writer about to clear-and-template a leaf root whose stamp names current MUST refuse: it is a legacy or stale writer against a versioned leaf, and clearing the root would delete every version and every earlier tag's referents with it.

Operator preconditions. Hive writers produce versioned leaves by default (output.leaf_versions; false opts a run out to the legacy leaf). A store written with versioned leaves MUST carry no bucket expiration rule over its hive tree: an S3 lifecycle filter selects by prefix, tag or size and cannot exclude the nested version subgroups, and the #388 touch refreshes only the pointer root, so any rule over the tree ages the current version out under a live pointer — version lifetime is owned by the §11.4 collector alone. Its readers MUST follow current (the one rule above): a reader that opens the root's arrays gets a 404 on a fresh versioned leaf and, on a converted legacy leaf, silently reads the last legacy write.

1.6 Succession

The ragged attrs block is /1's element contract. The candidate successor is the §6 typed-dtype revision (zagg-ragged/2, issue #210). The signaling mechanics are normative now:

• An array whose zarr data type is the typed vlen-ndarray dtype is zagg-ragged/2; on such arrays the ragged attrs marker is retired (not written).
• An array with the variable_length_bytes/bytes dtype and a spec: "zagg-ragged/1" attrs block is zagg-ragged/1.
• /1 remains valid indefinitely — existing stores never require rewriting, and every conforming reader supports /1 unconditionally, whatever timing the /2 implementation lands on.

2. Digest payload semantics

Status: contract (payload bytes); the digest algebra is deliberately NOT specified.

A t-digest field is a zagg-ragged/1 (or /2) array whose element declaration is {"dtype": "float32", "shape": [-1, 2]}. Source of truth in code: zagg.stats.tdigest.

2.0 The weights declaration

Contract (issue #422). A digest payload array declares the semantics of its weight column under the weights attrs key — a sibling of the §1.2 ragged block on the payload array, never a key inside it (the ragged block is retired wholesale under /2 — §1.6/§6.3 — so a sibling key survives that metadata-only migration untouched). Two values are defined:

• "counts" — weights are observation counts: integers ≥ 1 whose sum is the cell's exact observation count, per §2.1. An absent weights key MUST be read as "counts" — every store written before this revision is conformant verbatim, no byte rewritten.
• "flux" — weights are calibrated flux: positive finite float32 reals (a zero-weight observation carries no flux and MUST NOT produce a row); sum(weights) estimates the cell's detected photoelectrons, not an observation count. A flux-declared array MUST record its calibration provenance in the same attrs: a gain key carrying at minimum the gain constant's name and version (the operating point of any write-time clip rides alongside, writer-defined).

A reader MUST strict-check the value: an unknown declaration is a future revision of this section and MUST be refused, never read as either defined value. Merges are legal only between payloads carrying the same declaration (counts with counts, flux with flux — an absent key is "counts" for this rule too): a mixed merge would produce a weight column whose sum means neither thing, so a merging reader or writer MUST refuse it. The declaration rides the payload array only; a located field's {field}_locations sibling carries no weights key (§1.2's no-user-attrs rule for siblings is unchanged).

2.1 Centroid array

Contract. A populated cell's decoded payload is a (k, 2) float32 array of weighted centroids:

• column 0 is the centroid mean; column 1 is the centroid weight — under the "counts" declaration (§2.0, the default) the number of observations merged into it, an integer ≥ 1; under "flux" a positive real per §2.0;
• rows MUST be sorted ascending by mean;
• under "counts", sum(weights) MUST equal the cell's exact observation count — the number of finite source values the digest was built over (non-finite source rows are dropped before building) — while that count is representable in float32, i.e. <= 2^24 (16,777,216); above that bound the weights and their sum are the nearest float32 values to the true counts, so a reader recovering counts from weights (§3.3 tells it to, for N_signal) gets the exact integer at or below the bound and a rounded one above it. The bound is comfortable at leaf cell orders and is the one to watch at coarse overview orders (§4.4). For a stratified product (§3) each stratum digest's total weight is the exact stratum count, under the same bound;
• under "flux" (§2.0) sum(weights) is a float32 photoelectron estimate, not a count: the exact-count recovery above (and §3.3's) is undefined for a flux payload, and no integrality holds;
• an absent cell decodes as the zero-length (0, 2) array (the b"" fill).

2.2 The location channel

Contract. A located digest field (issue #87) carries one uint64 morton word per centroid row in its {field}_locations sibling (§1.1), row-aligned with the payload:

• A word's claim is keyed on its kind, carried by the word's own encoding (mortie spec §4), never by the payload's weights — under a "flux" payload (§2.0) sum(weights) is not a member count, so weight identifies nothing:
• a point word (order-29 by mortie's grammar) is the observation's reported position, carrying no area claim — not an assertion that the observation is dimensionless;
• an area word is a cell known to contain every observation beneath it — the finest such cell its producer could establish. Under this section an area word arises only from a fold (the deepest common ancestor of the fold's input words); ingesting one directly is the §9 declaration's grant, not this section's.
• Per-observation ingest under this section is order-29 point words (HealpixGrid.assign): an observation enters as its reported position's point word, and area words appear only as fold products. A writer whose observations are resolved only to a cell (a pre-gridded or pre-aggregated input) MUST NOT narrow them into points — §8.1's discipline, spatially — and therefore MUST write them under the §9 declaration, which grants area-word ingest; this section deliberately does not. The restriction is the freeze rule at work: these semantics predate the declaration, and a store read "as §2.2 verbatim" keeps exactly the meaning this section was published with.
• A merged centroid carries the deepest common ancestor of its members' words (mortie.common_ancestor): a cell containing every observation merged into it, and the finest one those words establish — a fold sees words, never the observations beneath them, so a coarse input word bounds how fine the result can be. A centroid folded from a single member carries that member's word unchanged. Point and area words share the same path prefix, so mixed inputs (a fresh point word folded with an earlier merge's coarser area word) compose under the one rule.
• Orders are heterogeneous, leaf arrays included. A reader MUST decode each word's order and kind from the word itself (mortie §1/§4) and MUST NOT assume a uniform order per array, per cell, or per store. A store whose leaf words happen to be uniformly order-29 point words (every shipped config's output today) is an observation about particular bytes, never an inference this section licenses — the same duty §9.1 spells out for a declared companion, stated here so the undeclared read path (§9's absent-key clause) carries it too.

A morton cell encodes containment, not calibrated uncertainty (informative): a word records the resolution at which the producer located the observation, never its error budget — a small error disk straddling a cell boundary is honestly enclosed only by a much coarser cell, so a reader MUST NOT read a word's cell as an uncertainty region, and an error-radius channel, if ever wanted, is a new companion declaration, not a reading of this one. This is a reader's limit, not a writer's obligation to coarsen: every position-resolved observation still enters as its point word.

A writer-side spill-block close (aggregation.streaming.mode: spill crossing its block threshold, issue #370) is an additional merge source under the same rule: an overflow shard's centroids may carry coarser common-ancestor words, with the stored layout and byte-level contract above unchanged.

Word semantics (bit layout, kind marking, coarsening) are mortie's specification §1/§4, not restated here. A located sibling MAY additionally carry the §9 located declaration, which self-describes that grammar in the store; its absence is this section verbatim, never a refusal (§9).

2.3 What is deliberately not specified (informative)

The build/merge algebra — Dunning's k1 scale function, the delta compression budget, merge order, merge_tdigests / merge_tdigests_kway — is zagg-owned and referenced informatively only (zagg.stats.tdigest docstrings). Readers do not need it to decode: the stored bytes above are the whole contract, and keeping the algebra out of the spec preserves zagg's freedom to optimize it (issue #279) without a spec revision. Two consequences a consumer should know (informative): digest merging is order-dependent (approximate composability class — np.isclose, not byte equality, across different fold orders), and quantile estimates from the centroids are approximations with the usual t-digest accuracy profile (tight tails, looser middle).

3. zagg-composition/1

Status: contract. Source of truth in code: zagg.stats.composition (issue #321). Rationale and narrative: signal_strata.md.

A composition field is one dense uint64 word per cell carrying eight 8-bit lanes of quantized fractions of the cell's signal stratum (N_signal = the signal digest's total weight — magnitude lives in the digest, composition here). An empty signal stratum packs to 0, and a composition array MUST declare fill_value: 0 so an unwritten cell decodes to the same word: readers key presence off lane > 0 (§3.2), so a nonzero fill would make every unwritten cell report spurious flag presence (a fill of 1 reads as lanes [1,0,0,0,0,0,0,0] — "land occurred exactly"). Enforced at config validation, alongside the §3.3 of/threshold cross-checks.

3.1 Word layout

Contract. Lanes are packed LSB-first: lane i occupies bits 8*i .. 8*i + 7 of the word. Lane order (LANES):

lane (byte)	meaning
0–4	per-surface fractions, signal_conf_ph column order: land, ocean, sea_ice, land_ice, inland_water — the count of signal photons whose per-surface confidence clears the threshold, over N_signal
5–7	low / med / high: signal photons whose strongest per-surface confidence is exactly 2 / 3 / 4, over N_signal

• The per-surface lanes are overlapping marginals (surf_type is multi-hot): they do not sum to 255 and cannot split the height distribution per surface.
• The level lanes are absolute — always conf == 2 / 3 / 4, never renumbered against the signal threshold. A product committing a higher threshold ships empty lower lanes rather than shifted ones (threshold=3 leaves low structurally 0; threshold=4 leaves low and med 0), so one lane layout serves every product. For ATL03 confidences (-2..4) the three level lanes partition the signal stratum exactly; a source with confidences above 4 is out of contract for this revision.

Golden word. For a single signal photon with per-surface confidences [4, -1, 0, 3, 1] at threshold=2, the lanes are [255, 0, 0, 255, 0, 0, 0, 255] (land, land_ice; strongest = 4 ⇒ high) and the packed word is exactly

0xFF000000FF0000FF

(an MSB-first layout would give 0xFF0000FF000000FF). Pinned by tests/test_composition.py::TestPackComposition::test_golden_word_pins_lsb_first_byte_order and the §7 kitchen-sink fixture.

3.2 Quantization: the presence floor

Contract. Lanes quantize as k = round(255 * c / N) — round-half-even, clipped to 0..255 — except any nonzero count quantizes to at least 1 (the presence floor). Consequences:

• lane > 0 means "this flag occurred" exactly, at every N, through arbitrary merge chains.
• Count recovery round(k * N / 255) is exact whenever N <= 254 (quantization error <= N/510 < 1/2).
• Above that, counts are within ±N/510 (plus O(N/510) per re-quantizing merge); presence stays exact.
• A cell with one signal photon has lanes in {0, 255} — the lanes are that photon's flags.

3.3 The attrs block

Contract. The composition array's attrs carry the versioned composition block; readers MUST bind to it, never to config conventions, and MUST strict-check spec per the conformance rule:

"composition": {
  "spec": "zagg-composition/1",
  "lanes": ["land", "ocean", "sea_ice", "land_ice", "inland_water", "low", "med", "high"],
  "of": "h_tdigest_signal",
  "threshold": 2
}

• lanes — the lane names in bit order (LSB byte first). For /1 the value is exactly the §3.1 order — lane order is not a product knob: the packer writes lane i at bits 8*i .. 8*i+7 in that fixed order, so a permuted, truncated, or renamed lanes value is out of contract and a writer MUST reject it rather than emit it.
• spec and lanes are writer-stamped, not author-declared: the store's values come from the writer's own constants (zagg.stats.composition.COMPOSITION_SPEC / LANES, stamped onto the array spec by grids.base.apply_field_attrs), the same posture as the §1.2 ragged block, and a config declaration that disagrees with either is rejected at config validation. of and threshold are per-product and author-declared, cross-checked at config validation (of must name a declared kind: ragged field; threshold must equal the reducer's own params.threshold).
• of — the name of the sibling digest field whose total weight is the N_signal the lanes are fractions of. The composition word is uninterpretable without it: readers recover counts by pairing the word with that digest's sum(weights).
• threshold — the committed signal cut (conf >= threshold; the ATBD predicate is > 1, i.e. threshold=2). Each stratum digest's payload array carries the companion provenance attrs stratum ("signal"/"noise") and signal_threshold, which MUST agree with this value.

3.4 Merge law

Contract (normative here, not a zagg implementation detail: a reader folding views — e.g. cells into a coarser cell — must reproduce it). Two (word, n_signal) pairs fold as the digest-weighted mean per lane, re-quantized with the same presence floor:

lane_merged = quantize((n_a * lane_a + n_b * lane_b) / (n_a + n_b))

where quantize rounds half-even, clips to 0..255, and floors a lane that is nonzero on either input to at least 1. The identity element is (0, 0); a pair with n <= 0 returns the other word unchanged. The operation is symmetric and, up to the bounded re-quantization error, associative — fold order never affects presence, and affects counts only within O(n/510). The n inputs come from the of digests' total weights (§3.3).

The k-way form. A writer folding many parts at once collapses them in ONE pass under the same law — lane = quantize(Σ nᵢ·laneᵢ / Σ nᵢ), the same presence floor, parts with n <= 0 skipped, an all-empty part set folding to 0 — quantizing once, so the result is order-independent (a permutation of the parts returns identical bytes) and the count error stays one quantization wide however many parts there are. This is zagg's writer-side fold everywhere a stored composition folds (the spill block close, issue

370, and every pyramid/overview fold site, issue #515;

zagg.stats.composition.merge_composition_kway); the binary law above stays the reader-facing contract, the k-way form being inside its documented associativity tolerance and strictly tighter than a binary chain.

Composability (§4.5). A composition field folds through the pyramid under D24 class packed, method composition_kway — the k-way form, with each contributor's n taken from its of digest's weight at the same cell. An overview's composition array carries the same §3.3 attrs block a leaf does, and its N_signal at every level is the folded of digest's total weight at that level — exact up to §2.1's float32 bound: past 2^24 pooled observations that sum is the nearest float32 to the true count (§4.6), and the fold's divisor rounds that float to an integer. The residual is relative ~1e-7, orders below one lane step of 1/255, so it never moves a published lane; it is stated because a reader recovering counts from N_signal is entitled to the same bound §2.1 gives the digest itself. One consequence a reader must hold: each fold re-quantizes once, so a cascaded level's counts drift by at most one quantization per level while presence stays exact through the whole chain — deterministic, but not byte-equal to a direct aggregation, which is why the class is neither exact nor approximate (§4.5).

4. Pyramid / overview declarations

Status: ratified design; implementation in flight (#201). The decisions this section records are ratified — D11/D22–D24 in design/sparse_coverage.md, plus three rulings on the #201 thread that this section's grammar traces to directly: all.zarr + role: overview confirmed, display schedule every 2 orders, the D24 none-class ruling (per-field exclusion the default, declared derived summary the opt-in, never the semantic core), and the A/B/C/D option space (C an espg-flagged opt-in phase, D rejected); the grammar below is what the #201 implementation lands and what moczarr's level-node reader plans against (espg/moczarr#15, the 8b seam). Any divergence discovered while landing #201 is resolved on this section first — the implementation conforms to the spec, never the reverse.

The level grammar is revision zagg-pyramid/2 (#382; design record #381, points (2)–(5), as collapsed by the espg grammar ruling on the declaring PR): the config declares leaf cell resolutions only, and everything above the shard is the fixed every-order ladder of §4.4; the manifest records the fully expanded (node, cells) list, of which the original constant-depth zagg-pyramid/1 grammar is the special case cells = [node + (cell_order - shard_order)]. /1 stores stay readable under their own rule forever (§4.5 — we are the sole consumer; there is no migration machinery). A /2 declaration is materialized by the leaf columns (#383, §4.6) and the staged sweep (#384): declared-but-unswept remains a legal recorded state (#381 point (11): declaring is free, sweeping is the operational decision), and since the issue #384 default flip a default declaration (no schedule spelled) is /2 at the grid's resolved chunk order whenever that order is strictly interior to the shard's resolution window (raster configs, explicit legacy orders/spacing schedules, and K == 1 grids keep /1).

A zagg-pyramid/2 store is therefore inherently a multiresolution statistical grid: every HEALPix order from 0 through the declared base, plus the native resolution, each level spec-guaranteed and individually addressable — a reader picks its resolution and reads it; skipping levels is a reader-side choice, never a store property. output.pyramid: false is the degenerate single-resolution opt-out.

4.1 Overview zarrs at ancestor nodes

An overview zarr is a sweep-built coarse materialization of a subtree's committed leaves, written at an ancestor digit node of the hive tree (tree layout and path grammar: mortie's specification). It has the same structure as a leaf (§4.4), one basename dialect (§4.2), and is classified by attrs alone (§4.3).

Overviews are regenerable caches, never load-bearing: deleting every overview MUST leave all leaf reads intact, and a reader MUST NOT require them. They are stale-detectable, not stale-proof — after a leaf re-run an ancestor overview may lag until the sweep regenerates it; the generation stamp (§4.3) is what makes that detectable.

4.2 Naming

Overviews inherit the leaf window-naming dialect (D23; grammar frozen on the mortie spec page): at an ancestor node an overview for time window {window} is {window}.zarr, and the reserved token all names the all-time fold (all.zarr — the same token that names a schedule: none store's leaves; excluded from the window grammar forever). Nothing about the name distinguishes an overview from a leaf — classification is §4.3's job.

A windowed leaf ({id}_{window}.zarr) is the one leaf form that stores no per-cell morton array: every window of a shard shares the shard's cells axis, so the coordinate is derived from the {id} and the rank (§1.5 "The cell coordinate"). The overview and column artifacts named in this section store theirs regardless.

A versioned leaf (§1.5; a stamp naming current) adds nothing at the node: its stable {id}.zarr / {id}_{window}.zarr entry is unchanged, and its versions are subgroups of that entry named run-{run_id}-{attempt} — so the D5 node invariant, the walker's child classification and every prefix-scoped rule see the same children as before. The stable root's stamp names the current version; the version subgroup carries the arrays. A version name is never a cell-order digit group (it always begins with run-), so a reader enumerating a leaf root tells the two apart by name.

4.3 The role and zagg_overview attrs

Contract. Classification is carried in the zarr's root-group attrs, never inferred from tree position or depth — a shallow zarr may equally be coarse source in a sparse region (D24: one product tree may carry regionally heterogeneous resolution).

• role — "overview" on every sweep-built overview. Source leaves carry no role key: absence means source. A reader MUST check role on every zarr it opens at an overview-carrying order; analysis readers reject or skip role: overview zarrs, display readers MAY stop at one.
• zagg_overview — the versioned provenance block, present exactly when role is "overview":

"zagg_overview": {
  "spec": "zagg-overview/1",
  "node": "-3111",
  "order": 3,
  "cell_order": 11,
  "source_shard_order": 5,
  "source_cell_order": 13,
  "window": "2019",
  "fields": {"count": {"class": "exact", "method": "sum", "nan_policy": "skip"},
             "h_tdigest": {"class": "approximate", "method": "tdigest_kway"}},
  "fold_source": "leaves",
  "generation": {"n_leaves": 16, "max_leaf_timestamp": "2026-07-20T00:00:00Z"},
  "content_hash": "…",
  "generated_at": "2026-07-21T00:00:00Z"
}

spec follows the conformance rule (strict-check, fail loudly on an unknown revision). node is the ancestor's morton decimal string and order its order — mortie's decimal grammar puts one digit per order after the base-cell digit, so order is always len(digits) - 1 for the node string ("-3111" is order 3; the leading - marks a southern base cell and is not a digit); cell_order the overview's own cell order (source_cell_order - (source_shard_order - order) — constant tree depth, §4.4); window the §4.2 window key ("all" for the all-time fold); generation the D22 staleness stamp (merged-leaf count + max leaf commit timestamp); content_hash a sweep-internal skip-if-current digest (informative — not the §5 O11 recipe and not part of the reader contract).

fields enumerates the materialized fields only — exactly the fields present as arrays in this overview, each recording the fold that was actually applied. A none-class field is absent from the zarr (§4.4) and so MUST be absent from this map: its recorded absence lives in the manifest's pyramid.overview.fields (§4.5), which is the map that enumerates every declared field. Consequently a reader MAY treat this map as the overview's variable list and MUST be able to open every array it names; cross-checking it against the arrays present is a valid integrity check. Each entry carries at least class and method, and MAY carry further fold provenance — an exact entry records the reduction's nan_policy ("skip": nan-skipping, never NaN-propagating) — so readers MUST tolerate additional keys.

fold_source names the regime that produced this level (#376) — the one piece of provenance a reader cannot recover from the arrays:

• "leaves" — folded directly from the subtree's source leaves: single quantization, and for the exact class byte-equal to a direct aggregation at this cell order (§4.4);

• "cascade" — folded from an already-materialized finer overview (fold-of-folds), whose order the entry then also names:

  "fold_source": "cascade", "fold_from_order": 3
  

The distinction is material for the approximate and packed classes: the exact merge laws are associative, so a cascaded sum/min/max is the same value either way, while a cascaded digest is a merge of merges — it inherits the merge's documented behavior once per level and carries no precision guarantee — and a cascaded composition word re-quantizes once per level (§3.4: presence stays exact through the chain; counts drift at most one lane quantization per level). That is in contract: overviews are display artifacts, and the precision promise stops at the levels declared exact (pyramid.overview.exact_levels, §4.5). A reader that needs the exact regime MUST check this key rather than the level's depth, and MUST read an overview that carries no fold_source as "leaves" — the only fold that existed before #376.

source_children records the cascade's coverage — present on a "cascade" overview only, and the companion a reader needs to interpret the level's fill cells:

"source_children": {"folded": 15, "missing": 1, "unreadable": 0}

A cascade folds the child overviews that are on disk, where a "leaves" fold folds every leaf the coverage MOC knows about. folded counts the children that contributed, missing those with no materialized (D4-stamped) overview, and unreadable those that failed to open or did not classify as an overview at fold_from_order. When either of the latter two is nonzero the level under-covers its subtree: the spans those children own hold the fill value, and a fill cell there is not evidence that the subtree is empty. Such a level is repaired by a later sweep, which sees the parent's summed generation change once the child exists. A reader MUST tolerate the key's absence (a "leaves" level, or a pre-#376 artifact) and MUST NOT read absence as missing: 0.

demotions records the packed guard rail firing at this node (#518) — present exactly when the fold that last wrote this artifact demoted a packed-class field, on both attrs revisions (zagg-overview/1 and /2, §4.4):

"demotions": [
  {"field": "composition", "class": "packed", "reason": "word-missing",
   "contributors": 1, "of": "h_tdigest_signal", "cells": 4}
]

One record per (field, reason), sorted by that pair. reason is one of:

• "word-missing" — a contributor carried the field's §3.3 divisor digest but not the word. The fold blanks the output cells that contributor's rows reach to the fill word (cells counts them): publishing the surviving siblings' word over an N_signal that counts rows the word never covered would be a §3.3 skew — absence over wrongness;
• "divisor-missing" — a contributor carried the word but not its of digest, so it contributes nothing for the field (the fold's n inputs are that digest's per-cell weights and are never guessed). This is the shape a mis-declared divisor leaves at every level above the leaves: a manifest naming an of whose own entry is class: "none" (§4.5 — manifests outlive their writer) materializes no divisor array in any overview or column, so every contributor fires this arm and the field silently never folds upward.

A demoted contributor is still a counted leaf (or child) on every fold path (since issue #620, which put the /1 cascade and the /2 ladder on one fold engine): its other fields fold, the level materializes and carries the record, and source_children counts the contributor as folded. The mis-declared-divisor shape above therefore lands as a level whose packed array is all fill beside a record naming every contributor. (Before #620 the /2 stage fold counted a fired contributor unreadable and dropped a level on which every contributor fired; a /2 store swept before then may lack such a level, and a reader MUST NOT read its absence as evidence the rail stayed quiet.)

field/class/reason/contributors are always present. A contributor is one source read the fold performed: a (child, window) staged column on a /2 merge (so one child missing a half in W windows counts W), a (leaf, window) column read on a /1 fold from leaves, and one child overview on a /1 cascade. contributors counts the ones this (field, reason) fired on, in that unit. of names the §3.3 linkage; cells is keyed whenever the demotion left output cells at the fill word that no surviving contributor covers — always in the word-missing direction, and in the divisor-missing direction only where contributors own disjoint spans of the level (a cascade of child overviews, /1 or /2, where a child skipped for the field leaves its whole span fill). Where many contributors share each output cell — a /1 fold from leaves, the all-time fold across windows — a dropped contributor blanks nothing and the key is absent. A reader MUST NOT read its absence as "nothing was blanked". Readers MUST tolerate additional keys, MUST tolerate the key's absence (a clean fold, or any pre-#518 artifact — a clean fold's attrs are byte-identical to a pre-#518 writer's), and MUST NOT read absence as "no demotion ever occurred" on a pre-#518 store. The record is the per-field refinement of source_children: those counters say a whole contributor was unusable, this key says which field folded short and why. The two counters are not in the same unit and MUST NOT be compared: source_children counts children, classifying each child once however many windows it contributed, while contributors counts the per-read instances defined above — they coincide only in the single-window, one-read-per-child case. A demoted field's cells hold the fill value, which §3.2 makes byte-indistinguishable from genuine emptiness, so the attrs are the only place the distinction can live. Like fields, source_children and fold_from_order beside it, a demotions state is as current as the artifact's own write and never a live claim about the manifest in force: the skip-if-current gates key on generations and regime, not on the field declarations, so after a manifest edit (the mis-declared of being introduced or corrected — §4.5, manifests outlive their writer) a level whose bytes have not changed stays current carrying the previous fold's answer. The demoted/ conformance fixture (§7) commits one fired and one clean level.

An overview also carries the standard D4 commit stamp as its final write: an unstamped overview prefix is debris, exactly as for leaves. The stamp carries the §5.3 content_hashes record over the overview's own arrays when one was computed (keyed only then — absence reads unverifiable, never tampered). Write order is pinned — template, arrays, role/provenance attrs, stamp LAST — so presence of the stamp certifies the role attr landed; a reader MUST ignore unstamped overview prefixes.

4.4 Structure

Contract. A pyramid is an ordered list of level entries {node, cells} (§4.5): node is the hive-tree ancestor-or-self order whose artifact carries the level — one artifact per (node, window), named by §4.2 — and cells the reader-facing cell resolutions stored there. The list is not free-form; it is determined by two declarations and one law:

• the leaf entry — node == shard_order (not an ancestor artifact at all: the leaf's own level column, #381 point (2)) carries every declared leaf resolution, each strictly between shard_order and cell_order. The set of levels whose cells are at or above the shard order MUST be contiguous from the finest leaf resolution down to the shard order (increments of one — [13, 12, 11, 10, 9] on the 19/13/9 geometry, never [13, 12, 10, 9]): that set is the §4.6 column tier, and contiguity guarantees the raw-fold boundary member exists whenever anything coarser is declared. The requirement scopes to that tier and nothing else: levels whose cells are below the shard order are the ladder's own, DERIVED by the every-order law below, so no gap can arise there to permit or forbid;
• the fixed every-order ladder — with d = base - shard_order (base the coarsest leaf resolution, so d >= 1), every order k from shard_order - 1 down to 0 inclusive carries exactly one member at resolution k + d. Two numbers — shard_order and d — determine everything above the shard; every store roots at order 0. (Non-normative: the ladder is cheap by construction — cell counts shrink 4× per rung, and digest bytes are cap-bounded per cell — which is why it is law rather than a knob.)

For each member resolution r the artifact holds one resolution group: the zarr group named str(r), with the same layout as a source leaf's cell group. Concretely, for a member r at an order-k node:

• the group's morton coordinate array holds the 4^(r - k) order-r words the node covers, in canonical nested order — its descendant words where r > k, its own word where r == k. For a manifest level member r > k always, by the window and ladder rules above; the one recorded r == k group is the §4.6 column's node-order member (the whole-footprint aggregate of #381 point (2)), which is a recorded group of that artifact and still never a manifest member. It is the source of the coarsest gather level only: every level whose cells are below the shard order is a cascade of its children's artifacts at the next finer declared level, four children folded 4^(r' - r)-to-one, and nothing above the first merge reads the leaf tier again (issue #620 — before it, every merge re-read the leaf columns' res-shard_order + 2 member, an input of 4^(shard_order + 2 - k) cells per node that put an order-0 node at 18.9M cells and the dense order-1 node past the 900 s wall). No separate partial grammar or partial/ path exists anywhere;
• each included field is the same array kind as at the leaves: dense fields as dense arrays, digest fields as zagg-ragged/1 (or /2) vlen arrays — §1–§3 of this page apply to overview arrays unchanged, including §2.1's float32 exactness bound: a coarse overview cell can pool more than 2^24 observations, and there sum(weights) is the nearest float32 to the true count rather than the count itself;
• field inclusion is gated by the field's composability class (§4.5): exact, approximate and packed fields appear, none fields are absent.

Under zagg-pyramid/1 every artifact holds exactly one resolution group, at the constant depth k_cell = c - (s - k) for shard order s and cell order c (cells coarsen 4× per order of ascent — the pyramid is the store's resolution axis, partially materialized): the /1 grammar is the special case cells = [node + (c - s)], and §4.1–§4.2 apply to both revisions unchanged. §4.3's per-artifact zagg_overview attrs block — in particular its single scalar cell_order = c - (s - k) — is specified for /1's single-group artifacts. Stage-written /2 ladder artifacts (issue

384) carry attrs revision zagg-overview/2: the same keys with

cell_order the entry's own cells member (k + d, not the constant-depth formula), the fold_source/fold_from_order pair replaced by the #381 point (7) provenance — regime (stage-gather | stage-merge), merges_from_raw (the fold depth: for a gather, the depth of the column groups it concatenates — 1 at or above the raw-fold boundary, 2 below it (§4.6) — and for a merge one more than the largest merges_from_raw among the children's artifacts it folded, and so on down to the root; on the 19/13/9 reference geometry the ladder records 1, 1, 2, 2 at orders 8–5 and 3, 4, 5, 6, 7 at orders 4–0. A reader MUST tolerate any value and MUST NOT bind to a ceiling: the value is a statement about the digest fold's generation, which exact and packed classes are indifferent to and approximate classes carry as their recorded accuracy — §4.4's accuracy doctrine below), and source_children (present in both stage regimes: a gather that under-covers says so exactly like a merge; a merge's counters are over its children's artifacts, a gather's over the child columns) — plus run_id, the sweep run that wrote the artifact, and a generation block summing the consumed children (the stage skip gate's ratchet key — {n_leaves, max_leaf_timestamp, run_ids}, composition in §4.5). §4.3's demotions key rides /2 artifacts unchanged (same grammar, same present-exactly-when-fired rule). The commit stamp of a stage-written artifact carries the same run_id key (additive to the stamp grammar; fleet-written stamps never carry it, and a reader treats its absence as "not a stage artifact", never an error): it is the residual-race backstop of the sweep-admission lease. The /2 leaf entry's artifact is the §4.6 column — declared-but-unmaterialized remains legal (§4.5).

The all-time fold of a windowed store (issue #586). Where the manifest declares pyramid.overview.all_time (§4.5), a windowed store's all.zarr at a ladder node is folded from that node's own per-window overviews ({window}.zarr, every window the node has), cell for cell at the level's resolution — one k-way merge per cell across windows under the field's §4.5 law. It is written after the node's per-window artifacts and never from the leaf tier, so its cost follows the window count, not the subtree. Its zagg-overview/2 attrs record:

• regime: "stage-merge" at every level, a gather level included;
• merges_from_raw one more than its sources': 2 where the level's per-window overviews are gathers of gen-1 members, and one more than their cascade depth where they are themselves merges — the same rule a merge level applies to its children. §4.5's per-entry actuals describe the per-window ladder, not the all-time fold;
• source_windows — {"folded", "missing", "unreadable"} over the window overviews it consumed. missing counts a window known to have data beneath the node whose overview was not committed when the fold ran (its stage unit failed, or has not landed): the fold then under-covers that window and the next sweep heals it. The key is present exactly on this artifact; a reader MUST tolerate it and MUST NOT require it elsewhere;
• source_children — the folded windows' own counters, summed: a window that under-covered its subtree makes the all-time fold short by the same children;
• generation — the consumed window overviews' blocks, summed, with the runs that stamped them (the fold's skip key, §4.5).

An unwindowed store has no such artifact: its single fold is already all-time (§4.2) and is an ordinary ladder artifact.

An overview's variable set may therefore be a subset of the leaf's — heterogeneous variable sets across level nodes are in contract, and a reader MUST NOT assume every leaf field exists at every overview order (the manifest declaration below is the zero-open way to know).

The fold regime (§4.3's fold_source) does not enter this contract: a cascaded level has exactly the layout above. It differs in the values its approximate fields carry and, when it under-covers its subtree, in which cells are populated at all — §4.3's source_children is the key that says so, in every class.

Accuracy doctrine (/2). For /2 levels the #376-era "display artifacts, no precision guarantee past the exact levels" posture is superseded (espg ruling on the declaring PR): exact-class fields are exactly correct at every order — the reductions are associative, so the pyramid is a true downsampling pyramid for them, provided the declared dtype holds the coarse reductions: every level carries the field's declared dtype (the per-field dtype of the §4.5 pyramid block below, copied from the build config), and a sum that exceeds it wraps rather than widens. zagg's packaged templates declare count as int64 (#626: an int32 count summed past 2^31 at orders 1 and 0 of a dense order-9 store); a reader MUST take count's dtype from the array metadata, never assume a width — and approximate-class fields are analysis-grade at their recorded generation: the per-entry merges-from-raw count (#381 point (7), recorded in the §4.5 actuals and per artifact in zagg_overview) is the contract a reader holds them to, not a blanket display-only caveat. The /1 operational caveats about the deprecated leaves/cascade regimes (§4.3, §4.5) stay as written for /1 artifacts.

4.5 The manifest pyramid block

Contract. The product manifest (morton_hive.json; manifest bootstrap semantics: mortie's specification and design/sparse_coverage.md §3) declares the overview family under the versioned pyramid block:

"pyramid": {
  "spec": "zagg-pyramid/1",
  "overview": {
    "spacing": 2,
    "orders": [3, 1],
    "all_time": false,
    "fold_source": "cascade",
    "exact_levels": 1,
    "fields": {
      "count":     {"class": "exact", "method": "sum", "nan_policy": "skip",
                    "dtype": "int64", "fill_value": 0},
      "h_tdigest": {"class": "approximate", "method": "tdigest_kway",
                    "dtype": "float32", "inner_shape": [2], "delta": 512},
      "photon_ids": {"class": "none"}
    }
  }
}

Under revision /2 (#382) the schedule is the block-level overviews list instead — the FULLY EXPANDED form of §4.4's leaf entry + fixed ladder; orders and spacing do not exist in a /2 block; every other key is unchanged:

"pyramid": {
  "spec": "zagg-pyramid/2",
  "overviews": [
    {"node": 3, "cells": [5, 4]},
    {"node": 2, "cells": [3]},
    {"node": 1, "cells": [2]},
    {"node": 0, "cells": [1]}
  ],
  "overview": {
    "all_time": false,
    "fold_source": "cascade",
    "exact_levels": 1,
    "fields": {"…": "…"}
  }
}

The split is deliberate: the block-level overviews list is the store-wide product declaration — what fleet writers, stage sweeps, declare-time forecasts, and external readers consume — while the singular overview dict remains the overview sweep family's execution regime (D22 per-family bookkeeping: all_time, the #376 fold keys, fields, the sweep's materialized actuals), so the two never re-nest. Per-entry materialization actuals (#381 point (7)) nest inside the block-level entries themselves — the actuals key below, written by the issue #384 staged sweep's finisher.

• orders (/1) — the ancestor orders that carry overviews (descending; empty = pyramid declared off). spacing records the schedule step (default 2 — the ratified display schedule). Schedules are per artifact family and deliberately decoupled from the tree's path_grouping.
• overviews (/2) — the level entries of §4.4, recorded at block level (a sibling of spec and overview, never inside the family dict) and always fully expanded: the leaf entry first, then the fixed every-order ladder down to node 0 — a reader never re-derives the ladder, the recorded list IS the contract. Every entry is a {node, cells} mapping with cells a list. The corresponding config grammar is leaf resolutions only: output.pyramid.overviews is an int (sugar for one) or a strictly descending list of ints, each strictly between shard_order and cell_order (a member at the shard's own order is the writer-side aggregate, never declared; a member at the base data's own order would be the base data), and consecutive — the fixed ladder fills [max(shard_order, d), base) by itself (its coarsest cell is node 0's, d = base - shard_order), so a gap in the list is a gap in the §4.4/§4.6 column tier and is refused by name (missing [11]). Where the ladder's floor is the coarser of the two — base > 2 * shard_order + 1, so d clears shard_order by more than one — the tier gaps below it whatever the list says, and the refusal names base instead; omitted, the default is one resolution at the grid's resolved chunk order — normative since the issue #384 default flip: a default declaration emits this /2 block for every new store whose resolved chunk order is strictly interior (raster configs, explicit legacy orders/spacing schedules, and K == 1 grids keep /1), and the worker column gate (§4.6) derives the SAME default from the grid, so declaration and artifact can never disagree. Consecutiveness is a writer-side rule and carries no marker bump — the /2 wire grammar is unchanged, only the declaration it admits is narrower — so a reader MUST still decode a stored declaration whose tier gaps (one written before the rule) by its recorded overviews list, member for member, exactly as always: such a store is not retroactively non-conformant. Only the acceptance harness's declaration leg reports the gap, and it reports it as a declaration finding. There is no above-shard configurability: no per-node spelling, no gather declaration, no member promotion — the ladder makes every within-footprint member spec-guaranteed (espg grammar-collapse ruling on the declaring PR).
• The declared-off form is smaller, and orders is the only key a reader may bind unconditionally. With the pyramid knob off the block is exactly

"pyramid": {"spec": "zagg-pyramid/1", "overview": {"orders": []}}

— spacing, all_time, fold_source, exact_levels, fields, and summarize are absent, not empty. Recording absence never needs the new grammar, so the declared-off form is always this /1 shape — a /2 block's overviews list is never empty. A reader MUST branch on spec first, then on the revision's schedule key: under /1 that is orders — an empty orders (or no pyramid block at all — pre-pyramid manifests) means no overview family exists and no other key of the block may be assumed — and under /2 it is the block-level overviews. When the schedule key is non-empty, all_time and fields MUST be present in the overview family dict (spacing too under /1; summarize stays optional), so the zero-open field query of §4.4 is well-defined exactly when there is something to query. - fold_source / exact_levels — the declared fold regime (#376): "cascade" (the default) folds each declared level from the next finer declared level's overviews, and exact_levels is how many of the finest levels are folded from the leaves instead — so under {"orders": [3, 1], "fold_source": "cascade", "exact_levels": 1} order 3 is exact and order 1 is a fold of order 3's folds. "leaves" is the deprecated exact-from-leaves regime, where every declared level folds the raw leaves and no exact_levels key is written (every level is exact there, so a boundary would name a distinction the store does not have). These two keys are written by #376-and-later writers; a reader that finds no fold_source MUST read the declaration as "leaves", which is the only regime that existed before. They declare what the next sweep will do — what is on disk is materialized.fold_sources below and, per artifact, §4.3's zagg_overview.fold_source, which is authoritative for a given overview. Under /2 the pair is declared identically; exact_levels counts level entries from the finest end, exactly as it counts orders under /1. (exact_levels predates the overviews key-naming convention and is deliberately untouched by it — ruled vestigial-in-waiting under the #381 regime law, which makes the exact/approximate boundary structural; the /2 accuracy contract is §4.4's doctrine, not this pair.) - fields — every aggregation field, keyed by name, with its composability class: exact (folds byte-equal — count/sum/min/max), approximate (t-digest merge — np.isclose equality class), packed (the §3 composition word — deterministic order-independent k-way fold, presence exact, counts within one lane quantization per fold), or none (non-composable). A "class": "none" entry is the recorded absence (the ruled D24 default, option A): the field exists only at native resolution, and this declaration is how a reader knows without opening anything. A none entry carries class only — no method, no dtype/shape metadata: there is no fold to name, and stamping a default fold method on an excluded field would declare a t-digest array that does not exist. exact/approximate/packed entries carry the fold method, any further fold provenance (an exact fold's nan_policy), and enough dtype/shape metadata to know the overview array's form up front. An approximate entry MAY additionally carry overview_delta — the compression budget overview folds run at when it is split from the leaf delta (issue #424; both budgets are fold algebra, informative per §2.3) — and, for a non-default §2.0 declaration, weights together with the gain calibration provenance §2.0 requires beside it (the overview writer reconstructs its arrays from this entry alone, so a declaration recorded here without its provenance would write an overview whose calibration is unrecoverable); a reader MUST tolerate entry keys it does not bind. This map is the all-fields view; the per-overview zagg_overview.fields attrs map (§4.3) is the materialized subset. - A located ragged field is approximate, and its entry carries location. A location: declaration (§9) does not exclude a field from the pyramid: the located k-way merge reduces the {field}_locations words over the same centroid partition the digest merge produces, so a located digest field folds through every level and the class is approximate like any other digest field (ruling 4 on issue #410). Its entry carries location — the source column the leaves' words were ingested from — keyed only when set, so an unlocated field's entry is unchanged. The key is load-bearing for the same reason weights/gain are: the overview writer reconstructs a level's arrays from this entry alone, so without it the overview template emits no sibling array and the fold has nowhere to write its words. An overview level of a located field therefore carries the {field}_locations sibling, its §9 located declaration, and the payload's §1.2 locations binding, exactly as a leaf does — with the words at the heterogeneous orders §9.1 makes normative. - A temporal ragged field composes the same way, and its entry carries temporal. A "per-centroid" temporal: declaration (§8.3) does not exclude a field from the pyramid either: zagg's digest pyramids keep the companion "per-centroid" at every level, symmetric with the located channel — the temporal k-way merge reduces the {field}_times words over the same centroid partition the digest merge produces (espg-ruled 2026-08-17, amending ruling 3 on issue #410; §8.4 records the amendment and keeps its per-cell reduction licensed but unused by this writer), so a temporal digest field folds through every level and the class is approximate like any other digest field. Its entry carries temporal — the companion shape ("per-centroid"), which is all a level above the leaf can act on, the leaf's ingest column never being re-read by any fold — keyed only when set, so a field without the companion has an unchanged entry. The key is load-bearing exactly as location is: the overview writer reconstructs a level's arrays from this entry alone, so without it the overview template emits no sibling array and the fold has nowhere to write its words. An overview level of a temporal field therefore carries the {field}_times sibling, its §8.3 temporal declaration, and the payload's §8.3 times binding, exactly as a leaf does. The §8.2 dense "per-cell" shape is the exception: a field declaring it is classed none whatever its reducer (zagg.semantics.field_composability), that shape's fold law being the word grammar's join over a cell group rather than the field's own reducer, so it exists at native resolution only. - A packed composition field is packed, and its entry carries of. The §3 composition word (issue #515; espg ruling: admit) folds by method: "composition_kway" — §3.4's k-way form over (word, n_signal) pairs, each contributor's n taken from its of digest's per-cell weight. The entry carries the dense dtype/fill_value (a packed field's fill is the 0 word, §3) plus the §3.3 linkage: of — the digest field whose weights are the fold's n inputs and whose folded weight is the level's N_signal — and, when declared, threshold. of is load-bearing exactly as location is above: the overview writer reconstructs a level's arrays from this entry alone, so without it the fold has no divisor and the overview array's §3.3 attrs block could not be written. A writer MUST declare a composition field packed only when the digest of names is itself declared approximate in the same map and carries the default §2.0 weights: counts — the law divides by that digest's weight at every level and N_signal is a photon COUNT, so a composition whose divisor is excluded from the pyramid, or whose divisor is flux-weighted (a different quantity), is declared {"class": "none"} instead. Readers of an older declaration finding an unknown class token MUST treat the field as non-composable for their purposes (fold nothing, read native resolution) rather than erroring — which is exactly how pre-#515 zagg behaves. - all_time — whether the all.zarr all-time fold is materialized at the declared orders (windowed stores only; a schedule: none store's single fold is already all-time). On a /2 store it is folded from the node's per-window overviews (§4.4 "The all-time fold of a windowed store"). A windowed store that declares an overview schedule (a non-empty overviews list or orders) MUST declare it true (issue #610 phase 4: the per-node rollups and the §4.10 companion ride the node's all-time close, so a windowed config that omits the key resolves to true and one that spells false is refused). A windowed store that declares none (pyramid: false, written as orders: []) declares no fold and carries no all_time key. A reader MUST still tolerate false on a windowed store written before that rule — it is §4.10's gated case, with no all.zarr to find. - summarize (optional) — the opt-in declared derived summary for none-class fields: a mapping from a new, different field name to its derivation (e.g. an auto-digest of a roster field's raw values), living in the pyramid block and never in the semantic core — leaf truth is unchanged, and overview schema never silently differs from source except by declaration. (Ruled on the #201 thread; deterministic seeded subsampling is deferred as an opt-in phase, not rejected — it shares this block's declaration grammar and stays declared/never-default when it lands (option space); roster concatenation is rejected.) - The sweep MAY additionally record materialized-actuals bookkeeping in the block; the template-time declaration above is never rewritten by the sweep, and the pyramid block is excluded from the manifest's frozen append-guard keys. zagg's sweep writes

"materialized": {"orders": [3, 1],
                 "fold_sources": {"3": "leaves", "1": "cascade"},
                 "generated_at": "2026-08-04T00:00:00Z"}

— the orders it has written and, per level, the §4.3 regime that wrote it (keys are the orders as strings, JSON having no integer keys). Both are actuals, accumulated across sweeps: a level swept under an earlier declaration keeps the regime it was made with until it is regenerated, so this map can disagree with the declaration above, and that disagreement is informative rather than an error. It is a convenience — a reader MAY instead open the overviews and read §4.3 — and, like every other actual, it says nothing about an overview still being present (overviews are regenerable caches, §4.1). On a /2 store this family-dict materialized map is the /1-era inventory, preserved verbatim across a declaration revision bump: a retrofit never discards actuals, and the overviews it names stay on disk as regenerable-cache debris; the staged sweep never writes this family-dict map on a /2 store — the per-entry actuals below are the one source of truth for /2 materialization. /2 materialization actuals nest inside the level entry that owns them (#381 point (7)), written by the issue #384 finisher's manifest RMW:

{"node": 2, "cells": [3],
 "actuals": {"regime": "stage-merge", "merges_from_raw": 2,
             "source_children": {"folded": 3, "missing": 0, "unreadable": 0},
             "run_id": "stage-20260809T000000Z-ab12cd",
             "generated_at": "2026-08-09T00:00:00+00:00"}}

— regime is leaf-column (the leaf entry: the fleet's own column; its merges_from_raw is the maximum over the entry's declared cells of the §4.6 per-group value — 1 when every declared leaf resolution is at or above the raw-fold boundary, 2 when the entry declares a resolution below it; no source_children — its source is complete by construction), stage-gather (a concatenation of the child columns' members, whose merges_from_raw is the gathered groups' depth, §4.4) or stage-merge (a cascade of the children's artifacts, §4.4, whose merges_from_raw is the maximum over the level's artifacts of their recorded depth — 2 at the first merge level, one more per level below it; the all-time fold of a windowed store records its own depth in its own attrs and never these per-entry actuals — §4.4). source_children accumulates the run's per-artifact coverage counts; run_id names the sweep run (stage entries only). The key is additive: a reader MUST tolerate additional keys on a level entry, and actuals says nothing about artifacts still being present (overviews are regenerable caches, §4.1). - The stage skip key (generation, recorded per artifact in §4.4's attrs, per stage column in §4.6, and — on an unwindowed store — in the sweep-internal node envelope; a windowed store's stage units read it off the artifact's own attrs) is the triple

"generation": {"n_leaves": 16,
               "max_leaf_timestamp": "2026-08-09T00:00:00+00:00",
               "run_ids": ["stage-20260809T000000Z-ab12cd"]}

— the summed leaf count of the consumed children, the newest child stamp, and the sorted set of run_ids those children's stamps carry, unioned with the ids their own recorded blocks relay (#417). A level is folded again exactly when this triple moves. The third term is load-bearing: stamps resolve to one second, so the first two alone read a child rewritten inside its own recorded second at an unchanged leaf count as current, and the /1 content-hash backstop cannot apply without doing the fold the skip exists to avoid. A run may not rewrite its own object mid-run (single-writer law, §4.8), so a same-second rewrite is a foreign run's and moves the set. run_ids is additive: absent, it MUST be read as the empty set (a pre-#417 entry, or children that are all fleet-written leaf columns — those stamps carry no run_id), never as a wildcard that matches any set, so an upgraded store folds once more rather than inheriting the blind spot. Fleet-written leaf columns contribute no run id, so at the finest tuple the term is empty and the gate rests on the pair alone.

4.6 Leaf column artifacts (zagg-column/1)

Status: contract — issue #383 (umbrella #381 points (1)–(3)). A column artifact is the leaf worker's own pyramid contribution, written at aggregation time while the shard's cell data is resident: one zarr per (leaf, window), a sibling of the leaf under the leaf's own node prefix. A column exists exactly when the writing run's pyramid declaration is zagg-pyramid/2 (§4.5), carries leaf-node levels — the expanded overviews list always places the declared resolutions at the shard node — and declares at least one composable field (§4.5): a declaration whose fields are all none-class writes no artifact at all (not a morton-only column), and clears any prior one exactly as the no-levels arm below does. When it does exist it is written by the same worker invocation that commits the leaf, after the leaf's own stamp. The gate is the config the unit carries, not a store read: workers never open the manifest, so the manifest's pyramid block is the reader- and sweep-facing declaration and MAY lag a config-only change until the store is re-templated or retrofitted (ensure_manifest deliberately excludes pyramid from the keys it freezes, so a re-run into an existing store never re-PUTs it). A reader that needs to know which leaves actually carry columns therefore reads the columns, not the manifest. A run whose declaration carries no leaf-node levels deletes any column and sidecar a previous declaration left at that (leaf, window), so a column never outlives the declaration that wrote it. Like overviews, columns are derived artifacts a reader MUST NOT require; unlike overviews they are not regenerated by a rollup sweep — the writer of a column is its leaf's worker (no locking anywhere), and repair is re-invoking the idempotent leaf, never a sweep-side fold from raw cells during a rollup pass.

The one sanctioned second writer is the /1 -> /2 column backfill (#520): an explicit upgrade pass that recomputes a committed leaf's column from that leaf's own stored arrays, for stores built before their fields were declared composable — which is what lets a published store take the /2 staged sweep without re-aggregation. It writes the same artifact under the same discipline, so it inherits the single-writer law rather than repealing it, under two conditions a reader MAY assume of any store:

• it is serialized against every other sweep by the §4.8 admission lease, held for the whole pass and heartbeated on the wall clock (a beat lands within a fixed fraction of ttl_s of the last one, however long a single leaf takes). §4.8's expiry rule is the residual and is not repealed here: a holder that stalls past its ttl_s without beating is claimable like any other, so what a reader MAY assume is the lease's guarantee, not a stronger one; and
• it MUST NOT run while an aggregation run may write the same (leaf, window). This is the one pass for which §4.8's fleet ∥ sweep disjointness does not hold, and no control-plane object can enforce it: it is an operator precondition, and running a backfill against a live fleet risks a chimera column exactly as two concurrent sweeps would.

The backfill changes no byte of this format. A backfilled column is byte-identical to the one the leaf's worker would have written from the same leaf — the two provenance timestamps (zagg_column.generated_at and the stamp's written_at) aside — and it is written only where the manifest's own /2 declaration (§4.5) carries leaf-node levels and at least one composable field; a store still declaring /1, declared off, or class: "none" on every field is refused and MUST be re-declared first, never guessed at.

• Naming. One column per (node, window), and its basename MUST be {window stem}.pyramid.zarr — the stem derived from the D23 window alone (the §4.2 overview dialect), never from the leaf's own basename stem, so the rule is independent of the store's leaf-naming revision: the unwindowed / schedule: none leaf takes the reserved token (§4.2), giving the §7 column/ fixture's committed pair 11213.zarr and all.pyramid.zarr side by side, and a /1 windowed leaf 11213_2019.zarr is paired with 2019.pyramid.zarr. The .pyramid.zarr suffix is the one name seam, and it is normative for name-grammar consumers (e.g. the root-MOC walker): a basename ending in .pyramid.zarr MUST NOT be read as a leaf or an overview. The seam is unambiguous because the frozen D23 window-label charset [0-9A-Za-z-]{1,32} (the grammar §4.2 inherits, generative labels being digits) admits no ., so no legitimate leaf or overview basename can end that way; classification for everything else is the attrs below.
• Structure. One zarr group per resolution group, named by its cell order (the {order}/{field} layout of §4.4): every declared leaf resolution, every within-footprint rung of the fixed ladder (node < cells ≤ base), and the node-order member — cells == node, one cell: the leaf's whole-footprint aggregate. This tier MUST be contiguous from the finest declared resolution down to the node order (§4.4's contiguity rule; only the stage-merge levels below the shard order MAY gap). (There is no partial/ grammar; a coarse level declared later never rewrites a leaf; the ladder's merge levels cascade from the artifacts above them, §4.4, and read no column group). Each group holds the morton coordinate (the node's order-r descendant words, ascending) and one array per composable field (§4.5 classes; none fields are absent), plus every channel sibling that field's §4.5 entry declares — the {field}_locations sibling for a location entry (carrying the §9 declaration), the {field}_times sibling for a temporal entry (carrying the §8.3 declaration) — each row-aligned with its payload, exactly as an overview level does. Each pair is written together or not at all: the words are exact only given the centroid partition the payload describes (§9.1, §8.3), so a group holding a populated payload against an empty sibling is non-conformant, not merely short.
• Layout. Every array of a group is one object per leaf, and its chunking is a function of the group's own geometry (node, r) alone — never of the leaf's chunk_inner or of any manifest key — so a writer, a checker and an external reader with only the manifest derive it the same way (issue #631; zagg.column_shards):
• a group of fewer than 4^5 cells (r - node < 5) is single-chunk and unsharded: chunk_shape equals shape (4^(r - node) cells), no sharding_indexed codec. This is every column written before the rule and the whole default tier below — unchanged byte for byte;
• a group of 4^5 cells or more (r - node ≥ 5) is sharded: its arrays carry the sharding_indexed codec with the outer chunk equal to shape (still one object), index_location: end, index codecs [bytes(endian=little), crc32c] — the §1.5 leaf shard layout — and an inner chunk of min(4^5, 4^(r - node) / 4) cells: at most 1,024 cells (the largest power of four whose saturated signal digest stays under ~0.5 MB at the 128-centroid overview_delta cap — ~470 B per cell, measured on the v4 ATL03 rebuild) and at least four inner chunks per object (an object that is one inner chunk has no piece to fetch). The inner codec chain is the member's own: §1.3's vlen-bytes + zstd for a ragged payload or companion, the leaves' bytes-only chain for a dense array. An inner chunk holding only fill is absent from the index (the §1.5 sentinel), for dense arrays as for ragged ones.

On the 19/13/9 reference geometry with overviews: [16, 15, 14, 13]: order 16 is 16 inner chunks of 1,024 cells, 15 four of 1,024, 14 four of 256, and 13 one whole 256-cell chunk. The inner chunk is always a power of four, so an aligned subtree span (§1.5) never straddles one, and a reader sizes its GETs from the array metadata exactly as §1.5 requires for a leaf (whole object below 4^5 cells; index suffix plus covering inner chunks above). The rule is pinned on committed bytes by the §7 column_fine/ fixture; a column whose fine group is laid whole (a writer predating the rule) is non-conformant and is re-laid by the column backfill, which reads the layout as structure drift. Under the default [chunk_inner] declaration on the 19/13/9 reference geometry a column carries groups {13, 12, 11, 10, 9}, all below 4^5. - Fold laws. The leaf tier has a raw-fold boundary at resolution order + 2 (the constant zagg.column.RAW_MEMBER_DEPTH = 2; a constant, never a per-store knob — issue #538). Groups at or above it (cells ≥ order + 2) fold directly from the leaf's resident cell data — exact classes by their §4.5 merge law (nan-skipping, §4.3), approximate classes by the order-independent k-way digest merge, packed classes by §3.4 — and record merges_from_raw 1. Column bytes there MUST equal the sweep-kernel fold of the committed leaf's arrays at that resolution — the from-leaves parity contract, which reads differently per class: for exact fields it is checkable from this page (the §4.5 merge law plus §4.3's nan policy, so an external reader can reproduce a group by direct aggregation), while for approximate fields the merge algebra is zagg-owned and deliberately unspecified (§2.3), so the MUST binds implementations sharing those kernels and is pinned on committed bytes by the §7 column/ fixture rather than derivable from spec text. The groups below the boundary (order + 1 and order) fold flat from the boundary group: one k-way merge per output cell over the 4^(order + 2 − r) boundary cells it contains (approximate payloads with their §9/§8.3 channels in the same call; packed words from the boundary group's (word, n) pairs, n the boundary of digest's weight) — never chained through the intermediate group — and record merges_from_raw 2. Their bytes MUST equal that flat fold of the column's own boundary group. Exact classes are unaffected in value (they reduce from raw at every group; the reductions are associative). Rationale: a from-raw fold of the member d orders above the node merges 1/4^d of the shard's centroid rows in one call, and at the node member that is the whole shard — ~100 B of peak memory per row, the 0.52 fleet's OOMs at 4 GB — whereas the boundary bounds any single merge to one boundary cell — a sixteenth of the shard only under uniform occupancy, and the bound is the cell, not the fraction. Measured across the 39 fattest shards of the CA ATL03 store: the largest res-(order+2) cell holds 6.04 M rows, 9.8% of its 61.7 M-row shard, with a p50 of 9.6% and a maximum of 14.0% — ~0.6 GB at ~100 B per row, against the 4 GB worker tier. When the boundary is not strictly below the cell order (order + 2 ≥ cell order), or the column carries no group at order + 2 (a finest declared resolution of order + 1; under the §4.4 contiguity rule a validated declaration cannot skip the boundary while declaring something coarser — [12, 10] on the 19/13/9 geometry, members {12, 10, 9}, is refused, though a hand-built manifest may still carry it and membership stays the predicate), every group folds from raw with merges_from_raw 1 — the §7 column/ fixture's 4/6 geometry is the first case, which is why its groups below record 1. - The role and zagg_column attrs. role is "column"; zagg_column is the versioned provenance block, present exactly when role is "column":

"zagg_column": {
  "spec": "zagg-column/1",
  "node": "11213",
  "order": 4,
  "source_cell_order": 6,
  "window": "all",
  "fields": {"count": {"class": "exact", "method": "sum", "nan_policy": "skip"},
             "h_tdigest": {"class": "approximate", "method": "tdigest_kway",
                            "delta": 16, "overview_delta": 16,
                            "dtype": "float32", "inner_shape": [2]}},
  "groups": {"5": {"regime": "leaf-column", "merges_from_raw": 1, "n_cells": 4},
             "4": {"regime": "leaf-column", "merges_from_raw": 1, "n_cells": 1}},
  "cells_with_data_order": 5,
  "generated_at": "2026-08-05T00:00:00+00:00"
}

node is the leaf's morton decimal and order its (shard) order; source_cell_order the leaf's own cell order; window the §4.2 window key ("all" unwindowed — the basename and this key round-trip); cells_with_data_order names the group whose populated-cell count the commit stamp's cells_with_data records (the finest group). fields follows §4.3's materialized-fields contract (approximate entries additionally carry dtype/inner_shape/delta — enough to decode without the manifest — and overview_delta, the budget this column's fold actually compressed at, which is the split pyramid-fold budget rather than the leaf delta (issue #424; both are fold algebra, informative per §2.3). A reader MUST tolerate entry keys it does not bind, exactly as in §4.5). groups carries the per-group provenance slots: the fold regime ("leaf-column" — folded from the leaf's own resident cells; source_children never rides this regime, its source is complete by construction), the merges_from_raw integer (1 at or above the raw-fold boundary, 2 below it — on the 19/13/9 reference geometry {"13": 1, "12": 1, "11": 1, "10": 2, "9": 2}; the example above is the 4/6 fixture geometry, whose boundary is the cell order), and n_cells — the group's grid size 4^(r - order), i.e. its arrays' length, not its populated-cell count (that is the stamp's cells_with_data, for the cells_with_data_order group only). n_cells is derivable and recorded as a convenience. - Write discipline. The leaf's own D4 order: template (wholesale — the column prefix, and any stale stats sidecar, are deleted first) → every group's arrays → role/zagg_column attrs → one commit stamp last covering the whole column. The stamp is the D15 morton_hive_commit root-attrs block, the same key and grammar a leaf carries:

"morton_hive_commit": {"spec": "morton-hive/1", "complete": true,
                       "cells_with_data": 3, "granule_count": 1,
                       "content_hashes": {"arrays": {"…": "…"}, "combined": "…"},
                       "written_at": "2026-08-05T00:00:00+00:00"}

cells_with_data is the populated-cell count of the group named by cells_with_data_order; granule_count is the leaf's granule count, not a column quantity; content_hashes is the §5.3 record over the column's own arrays, keyed only when one was computed; and a column stamp carries no coverage payload (a leaf's does), so a stamp reader MUST NOT require one. On a windowed store the column's stamp is spec: "morton-hive/2" and carries the D15 half exactly as the leaf's does — window plus the observed time_range — so a reader that strict-checks the spec marker must accept both revisions here. An unstamped column prefix is debris and MUST be ignored; an idempotent re-run rewrites the whole column to the same array bytes (provenance timestamps move). The D20 stats sidecar — {stem}.stats.json, e.g. all.pyramid.stats.json, carrying the §5 record for the column's own arrays — is a sibling object PUT after the stamp, fail-open: absence reads unverifiable, never tampered. - Failure identity. A column-write failure fails the worker unit; the retry rewrites leaf and column wholesale. A committed leaf whose column is absent or unstamped therefore reads as one of three things: a torn worker; a leaf whose writing declaration carried no column (the gate and the clear above); or — since the backfill sanctioned above — a leaf that a /1 -> /2 upgrade pass has not reached, or failed on. The manifest cannot always separate them, since its pyramid block MAY lag, and after a re-declaration it necessarily leads the leaves. Readers never require a column, so absence is never an error state; the three differ only in their repair: - where the writing declaration is known to carry leaf-node levels, absence is the torn-worker signature and the repair is re-invoking the idempotent leaf; - where the leaf-node levels arrived by RE-DECLARATION — the /1 -> /2 upgrade, whose whole point is that the leaf is not rewritten — absence is the backfill's signature instead, and the repair is re-running the idempotent backfill. Re-invoking the leaf would re-aggregate from source granules, the one cost the upgrade exists to avoid; - and a leaf that predates any column declaration needs no repair at all.

A backfill counts a leaf it cannot read or fold and carries on, so a partially-completed pass leaves exactly this state on the store; what distinguishes it is the pass's own summary (failed), not the artifacts.

Stage columns (issue #384). The staged sweep writes the SAME artifact shape at its dispatch nodes ({window}.pyramid.zarr under an ancestor node's prefix, zagg-column/1 attrs, D4 order, one commit stamp last, D20 sidecar after): every group is a pure gather of the child columns' members at the same resolution — groups entries record regime: "stage-gather" with the gathered member's merges_from_raw (the deepest the child columns record for that group) — and the artifact carries exactly the members some coarser gather level consumes (the cells at or above the shard order declared at orders below the dispatch node's). It carries no member for the merge levels: those cascade from the ladder artifacts (§4.4), so a dispatch node below which nothing gathers writes no column at all. (Before issue #620 every stage column also relayed the leaf columns' res-shard_order + 2 member upward for the merges; a /2 store swept before then may still carry that group, which nothing reads.) Stage-column attrs additionally carry generation ({n_leaves, max_leaf_timestamp, run_ids} summed over the consumed children — the parent's skip-gate key, §4.5), source_children (a gather that under-covered says so in the artifact), and run_id; the commit stamp carries run_id too. Cadence decides placement (columns sit at dispatch orders), so column EXISTENCE at a given ancestor order is orchestration, never contract — a reader binds to the ladder artifacts of §4.4, not to stage columns. The root tuple writes no column (nothing consumes it). A stage column's chunking is orchestration too (issue #586): a resolution group of at most 4^5 cells (res − node ≤ 5) is one chunk per array — as a leaf column's is below 4^5 cells; at exactly 4^5 the leaf column shards (§4.6 Layout) while the stage column stays one chunk; a wider group is laid on regular inner chunks of 4^5 cells, one object per chunk and no ShardingCodec, because the staged sweep gathers, writes and reads such a group one chunk at a time (it never holds a level whole). The arrays, their values, the attrs and the §5 record are the same either way — the O11 hash is over decoded values. On a windowed store every window has its own stage column ({window}.pyramid.zarr), written by that window's own stage unit; there is no all-time stage column (a column relays gen-1 content). Raster hive stores are column-less by construction: nothing in this section applies to them (issue #399 owns their overview regime).

4.7 What §4 does not cover (informative)

The sweep's other derived-artifact families (MOC regeneration, stats and sub-shardmap rollups, debris collection) are operational concerns recorded in design/sparse_coverage.md D22 — they add no new byte layouts (the stats rollup reuses the D20 sidecar schema; the sub-shardmap is ShardMap JSON) — except the §4.8 sweep-admission lease, which is control plane rather than data. The fold algebra for overview contents is zagg-owned per §2.3; a reader consumes overview arrays exactly as it consumes leaf arrays.

The staged sweep's run records (informative). A staged sweep leaves one run record at the store root, sweep_stats_{ts}_stages.json, and — on the fleet — one record per stage invoke plus the finisher's under the run's status prefix (zagg-sweep-stage-record/1). They are telemetry, not conformance material, but two of their keys are relied on by tooling and are stated here:

• run_id is the SWEEP's own identity (the §4.8 lease, the stage stamps' skip-key term). pipeline_run_id (issue #593) is a different thing: the id of the aggregation run the sweep completes — the run whose leaves it folded — stamped into the stage event by the dispatcher that chained the sweep and carried verbatim into every stage record and the root record. The key is always present; null means the sweep named no run and vouches for none (a standalone python -m zagg.sweep --stages pass, unless it is given --pipeline-run-id). A consumer deciding whether run R's ladder was built MUST match pipeline_run_id == R; the record's time alone does not link it to a run.
• A stage record's unit / window (issue #586) say which share of its dispatch nodes the invoke ran: "window" with the window label — one (node, window) fold; "close" — the per-node step that follows a node's window units (the all-time fold of §4.4); null — the nodes whole, which is the only form an unwindowed store has. Its per-tuple rows count window_units and close_units, name a unit that raised in unit_errors, and report the streamed fold's accounting (fold_blocks, fold_cells_read, fold_peak_cells — the most source cells any one fold block held). closes is whether the store's nodes take a close, as the worker read it off the store manifest (a windowed store declaring pyramid.overview.all_time). The fleet dispatcher fires the close units from it, not from its run config: pyramid is not a frozen manifest key, so the two can disagree, and the store is the source of truth.
• A stage record's families (issue #610) is present when the rollup families rode the same walk as the ladder (§4.7's first paragraph: they add no byte layouts). It maps each family's name to that family's block — its per-pass counts, the dispatch nodes whose span failed (node_failures), and, for moc, the §10 accumulator block the run's finisher composes the root §10.2/§10.3 section and its §10.5 sibling from, the same block a families-pass partition record carries, in the same currency. Every key of the mapping is a family name, so a reader may walk it without an allow-list. Still telemetry: the objects it composes are normative, this record is not, and the finisher reads which families rode off these records rather than off its own invoke.

4.8 The sweep-admission lease (zagg-sweep-lease/1)

Status: contract — issue #384 (espg admission ruling). Pyramid sweeps serialize per store: a column is a multi-object artifact whose D4 stamp-last discipline proves completeness only under a single writer, so two concurrent sweeps could interleave PUTs into a chimera column that validates as complete. Admission is one atomic conditional PUT (If-None-Match: *) of the store-root intent object sweep.lease.json:

{
  "spec": "zagg-sweep-lease/1",
  "run_id": "stage-20260809T000000Z-ab12cd",
  "scope": null,
  "acquired_at": "2026-08-09T00:00:00+00:00",
  "heartbeat_at": "2026-08-09T00:05:00+00:00",
  "ttl_s": 900,
  "claimed_from": "stage-20260808T230000Z-9f00aa"
}

— scope is the admitted run's node-prefix set (informative; the lease is store-granular by correctness: scope-disjointness does not imply write-disjointness, because disjoint-leaf sweeps converge on shared coarse ancestors). A live intent refuses admission naming the runner; a heartbeat_at older than ttl_s is claimable (crash recovery — claimed_from records the takeover), and the claimant simply completes the partial prior run under the ratchet. The finisher deletes the intent as its final act. Control plane, explicitly: no data object is ever locked — the lease is what makes "every data object has exactly one writer, ever" true across runs, extending (never amending) the no-locking law. Within one admitted run the single-writer law holds per object: on a windowed store a dispatch node's windows are folded by concurrent stage units (issue #586), each of which writes only its own window's artifacts and reads its skip key from the artifact it alone writes, so no object — the sweep-internal node envelope included, which a windowed store does not have — is shared between them. Fleets are unaffected: fleet ∥ fleet is governed by the leaf single-writer law and fleet ∥ sweep is allowed (the stage workers validate every column stamp before and after reading its groups and re-read on movement until a read has been served; after that a moved or vanished stamp fails the artifact being folded, which is folded once more from fresh reads and otherwise counted failed, never assembled from two writes; stage stamps carry run_id, and a skip-if-current read that sees a foreign stamp written after the run started aborts loudly). The same run_id is a term of the skip key (§4.5): the abort covers a foreign stamp written since this run started, and the key covers the foreign rewrite that landed before it — inside the second the timestamp cannot resolve.

4.9 The multiscales discovery mirror (zagg-multiscales/1)

Status: contract — issue #392 (espg re-scope ruling 2026-09-10: zagg-native convention metadata only). A zagg-pyramid/2 store is inherently a multiresolution grid (§4 intro), but that identity is discoverable only through §4.5's own grammar. The manifest therefore ALSO carries a machine-readable convention declaration of the ladder — the OME/zarr-style multiscales attr grammar adapted to the node-tree reality — so a reader discovers the whole ladder from one metadata read (morton_hive.json).

Contract. The manifest key multiscales (a top-level sibling of pyramid) is present exactly when the manifest's pyramid block declares zagg-pyramid/2: a /1, declared-off, or pre-pyramid manifest carries no multiscales key, and its absence means pre-convention (or single-resolution) store — never an error, and no spec marker bump anywhere. The value is a list of one multiscale object (the OME-style shape: one declared product, one entry):

"multiscales": [
  {
    "spec": "zagg-multiscales/1",
    "name": "SPEC_FIXTURE",
    "base": {"order": 3, "cells": [6]},
    "datasets": [
      {"order": 3, "cells": [5, 4], "artifact": "column"},
      {"order": 2, "cells": [3], "artifact": "overview"},
      {"order": 1, "cells": [2], "artifact": "overview"},
      {"order": 0, "cells": [1], "artifact": "overview"}
    ],
    "order2res": {"3": [5, 4], "2": [3], "1": [2], "0": [1]},
    "fields": {"count": "exact", "h_min": "exact",
               "h_tdigest": "approximate", "h_mean": "none"},
    "fold": {"fold_source": "cascade", "exact_levels": 1}
  }
]

• spec — the revision marker; conformance rule as everywhere (strict-check, fail loudly on an unknown revision).
• name — the manifest's dataset.short_name (null when the manifest records none): identity, not grammar.
• datasets — one entry per §4.4 level entry, finest first, in the recorded pyramid.overviews order: order and cells are copied verbatim from the level entry (cells keeps §4.5's vocabulary — the reader-facing cell resolutions stored at that node order), and artifact names the kind that carries the level — "column" for the leaf entry (node == shard_order, the §4.6 per-leaf {window}.pyramid.zarr), "overview" for every ladder entry (the §4.1 ancestor-node {window}.zarr). Artifact paths are not recorded here: node addressing stays the manifest + mortie hive-path grammar, and declared-but-unmaterialized remains legal (§4.5), so a path list would claim presence this block cannot promise.
• order2res — the flat per-order lookup {str(order): cells} (JSON has no integer keys): the cell resolutions of the pyramid levels at order k, whose key set is exactly the orders present. It is a projection of datasets and MUST agree with it entry for entry. It does not carry the base's native cells: those ride in base alone, which shares the shard order's key (in the example above, order2res["3"] is [5, 4] while base adds 6 at that same order 3). So the resolutions readable at order k are order2res[str(k)] unioned with base["cells"] when k == base["order"] — a reader assembling a complete per-order map MUST union the base in, never overwrite the entry with it.
• base — the native source data ({"order": shard_order, "cells": [cell_order]}): the finest rung of the resolution ladder a reader picks from, deliberately not a datasets entry — it is not a pyramid level (§4.5: a member at the base data's own order would be the base data).
• fields — {name: class} projected from the §4.5 D24 map: the zero-open composability answer (which fields exist at every ladder order, which are exact there, which exist only at native resolution). The class tokens and their meaning are §4.5's; the accuracy contract per class is §4.4's doctrine.
• fold — the declared fold provenance: fold_source always, and exact_levels exactly when the pyramid family dict declares it (§4.5's presence rule — the deprecated leaves regime records no boundary).

Derivation and precedence. The mirror is derived, never authoritative: zagg computes it from the pyramid block in the same write that installs that block (hive.build_manifest at template time; declare_pyramid on retrofit, which also installs the mirror on an already-/2 store whose declaration predates this section, and removes it when a retrofit leaves /2). On ANY disagreement between this block and the pyramid block, the pyramid block wins — a reader that consumes the mirror MAY cross-check it against pyramid.overviews and MUST resolve a conflict in the pyramid block's favor. Sweep actuals never enter the mirror: materialization truth stays in §4.5's actuals/materialized bookkeeping and §4.3's per-artifact attrs. A reader MUST tolerate additional keys on the multiscale object and on its entries (additive grammar, same rule as §4.5).

4.10 The multiscales companion group

Status: contract — issue #394 (espg re-scope ruling 2026-09-10: zagg-native, references only — never data copies). The §4.9 mirror makes the ladder discoverable; the companion group makes it walkable by stock zarr tooling: one literal zarr v3 group hierarchy at the reserved store-root path multiscales/, mirroring the coarse ladder — one child group per §4.4 ladder order, shard_order - 1 down to 0 — that a plain zarr reader opens with zero custom code. It holds metadata only: group documents and attrs. No arrays and no data bytes — every entry references node artifacts by store-root-relative path.

Contract.

• Layout. multiscales/zarr.json is a zarr v3 group document whose attributes carry the §4.9 multiscales list (derived fresh from the manifest's pyramid block at write time — the block wins over any recorded copy, §4.9) plus the zagg_multiscales provenance stamp, and whose consolidated_metadata block (kind: "inline", must_understand: false) inlines every child group document identically as JSON — the inlined copy and the standalone multiscales/{order}/zarr.json decode to equal documents, not to equal bytes (the nested copy is indented deeper), so one GET of that object walks the whole tree. Each ladder order additionally has its own multiscales/{order}/zarr.json group document, so non-consolidated walkers work too. Write order is child documents first, the root document LAST — the root doc is the commit marker; a prefix without it is debris.
• The root stamp —

"zagg_multiscales": {
  "spec": "zagg-multiscales/1",
  "window": "all",
  "members_source": "coverage.moc",
  "generated_at": "2026-09-11T00:00:00+00:00",
  "members_generated_at": "2026-09-10T23:12:04+00:00"
}

window is the §4.2 window token the member references resolve — "all" only in this revision: the companion mirrors the all-time fold (per-window mirrors are a declared non-goal of /1, issue #394 decision (2)). members_source records where the member node set came from: "coverage.moc" (the root MOC, one GET) or "run-records" (the D22 discovery fallback). generated_at is when the companion was WRITTEN; members_generated_at is the age of the INPUT the member sets were derived from — the root MOC envelope's own generated_at, present exactly when members_source is "coverage.moc" and that envelope carries the field. The two together are the staleness discriminator (see below). A reader MUST tolerate its absence (a "run-records" companion, or a MOC that carries no stamp). - Per-order groups — the child group named str(order) carries

"zagg_multiscales_level": {
  "spec": "zagg-multiscales/1",
  "order": 2, "cells": [3], "artifact": "overview", "window": "all",
  "members": {"-311": "-3/1/1/all.zarr/3"}
}

order/cells/artifact are the §4.9 dataset entry verbatim; members maps each node decimal the ladder owns at this order — the order-k ancestors of the store's occupied shards — to the store-root-relative path of that node's resolution group: {hive_path(node)}/all.zarr/{cells[0]} (path grammar: mortie's hive-path convention; a ladder entry has exactly one member resolution, §4.4). The leaf entry and the native resolution are deliberately not mirrored: their artifact count is the shard count, which is exactly the scale the hive grammar exists to handle — the companion is the coarse, cap-bounded tier (issue #394 decision (1)); both remain declared in the root attrs' §4.9 list. - References claim ownership, never presence. A member names where the node's overview artifact lives when materialized; overviews are regenerable caches (§4.1) and declared-but-unswept is legal (§4.5), so a reader following a member path MUST treat an absent or unstamped artifact as "not materialized", never as a broken store. An empty members map (a declared-but-unoccupied store) is likewise legal. - Presence and gating. The companion is written for a /2 store whose all-time fold exists by declaration: an unwindowed store always (its single fold IS all-time, §4.2), a windowed store only when pyramid.overview.all_time is true. Absence of the whole group is always legal (a pre-convention store, or a gated windowed store). The name multiscales is reserved at the store root: it is excluded from the D19 product-name grammar (like the base-component exclusion) so a multi-product root walker can never classify it as a product. The reservation is retroactive on a store predating it that holds a product literally named multiscales: that product is no longer discoverable or addressable and MUST be renamed — but the exclusion is reported, not silent (a product carries a morton_hive.json at its root and the companion group never does, so discovery discriminates the two and warns on the collision). - Writers. The staged sweep's designated finisher refreshes the companion after its manifest RMW, gated on the run having touched shards and fail-open (the finisher's load-bearing steps never fail on it); python -m zagg.sweep <root> --write-multiscales writes/refreshes it standalone. Single-writer discipline is inherited from the finisher's own (§4.8: one finisher per admitted run). - Removal. A store that stops declaring /2 MUST NOT keep a readable companion: the group asserts the ladder without consulting the manifest, so the §4.9 precedence rule has no reader to apply. The same write that drops the §4.9 mirror (declare_pyramid, on a retrofit that leaves /2) therefore deletes multiscales/zarr.json — the commit marker, so removing it alone un-commits the tree and any surviving child prefix is debris a later companion write overwrites. The delete is best-effort (the manifest is truth and is already written); a failure is logged, not unwound. - Normativity and staleness. The manifest + hive remain truth; the companion is a recorded mirror (D9 cache class): regenerable at any time, self-healing on the sweep ratchet (the finisher rewrites it each admitted run), and stale-detectable — but the test is members_generated_at, not generated_at: the write-time stamp only says when the mirror was last rewritten, so a companion derived from a stale root MOC carries a fresh generated_at. A members_generated_at older than the manifest's pyramid declaration or the newest §4.5 actuals means the member sets may lag store truth; its absence means the input age is unknown and the member sets carry no freshness proof at all. On any disagreement the manifest wins, exactly as §4.9. A reader MUST tolerate additional keys in both attrs blocks.

5. O11 content hashes

Status: contract — frozen on #342. The recipe was pinned by the moczarr verify reader (espg/moczarr PR #23, moczarr.stats.hash_arrays / combined_hash); zagg's writer, when it lands (#342), MUST adopt it verbatim. The O11 decision record (scope, compute-at- write, exact-bytes rationale) is design/sparse_coverage.md §8.2 O11.

The logical content hash of a leaf is per-array sha256 over decoded values — never stored object bytes, so codec and packaging changes (ShardingCodec inner chunks, compressor upgrades, §1.5 geometry) are invisible by construction, while any value change flips the hash (exact bytes, no float tolerance — interpretation pairs the hash with the recorded zagg version).

5.1 Scope and keys

Contract. The hash set covers every named zarr array beneath the leaf root — data fields, the ragged vlen payload arrays and their {field}_locations siblings, morton (where the leaf stores it — a windowed leaf does not, §1.5), every coordinate — keyed by the array's path relative to the leaf root (e.g. "8/morton").

The scope is therefore discovery-based on the verifier side: a verifier (moczarr, zagg's parity checks) enumerates (group.members(max_depth=None)), so the key set it hashes is whatever named arrays exist under the leaf root, not what the template declared. The leaf and leaf-column writers hash the node set they have just created — the arrays of their own template pass, written under a prefix they cleared immediately before ([issue

657](englacial/zagg#657); zero metadata reads) —

and under that cleared prefix the created set is the enumerated set, so the key set and every per-array digest are the same either way. The overview and stage-sweep writers still enumerate (group.members) before hashing; moving them onto their own node set is a follow-on. Their stamps carry the same §5.3 record either way. That has one normative consequence a verifier MUST honour: a key-set difference is a distinct outcome from a digest mismatch. Debris inside a leaf — a foreign array prefix, the issue #341 Bug A class, an issue #327 .zarr.status/ prefix — adds a key and so changes combined, which means a verifier comparing combined alone reports an intact leaf as tampered. A verifier MUST compare the per-array map first and report extra or missing keys as their own outcome ("extra array present" / "array missing"), reserving "mismatch" for a differing digest on a shared key. Verification and the overview and stage-sweep writers are the only paths that enumerate: the leaf and leaf-column writers hash their own node set, and the read and fold paths open leaf arrays by name (no member enumeration, no LIST — #344), so debris is inert everywhere else.

5.2 Per-array recipe

Contract.

• Fixed-width arrays: sha256 over the array's full decoded contents as raw C-order little-endian bytes at the declared dtype (a big-endian dtype is byte-swapped to little-endian before hashing).
• Vlen (ragged) arrays: an object-dtype array has no flat buffer (ndarray.tobytes() on it would serialize per-process pointer addresses). It is hashed instead as, over cells in flat C order:

sha256( for each cell:  u64_le(len(payload)) || payload )

where payload is the cell's decoded bytes — exactly the §1.4 payload_bytes for a zagg-ragged/1 array (an empty or unwritten cell contributes its zero length; a locations sibling's payload is its raw little-endian uint64 words). The 8-byte length prefix is what makes the digest injective ([b"ab", b"c"] and [b"a", b"bc"] must not collide), and it covers the cell grid, not just the payloads.

The element → bytes normalization is itself normative, because a /2 (§6) cell decodes to an ndarray rather than to bytes:

decoded element	payload bytes
None — an unwritten vlen cell may decode as None, not b""	zero-length
bytes / bytearray / memoryview (a /1 cell)	as-is
str (a vlen-utf8 future)	UTF-8 encoded
ndarray (a typed /2 cell)	C-contiguous, little-endian bytes at the declared element dtype
anything else	the recipe does not apply: a verifier MUST raise rather than hash

The last row is deliberate: a digest that is silently wrong is worse than no digest, so hashing a repr or a pointer buffer is forbidden. None and b"" hash identically by construction — a b"" fill is distinguishable from a missing cell only by position, which is the intent.

5.3 Combined hash and sidecar record

Contract. The combined hash is sha256 over the sorted per-array hex digests joined by "\n", hashed as ASCII — array names deliberately excluded ("hash of the sorted per-array hashes").

(Informative.) Because it sorts the digests, combined is immune to the order in which the §5.1 enumeration happened to yield arrays — a real robustness property, and the reason two implementations agree without agreeing on traversal order. The recorded arrays map is a different matter: a writer SHOULD record it key-sorted so a regenerated record diffs cleanly.

The hashes are recorded in the leaf's commit stamp (morton_hive_commit, issue #580) and in its D20 stats sidecar, both under content_hashes, in the structured shape:

"content_hashes": {
  "arrays": {"8/count": "…", "8/h_tdigest": "…", "8/morton": "…"},
  "combined": "…"
}

A windowed leaf's key set has no {cell_order}/morton entry — in the stamp and the sidecar alike — because the array is not stored (§1.5 "The cell coordinate"). The scope is discovery-based (§5.1), so this follows from the array's absence and is not a second rule; a verifier holding a record of the same window from a pre-revision writer reports the difference as §5.1's "array missing" outcome, never as a mismatch.

A writer MUST emit the structured shape. A reader SHOULD also accept the flat shape ({array_key: hash, "combined": hash} — combined is reserved and is not a legal zagg array name). A leaf with no recorded content_hashes is unverifiable, not tampered: verification MUST report "nothing recorded" as a distinct outcome from a mismatch (the conservative dedup posture — an unverifiable leaf is never a hit).

(Informative.) Beyond content_hashes and cells_with_data (§7), the sidecar's remaining keys are D20 telemetry, not conformance material. Two of them are clocks: duration_s is the worker's read + index + aggregate wall, and duration_total_s (issue #589) the whole unit's — handler entry to the record — which gb_seconds / est_cost_usd price from. A record without the key (the column sidecar, failure rows, older writers) is priced from duration_s. Pricing happens only where a Lambda config is present: local records carry the total, but gb_seconds stays null.

Contract (the stamp copy). The writer computes the record from the arrays it just wrote, before the stamp lands, so the D4 seal certifies the digest of the bytes it seals; the stamp copy and the sidecar copy are the same record. A reader verifying a leaf SHOULD prefer the stamp's copy (one root-metadata GET, already read for the stamp) and fall back to the sidecar for leaves written before this key existed. A stamp without the key is a pre-#580 writer or a writer that could not stand behind a digest (§5.2's raise gate) — unverifiable, never tampered.

Contract. A sweep-built overview (§4) records its hashes in a sidecar the same way, and that sidecar is named from the overview's own basename — the stem plus .stats.json (all.zarr → all.stats.json, 2019.zarr → 2019.stats.json), a sibling object at the ancestor node. Unlike a source leaf's sidecar name, which is keyed to the store's spec revision, the stem grammar applies to an overview sidecar unconditionally, at every revision: one ancestor node holds every window's overview (§4.2), so a revision-keyed bare name would resolve all of them to a single stats.json at that node.

(Informative.) The O11 hash is the verification half of the D19 identity split — the semantic_hash says two leaves were intended identical; O11 says they are byte-identical — and doubles as the mismatch localizer ("only h_tdigest differs in this leaf") and the detection mechanism for stamped-but-torn leaves under the concurrency contract's out-of-contract case.

(Informative.) The semantic_hash itself is not part of this specification: its derivation is the D19 record (design/sparse_coverage.md) as implemented by zagg.semantics, it is subject to hash epochs (a change to the canonicalized subset moves every digest by design — the 2026-08-17 epoch, issue #415, and the 2026-09-13 index-exclusion epoch, issue #499, under which data_source.index is read machinery outside the core), and no fixture's *.expected.json records one — the committed §7 fixture manifests do carry the digest, and tests/test_spec_conformance.py reproduces each from its build config, so an epoch that moves a fixture config's digest regenerates those manifests. A reader should treat it as an opaque frozen manifest key: compare it for equality rather than recomputing it. Operator consequences of an epoch live in hive_layout.md.

6. zagg-ragged/2

Status: specified, implementation pending (#210; timing ratified — the dtype package ships on its own release train, the zagg writer knob and reader dispatch are gated on 1.0). /2 adds to /1, it does not replace it: /1 is the pinned 1.0 wire contract (§1), existing stores keep it forever, and every conforming reader supports /1 unconditionally.

/2 moves the element declaration out of the §1.2 attrs block and into the zarr data type itself: a parameterized typed vlen dtype, so a generic zarr stack knows the element interpretation without zagg's attrs convention.

6.1 The typed dtype

Contract. The /2 data type is the registered zarr v3 extension vlen-ndarray (espg-ratified name; reference implementation: the zarr-vlen-ndarray package under github.com/espg), parameterized by the element dtype and trailing inner shape — exactly the pair the /1 attrs block declares:

• element dtype float32, inner shape (2,) for a digest payload array;
• element dtype uint64, inner shape () for a locations sibling (§2.2), and the same pair for a {field}_times temporal sibling (§8.3) — the two companions are one dtype, differing only in the declaration they carry.

A cell's logical value is the (n, *inner_shape) array itself rather than its raw bytes; everything else about the array — shape, cells axis, fill_value (the empty cell), located sibling alignment (§1.1), storage geometries (§1.5) — is unchanged from §1.

6.2 Byte identity

Contract. The /2 codec chain MUST produce chunk bytes byte-identical to /1's: the §1.4 wire framing and the §1.3 […, zstd(level=3, checksum=false)] chain are unchanged, with the typed array↔bytes codec serializing each cell as the same np.ascontiguousarray(value).tobytes() little-endian payload. The typed dtype changes interpretation only, never stored bytes. Consequences (the point of the revision):

• migrating an existing /1 store to /2 is a metadata-only rewrite (zarr.json objects; no data object is touched);
• the §7 conformance fixtures serve both revisions — a /1 fixture's chunk objects re-labeled /2 MUST decode identically through the typed path;
• the §5 O11 vlen recipe is unaffected (it hashes decoded payload bytes, which are identical by construction) — this is exactly what §5.2's element→bytes normalization buys: a /2 cell decodes to an ndarray, whose C-contiguous little-endian bytes are the /1 cell's bytes.

6.3 Revision signaling

Contract (restating §1.6 from the /2 side):

• An array whose zarr data type is vlen-ndarray is zagg-ragged/2; the ragged attrs marker is retired on such arrays (not written). The element declaration lives in the dtype configuration alone — a reader MUST NOT require the attrs block on a /2 array. (§8.3's times binding and §2.0's weights are already top-level keys beside the block, so they ride the migration unchanged — this clause is about the bindings that live inside it.)
• A located /2 payload array still declares its sibling binding in metadata, never by naming convention (the §1.2 rule survives the revision), and it does so under a new top-level attrs key — not a residual ragged block with spec/element dropped. "Retired" is literal: no ragged key is written on a /2 array. That is a ruling, not a leftover choice — a ragged block carrying only locations would be, by §1.2's own words, an array with no well-formed element declaration, which a /1-only reader MUST refuse with a pointed "not a zagg-ragged/1 array" error, i.e. the misleading path instead of the actionable "install zarr-vlen-ndarray" one below. The ragged key stays reserved but unwritten under /2 (config-declared attrs still MUST NOT shadow it), and the new key's exact name is for the /2 implementation PR to pin in this section before any /2 store exists. The sibling-alignment semantics of §1.1 are unchanged.
• An array with the variable_length_bytes/bytes dtype and a spec: "zagg-ragged/1" attrs block is zagg-ragged/1.
• A reader without the vlen-ndarray extension installed MUST surface an actionable "install zarr-vlen-ndarray to read this store" failure, not a silent mis-decode (and cannot half-parse: the dtype is unknown to its zarr stack by construction).

(Informative.) Writing /2 will be a per-product opt-in (output.ragged_encoding: typed), which shifts the product's semantic_hash — a new product identity, by design. That shift is not automatic: output.* keys reach the semantic core only by being listed as leaf-shaping (zagg.semantics.OUTPUT_LEAF_SHAPING_KEYS, issue #415), so the /2 implementation PR must add the knob there in the same change. The default stays /1; flipping it is a schema epoch deferred to its own ruling (public/interop stores may deliberately stay /1 for vanilla-zarr openability).

7. Conformance fixtures

Status: contract. The committed stores under tests/data/spec/ are part of this specification: a reader implementation that reproduces their expected decoded values and content hashes conforms to §1–§3, §5, §8, §9 and §10. They are generated by tools/generate_spec_fixtures.py through zagg's production write path (manifest, sharded leaf template, dense + ragged writes, coverage sidecar, commit stamp), so writer↔spec drift fails zagg's own suite (tests/test_spec_conformance.py) on whichever side moved. moczarr vendors the same fixtures for its parity gates (espg/moczarr#19/#20).

Every fixture that declares count was regenerated for issue #626 (count is int64, §4.4/§4.5), so those stamps carry the §5.3 copy of content_hashes. Every committed dggs block carries the §1 latitude token (since #616), so §1's absent-token rule has no committed golden; §1's evidence paragraph is its record. raster_toc/ (no count) is unregenerated: its stamp without the §5.3 copy is the committed absent-key ⇒ pre-#580 pin (the sidecar copy and §1's own evidence paragraph are the record for what that artifact means). The one versioned leaf (§1.5; a stamp naming current) is versioned/ below, added standalone with the writer that produces it (issue #582); the other leaf fixtures are unversioned and pin the absent-current rule. Two older-era pins are produced deliberately by the generator (its STALE BY DESIGN note): column/'s builder drops the §4.9 multiscales mirror from its manifest, and kitchen_sink/'s builder strips the §9 located attrs from its two leaf *_locations siblings, so a regeneration reproduces both.

Ten tiny single-shard hive stores plus four metadata-only ones (the pyramid/ declaration, the multiscales/ companion, the icechunk/ repo root and the uncounted/ root objects), all but one on the same deliberately small geometry — shard order 4, inner-chunk order 5, cell order 6 (16 cells, K = 4 inner chunks of 4 cells), sharded (the hive default; raster_toc/ is the one exception — a (time, cells) product is never sharded, §8/#247; column_fine/ keeps the shard order and deepens the cell order to 10, the smallest leaf that can declare a §4.6 fine tier):

• minimal/ — one unlocated digest field (h_tdigest) plus count. The smallest thing that is a conforming store.
• versioned/ — minimal/'s inputs written as a versioned leaf (§1.5, issue #582): the stable root {id}.zarr/zarr.json is a pointer stamp naming current (run-{run_id}-{attempt}), and the arrays, the coverage sidecar and the version's own stamp sit in that subgroup — nothing else at the root. versioned.expected.json records pointer (the root), version (the current value) and run_id, with leaf pointing at the version so every leaf-shaped assertion applies unchanged; the conformance suite decodes the pointer from the on-disk JSON alone and pins that one reader rule (open the root; follow current when named, else read the root) resolves minimal/ and versioned/ alike. The pointer's spec is the legacy token (/1): versioning is the current key, never a token.
• flux/ — the §2.0 weights declaration surface: one flux-declared digest field (rx_flux, weights: "flux" stamped beside the ragged block, gain provenance attrs) plus count. Its payloads carry fractional positive weights whose per-cell sums are not integers — the pin that a flux reader must not round-trip weights through counts — while minimal/, which declares no weights, pins the absent-key ⇒ "counts" default.
• kitchen_sink/ — the full stratified-product surface: located signal/noise digest strata (payload + {field}_locations siblings, stratum/signal_threshold provenance attrs), the composition word (including a single-photon cell packing the §3.1 golden word 0xFF000000FF0000FF and a noise-only cell whose signal payload is the empty (0, 2) array), and count.
• pyramid/ — MANIFEST ONLY: the §4.5 zagg-pyramid/2 declaration grammar. The committed morton_hive.json was produced by the production declaration paths end to end — templated /1 (hive.build_manifest), given sweep actuals by the production bookkeeping writer, then retrofitted to /2 with declare_pyramid — and carries every /2 reading a decoder must tell apart: a multi-resolution leaf entry ({"node": 3, "cells": [5, 4]} — this fixture is shard order 3, not the leaf fixtures' 4, so the leaf window has two interior resolutions), the §4.4 fixed every-order ladder rooted at node 0, the #376 fold keys (fold_source, exact_levels), and the preserved /1-era materialized.fold_sources actuals — and, via the committed pyramid.expected.json, which records the raw config knob, the leaf-resolution declaration the expansion was derived from. It writes no store beneath it on purpose: the pyramid block is a template-time manifest artifact, decodable from morton_hive.json alone — the /2 artifacts a fleet writes are column/'s job below (sweep-side levels are #384's). The same manifest carries the §4.9 multiscales discovery mirror the retrofit installs beside a /2 block; column/'s manifest, whose mirror the generator drops on purpose (stale by design), is the absent-key ⇒ pre-convention pin.
• column/ — the minimal/ inputs plus an explicit output.pyramid.overviews: 5 knob, so the same worker invocation that committed the leaf also wrote its §4.6 column: all.pyramid.zarr beside the leaf, groups {5, 4} (the declared base and the node-order member — this geometry has no interior ladder rung between them), role: column + the zagg_column attrs grammar, its own commit stamp, and the all.pyramid.stats.json D20 sidecar. column.expected.json records the decoded group values, the attrs block verbatim, the stamp's clock-free fields, and the column's §5 hashes; the conformance tests additionally re-derive the base group from the committed leaf through the §4.4 fold kernels — the §4.6 from-leaves parity contract, pinned on committed bytes. The two-group set is forced, not chosen: on this ⅘/6 geometry §4.4 admits only overviews: 5, so no committed golden here can carry an interior ladder rung (a three-or-more-group column, its group ordering, or a declared base distinct from an implied rung). That case is pinned zagg-side by tests/test_column.py; a committed multi-rung golden arrives with the sweep-side fixtures of #384.

• column_fine/ — the §4.6 fine-tier layout surface (#631): the minimal/ fields on a deeper leaf — the one fixture not on the ⅘/6 geometry: shard order 4 (the same 11213 key), inner-chunk order 7, cell order 10 (4,096 cells, K = 64 inner chunks of 64 cells) — under output.pyramid.overviews: 9, the smallest declarable fine tier (a base at node + 5 is a 4^5-cell group, and §4.4's contiguity rule caps the base at 2 · node + 1; both hold only from node 4 up). The column carries groups {9, 8, 7, 6, 5, 4}; group 9 is sharded — each array one sharding_indexed object on four 256-cell inner chunks, index at the end — and group 8 and below are one whole unsharded chunk each. The four populated leaf cells (0, 2, 1,089, 3,263) fold to group-9 rows 0, 0, 272 and 815, so in the count and h_tdigest objects inner chunks 0, 1 and 3 are present and inner chunk 2 is absent from the shard index (the §1.5 sentinel on a column, for the dense array as for the ragged one), while the morton coordinate — written for every cell — holds all four. column_fine.expected.json carries the column/ record plus column.layout — per group {sharded, chunk_cells} spelled from the §4.6 formula, never read back — and the conformance tests decode group 9 from the committed bytes by the §1.5 index recipe alone (suffix, ranged inner chunks, zstd + §1.4 framing for the digest, raw little-endian words for morton and count) and hold every group's zarr.json to the rule. Its groups 5 and 4 are byte-identical to column/'s (the same observations fold to the same res-5 rows), which pins that the deeper leaf and the fine-tier fold leave the coarse groups' values unchanged (groups 5 and 4 fold flat from the unsharded member 6 and never read group 9, so layout parity is pinned zagg-side by tests/test_column_shards.py::TestLayoutParity).

• multiscales/ — the §4.10 companion-group surface: METADATA ONLY — a zagg-pyramid/2 manifest on pyramid/'s grid, a production root coverage.moc naming two occupied order-3 shards, and the companion tree at the reserved root path multiscales/: a consolidated root group document (the §4.9 mirror + the zagg_multiscales stamp) plus one child group per coarse ladder order, each mapping the order's ancestor nodes to store-root-relative resolution-group paths. No arrays, no leaves, no overview artifacts — the committed golden pins that §4.10 references claim ownership, never presence, and that the tree opens with stock zarr alone.

• icechunk/ — the §11.1 Icechunk repo-root surface (#615): METADATA ONLY — the group documents the once-per-run init commits, written by the production spec builder (icechunk_refs.repo_group_spec) on pyramid/'s grid and knob: the root zarr.json (the zagg_icechunk block, the zarr_conventions registration and the multiscales object with its layout) plus one {cells}/zarr.json per level group (the artifact's dggs block and zarr_conventions, verbatim). No Icechunk repository, no arrays, no refs — the committed golden pins the documents a conformant reader validates: the root group document against the multiscales v0.1 schema, and every level group against the dggs v1 schema (the semi_major_axis spelling, since #616), both vendored under tests/data/conventions/. icechunk.expected.json spells the layout from the generator's inputs (the §11.1 rules over pyramid/'s expanded levels), never read back through zagg.

• raster_toc/ — the §8 temporal declaration surface: one raster (time, cells) hive leaf whose time coordinate is uint64 toc words carrying temporal: {"spec": "zagg-toc/1", "shape": "coordinate", …} and no CF units/calendar attrs, beside morton and two band arrays. Its axis mixes both word variants deliberately — one single-item timestep encoded as an exact timestamp word and two multi-item acquisition groups encoded as range words — so a reader that implements only one variant fails a §7 fixture. raster_toc.expected.json records the words as decimal strings (JSON numbers cannot carry uint64 faithfully) together with the (start, end) nanoseconds a conforming decode yields and the real acquisition spans they must contain, which is the §8 conservative containment claim pinned on committed bytes. The leaf fixtures that carry no temporal key anywhere are the absent-key ⇒ legacy pin.

• temporal/ — the §8.2/§8.3/§9 companion surface: minimal/'s geometry and cell plan with one digest field (h_tdigest) carrying both companions — the located sibling h_tdigest_locations (declaring located, §9) and the temporal sibling h_tdigest_times (declaring temporal at shape: "per-centroid", bound from the payload's times key) — plus the dense uint64 observed array declaring shape: "per-cell", and count. Both toc variants are committed in both shapes: the single-observation cell is an exact timestamp word and every multi-observation cell and merged centroid a conservative range, so a reader implementing one variant fails a §7 fixture. The unpopulated cells of observed hold the §8.2 reserved 0. temporal.expected.json records the words as decimal strings, the three declarations verbatim, and — derived from the generator's inputs, never transcribed — the real member instants each word must contain and each centroid's true member run, so the conformance suite asserts §8.2/§8.3's containment (and that a cell's per-cell envelope encloses every per-centroid envelope beneath it) on committed bytes. Its located sibling carries heterogeneous orders — order-29 point words on unmerged centroids, coarser ancestors on merged ones — which is §9.1's claim pinned. kitchen_sink/, whose two located siblings the generator writes without the §9 attrs on purpose (stale by design), is the absent-located ⇒ §2.2 pin, exactly as minimal/ is §2.0's.

Its §4.6 leaf column carries the folded companions: every resolution group holds h_tdigest with both siblings, each declaring per-centroid, row-aligned with the folded payload. That makes it the fixture set's only golden for a companion produced by a merge rather than by ingest — both channels reduced over the centroid partition that merge produced, at every level (espg-ruled 2026-08-17, amending ruling 3). The dense observed array is deliberately absent from those groups: the "per-cell" shape's fold law is the grammar's join over a cell group rather than the field's own reducer, so it exists at native resolution only.

It is also the fixture set's only leaf-bearing store with a temporal root coverage.moc, and so §10's golden: the object was written by the production sweep writer (the MOC family's leaf read plus its finisher) and carries the zagg-coverage-toc/1 section — the shard's tier-1 envelope word and the tier-2 root counted cover (§10.3): two row-aligned uint64 buffers, the occupied bucket words at the pinned order and their per-bucket observation counts. There is no digest key, and the suite asserts its absence — the counted cover replaced the time-digest outright (#575), so a reader implementing §10.3 from this fixture meets no digest grammar at all. temporal.expected.json's root_coverage block records the tier-1 word derived from the generator's inputs (the join over every per-centroid word it fed the writer, so the writer is pinned rather than self-certified), the decoded words/obs buffers read back — the same exception column/'s group values are — and obs_total, the cell plan's own observation count, which §10.3's count rule says the block's obs_total MUST equal at the uncounted_shards: 0 the block carries. The one leaf also carries the §10.6 record, temporal.toc (#575), written by the production worker path; temporal.expected.json's leaf_temporal block records the object name, the required keys, the envelope word (equal to the root section's shard word), and the decoded counted cover and cover — each derived through §10.3's laws from the generator's own instants, never transcribed from the object. The other leaf fixtures have no root coverage object at all: none of them declares a temporal field, so a sweep of one produces no section, and their committed trees carry no such object — which is exactly §10's absence rule, pinned as bytes. From issue #489 the same fixture also carries the §10.5 coverage.toc sibling, PUT by the same production finisher beside the carrier (whose section now bears the §10.1 cover marker); temporal.expected.json's cover block records the derived word set — the §10.5 quantization of the same inputs the tier-1 word is derived from — the object's declared cap/fields/element/encoding and the block's count, and gap_ns, an interval on the internal scale the committed cover MUST NOT claim. The fixture's last cell is deliberately clocked whole buckets past the rest of the plan, so the cover is a MULTI-word set with a real hole in it: a reader that quantized every word into one bucket would satisfy the parity and containment claims and fail gap_ns. The other leaf fixtures carry no coverage.toc either, the §10.5 absence pin.

• uncounted/ — the §10.3 coverage-only surface (#575): metadata only — the manifest and the two root objects the production sweep writes over temporal/'s leaf when that leaf carries no §10.6 record (the generator builds the temporal/ store in a scratch directory, deletes its temporal.toc, and runs the same leaf read and finisher; nothing writes the record back). Its coverage.moc is the one committed counted cover whose every obs is 0 — the same five bucket words temporal/ counts, occupied and uncounted — under uncounted_shards: 1 and obs_total: 0, so a reader that drops zero-count rows, or computes "occupied" as obs > 0, or reads obs_total as the store's count, fails a §7 fixture. Its coverage.toc is the same word set as temporal/'s: the cover is the key set, zero counts included. uncounted.expected.json records the shard word, the bucket words and the cover — derived from the generator's per-centroid words, each keyed at its representative instant, never read back — the marker, clocked_obs (the leaf's real observation count, which obs_total is a lower bound on and no object in the fixture carries), and leaf_content_hash, the §5 combined digest of the leaf the objects describe, equal to temporal/'s: the two fixtures are two readings of one leaf, and the leaf is committed once.

• windowed/ — the §1.5 derived-coordinate surface (issue #586): minimal/'s geometry and cell plan with one located digest field (h_tdigest + h_tdigest_locations) and count, written as one windowed leaf — {id}_2019.zarr, a morton-hive/2 stamp naming window, under a manifest carrying the D15 temporal block. It is the one fixture leaf with no morton array: the group holds three arrays, and the stamp's §5.3 key set has no 6/morton entry. windowed.expected.json records, in a derivation block, the sixteen words a reader must derive for the cells axis — computed from mortie, never read back — their stride at this cell order (2^48), and the order-19 instance of the law: for one order-9 shard in each hemisphere, the first and last of its 1,048,576 cell words and the one stride between them, 4194304 (the southern shard's word sets bit 63). Each populated cell records its derived morton, its location words and their decoded orders, which are heterogeneous on these bytes — order-29 point words beside merged centroids' ancestors, some of them the cell itself — so the conformance suite asserts §1.5's containment check on committed bytes: every location word's ancestor at the cell order is the derived word, and a neighbouring cell's word is not. The shipping readers are run over it too, and report the read-chunk ids a stored coordinate would give. Its §4.6 column (2019.pyramid.zarr) stores morton in every group, the pin that the artifacts are unchanged. Every other fixture leaf is unwindowed and keeps its stored array, byte for byte.

• demoted/ — the §4.3 demotions surface (#518): the kitchen_sink/ store swept under a hand-installed /1 cascade manifest whose composition of divisor is mis-declared class: "none" (§4.5: manifests outlive their writer). Two committed overviews pin the pair of readings: the exact-from-leaves level folds composition cleanly (the leaf arrays carry the divisor regardless of its declared class) and carries no demotions key — the absence pin — while the cascaded level above it fires divisor-missing on its one contributor, holds the all-fill word slab, and records the demotion — cells: 4, the skipped child's whole disjoint span of the 16-cell level — in its zagg_overview attrs. demoted.expected.json records both levels' paths, fold sources and the demotions list beside the leaf record.

minimal/ and kitchen_sink/ pin the layout edge cases a reader must handle (column/'s leaf is minimal/'s, so it pins them again): inner chunk ordinal 2 is empty (absent from the shard index — the §1.5 sentinel, and that sparsity reaches the dense arrays too: the morton coordinate and count hold their fill across that chunk, so a reader MUST NOT assume a stored coordinate is dense across a shard — a derived one, windowed/, always is), populated chunks contain empty cells (the b"" fill), and one cell's digest carries merged centroids whose location words are common ancestors (§2.2).

Every fixture leaf is sharded, so §7's leaf conformance claim is scoped to the §1.5 sharded geometry. The per-inner-chunk geometry — identical §1.4 framing, one object per inner chunk, no shard index — now has a committed golden: column/'s resolution groups are single-chunk unsharded arrays (§4.6), including a zagg-ragged/1 payload array (h_tdigest, chunk_shape == shape, no sharding_indexed), so a reader that hard-codes the shard-index suffix fails a §7 fixture instead of sailing through. The unsharded multi-chunk case remains pinned zagg-side by tests/test_processing.py::TestRaggedVlenLayout. A reader that derives the stored span from the array's own metadata, as §1.5 requires, reads all of these from one code path.

Each fixture ships a sibling {name}.expected.json recording the shard key, leaf path, geometry, every populated cell's decoded values (digest centroids, location words and composition words as decimal strings — JSON numbers cannot carry uint64 faithfully), the per-stratum exact counts, and the §5 O11 content_hashes (per-array + combined). The expected leaf decoded values were computed from the generator's inputs — the arrays handed to the writers — never read back through a zagg reader, so the reader is pinned, not self-certified. The column record's group values are the one exception: they are the writer's committed output, read back. Their independence comes from elsewhere — the conformance suite re-derives the base group from the committed leaf through the §4.4 fold kernels and asserts byte equality (the §4.6 from-leaves parity contract), so the recorded values are a regression pin over an independently derived result, not a self-certification. The content_hashes are necessarily computed from the written leaf (a content hash of nothing else would mean anything), so they are pinned differently: the suite carries the combined digest and one per-array digest per §5.2 element kind as frozen hex literals, and recomputes the vlen digests a second time from the shard objects alone (§1.4/§1.5 byte recipes, no zarr). A recipe change — prefix width, joiner, key set — therefore fails a test instead of agreeing with itself on both sides, which is also the only mechanism that catches a future zagg↔moczarr divergence (neither side's fixture can: espg/moczarr#23).

Conformance criteria for an external reader: decode every ragged array per §1–§2, derive the per-cell coordinate of a leaf that stores none per §1.5 (windowed/), the composition array per §3, and every declared word-typed array — the time coordinate and both temporal companion shapes per §8, the located companion per §9 — reproducing the expected decoded values exactly (byte-exact float32/uint64 — no tolerance), and reproduce content_hashes per §5. zagg's own suite additionally decodes the shard objects with spec-text-only decoders (struct + zstd, no zagg read path) to prove the byte recipes in §1.4/§1.5 are sufficient on their own.

Regenerating the fixtures reproduces the same logical values (seeded rng); stamp timestamps and compressed bytes may differ across zstd versions — conformance is over decoded values, never stored object bytes (the same principle as §5). In the committed D20 sidecar (column/…/all.pyramid.stats.json) only content_hashes and cells_with_data are pinned: timestamp, zagg_version, run_id and the run counters are informative provenance, they churn on every regeneration, and conformance never asserts them.

---

8. zagg-toc/1

Status: contract (issue #443 — the first shape of the temporal series, #410).

An array whose elements are packed words rather than self-describing scalars carries a word-typed coordinate declaration: a spec-owned attrs block, keyed by its domain, holding {spec, shape, grammar} — the convention revision, where the words sit relative to the store's cells, and which word grammar the values follow. The key is spec-owned: the writer stamps it, never author-transcribed (§1.2's reserved-key discipline, extended to coordinate arrays). The shape vocabulary is domain-neutral and defined once, here:

• "coordinate" — the declaring array is the coordinate variable of a dimension (CF/xarray sense): one word per index along that dimension, row-aligned with every array sharing it.
• "per-cell" — one word per cell of the store's cell grid, aligned with the morton coordinate.
• "per-centroid" — one word per centroid inside a cell's ragged payload, aligned element-for-element with the §1/§2 digest it accompanies.

This section instantiates that pattern for the temporal domain: attrs key temporal, spec: "zagg-toc/1", grammar: "mortie-toc/1".

An absent temporal key MUST be read as the legacy encoding — signed int64 microseconds since 1970-01-01T00:00:00 UTC, self-described by the CF units/calendar attrs the writer stamps beside it. Every store written before this revision is conformant verbatim, no byte rewritten, and a reader MUST NOT refuse a store for lacking the declaration.

"temporal": {
  "spec": "zagg-toc/1",
  "shape": "coordinate",
  "grammar": "mortie-toc/1"
}

Three keys, and deliberately no more — ruled on #410: the declaration is {shape, grammar revision} under the spec marker, with no per-store epoch or quantization guards.

• spec — the convention revision. Readers MUST strict-check it: an unknown or future revision raises, never half-parses under a guessed layout.
• shape — the vocabulary value above. This revision defines all three for temporal, each with its own contract section: "coordinate" (§8.1 — the declaring array is the time coordinate of a (time, cells) product), "per-cell" (§8.2 — one word per cell of the cell grid), and "per-centroid" (§8.3 — one word per centroid of the digest it accompanies). A reader MUST refuse a shape it does not implement — including one a later revision adds, which is why refusing is the required behavior and guessing never is. Adding a shape is therefore additive within the revision: spec does not move, stores already written stay valid verbatim, and a reader that implements only some shapes fails loudly on the rest rather than mis-decoding them.
• grammar — the word grammar the values follow, named as a grammar revision in the {name}/{major} style this document uses throughout (zagg-ragged/1, morton-hive/2). It is a fixed token of this revision — not a documentation URL, and not a stamp of the writer's installed mortie — so a store's bytes move neither when a dependency floor moves nor when the documentation that describes the grammar moves. The pointer to that documentation is prose (below), which is where it can be updated freely. A reader MUST refuse a grammar it does not implement, and SHOULD record it as what it decoded against.

The declaring array is the array that holds the words. A block declares the values of the array it sits on, never a neighbour's: the time coordinate declares its own axis (§8.1), a per-cell companion its own dense array (§8.2), a per-centroid companion its own ragged sibling (§8.3). Where a companion is reached through another array, that array carries a binding (a sibling array name) and the companion carries the declaration — the same split §1.2 already makes for the located sibling.

Instantiations (#410). The temporal companions — one word per cell, one word per centroid — are the "per-cell" (§8.2) and "per-centroid" (§8.3) shapes under this spec and this grammar. The located (spatial) companion family declares under the same pattern with grammar: "mortie-morton/1" and its own domain key, in §9. None of them re-declares the store's primary morton axis: that surface stays declared by the morton-hive/{1,2} manifest grammar (morton_hive.json, docs/hive_layout.md) and the store's DGGS-convention dggs attrs (docs/morton_arrow.md). This pattern is for word-typed coordinate and companion arrays beyond that primary surface, never a second, competing declaration of it.

The epoch, the timescale, and the range variant's rounding quanta are properties of the cited grammar and are deliberately NOT echoed here. A store cannot re-base them — a differing origin would be a different grammar, which is precisely what grammar (and spec above it) already discriminates — so a per-store copy could only ever restate a constant, while entering the committed conformance bytes and the §5 content hash and forcing a fixture regeneration for any upstream constant that changed.

A writer MAY add further informative keys to the block (source-time lineage and the like). They are non-normative: nothing in §8 is decoded from them, and a reader MUST ignore keys it does not recognize rather than refuse the store.

The word grammar is mortie's

A zagg-toc/1 value is a mortie toc word (temporal order coverage): one uint64 that is a tagged union of an exact nanosecond timestamp and an outward-rounded, conservative range, on a continuous, leap-free, GPS-aligned timescale with a fixed 1850-01-01T00:00:00 epoch. The bit layout, the flag position, the unsigned sort order, and the semilattice merge law are normative in mortie's mortie.toc reference and its decision ledger (espg/mortie#175), and are not restated here. What follows is zagg's half of the contract.

The stored token is the grammar revision mortie-toc/1; the URL above is the documentation pointer, and it lives in this prose precisely so it can be re-pointed without moving a store byte. It is release-pinned deliberately: mortie's documentation is mike-versioned and the unversioned /api/toc/ path was never published, so only a versioned URL resolves; the pin is 0.9.6, the earliest release carrying the reference, and the page's normative words are unchanged through the current dependency floor.

It is also not yet the same class of citation as §2.2's, and §8 does not claim it is. §2.2 defers the morton word's layout to mortie's docs/specification.md §1/§4 — sections that document's §10 "Frozen for 1.x" enumerates as immutable within the major version. Mortie's frozen specification contains no toc section: the grammar above is normative as a module reference pinned by mortie's own golden fixtures, which is a weaker guarantee. espg/mortie#193 tracks adding the frozen section; when it lands, this pointer swaps to it, the stored grammar token does not move, and nothing else in §8 changes.

8.1 shape: "coordinate"

Contract.

• The declaring array's element type is uint64, one word per timestep.
• A timestep whose real acquisition is a single instant MUST be encoded as a timestamp word, exact to the nanosecond; a timestep covering a real interval MUST be encoded as a range word whose envelope conservatively contains that interval. A conforming writer therefore never widens an instant into a range, and never narrows a real interval into an instant.
• Row order is the acquisition-group order: timesteps are ordered by the group's earliest member observation time — the order the time axis has always had, the order the (time, cells) slabs were indexed by, and identical under both encodings, so a row assignment never drifts with the encoding.
• Stored word order is not that key, and a reader MUST NOT assume the stored words ascend. Unsigned word order is order by the encoded start, which is the conservative envelope start — and an envelope may begin before its group's earliest member observation time, by an amount nothing in this section bounds. Where it does, the word leads the row key and the stored axis is materially out of order: np.sort(words) yields ascending envelope-start order, which is not guaranteed to equal row order. In particular a reader MUST NOT bisect the stored axis to resolve a time window; use the overlap predicate below, which is correct regardless of stored order.
• Decoding to wall time yields (start, end) per timestep: for a timestamp both bounds are its exact instant; for a range end is the envelope's exclusive upper bound. A reader that must present one instant per timestep SHOULD present start, and MUST NOT present a midpoint as if it were the observation time — the midpoint of a conservative envelope is not an observation.
• Temporal window selection is the grammar's overlap predicate applied to the stored words directly: it over-reports by at most one quantum at a window edge and never under-reports, so a selection is a conservative superset of the timesteps whose real acquisition intersects the window.

Composition. Time axes compose only between matching declarations — zagg-toc/1 with zagg-toc/1, legacy with legacy (an absent key is legacy for this rule too). A concatenation across encodings would produce an axis whose values mean two different things, so a reader or writer joining two stores' time axes MUST refuse a mismatch. This is a join rule only: reading either store on its own is always legal.

What §8.1 does not cover (informative). Its clauses are the "coordinate" shape's alone: the companion shapes carry their own in §8.2 and §8.3, and a clause here binds a companion only where that section restates it. Nothing in §8 constrains the (time, cells) band arrays themselves, which are unchanged.

8.2 shape: "per-cell"

Contract (#410). One word per cell of the store's cell grid: a dense uint64 array on the cells axis, index-aligned with the morton coordinate (row i is the cell morton[i] addresses), sharing that array's shape, chunk grid and storage geometry (§1.5). It is an ordinary dense array — not a zagg-ragged/1 array — and carries the temporal block in its own attrs.

• The array's fill_value MUST be 0, and 0 is reserved: it marks a cell the writer left unobserved and MUST NOT be read as an acquisition. A writer MUST NOT store 0 for an observed cell. The reservation is cost-free: 0 is not a value the grammar's encoders can produce, and it is empty where it is decoded, so the sentinel excludes nothing.
• No encoder emits it. The epoch instant encodes as 2147483648 (the flag bit sits at position 31, not at the bottom of the word), and the shortest range word is 1, because the range encoder's end code is a strictly-greater ceiling and is therefore ≥ 1 for every input, (0, 0) included.
• toc_merge cannot introduce it. The join is over encoded words, so a reduction over words no encoder produced as 0 does not produce 0.
• It is empty, not an instant. 0 decodes as a range word whose envelope is the half-open [0, 0) — it overlaps no window, including one containing the epoch — so a reader that meets it under the grammar's overlap predicate selects nothing, with or without this reservation.
• Each observation enters under §8.1's discipline, restated here with its force (§8.1's clauses bind a companion only where its section restates them): a writer MUST encode an instant as a timestamp word, exact to the nanosecond, and a real interval (an integration window — an observation need not be instantaneous) as a range word conservatively containing it. A cell's stored word is the join of its observations' words (below): a timestamp word exactly when that join is a single instant — a cell covering one instantaneous observation, or several sharing one instant — and a range word conservatively containing every observation pooled into it otherwise. Instants never widened, intervals never narrowed, per observation, not per count.
• The pooled word is the grammar's join. A cell's word over a set of observations is toc_merge (the grammar's semilattice join) reduced over their individual words: the conservative envelope. The join is associative, commutative and idempotent, so the stored word is independent of the order and tree any producer reduced in — bit identity, not approximate agreement. Whether that reduction happens in one pass, in spill blocks, or up a pyramid is a producer's business and is not specified here.
• The claim a reader may make from the word is exactly the envelope: every observation the cell summarizes fell inside [start, end) (both bounds equal for a timestamp). It is not a claim that the cell was observed throughout that interval, and the envelope's midpoint is not an observation.
• Stored word order is not the cells' order and carries no meaning: the axis is ordered by morton (§1.5), so a reader MUST NOT sort or bisect this array to resolve a time window. Selection is the grammar's overlap predicate applied to the stored words directly — a conservative superset of the cells whose real observations intersect the window, over-reporting by at most one quantum at an edge and never under-reporting.

8.3 shape: "per-centroid"

Contract (#410). One word per centroid of a digest field: a zagg-ragged/1 vlen sibling array (element dtype uint64, empty inner_shape) with the same shape and chunk geometry as the payload array, row-aligned with it exactly as the located sibling is (§1.1) — cell i of the sibling holds one uint64 word per payload row of cell i, so the two arrays share the payload's per-cell row counts. The sibling carries the temporal block in its own attrs.

• Binding. The payload array declares the sibling by name under a spec-owned times attrs key — a sibling of the §1.2 ragged block on the payload array, exactly as §2.0's weights is, and for the same reason: the ragged block is retired wholesale under /2 (§1.6/§6.3), so a key outside it survives that metadata-only migration untouched. The zagg-ragged/1 block grammar is therefore unchanged by this revision — no key is added to it, and a /1 reader that ignores unknown top-level attrs decodes such a store exactly as before, minus the companion.

{
  "ragged": {
    "spec": "zagg-ragged/1",
    "element": {"dtype": "float32", "shape": [-1, 2]},
    "locations": "h_tdigest_locations"
  },
  "times": "h_tdigest_times"
}

(The asymmetry with locations — inside the block — is deliberate and historical: locations predates the sibling-key ruling and cannot move without a /1 revision, while a new key has no such constraint.)

A reader MUST bind the sibling by that declaration, never by reconstructing a naming convention, and MUST read the temporal block off the sibling it binds — a payload array carries the binding, never the declaration. An absent times key means the field has no temporal companion. The key is spec-owned: a writer stamps it from the field's declaration, and config-declared attrs MUST NOT shadow it (§1.2's reserved-key discipline, enforced at config validation). - A word's claim is keyed on its variant, carried by the word itself, never by the payload's weights — under a "flux" payload (§2.0) sum(weights) is not a member count, so weight identifies nothing. Each observation enters under §8.1's discipline, restated here with its force (§8.1 binds a companion only where its section restates a clause): a writer MUST encode an instant as a timestamp word, exact to the nanosecond, and a real interval (an integration window) as a range word conservatively containing it. A merged centroid carries the grammar's toc_merge join over its members' words, with the same order-independence §8.2 states, and a centroid folded from a single observation carries that observation's word unchanged. (Zagg's own writers ingest per-observation instants today — range ingest is legal, not emitted; informative.) Both variants therefore coexist in one array, and a reader that implements only one is not conforming. - Row order is the payload's, and it is not time order. §2.1 sorts payload rows ascending by mean, and the sibling is row-aligned with them, so the stored words are in value order. A reader MUST NOT assume the words ascend and MUST NOT bisect them; window selection is the grammar's overlap predicate on the words directly, exactly as in §8.2, and its result is a conservative superset of the centroids whose observations intersect the window. - The digest payload it accompanies is approximate across fold orders (§2.3) while this companion's join is exact. The companion's exactness does not lift the field's composability class: a merged payload's centroids may differ, and the words are exact given the centroid partition they describe, not independently of it. - Provenance attrs still ride the payload array only (§1.2); the temporal sibling carries the spec-owned temporal block and no user attrs.

Per-level shapes need not match (contract). The two companion shapes describe the same information at different granularities, so one product may carry "per-centroid" on its leaves and "per-cell" on the summaries folded above them — a per-cell range is the honest temporal statement for a lossy summary, and it is the envelope of the per-centroid words beneath it. Each array is read through its own declaration; nothing requires a store's levels to agree, and a reader MUST NOT infer one array's shape from another's.

8.4 Composition and merge legality

Scope. §8.4 governs the composition of the companion shapes — "per-cell" (§8.2) and "per-centroid" (§8.3). It does not govern the "coordinate" shape: two time axes join under §8.1's Composition paragraph, which is the stricter rule, and §8.1 takes precedence wherever both could be read to apply. The two differ exactly on absence, and deliberately: an undeclared companion carries no information, so §8.4 composes with it and drops the channel; an undeclared coordinate array carries the legacy encoding (§8), which is information — a rival encoding of the same axis — so §8.1 MUST-refuses a legacy ↔ zagg-toc/1 join. Nothing in §8.4's absent-key clause below licenses that join. This is the same boundary §8.1 draws from its own side under "What §8.1 does not cover".

§9.2 imports this section verbatim with located in place of temporal, and inherits this scope with it: it governs the located companions — whose only shape this revision defines is "per-centroid" (§9) — and never a coordinate array.

Peer joins (contract). Two payloads compose as peers — like joined with like, at one granularity — only when their temporal declarations match on {shape, grammar} — the §2.0 weights-gate discipline, for the same reason: words from two grammars, or from two shapes, describe different things, and a merged array carrying either would mean neither. A reader or writer joining declared payloads whose shape or grammar differ MUST refuse, and the composed result carries its contributors' shape unchanged. (spec is strict-checked before this rule is reached: an unknown revision is already a refusal.)

Shape-coarsening reductions are licensed, and are not peer joins (contract). A reduction folds a finer companion into a coarser one, so by construction its output shape differs from its contributors' and it can never satisfy the peer gate above. This revision licenses exactly one — the "per-centroid" → "per-cell" fold that §8.3's closing "Per-level shapes need not match" clause describes and that ruling 3 on #410 first called for ("per-cell toc range at overview levels, even where leaves are per-centroid"). That ruling was amended on 2026-08-17: zagg's own digest pyramids keep companions "per-centroid" at every level, symmetric with the located channel, so the reduction below is licensed but unused by this writer — the terms stand unchanged for any producer that wants it, and a reader must still implement the mixed-level case §8.3 permits. Its terms:

• The contributors MUST be peers of each other: every one declares shape: "per-centroid" under the same grammar. A reduction over contributors that fail the peer gate is itself a refusal.
• The output is a §8.2 array declaring shape: "per-cell" and the same grammar — a reduction coarsens the shape, never the word grammar, and never the spec.
• The output word for cell i is toc_merge reduced over the per-centroid words of cell i: the envelope §8.3's closing clause requires ("it is the envelope of the per-centroid words beneath it"), exact and order-independent by §8.2's join clause.
• No other cross-shape combination is defined. A writer MUST NOT invent one, and a reader MUST refuse any it meets under the peer rule above.

So "matching" is never ambiguous: a peer join matches its contributors against each other and inherits their shape; a reduction matches its contributors against each other and declares its own, coarser output shape, which by construction does not match theirs. Which of the two is in play is a property of the operation being performed, never something a reader infers from the declarations it finds.

An absent temporal key is never a refusal. A payload, cell array or digest that declares nothing composes freely with anything — this is the schema-evolution rule that keeps every store written before this revision conformant verbatim. Two consequences, both normative, and both binding on reductions as well as peer joins:

• The composed result MUST NOT carry a temporal declaration unless every contributor carried one matching the others'. Dropping the channel is the honest outcome: a word that omits an undeclared contributor's observations is not a conservative envelope, and silently narrowing one is the failure this clause exists to prevent.
• An undeclared store supports no temporal subsetting. A reader MUST NOT infer an encoding, an epoch, or a window from anything else in the store, and MUST report that the query is unanswerable rather than approximate it.

---

9. zagg-located/1

Status: contract (issue #410). The spatial instantiation of §8's word-typed coordinate declaration: attrs key located, spec: "zagg-located/1", grammar: "mortie-morton/1". Everything §8 says about the pattern — the three keys and no more, strict-checking, the declaring array being the array that holds the words, informative extra keys ignored rather than refused — applies here unchanged and is not restated. The marker is named for the domain, not for the word type, precisely so it cannot read as a second declaration of the store's primary morton axis: it is not one, per §8's carve-out.

"located": {
  "spec": "zagg-located/1",
  "shape": "per-centroid",
  "grammar": "mortie-morton/1"
}

shape — "per-centroid" is the only value this revision defines: the declaring array is the {field}_locations sibling of §1.1/§2.2, one word per centroid row of the payload it accompanies, bound from the payload array's ragged block locations key (§1.2). A reader MUST refuse a shape it does not implement.

An absent located key MUST be read as §2.2 verbatim — kind-keyed words (a point word a reported position carrying no area claim, an area word a cell containing everything beneath it), order-29 point-word ingest, area words only as fold products (the deepest common ancestor of the members' words) — never as an unknown encoding, and never as grounds to refuse. Every located store written before this revision is conformant as it stands, no byte rewritten. What the declaration adds is self-description (a generic reader learns the word grammar from the array rather than from this page), the §8 shape vocabulary, the coarse-ingest grant (§9.1 — a latitude §2.2 deliberately withholds, so its published semantics never move), and the overview clause below, which §2.2 does not cover.

The word grammar is mortie's

A zagg-located/1 value is a mortie morton word: one uint64 packing the HEALPix nested cell and its order, with the order-29 point/area kind carried by the encoding. The bit layout, the order range, the kind convention and the raw-sort Z-order property are normative in mortie's docs/specification.md §1 and §4, and are not restated here. Unlike §8's toc citation, this is the same class of citation §2.2 already makes: mortie's §10 "Frozen for 1.x" enumerates §1's bit layout and §4's encoding-carried kind convention as immutable within that major version, so the mortie-morton/1 token names a grammar that cannot move under a conforming reader.

9.1 Leaf and overview words

Contract.

• A word's claim is keyed on its kind (§2.2): a point word is the observation's reported position, carrying no area claim; an area word is a cell known to contain every observation beneath it — the finest such cell its producer could establish, from cell-resolved ingest or from a fold. A centroid folded from a single observation carries that observation's word unchanged.
• Coarse ingest is this declaration's grant. An observation located to a position enters as its point word; one resolved only to a cell enters as that cell's area word — positions never narrowed into points, §8.1's discipline spatially. A store whose ingest words include area words MUST carry this declaration: undeclared ingest is §2.2's (order-29 point words only), so the absent-key route never widens under a reader's feet. (Zagg's writers ingest point instruments as order-29 point words, HealpixGrid.assign; no shipped config emits area-word ingest today — informative.)
• A merged centroid — at a leaf, at a spill-block close, or at any level of a pyramid — carries the deepest common ancestor of its contributors' words: a cell containing every observation merged into it, and the finest one those words establish (a fold reduces words, so an ingested coarse word bounds the result). Containment is the whole claim a reader may make from the word: it is not a centroid position, not a mean, not a cell the data fills, and not the finest cell containing the observations themselves.
• Overview words sit at heterogeneous orders, within one array. A fold coarsens only as far as its contributors force, so one overview array's words routinely carry different orders — an unmerged centroid's order-29 point word beside a merged centroid's coarse area word. A reader MUST decode each word's order from the word itself (mortie §1/§4) and MUST NOT assume a uniform order per array, per level, or per store, nor infer one from the level's cell order. No order uniformity is promised anywhere — leaf arrays included (this declaration admits cell-resolved area-word ingest, and even §2.2's strict ingest leaves spill-folded leaf centroids coarse): a uniform order is an observation about particular bytes, never an inference.
• The ancestor reduction is exact and order-independent — point and area words share a path prefix, so mixed inputs compose under the one rule (§2.2) — but, as in §8.3, it does not lift the accompanying digest's composability class (§2.3): the words are exact given the centroid partition they describe.

9.2 Composition and merge legality

Contract. §8.4 applies verbatim with located in place of temporal: declared companions compose only on matching {shape, grammar} and MUST be refused otherwise; an absent declaration is never a refusal. The one difference follows from the absent-key rule above — an undeclared located sibling is not information-free the way an undeclared temporal one is, since §2.2 already pins its word semantics, so composing a declared companion with an undeclared one is legal and the result MAY carry the declaration: the words mean the same thing on both sides. A reader that cannot establish that — because §2.2 does not apply to the array in hand — MUST drop the declaration rather than assert it.

---

10. zagg-coverage-toc/1

Status: contract (issue #480).

The temporal section of the store-root coverage sidecar — the one object a reader GETs to bootstrap discovery. Its carrier is the morton-moc/1 root envelope, whose body ({spec, encoding: "ranges", order, source, generated_at, ranges}) is described in hive_layout.md and whose word and decimal grammars are mortie's; this page owns exactly one addition to it — a temporal key — so that a spatiotemporal candidate query resolves from metadata alone, before any leaf is opened.

Two tiers. Tier 1, and the buckets tier 2 keys, are derived from the §8.3 "per-centroid" companions the leaves already carry; tier 2's counts are summed from the leaves' §10.6 worker records and come from nowhere else:

tier	key	what	answers
1	shards	one toc word per populated shard — the join over that shard's sibling words	which shards hold data during a window
2	counts	the store-wide counted cover — the observation count in every aligned time bucket the store's observations fall in, exact when the block's uncounted_shards is 0 and a stated lower bound otherwise (§10.3)	how much data falls in a window

Tier 1 and tier 2's occupied buckets are not new information: like the spatial ranges beside them they are a regenerable accelerator over the leaf arrays (§8.3, D9). Tier 2's counts are not regenerable — they are leaf-written data that exist only in the §10.6 records, so a lost record lowers the bound (§10.3 uncounted_shards) and no refresh rebuilds it. Neither tier is truth: both are written at end of walk while leaves stamp continuously, so a reader MUST treat them under the same staleness posture as the ranges — a shard the section does not list is not proof the shard has no data in the window.

Absence is the whole-section rule. A store with no temporal channel carries no temporal key, and its root object is byte-identical to one written before this revision. A reader MUST read that absence as "this store publishes no temporal coverage" and MUST NOT refuse the store, the sidecar, or a windowed query because of it — the standing absence posture of §8/§9, restated here with its force. Absence of the counts sub-block alone says the same thing one tier down: tier 1 stands without it.

Versioned key discipline. The section carries its own spec marker, independent of the carrier's. A reader MUST strict-check it. Unlike the array-level sections of this page, an unknown revision here reads as absent, not as a hard failure: the section is an accelerator whose truth is elsewhere, and this matches the strict-gate-then-degrade rule the sibling coverage envelopes already use (the leaf stamp's coverage payload, hive_layout.md). A future revision is therefore a new spec string and a new section here; keys are never repurposed in place.

10.1 Section grammar

Contract.

"temporal": {
  "spec": "zagg-coverage-toc/1",
  "source": "sweep",
  "generated_at": "2026-08-17T22:59:35+00:00",
  "fields": ["h_tdigest"],
  "cover": "zagg-coverage-toc-cover/1",
  "shards": {"11213": "10689250968998768172"},
  "counts": {
    "temporal_order": 24,
    "cap": 512,
    "element": {"dtype": "uint64", "shape": [-1]},
    "encoding": "base64",
    "words": "…",
    "obs": "…",
    "count": 5,
    "obs_total": 346,
    "uncounted_shards": 0
  }
}

• spec (required) — "zagg-coverage-toc/1", gated as above.
• source (required) — which producer wrote the section: "sweep" (the sweep's leaf walk) or "refresh" (the explicit whole-store rebuild). The vocabulary is open, exactly as the carrier's source is.
• generated_at (required) — ISO-8601 UTC. The section's own clock: it and the carrier's may differ, because a producer with no temporal contribution rewrites the ranges and leaves the section standing (§10.4).
• fields (required) — the sorted payload field names whose §8.3 companions the shards map was derived from. It is the map's provenance, and it composes as a union across producers (§10.4): after a merge it names every field any contributing producer read, which is not necessarily the set the installed counts was built over — an upper bound, not a per-block field list. It carries no multiplicity: §10.3 counts an observation once however many fields are named here. Informative for tier 1 (the words are already unioned across fields by construction).
• shards (required) — tier 1, below.
• counts (optional) — tier 2, below (a digest key under this spec is the block this revision retired before any published store carried one; a reader MUST ignore it under the unknown-keys rule).
• cover (optional, added under this revision — issue #489) — the presence marker for the word-set cover sibling object (§10.5): its value is that object's spec string ("zagg-coverage-toc-cover/1"), telling a temporal consumer one GET ahead that a coverage.toc stands beside this root, and at which revision. It is a hint under the sidecar staleness posture, never a promise: a reader MUST fall back to tier 1 (or to opening leaves) when the object turns out to be absent, unreadable, or at a revision it does not implement — and, conversely, MAY probe for the object even without the marker (a marker-less root simply predates it). Pre-marker readers ignore the key under the unknown-keys rule above, which is what makes this an in-revision addition rather than a spec bump: no meaning of any existing key changed.

A reader MUST ignore keys it does not recognize; a producer MUST NOT put anything under temporal that is not defined here or by a later revision.

10.2 Tier 1 — the per-shard envelope word map

Contract. shards maps a shard id to one toc word:

• keys are D1 decimal shard ids at the carrier's order, spelled exactly as the ranges endpoints are;
• values are toc words as decimal strings, for the same reason the range endpoints are strings: a uint64 word exceeds 2^53 and a float-based JSON parser would silently mangle a raw number;
• the word is the grammar's join (mortie toc_reduce) over every §8.3 companion word the shard's leaves hold, unioned across all of the store's temporal-carrying fields — coverage means "any data", not "data in this field". Where a shard is split into window leaves (morton-hive/2), the one word is the join over all of them: the map is shard-keyed, never window-keyed.

The word therefore conservatively contains every observation instant in that shard, exactly as §8.2's per-cell envelope does one level down, and the grammar's own predicates apply unchanged: toc_overlaps never under-reports (every shard whose real content intersects the window tests true, possibly with edge over-report of up to one quantum) and toc_contains never over-reports. A reader MUST use those predicates on the words rather than decoding to bounds and comparing itself.

The map lists only shards a producer actually walked. Its key set therefore need not equal the carrier's ranges — a shard listed by ranges but absent from shards is one whose temporal contribution has not been rolled up yet (or one carrying no companion at all), and MUST be treated as unknown, i.e. a candidate, never as empty.

That escape hatch is the only one: containment is a claim about a listed shard, so a producer whose read of any input behind a shard's word FAILED — one window leaf of several, one unreadable companion — MUST omit that shard from shards rather than publish the word it managed to join over the rest. Dropping the shard costs a reader one candidate it must open; publishing a partial word costs it data it will never look for. A field a leaf simply does not carry (one added to the store after that leaf was written) is absence, not failure: it holds no observations, so the word over the remaining fields is still whole.

Open question, flagged not decided (#480): whether a multi-field store should additionally publish per-field maps instead of (or beside) this union. The union is the default because coverage is a "is there any data here" question; a store whose fields have genuinely different temporal extents would be better served by per-field maps. If that case earns it, it arrives as an additional key under temporal (e.g. shards_by_field) in a later revision — shards keeps this meaning unchanged. The count side of this question is closed (espg rulings of 2026-10-01 on #575): §10.3's counts come from the leaf's worker alone, which folds the one clock every declared field shares — an observation is counted once however many fields are declared — and a leaf read from its per-field companions contributes coverage and no count, so there is no field to choose a count over. The union map and the counted cover both describe "any data"; a per-field count breakdown, if a store ever earns one, would likewise arrive as an additional key and leave counts unchanged.

10.3 Tier 2 — the root counted cover

Contract, optional (espg ruling of 2026-09-17 on issue #575, replacing the root time-digest this block carried under the same spec — no published store carried a section at the time, and the §7 fixtures regenerate). The counts block is the store-wide counted cover: the observation count in every aligned time bucket the store's temporal observations fall in, carried in the §10.5 word grammar so a reader needs no decoder it does not already have for the cover sibling:

"counts": {
  "temporal_order": 24,
  "cap": 512,
  "element": {"dtype": "uint64", "shape": [-1]},
  "encoding": "base64",
  "words": "…",
  "obs": "…",
  "count": 5,
  "obs_total": 346,
  "uncounted_shards": 0
}

• temporal_order (required) — the bucket order of words, on §10.5's grid: aligned buckets of 2^(63 − o) ns. Producers under this revision count at §10.5's one pinned order, 24 (2^39 ns ≈ 9.2 min), and coarsen below it only under cap; a block declaring an order above the pin was written by a producer this revision does not know, and a reader MUST refuse it.
• cap (required) — the overflow cap the producer enforced, with §10.5's meaning: the block holds at most cap words.
• element / encoding (required) — the one byte grammar for BOTH buffers: count little-endian uint64 values each, base64'd.
• words (required) — the bucket words: the aligned order-temporal_order bucket range words (§10.5's quantization of an instant is exactly one such word), sorted, unique, and un-coalesced — abutting occupied buckets stay distinct words with their own counts. toc_normalize's coalescing is never applied to this buffer: a coalesced range would no longer name a bucket a count could belong to.
• obs (required) — the observation count in each bucket, row-aligned with words. Integers, and possibly 0: a word marks its bucket occupied, and a leaf that carries no worker record contributes its occupied buckets with nothing counted (coverage only, below). A reader MUST accept a zero count and MUST NOT read it as an empty bucket; "is there data in this window" reads words, "how much" sums obs.
• count (required) — the bucket count k. A reader MUST refuse a block whose buffers disagree with it or with each other (§1.1's row alignment, broken).
• obs_total (required) — the sum of obs, which a reader MUST refuse a block for disagreeing with. It is the number of temporal observations counted, once per observation however many fields fields names: the clocked-observation count of the shards shards lists when uncounted_shards is 0, a lower bound on it otherwise.
• uncounted_shards (required in the root block) — how many of the shards shards lists hold at least one leaf that contributed coverage only (below): its buckets are keyed in words and none of its observations is in obs. An integer ≥ 0; a reader MUST refuse a root block that lacks the key or carries anything else.
• 0 — every obs is the exact number of clocked observations those shards hold in its bucket, and obs_total their total.
• greater than 0 — every obs, and obs_total, is a lower bound: that many shards hold observations no producer counted. A bucket at obs: 0 is occupied by such observations alone; a bucket with a positive count may hold more than it says. The block does not name the shards, so a reader that needs an exact count for a window MUST treat a nonzero marker as "at least this many", never as the count.

It describes the walk that built the block, and travels with the block (§10.4 replaces tier 2 whole). A §10.6 record's counts block omits it: a record is one leaf's exact count.

(Informative.) No object of this contract names the uncounted shards. zagg's sweep names them in its own per-pass run record (sweep_stats_*.json, telemetry outside this specification) as pass_uncounted: {count, shards, truncated} — the decimal ids of the shards that pass read coverage-only, capped, with the exact count and a truncated flag beside them (issue #598). That list is a pass's tally, scoped to the shards the pass visited (one partition's, on a partitioned pass); it is not this marker, which is the standing figure for the shards shards lists. The two are not interchangeable.

What is counted (normative). Counts have one source: the leaf's own worker, which folds the ONE clock column every declared field shares and writes the result as the leaf's §10.6 record. Each clocked observation the leaf aggregated is one count in the bucket of its own instant, whatever its payload values were — a row whose value is non-finite still has a clock — and however many fields are declared. A producer composing the root block sums those records (the merge law below) and derives no count of its own.

Coverage only (normative). A leaf with no usable record (§10.6 — written before the record existed, or whose record was lost or damaged after the stamp, or is bypassed or unreadable on this pass; a writer never stamps a leaf whose record write failed) is read from its committed §8.3 companions instead, and contributes its coverage and no count (espg ruling of 2026-10-01 on issue #575, retiring the weight-derived count this section carried for such leaves):

• every declared field's companion words join the §10.2 envelope, and the bucket of each word's representative instant (below) is keyed in words — the union across fields that §10.2 and §10.5 require ("any data");
• nothing is added to obs, for any field. A producer MUST NOT derive an observation count from a payload. A digest's weights are one field's: the rows its where predicate admits (a stratum digest, §2.1), with a finite value (a digest drops non-finite rows; the clock does not), and calibrated flux rather than counts under weights: "flux" (§2.0). No choice of field makes them the leaf's clocked observations — the shipped atl03_tdigest_strata_healpix.yaml declares two complementary where strata, so either field alone is one stratum — so no count is published for such a leaf, rather than a one-field figure beside the workers' exact ones;
• each field's payload is still read, for §1.1's row alignment only: a cell whose companion word count disagrees with its centroid count is refused, and the shard omitted under §10.2's whole-word rule. A field the leaf lacks (one added to the store later) or holds no word for contributes nothing;
• the leaf's shard is counted in uncounted_shards.

How a word lands in a bucket. Its instant is its §8.3 word's representative instant: the instant itself for a timestamp word, the envelope's midpoint for a range word. A worker record (§10.6) folds per-observation words, every one a timestamp, so its buckets are exact as its counts are. A coverage-only leaf folds per-centroid companions: a weight-1 centroid's word is its observation's own instant, and a merged centroid's is a range, keyed at the ONE bucket of its envelope midpoint. That is the one approximation this tier carries, it is confined to such leaves, and §10.5's parity paragraph states what it costs the cover.

Laws (normative). All exact, none a merge law:

• Merge of two covers on one grid — the window leaves of one shard, the partitions of a sweep, the children of a ladder node: the per-word sum over the union of keys. Same order ⇒ same grid ⇒ no rounding. Covers at two orders merge by first coarsening the finer to the coarser. A coverage-only leaf merges like any other: its keys join the union and its zeros add nothing.
• Coarsen to an order o' < o: each word maps to its order-o' ancestor bucket, and counts sum per ancestor — exact at the coarser rung. A producer applies it only to fit cap, by whole orders, recording the surviving order in temporal_order; it MUST NOT truncate the word list instead (§10.5's rule, the same reason: silently dropped coverage).
• The §10.5 cover is derived from it: take the keys, toc_normalize. This MUST equal §10.5's quantization over the same instants at the same order — the two are one law on one grid — so a cover is never computed separately from its counts. The derivation reads the counts before this section's cap: the cap here counts un-coalesced buckets, §10.5's counts the cover's words (abutting buckets coalesce into one), and the two are different quantities. A producer therefore derives the cover from the uncapped counts and then applies §10.5's own cap to it, so a shard whose coalesced cover fits stays at the pinned order however many buckets it occupies. The one exception is a §10.6 record, whose cover is derived from the counts the record actually carries (below).

Why a counted cover and not a digest. The sources are spikes: a pass crosses an order-9 shard in about a second, so a shard's time distribution is a weighted point set of tens (ATL03) to a few hundred (GEDI) instants. A t-digest keeps approximate density in adaptive bins and places merged mass at midpoints where nothing was observed; the counted cover keeps exact occupancy and exact counts per bucket, composes by addition, and coarsens exactly. Because a pass is ~1 s wide, the word count does not grow with finer order until buckets approach pass duration (about order 33): 49 passes are 49 words at order 18 and still 49 at order 24, so the one pinned order (§10.5) costs nothing over a coarser one and answers "within the hour?" from the block alone; a time-dense source hits the cap sooner and coarsens exactly as §10.5 specifies. Everything the digest answered is derivable from the counted cover — the mass in a window is the sum of obs over the buckets it covers, with at most one partial bucket at each edge, and a reader that wants a digest builds one from (bucket midpoint, count) — and the reverse is not true.

Gaps stay visible exactly as in §10.5: a bucket with no data has no word. "Is there data in this window at all" reads the words; "how much" sums obs over the buckets the window covers, resolved to the bucket and nothing finer.

10.4 Composition

Contract. The root object is written GET-union-PUT (incremental runs accumulate; concurrent runs race benignly), and the section composes across that seam as follows:

• Tier 1 unions elementwise under toc_merge: a shard on both sides merges to the join of its two words, a shard on one side carries over unchanged. The join is idempotent, so re-walking unchanged leaves reproduces the identical map.
• Tier 2 is never summed at this seam. Its values are counts, and adding two covers over overlapping shard sets would double-count them (§10.3's merge law is for DISJOINT contributors — window leaves, partitions, children). It is replaced — the whole block, its uncounted_shards with it, since the marker describes the walk that counted — and only by a producer whose own map covered every shard the merged map lists; a producer that covered only part of the store publishes no counts and leaves the standing block alone, and a merge that can find no whole-covering block on either side drops it rather than publish a partial one.
• A producer with no temporal contribution at all leaves an existing section untouched — it is not evidence of absence, only of a walk that did not look. Conversely a producer that overwrites the carrier wholesale (an unparsable or incompatible existing root — the D9 regenerable-cache rule) discards the stale section with it.
• The cover marker carries through the merge: the incoming section's value wins when both sides carry one, and a marker on either side alone survives — the object it points at composes under §10.5's own seam, so a producer that did not write the sibling must not erase the standing pointer to it. One exception, the succession rule below applied to the marker: an incoming marker naming the producer's OWN cover revision MUST NOT overwrite a standing marker naming a revision that producer cannot read. §10.5 preserves such an unknown-revision object on every write path, so the standing marker still names the object that is actually there; downgrading it would leave the carrier advertising a revision the sibling does not carry, contradicting §10.1's definition of the value, and would leave producers at two revisions rewriting the root object forever.
• An existing section at an unknown revision is preserved verbatim, and never downgraded. The strict spec gate above makes such a section read as absent, but composition is a write, and dropping the key is not the same as ignoring it: a producer MUST copy an unreadable standing section through unchanged, both when it has no section of its own and when the section it holds is at an older revision. This is the page's standing succession rule (readers add revisions, they never drop them) applied at the one seam a mixed-version fleet actually meets — without it the oldest producer in the fleet wins. An unmarked carrier (a temporal value with no spec string at all) claims no revision and is debris a producer MAY replace.

Conformance for an external reader is §7's temporal/ fixture: its root coverage.moc carries this section, and the fixture's temporal.expected.json records the shard word and the decoded counted cover so the containment and count claims above are pinned on committed bytes — exact counts, under uncounted_shards: 0. §7's uncounted/ is the same leaf read without its record: the same buckets at obs: 0 under uncounted_shards: 1, which is §10.3's zero-count grammar and lower-bound marker as bytes.

10.5 The word-set cover sibling — zagg-coverage-toc-cover/1

Status: contract (issue #489).

Tier 1 gives one envelope word per shard, so gaps are invisible below shard granularity: a shard observed in 2019 and again in 2025 answers every window in between. The cover restores the grammar's never-bridge law at sub-shard resolution — per shard, a canonical multi-word set (mortie.toc_normalize over that shard's §8.3 companion words, quantized as below) whose surviving decoded gaps are floors on the true gaps. It is the multi-order-MOC analog on the time axis: the §10.2 word is the overview, this is the index.

It lives in its own root object, {store_root}/coverage.toc, beside the bootstrap sidecar — never inline: at the published CA store's shape it is ~50–60 words per shard (~1.5 MB store-wide, ~15× the tier-1-carrying bootstrap object it sits beside, and ~300× a spatial-only pre-#480 root), and a spatial-only reader must not pay that on every bootstrap GET. Temporal consumers GET it on demand, discovering it through §10.1's cover marker. Like the section it refines, it is a regenerable accelerator over the leaf arrays (D9), under the same staleness posture: a shard it does not list is unknown, a candidate — never empty.

Contract.

{
  "spec": "zagg-coverage-toc-cover/1",
  "source": "sweep",
  "generated_at": "2026-08-23T02:41:00+00:00",
  "order": 4,
  "temporal_order": 24,
  "cap": 512,
  "fields": ["h_tdigest"],
  "element": {"dtype": "uint64", "shape": [-1]},
  "encoding": "base64",
  "shards": {
    "11213": {"words": "…", "count": 3}
  }
}

• spec (required) — "zagg-coverage-toc-cover/1", the object's OWN marker (it is a root object, not a section, so it gates itself the way the carrier envelope does). A reader MUST strict-check it, and an unknown revision reads as absent — degrade to tier 1 or open leaves, never refuse the store. A future revision is a new spec string; keys are never repurposed in place.
• source / generated_at (required) — as §10.1's, the object's own provenance and clock.
• order (required) — the shard order of the shards keys, spelled as the carrier's order is; keys are D1 decimal shard ids at that order, exactly as §10.2's are.
• temporal_order (required) — the object's pinned quantization order, below. This revision's producers write 24.
• cap (required) — the overflow cap the producer enforced, below. This revision's producers write 512.
• fields (required) — provenance, with §10.1 fields semantics verbatim: the union of payload field names whose companions any contributing producer read; an upper bound, not a per-shard list.
• element / encoding (required) — the one byte grammar, declared once for every shard block: each words value is base64 of the words' §1.4 element bytes — count little-endian uint64 toc words, exactly what one zagg-ragged/1 uint64 element holds — so a reader decodes it with the leaf decoder it already has.
• shards (required) — one block per covered shard:
• words — the base64 word set: the canonical (toc_normalize-form) cover of every §8.3 companion word the shard's leaves hold, unioned across the store's temporal-carrying fields (§10.2's union rule), quantized at the shard's effective temporal order. Sorted, duplicate-free uint64, mixed timestamp/range members legal under the grammar's kind-keyed semantics.
• count — the word count. A reader MUST refuse a block whose buffer disagrees with it (§10.3's centroids rule, one buffer instead of two).
• temporal_order (optional) — present iff this shard coarsened below the object's pinned order under the cap; absence means the object's temporal_order (not any order a reader's own build pins), which is also the ceiling: always ≤ the object's temporal_order, and a reader MUST refuse a block declaring an order above it.

Quantization. Temporal order o partitions the toc scale (2^63 ns from the grammar's 1850 epoch — the span the 31-bit end code at 2^32 ns closes) into 2^o aligned buckets of 2^(63 − o) ns. A word enters the cover with its conservative decoded envelope (toc2time) widened to the bucket grid — start floored, end ceiled — re-encoded as a range word, and the set canonicalized with toc_normalize. Bucket bounds are exactly representable on the grammar's own encoding grids for every o ≤ 31 (a bucket start is a multiple of 2^31 ns and its end of 2^32 ns), so the re-encoding adds no rounding of its own; the one exception is the scale ceiling, where the top bucket's end clamps to the grammar's maximum encodable end (TOC_MAX_NS) — still containing every encodable input word.

The pinned temporal order is 24: bucket span 2^39 ns ≈ 9.2 min (espg-ruled on issue #575, 2026-09-17, retiring the order-18 pin of issue #489). It is the ONE rung of the temporal path: the leaf record's counted cover (§10.6), the root counted cover (§10.3) and this object all quantize at it, and the cap below is the only coarsening mechanism — coarser effective rungs (a root over years of daily passes, a coarse ladder node) are data-driven, recorded per block, never pinned. The ladder is the grammar's own power-of-two structure; the rung on it is chosen for the consumer, because nothing else constrains it: correctness is order-independent (quantization only widens at any order), and storage does not depend on it, because the sources are spikes — a pass crosses an order-9 shard in about a second, so a shard's time distribution is a point set of tens to a few hundred instants and holds the same ~50–60 words at order 24 as at 18 (the word count only grows once buckets approach pass duration, near order 33). Order 24 keeps consecutive orbits (~95 min) and a day's ascending and descending passes distinct, resolves time-of-day, and holds the epoch error of a cover-bucket midpoint to ±half a span ≈ ±4.6 min — so the closest-observation consumer's argmin over |t_other − t_epoch| cannot flip on a pass that falls near the midpoint between two acquisitions, and "within a day of the other sensor" is answered by the candidate set itself. A time-dense source (hourly data hits the cap in months rather than years) is absorbed by the cap below, coarsening by whole orders as specified there, never by re-pinning. Note the bucket span is not the gap promise: a surviving gap needs a whole aligned bucket to itself (below), so the guaranteed gap floor is two spans, 2 × 2^39 ns ≈ 18.3 min.

Three consequences, all normative:

• Widening only. The cover's decoded coverage contains the shard's true observation set: quantization widens, toc_normalize is coverage-exact, and the union across leaves and fields is a union. A cover may over-claim — by strictly less than one bucket at each cluster edge — and MUST NEVER false-negative. toc_overlaps on the words therefore never under-reports, and toc_contains never over-reports, exactly as in §10.2.
• Gaps survive at bucket resolution. The never-bridge law holds on the quantized words, with an exact criterion: a gap survives iff it contains a whole aligned bucket. Widening lands the instants either side of a gap in their own buckets, and toc_normalize coalesces ranges that merely abut, so a gap of exactly one bucket span always closes; a gap of two spans or more always survives; and a gap between one and two spans survives only when it happens to straddle a bucket boundary the right way — alignment, i.e. data-dependent, so a consumer MUST NOT reason on it. The guaranteed floor a consumer may rely on is therefore 2 × 2^(63 − o) ns — at the pinned order 24, ≈ 18.3 min. "Is there data in [t0, t1)" answers per shard from this object alone, down to that floor.
• Quantization commutes with union and with the envelope join, which is what makes the per-leaf fold exact (the cover of a union of leaves is the normalize of the union of their covers) and the parity invariant below well-defined.
• The cover is the counted cover's key set (§10.3, §10.6): a producer holding a leaf's or a shard's counted cover derives this object's words by taking its keys and normalizing, and MUST get exactly what quantizing the instants directly gives at the same order — one law on one grid. It derives them from the counts as counted, before §10.3's cap on the buckets, and then applies the cap below to the resulting words: the two caps count different things (buckets there, coalesced words here), so a shard holding more than cap occupied buckets in one unbroken run is one word here and stays at the pinned order.

The cap. A shard's block holds at most cap words. A producer whose cover lands above it MUST coarsen by order — re-quantize at o − 1, halving the bucket count — until it fits, recording the landing order in the block's temporal_order. The same widening law (order 0 is a single bucket, so the loop terminates); a producer MUST NOT truncate the word list instead, which would silently drop coverage.

Parity invariant. For every shard listed both here and in §10.2's map, the words MUST satisfy

toc_reduce(words) ⊆ toc_reduce(quantize({§10.2 word}, o))

at the shard's effective order o — the cover's own envelope lies inside the quantized tier-1 envelope — with equality whenever every leaf behind the shard contributed per-observation words (a worker-written §10.6 record, or a raw read of a leaf whose centroids are all weight-1): the cover and the quantized join are then one law over one set, which is the commutation above. A leaf with no record is read from its per-centroid companions, and a merged centroid is keyed at the one bucket of its envelope's midpoint (§10.3), so a shard holding such a leaf may have a cover that sits strictly inside its tier-1 envelope — and for that shard the widening-only law above is not guaranteed either: an instant a merged centroid absorbed can fall in a bucket the cover does not list, though never outside the shard's §10.2 word. The root block's uncounted_shards (§10.3) is the count of shards this can affect; at 0 the three consequences above hold on every shard. Either way the relation is the cross-object consistency check a reader MAY apply cheaply; zagg's suite asserts equality on every shard the fixture writes. (Plain equality with the §10.2 word itself does NOT hold: that word lives on the grammar's native 2^31/232 grids, the cover on the bucket grid.)

Composition. The object is written GET-union-PUT across the same seam as the carrier (§10.4), composing per shard: the union of the two word sets, re-quantized at the coarser of the two effective orders (a union alone could interleave two orders' bucket bounds and break the parity claim), then re-capped — which may coarsen further, under the same widening law. A shard on one side carries over unchanged. The §10.4 succession rules apply verbatim at object level: an unknown-revision standing object is preserved and never downgraded (on the refresh's replace path too); an unmarked or unparsable one is debris a producer replaces — "unparsable" including a standing object that is valid JSON but not a decodable cover (a block disagreeing with its own count, a corrupt words buffer, a non-object block), since the truth is in the leaves and this object is regenerable; a producer whose read of any input behind a shard FAILED omits that shard (§10.2's whole-word rule); a producer with no temporal contribution leaves the standing object untouched.

order gates the union, exactly as it does for the carrier (§10.4): the shards keys are D1 ids at that order and ids at two orders are not comparable, so when the standing object's order differs from the incoming one's the standing object is incompatible debris and the incoming side REPLACES it wholesale rather than unioning. A re-shard therefore moves both objects together instead of leaving the sibling holding two orders' ids under one declared order.

The sibling's life is tied to the carrier's. A producer that DELETES {store_root}/coverage.moc wholesale — the refresh path, when no stamped leaf remains — SHOULD discard {store_root}/coverage.toc with it: the §10.1 cover marker lives in the carrier, so a surviving sibling is an orphan no reader discovers, and a later carrier at a different shape would find ids it cannot key. A reader that reaches the sibling anyway is still safe: an orphan only over-claims, and a cover shard absent from the carrier's map composes under the standing staleness posture — unknown, a candidate, never authoritative, exactly as a shard the cover does not list is unknown rather than empty.

That licence is the deletion case only. A producer that merely REWRITES the carrier while producing no cover of its own does not discard the sibling: it has looked at no temporal input, which is not evidence of absence, and the composition rule above governs instead — the standing object is left untouched. Otherwise a defect that costs a walk its inputs (a manifest missing cell_order, companions not yet written) would destroy a perfectly good cover on the strength of the defect, and the two producers would answer differently on identical evidence.

Conformance is §7's temporal/ fixture again: it is the only fixture carrying a coverage.toc, written by the production sweep writer beside its coverage.moc, and temporal.expected.json's cover block records the decoded word set — derived from the generator's inputs through the quantization law above, never transcribed — plus the parity claim on committed bytes, the required keys above, and the gap_ns interval the fixture's two clusters leave uncovered, which is what makes the object a test of the never-bridge law rather than of a single bucket. The other fixtures carry no coverage.toc at all, which pins the absence rule as bytes, exactly as §10's section absence is pinned.

10.6 The leaf temporal record — zagg-leaf-temporal/1

Status: contract (issue #575; espg rulings of 2026-09-17 on the record's channels and of 2026-10-01 on its one producer and its fail-closed write).

§10.2, §10.3 and §10.5 describe the root section's outputs; this section describes where their per-leaf inputs are kept. All three were originally derived at sweep time by reading every leaf's raw §8.3 companion column back, which at the published California store's shape (a million-row ragged array per field per leaf, ~2,964 leaves) cannot finish inside one invoke at all — while the worker that wrote the leaf held every observation's toc word in memory, per chunk, and threw it away. The record keeps what that worker knew: one small JSON object per shard leaf, {leaf}/temporal.toc, beside the leaf's coverage.moc occupancy bitmap (hive_layout.md), carrying the leaf's §10.2 envelope word, its §10.3 counted cover, and the §10.5 cover derived from it. {leaf} is where the leaf's arrays live under §1.5's one reader rule: the stable root {id}.zarr/ of a legacy leaf, the version subgroup {id}.zarr/{current}/ of a versioned one — never the pointer root (which holds the pointer stamp and its versions, plus, on a converted legacy leaf, the last legacy write's arrays and sidecars, which a versioned reader ignores). A reader of the record resolves current first, exactly as a reader of the arrays or the bitmap does.

It has the bitmap's placement and its posture. It is never truth; it is written before the commit stamp, so an unstamped prefix's record is debris with everything else (D4) and the stamp stays the leaf's final write; and it is additive — a leaf written before this revision simply lacks it, and a reader treats absence as "read the leaf". Its envelope word is a regenerable accelerator (D9) over the leaf's own arrays, exactly, and its cover up to §10.5's merged-centroid approximation. Its counts are not regenerable: the leaf stores no per-observation clock, only per-centroid companions, so the counts exist nowhere but here, and reading the leaf in the record's absence yields the leaf's coverage and no count (§10.3). Like the bitmap it is a foreign key inside the otherwise-vanilla leaf: zarr data reads are unaffected, and member enumeration warn-skips it. It is not a data object in §1.5's write-once sense — it is a leaf-internal sidecar sealed by the same stamp — and the §11 Icechunk companion never references it: the ref plan enumerates the template's named arrays and reads their chunk keys only, so neither this record nor coverage.moc is referenced or checksummed, in a version or at a legacy root.

Written by the leaf's worker, and by nothing else. The record is written once, by the worker that wrote the leaf, before that leaf's stamp (a versioned leaf's: before the version stamp) — or, for a leaf holding no clocked observation, not at all. No other producer creates one or replaces one: a sweep or refresh that had to read a leaf raw MUST NOT write a record for it, and MUST NOT replace a record it finds — not a missing one, not debris, not one at an unknown revision (espg ruling of 2026-10-01 on issue #575: no backfill). Two reasons, either sufficient:

• Counts have one source (§10.3). A record composed from a leaf's companions would carry zero counts, and the next reader would take it for the worker's — the store's counts would be incomplete with nothing left to say so.
• A stamped version is never written into. §1.5 fixes a version subgroup's objects once its stamp lands, and this record is one of them.

A leaf without a usable record therefore reads as absence does everywhere: the raw route on every pass, coverage only, its shard counted in the root block's uncounted_shards (§10.3), until the leaf's next replacement, whose worker writes a fresh record. A store written before this revision stays record-less — its root section carries tier 1 and a counts block of zero-count buckets — and is brought up to date by rewriting its leaves, not by a sweep. The rest follows from the placement: the #388 lifecycle touch never reaches the record of a versioned leaf (it touches no version object, §1.5; on a legacy leaf the record rides the leaf tree like the bitmap), and the §11.4 collector reclaims it with its version, which it deletes whole.

The write fails closed. The record takes the bitmap's slot and its failure posture, for its own reason. The commit stamp points AT the bitmap (§ the stamp's coverage), so a stamp published without one is a false claim. Nothing points at this record — but its counts exist nowhere else, no later producer writes it (above), and a leaf stamped without it is, to every later run, a finished leaf: zagg's rerun identity compares the configuration and the input set, not the record, so the leaf would be skipped as current and stay uncounted until something else forced its rewrite. A writer under this revision therefore MUST NOT stamp a leaf whose record could not be built or written, when the leaf's fold saw at least one clocked observation (espg ruling of 2026-10-01 on PR #578). The failed write fails the unit before the stamp: the prefix is unstamped debris (D4); on a versioned leaf the pointer is not swapped, so the previous current, if any, keeps serving and the attempt's version is reclaimed as any dead attempt's is (§1.5, §11.4); and the unit is retried as any failed leaf write is — a legacy leaf cleared and rewritten in place, a versioned one under a fresh version. A fold that saw no clocked observation writes no record and stamps: that is the absence rule below, not a failure.

One record per window leaf. On a windowed store (§4.2) every {id}_{window}.zarr is a leaf, so each carries its own record, folded from that window's observations alone: its envelope word lies inside the window (its decoded bounds may pass the window's ends only by the word grid's outward snap, §8), and the shard's contribution to the root section is the composition of its window leaves' records (§10.2, §10.3). The rule does not depend on which unit wrote the leaf. A writer that emits several windows of a shard from one read (zagg's bulk per-shard unit, issue #586) MUST count each observation into the record of the one window that holds it, MUST write each leaf's record before that leaf's stamp, and MUST treat a record that cannot be built or written as that window leaf's failure — the leaf is left unstamped and the shard's other window leaves are unaffected. The record is the one the window's own unit would have written, byte for byte but for generated_at.

A reader's rule is unchanged: an absent or unusable record is coverage only (§10.3), never a refusal of the leaf. With the writer failing closed, the cases that still reach it are a leaf written before the record existed; a record damaged or lost after its leaf was stamped; a record at another revision; and a record whose read failed on this pass. A record's fields list is not among them (below). Each is reported the same way — the shard in the root block's uncounted_shards — instead of a total that silently omits it. (Informative.) The marker is a count; a zagg sweep pass names the shards it read coverage-only, up to a cap, in its own run record (§10.3).

Absence is the rule for non-temporal stores. A leaf is written with a record iff its config declares a §8.3 "per-centroid" field and the leaf holds at least one clocked observation. A "per-cell" or "coordinate" declaration alone arms nothing (it is a different array grammar and contributes nothing to §10), and an empty leaf publishes no temporal claim. Every other fixture leaf under §7 carries no such object, which is that rule pinned as bytes.

Contract.

{
  "spec": "zagg-leaf-temporal/1",
  "source": "worker",
  "generated_at": "2026-09-17T18:02:11+00:00",
  "fields": ["h_tdigest"],
  "n_obs": 346,
  "word": "10689250968998868755",
  "temporal_order": 24,
  "cap": 512,
  "counts": {
    "temporal_order": 24,
    "cap": 512,
    "element": {"dtype": "uint64", "shape": [-1]},
    "encoding": "base64",
    "words": "…",
    "obs": "…",
    "count": 5,
    "obs_total": 346
  },
  "cover": {
    "element": {"dtype": "uint64", "shape": [-1]},
    "encoding": "base64",
    "words": "…",
    "count": 2
  }
}

• spec (required) — "zagg-leaf-temporal/1", the object's OWN marker (it gates itself, as the §10.5 sibling does). A reader MUST strict-check it; an unknown revision reads as absent — read the leaf — never as a refusal. No producer overwrites it, nor any other body it finds in the record's place — an unmarked or unparsable one included (above).
• source (required) — "worker": the leaf's own worker, from per-observation words at commit, which is this revision's only producer of the record (above). The key stays required and its vocabulary open, as §10.1's is — it is provenance, and a later producer of exact counts would name itself here — so a reader MUST NOT gate on its value.
• generated_at (required) — ISO-8601 UTC, the object's own clock.
• fields (required) — the sorted payload field names the writing config declared "per-centroid". Provenance, with §10.1 fields semantics, and nothing else: a reader MUST NOT gate on it. A record that passes this section's checks is used whether its fields equal, omit or exceed the set the manifest declares, because nothing else in the record depends on the list — the worker folds one shared clock column (n_obs below), so the word, the counts and the cover are the same under any declared set — and because a record set aside is never replaced (above): the leaf would contribute coverage only (§10.3) until it is rewritten, its exact count discarded for none.
• n_obs (required) — the leaf's temporal observation count, which MUST equal the counts block's obs_total: the number of clocked observations the leaf aggregated, once each (§10.3's count rule). The worker folds one shared clock column, so a second declared field adds nothing, and a row whose payload value is non-finite still counts.
• word (required) — the §10.2 envelope word for THIS leaf, as a decimal string: the grammar's join (toc_reduce) over every observation word the leaf holds. Because the join is a semilattice, it is identical whether folded over the per-observation words (the worker) or over the leaf's per-centroid companions (a raw read), and a shard's §10.2 word is the join over its window leaves' words.
• temporal_order / cap (required) — the §10.5 pin and cap the record was written against, with §10.5's meanings verbatim (24 and 512 under this revision). One rung serves both blocks: counts is counted at it, and cover is its key set.
• counts (required) — the leaf's counted cover, in §10.3's block grammar: temporal_order (the record's pin, lower only when coarsened under cap), cap, the two row-aligned buffers, count and obs_total, with §10.3's MUST-checks — and without uncounted_shards, which is the root block's alone. Counted from the per-observation words: each instant in its own bucket, exact.
• cover (required) — the leaf's §10.5 word set, derived from the counts this record carries — after their cap, not before it — by §10.3's law (its keys, toc_normalized, at its order), and carried so a reader wanting only the word set needs no decoding of the counts. This is §10.3's one exception to deriving the cover from the uncapped counts, and it is what makes the record self-checking: a reader recomputes the cover from the block it is handed and refuses a record whose two blocks disagree. A producer composing shard blocks for §10.5 from records still applies §10.5's own cap to the composed words, not this one. In the §10.5 block grammar (words base64 of count little-endian uint64 words; temporal_order present iff the leaf coarsened below the record's pin, absence meaning that pin, never an order above it) plus its own element/encoding declaration, since the record carries no object-level one. A reader MUST refuse a block whose buffer disagrees with its count, and MAY check the derivation; zagg's reader does, and treats a record failing it as debris.

Per-leaf containment. §10.5's parity relation holds on every record at the cover block's effective order o: toc_reduce(cover) lies inside toc_reduce(quantize({word}, o)), with equality for a worker record: every instant is an observation word, so the cover and the quantized join are one law over one set. A reader MAY check it and MUST treat a record that fails it as debris — absent, read the leaf.

Composition into the root. A record is one leaf's contribution to the §10 section and its §10.5 sibling, composing exactly as a raw-read leaf does: a shard's shards word is the join over its window leaves' words; the root counts and each shard's cover block come from the per-word sum of the leaves' counts (§10.3's merge law), capped, the cover then derived. A producer that reads records mixes them with raw-read leaves in one walk — the two feeds meet the same laws, a raw-read leaf adding its word, its zero-count buckets, and its shard to the root block's uncounted_shards — and the whole-word rule of §10.2 applies unchanged: a shard any of whose leaves failed to read (record or raw) is omitted, never published partial.

Conformance is §7's temporal/ fixture: its one leaf carries the record, written by the production worker path, and temporal.expected.json's leaf_temporal block records the object name, the required keys, the envelope word (derived from the generator's per-observation instants, and equal to the root section's shard word), the decoded counted cover and the decoded cover — each derived through §10.3's laws from those same instants, never transcribed — so the record is pinned against the generator's inputs, not against itself. uncounted/ is the absence side: the root objects over that same leaf with the record gone, which no producer wrote back.

---

11. Icechunk companion repo

Status: contract (zagg-icechunk/2, issue #580, stage 1 — refs-only, additive; issue #582, stage 2 — the two-plane statement, run tags, finalize and the metadata operations below; issue #584 — the row dimension of §11.2, the /2 array model). The leaves remain the normative, self-describing data plane (§1–§10); the companion is a derived index over their bytes.

Two planes. A hive store is two planes with different mutability:

• the data plane is the leaf shard objects — write-once under §1.5: no byte at a stamped leaf key is modified in place, and the one legal change is wholesale replacement under the same keys (a replaced leaf's earlier references fail loudly by checksum, §1.5/§11.3, so a run tag guarantees the leaves not replaced since it); a leaf's content identity is its §5.3 O11 digest in the commit stamp plus the ETag every virtual reference carries (§11.3);
• the metadata plane is the one repository — authoritative for what evolves: convention and attrs blocks (the dggs latitude token, spec markers — a leaf stamp's or a convention block's /1→/2 revision carried in attrs, never the repo's own array model), the pyramid declaration mirrored as multiscales, and the run history (tags, §11.4). A leaf's own zarr.json keeps being written (a leaf stays a valid standalone zarr) but is frozen with the leaf: shape, dtype, chunking and codecs never diverge from the repo's array model; attrs may, by design, and the repo's are the ones a reader binds to.

"Backfill" is therefore not a category: an evolving fact is written to the repo in a commit, never by rewriting leaves — the §11.4 operations (zagg.icechunk_ops: set-attrs, declare-pyramid) are how an operator writes one. Stage 2's remaining items are tracked on issue #582: native overview chunks (issue #584 phase 3, on the /2 array model), moczarr's reads through the repo (phase 6) and the browser (phase 7).

Succession. A change to the repo's ARRAY MODEL is a new revision, declared in the zagg_icechunk.spec token (§11.1), so a reader discriminates revisions from the repo's own attrs. zagg-icechunk/2 (issue #584) is the row model: every level array gains a leading row dimension (§11.2), which is what lets one repo hold a windowed store's per-window leaves and an unwindowed store alike. zagg-icechunk/1 (the same arrays without the row dimension; run tags and the finalize commit were additive on it) stays valid and readable as written — its arrays are exactly the /2 arrays' single row — but it is never written again: the /2 writer refuses a /1 repo at every entry point (the init, every ref commit, finalize, the operations and the virtual-target collector) with an error naming the remedy, and there is no in-place upgrade, because no published store carries a /1 repo. The remedy is to clear {store_root}/icechunk/: the next run's init re-creates the repo as /2 and a staged sweep over the store re-gathers its refs (§11.4; a leaf's ref carrier written before /2 names no row and is read as the all row). That re-gather reads the ladder's carriers, so it covers leaves written under commit: "ladder"; a leaf committed per leaf (commit: "leaf") wrote no carrier, and is indexed again only when a run rewrites it.

A morton hive is many leaf zarrs. The companion presents every leaf of one order as one zarr hierarchy by recording each leaf's inner chunks as Icechunk virtual chunk references — (location, offset, length) byte ranges into the leaf objects that already exist — so an Icechunk reader (icechunk-py, icechunk-js in a browser) opens the hive as a single array per field without moczarr's hand-built virtual store, and every hive commit maps onto an Icechunk snapshot.

11.1 Placement and naming

Contract. One repository per store, at the store root, with one zarr group per level, named by the level's CELL order — exactly the manifest's zagg-multiscales/1 datasets (§4.9) plus the base:

{store_root}/icechunk/          <- the Icechunk repository
   /19                           <- the base: the source leaves (node order 9, cells 19)
   /13                           <- the §4.6 leaf columns' declared member (node 9, cells 13)
   /12, /11, … /4                <- the §4 overviews, one level per ancestor order (node 8 … 0)
   /window_start, /window_end    <- the row coordinate, shared by every level (§11.2)

level (group)	artifact per node	node order n	cells per object 4^(c−n)	object arrays
/19	{leaf}.zarr/19/…	9 (shard)	4^10, as K = 256 inner chunks of 4^6 (§1.5, sharded)	the leaf template's
/13	{node}/all.pyramid.zarr/13/… (the leaf's sibling column)	9	256, ONE unsharded chunk (a column member of 4^5 cells or more — /14–/16 under overviews: [16, 15, 14, 13] — is one sharded object on the §4.6 inner chunk; its level is chunked at that inner chunk)	the column's declared member
/12 … /4	{node}/all.zarr/{c}/… (the ancestor's overview)	8 … 0 (= c − 4)	256, ONE unsharded chunk	the overview's

(production geometry: shard 9 / chunk 13 / cell 19, declared leaf-node cells 13 — every ladder level keeps c − n = 4; the general rule is the manifest's own datasets). A column holds more members than its declared one — the within-footprint intermediates and the node-order partial the stage gather reads (13, 12, 11, 10 on the live California store) — and those are deliberately not levels: they overlap the overview levels for the same cells and are sweep inputs, not reader-facing artifacts. A level whose artifact a node never wrote (a column exists only under a /2 declaration with composable fields; an overview only once swept) simply has no refs there and reads as fill. Every group is created by the once-per-run init (§11.4) from the manifest's declaration, and the repo's root attrs carry the manifest's zagg-multiscales/1 block as the root multiscales object — its keys verbatim, plus the zarr-conventions layout below — so a reader opens one repo and discovers every level — its node order, cell order and artifact kind — from its root (the GeoZarr/OME-NGFF-style multiscales convention; earth-mover/icechunk-multiscales-demo; what gridlook's level resolver reads). icechunk/ is a reserved store-root child name on the same footing as the §4.10 multiscales/ companion: it is excluded from the D19 product-name grammar (like the base-component exclusion), so a multi-product root walker can never classify it as a product, and zagg.hive.validate_product_name refuses the name outright — a product MUST NOT be named icechunk.

The repo's root group carries a zagg_icechunk attrs block that makes it self-describing:

"zagg_icechunk": {
  "spec": "zagg-icechunk/2",
  "shard_order": 9, "chunk_order": 13, "cell_order": 19,
  "url_prefix": "s3://bucket/product/",
  "rows": ["all"],
  "commit": "ladder", "commit_order": 6, "split_order": 6,
  "levels": {
    "19": {"node_order": 9, "artifact": "leaf",     "chunk_order": 13, "cell_order": 19, "split": {"chunks": 16384, "order": 6}},
    "13": {"node_order": 9, "artifact": "column",   "chunk_order": 9,  "cell_order": 13, "split": {"chunks": 16384, "order": 2}},
    "12": {"node_order": 8, "artifact": "overview", "chunk_order": 8,  "cell_order": 12, "split": {"chunks": 16384, "order": 1}},
    "…":  "one entry per level, keyed by cell order",
    "4":  {"node_order": 0, "artifact": "overview", "chunk_order": 0,  "cell_order": 4,  "split": {"chunks": 1,     "order": 0}}
  }
},
"zarr_conventions": [
  {"schema_url": "https://raw.githubusercontent.com/zarr-conventions/multiscales/refs/tags/v0.1/schema.json",
   "spec_url": "https://github.com/zarr-conventions/multiscales/blob/v0.1/README.md",
   "uuid": "d35379db-88df-4056-af3a-620245f8e347",
   "name": "multiscales", "description": "Multiscale layout of zarr datasets"}
],
"multiscales": {
  "spec": "zagg-multiscales/1",
  "name": "…", "base": {"…": "…"}, "datasets": ["…"], "order2res": {"…": "…"}, "fields": {"…": "…"}, "fold": {"…": "…"},
  "layout": [
    {"asset": "19", "dggs": {"… the /19 level group's dggs block, verbatim …": "…"}},
    {"asset": "13", "derived_from": "19", "transform": {"scale": [4096.0]}, "dggs": {"refinement_level": 13}},
    {"asset": "12", "derived_from": "13", "transform": {"scale": [4.0]},    "dggs": {"refinement_level": 12}},
    {"asset": "11", "derived_from": "12", "transform": {"scale": [4.0]},    "dggs": {"refinement_level": 11}},
    {"…": "one entry per listed level, finest first, down to /4"}
  ]
}

shard_order / chunk_order / cell_order mirror the manifest and the base grid; url_prefix is the virtual chunk container's prefix (§11.3); rows is the ordered list of row labels — row w of every level array is rows[w] (§11.2: ["all"] on an unwindowed store); levels carries, per level group (keyed by cell order), its node order, its artifact kind, its chunk-axis order (the inner-chunk order for the base, the node order for a one-chunk-per-node level), its cell order and its manifest split (§11.5); retired, present only once a declare-pyramid operation has delisted a level (§11.4 Operations), maps each delisted level's cell order to its last levels entry, so its surviving group keeps its manifest split; commit, commit_order and split_order are the ladder's knobs (§11.4, §11.5), read back by every stage node so the sweep needs no config. The repo root group's attrs are exactly these three keys — zagg_icechunk, zarr_conventions and multiscales (the latter two are absent, together, only on a store whose manifest declares no zagg-multiscales/1 block) — and its members exactly the level groups and the two row-coordinate arrays (§11.2). A leaf root group carries only its own commit stamp, which is a per-leaf fact and so has nothing to mirror. Each level group mirrors its artifact's resolution-group attrs verbatim (the dggs block, zarr_conventions), and never the commit stamp.

The root is a conformant multiscales group composed with dggs (#615). The root group document validates against the zarr-conventions multiscales v0.1 schema and composes with the dggs convention per the grammar of zarr-conventions/dggs#25. The level groups, and the base layout entry's dggs object, validate against the dggs v1 schema once #616 lands the semi_major_axis spelling (until then the ellipsoid key alone fails; the conformance test pins exactly that gap, pending #616). Both schemas are vendored under tests/data/conventions/ (in CI only the hand checker runs: jsonschema is not in the test extra), and the icechunk/ fixture (§7) pins the documents.

• zarr_conventions — the convention's registration: the multiscales v0.1 Convention Metadata Object, copied verbatim from the convention's README (the schema pins every key as a const; the UUID d35379db-88df-4056-af3a-620245f8e347 is permanent across versions). Only the multiscales entry rides the root: the tagged dggs v1 schema requires a top-level dggs wherever its own entry is claimed, and the root has none — the per-level dggs blocks live on the level groups.
• multiscales — the manifest's single zagg-multiscales/1 entry as an object (the v0.1 schema requires an object with layout; its additionalProperties: true admits the /1 keys — spec, name, base, datasets, order2res, fields, fold — which ride verbatim, and §4.9's tolerate-extra-keys rule already covers readers of the mirror) plus layout: one entry per listed level (a retired level's surviving group has none), in descending cell order — which is the base, then the manifest's datasets order, because §4.9 requires datasets finest first — asset the level group's name (str(cell_order)), so asset keys and zagg_icechunk.levels keys are the same set in the same order.
• The base entry carries that level group's dggs block whole (dggs#25: "specify the full dggs object for the original data") — a verbatim copy of the live base level group's block, so whatever that group's dggs says is what the layout says: a set-attrs on it (§11.4) refreshes the entry in the same commit, and the init and declare-pyramid rebuild it from the group, never from the grid.
• Every derived entry carries the reduced {"refinement_level": c}, derived_from (another entry's asset) and transform: {"scale": [4^(c_from − c)]} — a single-element scale on the cells axis (nested order: four contiguous children per parent), and never a translation (dggs#25: rotations are not representable). The level arrays are (window, cells) (§11.2): dggs.spatial_dimension names the spatial axis, and the one scale element indexes that axis, not the leading row axis, whose extent every level shares.
• derived_from is the declared fold provenance (§4.5's fold block, mirrored in the object's fold key): the column level folds the base's own leaves within each leaf's footprint, so it derives from the base; so does every level entry inside the exact_levels boundary and every level under the leaves regime; a cascaded level derives from the next-finer level (its scale is then 4.0 on the every-order ladder). dggs#25's example derives every level from the base; a reader resolving a level's absolute resolution needs only dggs.refinement_level either way.
• No top-level resampling_method: it is one string, and zagg's resampling is per field (fields is the class map — count an exact sum, digests merged, none-class fields absent) — the /1 keys carry the per-field truth.
• Derived, never authoritative — the same precedence as §4.9: the object is computed from the manifest's pyramid block (through its mirror) and the level groups' attrs at the init (§11.4) and by declare-pyramid (which refreshes multiscales and zarr_conventions together, and removes both when the manifest leaves /2); the manifest wins any disagreement. The zagg-icechunk/2 marker is not bumped by the object shape: no reader breaks on it — gridlook's parseMultiscales accepts an object or a list and branches on spec, and moczarr reads pyramid and passes the mirror through — and the zagg_icechunk block the compatibility check vets is unchanged.

11.2 Array model

Contract. For every named array a level's template declares (a leaf array {cell_order}/{name} for the base, a column's declared-member array {c}/{name} for the column level, an overview's {c}/{name} for an overview level), the repo holds /{c}/{name} whose metadata is the template array's, re-rooted on the whole sphere at that cell order under a leading row dimension (n the level's node order, n_shards = 12·4^n, the level's global cell extent 12·4^c, n_rows the length of the block's rows):

field	repo array	derivation
shape	(n_rows, n_shards · L₀, *L[1:])	L the level's per-node object array's shape (the leaf's for the base); n_shards = 12·4^n, n the level's node order; n_rows the same for every array of the repo
chunk shape	(1, *inner) — one row deep, the object's inner chunk shape across	inner is the sharding_indexed codec's chunk_shape when the object array is sharded (every base leaf, §1.5, and every fine column level, §4.6 — its inner chunk), else its own chunk_grid — the whole object for an overview level or a column level below 4^5 cells, one chunk per node
codecs	the inner codec chain	the sharding_indexed wrapper is absent; [bytes] for dense fields, [vlen-bytes, zstd] for zagg-ragged/1 (§1.3)
dimension_names	("window", *names)	the object array's names behind the row dimension's
data_type, fill_value, attributes	verbatim from the object array	so a §1.2 ragged block, §2.0 weights, §8/§9 declarations bind identically

A (1, C) chunk holds exactly the bytes the object's (C,) inner chunk does (C order; none of the inner codecs depends on the chunk's rank), so a reader that decodes a leaf array per §1–§3 decodes the repo array the same way, chunk by chunk, and a virtual reference needs no re-encoding.

The level group's dggs block is the artifact's, verbatim, so it names spatial_dimension: "cells" and its cell coordinate (morton) as the artifact does: it binds to each row slice array[w, …], which is exactly the artifact's 1-D array on the whole sphere. morton is a function of the cell alone, so every row holding a node's object holds the same words there (a row with no object at that node reads fill); a convention-following reader selects a row first, then reads the slice as the dggs block describes.

Contract — rows. A row is one window of the store's schedule (§4.2, the manifest's temporal block), or the reserved row all — the same token the leaf and overview names reserve. Which rows a store has follows its kind, fixed at birth with its schedule:

• an unwindowed store (schedule: none) has the single row all, its only row at every level — it is the one-row case of the model, not a different one;
• a windowed store has one row per label its runs have named: a row per window written, and all for its all-time fold once a run names it (populated at the overview levels only — there is no all-time leaf at the base). Nothing is allocated that no run named: which labels a windowed run names is the dispatcher's (the all fold follows pyramid.overview.all_time, §4.5) and lands with the windowed writer (§11.6), and an init that names no label on a windowed store is refused rather than creating a repo with no rows.

The two meanings of all never meet in one store. The row law: a label's row is allocated once, by label, in order of first appearance, and never moves — a window that sorts before an existing one is still appended after it (sorted rank would move every later reference). The block's rows list is the authority: row w is rows[w], the list is only ever appended to, and a reader looks a row up by label or sorts by the coordinate below, never assumes chronological order. (A chronological listing, if ever wanted, is a metadata operation — Icechunk's Session.reindex_array rewrites an array's manifests and no chunk.)

Contract — the row coordinate. The repo root holds two int64 arrays of shape (n_rows,) on the window dimension, window_start and window_end: row w's half-open [start, end) as integers in the manifest's temporal units on its scale since its epoch (the encoding the leaves' time_field values are in). Their attrs mirror that encoding — units as the CF string "{units} since {epoch}", calendar, and scale (utc / gps / tai, which CF does not carry) — and are empty on an unwindowed store. The all row carries no bound: both arrays read their fill_value, −2^63, there (never written). A window boundary that is not a whole number of the declared units is refused at allocation, not rounded. The arrays are chunked 1,024 rows deep ([bytes, zstd]), so the coordinate is one small chunk for any store's lifetime of windows.

11.3 Chunk index law and refs

Contract. Every level's chunk axis is in canonical nested order (§1.5 "Subtree spans"), so one node's chunks are one contiguous run. For a level of cell order c and node order n, an object at nested rank r at order n (r ∈ [0, 12·4^n), the rank block_index gives) holds 4^(c−n) cells; its chunk j (C-order within the object's chunk grid along the cells axis, j ∈ [0, C), C the object's chunk count) sits, in row w (§11.2 — the row of the object's window; all for an unwindowed leaf, a leaf column and an all-time overview), at global chunk index

(w, r · C + j)   (trailing axes keep their object-local chunk index, 0 for a single-chunk payload dim)

For the base level C = 4^(chunk_order − shard_order) inner chunks; for an overview level, and for a column level of fewer than 4^5 cells, the object is ONE chunk, C = 1, and the cell-axis index is the node's rank itself; for a fine column level (cells − shard_order ≥ 5, §4.6) the object is sharded and C is its inner-chunk count — the level's chunk_order is the inner chunk's order (cells − 5, or shard_order + 1 at exactly 4^5 cells), so the same r · C + j law applies with the base level's C = 4^(chunk_order − shard_order) (16 / 4 / 4 at /16 / /15 / /14 on 19/13/9). The row index is the repo's, not the object's: a writer names an object's row by label and the commit resolves it against the block's rows (§11.4), so the same object bytes land at whatever row the repo allocated that label.

At the production geometry (shard 9 / chunk 13 / cell 19) C = 256.

Each populated inner chunk is recorded as one virtual reference:

• sharded array (every hive leaf, §1.5, and every fine column level, §4.6): location is the array's single shard object — its outer-chunk key, {leaf}/{p}/c/0 for the 1-D cells arrays the hive writes — and offset/length are the chunk's entry in the shard index suffix, the same two u64 words the §1.5 2-GET recipe reads — the writer fetches that suffix itself, once per array (§11.4). An inner chunk the index marks absent (the 2^64 − 1 sentinel in both words) gets no reference and reads as fill_value.
• regular (unsharded) array — a leaf array on a chunk_inner-less grid, every overview array, and every column array of fewer than 4^5 cells (one chunk per object): location is the chunk object ({leaf}/{p}/c/{j}; {object}/{p}/c/0 for a single-chunk array), offset 0, length the object size (§11.4); a missing object gets no reference.

Chunk keys are the array's own, under the chunk_key_encoding its zarr.json declares — zagg emits the default encoding with the / separator throughout — so the chunk at grid index (i₀, i₁, …) is

{leaf}/{p}/c/{i₀}/{i₁}…

with the trailing axes at their leaf-local chunk index (0 for a dimension the array holds in a single chunk). A 1-D array's key is therefore c/0 for its one chunk, and c/{j} where the cells axis is chunked. These are the object's keys, and an object's arrays have no row dimension: the row appears only in the repo array's own chunk index, (w, r·C + j, …), never in a location. Keys are used verbatim as the path part of location (// and ./.. are preserved), so a character that is reserved in a URL but part of the key MUST be percent-encoded — ? → %3F, # → %23, % → %25 (set_virtual_refs_arr). zagg's own keys contain none of the three; the rule is normative for a reader reconstructing a key from a recorded location.

Every reference also carries a checksum, in the form its container validates. Into an object-store container it is the ETag of the object it points into, read from the HEAD the writer performs against that object after the leaf write (§11.4). Into a file:// container it is that object's last_modified, ceiled to the next whole second: Icechunk compares a recorded datetime against the object's modification time at whole-second granularity, so the exact sub-second mtime fails the very object it was read from while the ceiling passes it. Icechunk verifies either form on read, so a reference into a legacy leaf that has since been wholesale-replaced (§1.5 — same keys, new bytes) fails loudly rather than decoding the replacement at a stale offset. That granularity is the local form's one caveat: a replacement landing within the same second as the original passes the check (an object store's ETag has no such window). On a versioned leaf (§1.5; a stamp naming current) the recorded location is the version subgroup's object — {leaf}/run-{run_id}-{attempt}/{p}/c/0 — which is never rewritten and never touched (§1.5), so the checksum is a guard against out-of-band tampering only and neither a replacement nor a lifecycle touch invalidates an earlier reference. The writer records which form it used under icechunk.checksum in the leaf's stats sidecar ("etag" or "last_modified").

location is url_prefix + key, key the object's path relative to the store root. The repo declares exactly one virtual chunk container whose url_prefix is the store root URL with a trailing / (s3://bucket/product/, file:///…/product/); a reader authorizes that prefix with the same credentials it reads the leaves with. That prefix is absolute, so the companion does not relocate with the store the way the root-relative manifest and MOC sidecars do: a mirrored or moved store must declare the container prefix afresh (and the recorded url_prefix, §11.1) at its new location before its refs resolve.

11.4 Commits

Why not one commit per leaf (informative). A per-leaf commit does not scale. At the full-globe worst case (3,145,728 order-9 leaves, 49,152 order-6 cells) it is 3.1M commits — and the real limit is not the commit count but the snapshot: an Icechunk snapshot lists every manifest, so its size is set by the manifest count. One manifest per order-6 cell per array is ≈442k manifests ≈ 44 MB read on every open, every rebase and every commit (≈2.2 TB of snapshot traffic over one run). Balancing snapshot bytes (≈100 B per manifest entry) against per-manifest bytes (≈2.5 KB on disk per leaf-array) gives leaves-per-manifest ≈ 0.6·√N: ≈30 at California scale (order 6–7 cells), ≈1,000 at the full globe (order-4 cells → 27k manifests, a 2.7 MB snapshot and 2.6 MB of manifests — the shape of the live ISMIP repo, 27,423 manifests / 2.2 MB snapshot). So both the manifest split and the commit granularity are configurable and derived, and a base manifest is written by exactly one commit (§11.5).

Contract — the ladder. Refs travel up the §4 pyramid the way the digest columns do:

1. The leaf worker writes a ref sidecar, not a commit. After the leaf's stamp, its granule-id sibling and the §4.6 column fold — last in the unit, behind every post-stamp phase that can still fail it — the worker computes the leaf's ref plan (the reads below) and writes it as one compact object beside the leaf, named by the stats sidecar's sibling grammar with the base icechunk_refs.json ({stem}.icechunk_refs.json under morton-hive/3). The carrier is JSON declaring zagg-icechunk-refs/1, the writer's geometry (shard_order, chunk_order, cell_order — never the container prefix, which is vetted at the repo) and the entries (≈40 KB per leaf at production geometry), grouped into units that each name the row they land in by label (row: all for an unwindowed leaf and its column, the window's label for a windowed leaf — one leaf, and so one carrier, per window; a carrier written before /2 names none and is read as all) — never a row index, which only the repo knows (§11.2), so a carrier stays valid whatever rows other runs allocate. It is a writer-internal carrier, not part of the reader contract — the repo is. No Icechunk session is opened on the leaf path.
2. Stage nodes gather. Each dispatch node of the staged sweep (§4, zagg.sweep_stages) reads its subtree's carriers — the leaf sidecars at the finest tuple, one per (leaf, window) on a windowed store, its children's node ref columns (the same carrier at {node}/icechunk_refs.json) above — and adds the refs of the overview objects at every order of its tuple beneath it (read the same way a leaf's are). What it reads and plans is scoped to the rows THIS RUN writes under the node — the window labels dirty beneath it plus all when it closes — so an append costs the new rows, not the store's history. A missing carrier is counted (icechunk_missing) and tolerated — under-coverage, never a failure — and a carrier whose recorded geometry disagrees with the repo's block is refused.
3. One tuple commits. Let c be commit_order, d a tuple's dispatch order and d′ its child order (the tuple covers orders [d, d′)):
4. d ≤ c < d′ — the committing tuple: every node commits all it gathered in one commit — every level's refs into its group of the same repo, keyed by cell order (§11.1): the base leaves into /{cell_order}/… (/19), the column's declared member into /{c}/… (/13) and each overview level into /{c}/… (/12 … /4) — each unit at the row its label resolves to in the block's rows as the committing session reads it (a unit naming a row the repo never allocated fails the node's commit, fail-open) — message node {decimal};
5. d′ ≤ c — a coarser tuple: its nodes commit only their own overview refs (their children already committed);
6. d > c — a finer tuple: its nodes write their node ref column and commit nothing.

Commits rebase on conflict exactly as before (ConflictDetector; nodes touch disjoint chunk ranges). One conflict no rebase reconciles is another run's row growth (its init, below) landing while a node holds staged refs: Icechunk reports Chunks updated in updated array on every array the node touched and the rebase fails. That class, and only that class — every conflict of the failed rebase is one, nothing else, and no commit between the session's base snapshot and the branch tip updated a chunk the session staged (the rebase stops at the first commit it cannot reconcile, so a double update landing after the growth is caught by that diff, not by the conflict list) — is replayed: the node writes the same units into a fresh session based on the grown arrays and commits them there (each label resolves to the same row index, §11.2 — a row never moves; the ladder streams the subtree again rather than holding it), at most once (REPLAY_TRIES), after which the failure propagates to the fail-open hook. A double update of the same chunks by two writers is never replayed over, whichever of it and the growth landed first; the rebase failure propagates. A replayed node's rebases and commit time sum every session's, the defeated one included, and a replay counts whether or not its re-read then staged anything to commit. Each stage row records icechunk_commits, icechunk_rebases, icechunk_replays, icechunk_commit_s, icechunk_refs, icechunk_missing, icechunk_failed, icechunk_skipped_levels, icechunk_clean, icechunk_regathered and icechunk_s — every one pre-seeded, so a row's key set does not depend on whether a node did work — plus icechunk_nodes, one entry per node that did, carrying the snapshot it committed (the leaf→snapshot join).

The ladder runs over the dirty set — the nodes with a dirty leaf beneath them — not over every candidate node: a node whose whole subtree is clean had its refs committed by the run that dirtied it, so it is skipped whole and counted (icechunk_clean). An append therefore costs O(dirty), not O(store). A dirt-only leaf (§11.6) joins the dirty set for the ladder alone: a node with dirt-only leaves and no dirty one runs the gather and commit, folds nothing, and is counted (icechunk_regathered). A full re-gather (repairing a repo against the leaves, after a run whose commits were lost) is a manual staged sweep over the whole store, where every leaf is dirty by construction.

commit_order defaults to the finest dispatch node of the staged sweep (shard_order − tuple_width when the shard order is a multiple of the width: 6 at production), split_order to commit_order. Both ride the zagg_icechunk block, with different standing: split_order is the store's value and ratchets (§11.5); commit and commit_order are per-run — the init writes this run's values so its stage nodes can read them, and they are never a compatibility key. commit: "leaf" keeps the per-leaf commit of the first revision — the leaf commits leaf {decimal} itself and the ladder commits overview refs only. An unset commit resolves to the ladder only when the run walks it — the dispatcher chains the staged sweep (output.sweep: "stages") and a /2 ladder with a composable field is declared — and to "leaf" otherwise, on every backend: a ladder-mode run that walks no ladder would leave the repo empty while every leaf reported a sidecar.

Init. init {run_id} — the once-per-run initialization, before the fan-out: the repo exists with every level group (§11.1 — the base, the column's declared member and one per declared overview level, keyed by cell order, created here once because Icechunk's create is not safe under concurrent callers), its array nodes (§11.2) defined, the row coordinate at its root and the multiscales mirror in its root attrs. Idempotent: a repo that already carries a matching block is reopened, never re-templated — the one thing a reopen may do to an array is grow its rows; a block for another geometry or container is refused, and so is a block of another revision (a /1 repo: Succession, above). Every run commits its init {run_id} — empty when neither the block nor the rows change, labelled split ratchet {from}->{to} {run_id} instead when its init re-cuts (§11.5), with run_id in the commit metadata either way — so the ancestry brackets each run between its init and its finalize (the repo is its own run log, and finalize's newest-run check below reads it). The ladder settings are not compared: split_order follows the §11.5 ratchet (a finer config adopts the store's value, a coarser one re-cuts), and commit / commit_order are per-run.

Row allocation. The init allocates the run's rows inside that same init {run_id} commit, before the fan-out, so every commit of the run finds its row. The dispatcher names the run's row labels — ["all"] for an unwindowed run — and they ride the init (rows on the icechunk_init event; the local backend passes them in-process). Each label the block's rows does not hold is appended, in the order given (§11.2, the row law): every array of the repo with the window dimension — each level's, listed or retired, and the coordinate — grows by one row per new label, the new rows' window_start / window_end are written, and the block records the longer list. A label the repo already has is a no-op, no existing row moves, and growing rows rewrites no manifest (a resize is metadata only). An unwindowed store's one row all is allocated at creation whatever the event names, and a window label on an unwindowed store — or one its schedule does not declare — is refused. Two runs allocating at once both resize every array, which Icechunk's rebase does not reconcile: the loser's init retries in a fresh session (5 attempts in all), re-reads the rows the winner recorded and appends after them. The other side of that race — a node of a run already committing when the growth lands — replays its units in a fresh session (One tuple commits, above). A retry re-applies the loser's own block changes (a ratchet, the per-run knobs) only when the winner changed nothing else, or made the same changes: a block the winner moved otherwise — a different ratchet, other knobs — is not overwritten with changes computed from the block before it, and the loser's init raises (and fails open). An init that raises after saving a ratcheted splitting config (§11.5: the save precedes the commit and is not part of it) first re-saves the splits of the block as it stands. That covers an error the init can catch, not an init killed between the save and the commit (a timeout, an out-of-memory kill): it re-saves nothing, and the saved config stays ahead of the block until the splits are saved again (a further ratchet, or declare-pyramid adding a level).

Finalize. finalize {run_id} — the once-per-run close (mode="icechunk_finalize" on Lambda, in-process on the local backend), invoked by the dispatcher AFTER every commit of the run has landed: after the staged sweep returned under the ladder (its finisher makes the last commit; every dispatcher that chains the sweep — the CLI, the local backend and the client facade — finalizes at this point), after the fan-out drained under commit: "leaf". Never by a stage node — tags, expiry and collection are singleton repo operations, and never inside the staged sweep's finisher, which is lease-scoped, load-bearing store-root machinery while the repo is fail-open (and which a per-leaf run does not have). On the Lambda backend the stage nodes are Event invokes, so a staged sweep that did not complete — its dispatch failed, a barrier expired or the finisher did not land — may still have node commits in flight: the dispatcher then does NOT finalize and records icechunk_finalize: {skipped: <reason>} and the run stays untagged (its commits are kept; the next run's tag covers them; the finalize operation below — this same finalize, invoked by an operator and run by a worker — tags it once the ladder is complete). One finalize, in order:

1. retention — output.icechunk.retain_runs = K. 0 (the default) keeps every run and does nothing here. K > 0: the run tags beyond the K − 1 newest are deleted (only run- tags, only here), snapshots older than the oldest retained run's finalize commit expire (expire_snapshots), and repo objects no retained snapshot references and older than it are collected (garbage_collect) — the cutoff is always a run tag's commit time, never "now", so a concurrent writer's in-flight objects are never collected. Its session is not protected: a writer (or reader) whose base snapshot is older than the cutoff loses that base, and its commit fails on rebase (a storage error, not a conflict — no retry recovers it; the leaf's or node's refs are lost fail-open until a re-gather). The cutoff is never newer than the previous run's finalize (K = 1 or 2; a larger K reaches further back), so the only casualty is a session opened before the previous run's finalize and still open across this one — overlapping runs on one store. The default K = 0 never expires anything. Expiry squashes every commit older than the oldest retained run's finalize — that run's own init, leaf and stage-node commits included — into that finalize snapshot; the runs newer than it (the K − 1 newest, this one included) keep their intermediate commits until a later finalize's cutoff passes them. History therefore reads one snapshot per run only up to the oldest retained tag, and per commit after it. Neither step touches a virtual target: Icechunk manages none of the leaf objects (the virtual-target collector below is the operator's tool for those). Retention is fail-open inside finalize: an error (two finalizes racing on one tag delete, a collection cut off by the invoke's ceiling) is recorded as retention_error and steps 2 and 3 still run;
2. one content-free finalize {run_id} commit whose commit metadata identifies the run — run_id, semantic_hash (the D19 digest the leaves were stamped with), zagg_version, the block's commit / commit_order / split_order — and records the retention counts (retain_runs, tags_deleted, snapshots_expired, gc, retention_error), so the repo is its own durable run record (ancestry(tag=…) answers "which config built this");
3. the tag run-{run_id} on that commit. Tags are immutable and keyed by the run, not the semantic hash: every append under one template shares a hash, so a hash-named tag would collide on the second run.

Idempotent: a finalize whose tag already exists returns it and writes nothing, and two finalizes of one run are safe: the loser reads the winner's tag. A reattached client (Run.attach) fires a finalize of its own, off the config's knob (it never held the init record, so the event carries icechunk_init: null and rewrite_pending is always null there), and only for a pinned commit: "leaf" run with no sweep: "stages": a sweep: "stages" run's finalize is its dispatcher's, after the staged sweep (whose nodes commit under either mode), and attach records {skipped}. It finalizes only while its run is the newest on the repo (no later init — init or split ratchet — or finalize commit on main), else it writes nothing and records {skipped}, so an untagged old run stays covered by the next run's tag as before. Fail-open (D9) at the dispatcher like the init; the record rides the run summary as icechunk_finalize ({path, tag, snapshot, tagged, retain_runs, tags_deleted, snapshots_expired, gc, retention_error, rewrite_pending, commit_s}, {error} or {skipped}), not the run parquet, whose write precedes the staged sweep on both backends; the tag itself is the durable outcome. rewrite_manifests is NOT run by finalize: a §11.5 split ratchet is a rare, deliberate, whole-repo operation with no run to attach to (at the globe it may exceed one invoke), so finalize reports it as rewrite_pending: {from, to} for the operator step.

A replaced leaf is re-indexed by the run that replaces it (per-leaf commit or the next staged sweep): the new refs supersede the old ones on main. On a legacy leaf (§1.5) snapshots older than that commit still reference the replaced keys — and because clear-then-template reuses those keys, their refs resolve to a live object rather than 404ing; the per-ref checksum (§11.3) is what makes that a loud failure instead of a silent mis-decode. On a versioned leaf the old refs name the superseded version's objects, which stay in place, so every earlier snapshot and run tag keeps reading. Icechunk's garbage collection never touches a virtual target; the virtual-target collector (tools/icechunk_gc_targets.py, operator-run, dry-run by default) deletes a leaf's version subgroups that are neither its current nor referenced by any retained snapshot, reports the reclaimable bytes, never touches a legacy leaf, and refuses a store without a repo. Retained means every snapshot in the ancestry of every branch and of every tag — all tags, not only run- ones — after expiry; a version only an expired snapshot references is collectable. In-flight guard: the collector never deletes a version whose stamp written_at is newer than the newest run- tag's finalize commit, since an in-flight run's versions have no tag yet. An unstamped version (an attempt that died before (2)) has no written_at; the collector reclaims it only once its own run's run-{run_id} tag exists. A crash between (3) and (4) of the §1.5 write order therefore leaves a version that is either referenced (per-leaf mode — retained until expiry drops it) or unreferenced but young (ladder mode — guarded); once its run is finalized without the version being gathered it is garbage, and the next collection reclaims it. A converted legacy leaf — one whose first versioned write left the legacy {cell_order}/… arrays at its root — keeps those root arrays forever: they are not a version, the collector never reclaims them, and any earlier snapshot that references them keeps reading, since nothing rewrites them.

Operations. An evolving fact of the metadata plane is written by an operation: one commit on main (zagg.icechunk_ops; python -m zagg.icechunk_ops <store> <operation> …), operator-run: set-attrs and declare-pyramid commit from the operator's host, never from a worker; finalize is invoked by the operator and written by a worker (its row). An operation's commit message names it, its commit metadata carries operation, zagg_version and the operation's own keys — so ancestry() reads as a log — and it touches no leaf. Before the commit the session is validated: the array model of every array (shape, dtype, chunk grid, codecs, fill value — everything but attrs) MUST be identical before and after except for row growth — the block's rows may gain labels at its end (never lose or reorder one, nor hold one twice, §11.2), and then every array with the window dimension MUST hold exactly that many rows, its cell extent and everything else unchanged; rows grow for the whole repo or not at all — the block's spec / shard_order / chunk_order / cell_order / url_prefix MUST hold, and every level the block lists MUST have its group; a session that fails is discarded and nothing lands. (Neither validated operation below allocates a row — the init does, §11.4 Row allocation — so for them the allowance is the invariant: a group declare-pyramid adds is built at the repo's rows — those the operation's own session reads, so an init allocating rows after the operation first read the block does not refuse it.) moczarr's validator (issue #582 phase 6) runs in addition when it lands; the check above is zagg's own. An operation that would write nothing commits nothing. The operations:

• set-attrs <path> <json> merges the JSON object into the attrs of the root group (/), a level group (/{cells}) or an array (/{cells}/{array}); a null value deletes the key. The root's zagg_icechunk block is the writer's and multiscales / zarr_conventions are declare-pyramid's: all three are refused here. This is how a convention block evolves — the dggs latitude token, a spec marker — without a leaf rewrite; the leaf's own attrs stay as the leaf was stamped, by design (the head of this section). A base level group's dggs change also refreshes the root multiscales base layout entry in the same commit (§11.1).
• declare-pyramid <config> brings the repo's levels and its multiscales / zarr_conventions root attrs (§11.1) to the manifest's declaration (§4.9): every level of §11.1 (level_grids of the manifest — the base plus the /2 block's datasets) that the block does not list gains its group, written from the same spec the init writes at the repo's rows (§11.2), and its manifest split (§11.5) is persisted with the repo; a level the manifest no longer declares is delisted — dropped from levels — but its group stays, since every snapshot and tag that references it keeps reading (a later re-declaration relists the group after checking its array model); a listed level whose geometry the manifest would change is refused: an array-model change is a new revision of §11, never an operation. A delisted level's entry moves to the block's retired map (§11.1), whose splits every later split save — this operation's and the §11.5 ratchet's — persists with the listed levels', and a re-declaration moves it back to levels. The commit metadata records the manifest's semantic_hash and the levels added / dropped. The manifest retrofit (zagg.sweep_overview.declare_pyramid, python -m zagg.sweep <root> --declare-pyramid <config>) performs this operation itself after its manifest write when the store has a repo (fail-open, D9; its summary's icechunk key carries the report or the error), so one operator step declares both planes; the standalone form re-runs it.
• finalize <run_id> tags a ladder run its dispatcher left untagged — the staged sweep completed but the dispatcher died before its finalize (the state described under Finalize above). It writes where the run's writer is, never from the operator's host. On an object-store root the command fires ONE synchronous mode="icechunk_finalize" invoke whose event is {mode, store_path, run_id, newest_only: true, operator_checks: true} — no config — and prints the worker's report: every read below, the commit, the tag and the retention are the worker's, under its execution role and in the store's region. The operator's host reads nothing from the store and writes nothing — it may hold invoke rights and no write credentials (a store whose only writer is the worker role), and a retention pass run elsewhere would read the repo out of its region. Only a local store root runs in-process. The function is --function-name, else the ZAGG_LAMBDA_FUNCTION_NAME environment variable, else process-shard, an empty value counting as unset — the dispatchers' own default for a config with no worker: block, though they take an empty value verbatim (the run config's worker: suffix is never applied: that config is in the manifest, which the host does not read); the report names the function invoked as function_name, and the command never falls back to the host. Because the event carries no config, a deployed worker that predates operator_checks fails on the missing key before any write; had the config ridden along, it would have tagged newest_only with none of the checks below. A check that does not hold is returned as {ok: false, refused: <reason>} with nothing written, and the command raises it. Under operator_checks the worker reads the run's dispatch manifest (<store>.status/run-<run_id>/manifest.json) for the run's own config — its retain_runs and the D19 hash — refuses without it (a Lambda-dispatched run has one; a large hive run's is slim — the shard list left out so the block fits the setup event — and carries the config all the same. The write is best-effort: a lost setup invoke or failed write leaves none, as does a block that does not fit even slim; such a run is not finalizable here and the next run's tag covers it), refuses unless the newest sweep_stats_*_stages.json that names this run — its pipeline_run_id (§4.7) equal to <run_id> (issue #593) — and was written since the run's init commit (its written_at on main, the repo's clock — a run with no init commit is refused) shows a completed sweep (the finisher wrote it, and no barrier expired), and then runs the Finalize above newest_only: it can only ever tag the repo's newest run — an older untagged run stays covered by the next run's tag, and tagging it would name later commits — and an existing tag is a no-op. It is the Finalize above invoked by an operator, listed here for discoverability — not a validated array-model operation, and an exception to this list's rules: its commit metadata carries run_id / semantic_hash / the retention counts, not an operation key; it commits even though content-free; and with retain_runs > 0 it runs the retention (tag deletion, expiry, collection). There is no --force (a run whose sweep did not complete has no tip that means "this run": python -m zagg.sweep <store> --stages --scope … --pipeline-run-id <run_id> completes the ladder first) and no retention override. The record is tied to the run by both conditions: the name and the time. A record naming another run — a sibling run's sweep landing after this run opened, the retention casualty case above — or naming none (pipeline_run_id: null: a standalone --stages pass that was not told which run it completes, or a record written before the key existed) vouches for nothing here, however recent. The newest record that does name the run decides; a later record that names no run neither vouches for the run nor blocks it.

The reads the writer performs. Recording refs costs I/O — the offsets come out of the object's own index, but the sizes and checksums do not. For each array of a leaf (at the leaf worker) or of an overview object (at its stage node) the writer issues:

• sharded array: one ranged GET of the shard index suffix (the §1.5 recipe's second read, which yields every inner chunk's (offset, length) at once) and one HEAD of the shard object, for its checksum — the ETag, or last_modified on a local store (§11.3);
• single-chunk array — every overview array and every column array of fewer than 4^5 cells (one chunk per object): one HEAD of its one chunk object, for its size (the ref's length) and checksum; a fine column array (§4.6) is a sharded array and takes the suffix read above;
• multi-chunk regular (unsharded) array — a leaf array on a chunk_inner-less grid: one LIST of the array's c/ chunk prefix, which yields every chunk object's key, size and checksum in one request — and, unlike probing, discovers which chunks were actually written.

Plus, per stage node, one small GET per child carrier. Small beside the leaf write, but not nothing.

Writing refs is fail-open (D9) at every rung: a leaf's sidecar failure is logged and recorded in its D20 stats sidecar (icechunk.error) and never fails the leaf; a stage node's failure counts icechunk_failed and never fails the sweep — the leaf is normative, the index is regenerable (a later staged sweep re-gathers). The leaf's icechunk block records the sidecar it wrote — {sidecar, bytes, refs, arrays, checksum} — or, under commit: "leaf", its commit {path, snapshot, arrays, refs, rebases, replays, commit_s, checksum}; {skipped: reason} for a unit stage 1 does not index (§11.6); {error: message} on failure. The run parquet flattens the same fields to icechunk_* columns.

11.5 Manifest splitting

Contract. Manifests are split along both axes of a level array: the cell axis into runs of 4^m chunks, and the row axis into blocks of rows_per_manifest rows (§11.2), default one row per manifest. m is fixed once, from the base level:

m_base = chunk_order_base − split_order          (7 at production: 4^7 = 16,384 chunks)
m      = min(m_base, chunk_order_level)          (per level)

Both axes are named explicitly, which is required rather than decorative: Icechunk splits an axis its config does not name at one chunk, so an unnamed cell axis would cut a manifest per chunk.

Why the row axis is split. With one manifest per row an append only adds manifest files: the rows already written are never rewritten, so a reader's cached manifests stay valid and a reader after one window downloads that window's refs alone. The alternative — one manifest spanning every row of its cell run — makes each append rewrite the whole manifest of every cell it touches (at 7 yearly windows, ≈1.05 MB rewritten per order-6 cell per array to add a row's refs) and makes a one-window read download every window's refs. The cost of splitting is the snapshot, which lists every manifest and so grows with the row count: negligible at regional scale, and the reason the cut is a knob (output.icechunk.rows_per_manifest, default 1) rather than a law — a global store sets blocks of several rows. One row per manifest is the finest cut a store can carry, which is the side a later coarsening can still move from.

Because the chunk axis is in nested order, one run of the base is exactly the chunks of one HEALPix cell at order split_order, so the split is stated as "one base manifest per order-split_order cell". Every other level holds the same number of chunks per manifest — not one split_order cell — capped at its own chunk axis (chunk_order is the group's own: the inner-chunk order for the base, 13 at production, and for a fine column level, §11.3; the node order for any other column or overview group, one chunk per node). At the defaults that is one order-2 cell per manifest at /13, one order-1 cell at /12, and one base cell (12 manifests per array) at /11 and every coarser level; no manifest ever spans more than a base cell. Holding the chunk count constant is what keeps the manifest count — and with it the snapshot, which lists every manifest — at the base's: applying "one manifest per split_order cell" to every level would multiply both by the number of levels (≈5× at production).

The split is configured per order group (Icechunk's path-matched split conditions, ^/{order}/) and recorded per level as levels.{order}.split (chunks = 4^m, order = chunk_order − m — the cell order one manifest spans, which differs per level by design — and rows, the row-axis block, the same for every level) and once each as split_order and rows_per_manifest in the zagg_icechunk block (§11.1), never assumed by readers. A /2 repo whose block carries no rows_per_manifest was written before this rule — an interim repo, never published — and is refused with the remedy (clear it; the next run's init re-creates it), never operated on: its rows-less level records are compared with nothing. split_order MUST satisfy commit_order ≤ split_order ≤ shard_order: the committing node (§11.4) owns every leaf under its order, so a base manifest keyed to a cell at or below it is written by that one commit and no other — zero rewrite amplification at the only heavy level. A coarse-level manifest spans more than one committing node's subtree and is rewritten by each node that touches it (a rebase on disjoint chunks, ConflictDetector); that is deliberate: those manifests are tens of KB, so the amplification is negligible in bytes, while their count is what every snapshot read pays for.

(Informative — what the row axis costs; issue #584 phase 0, icechunk 2.2.2, one order-6 cell's 64 leaves at production geometry.) Growing the row axis rewrites nothing: a resize is metadata only, and every manifest keeps its id. Refs written into a new row rewrite, per array, exactly the manifests of the split cells they touch — whole, the earlier rows' refs included — and no other; every earlier reference stays byte-identical. The one-row case costs nothing over a row-less array (2.10 MB of manifests for 147,456 refs against 2.14 MB). Under the default one-row cut, refs written into a new row therefore touch only that row's manifests and no earlier one. Measured on the issue #586 Sierra stores, a yearly-windowed leaf occupies ≈62 % of its 256 inner chunks (mean 158, min 34, max 234), so one shard's 7 yearly windows carry ≈4.3× the occupied chunks of one dense row — ≈8.4 MB of manifests for one order-6 cell: 8 files of ≈1.05 MB spanning every row, against 56 files of ≈150 KB at one manifest per row. Row-spanning manifests rewrite that 1.05 MB per append; the one-row cut writes only the new row's ≈150 KB and supersedes nothing, paying ≈33 B of snapshot per manifest (≈4.8 KB per order-6 cell against ≈1.4 KB). Occupancy did not track a leaf's pass count, so these numbers do not project to finer schedules — issue #602 measures those.

Contract — the ratchet. The store's recorded split_order is authoritative and moves one way, toward coarser. At init, the run config's split_order is compared with the block's: a config value FINER than the store's (numerically higher) is not applied — the init adopts the store's value and logs a warning naming both (templates are hash-pinned build configs reused by appends; refusing would break every append after a ratchet); an equal value is a no-op; a COARSER value is an intentional ratchet: the init records the new split_order and per-level splits in the block and in the repo's saved splitting config before any commit of the run, so every manifest the run writes is already at the new cut, and flags the run record with icechunk_split_ratchet: "{from}->{to}", which the run's finalize (§11.4) reports as rewrite_pending for an operator rewrite_manifests pass over the old manifests (not run by the writer or by finalize; mixed cuts are valid — each manifest carries its own extents). A repo is never re-split finer. split_order is a layout knob outside the D19 semantic core: it changes no leaf byte. commit_order is per-run and unchecked beyond split_order ≥ commit_order.

rows_per_manifest does not ratchet: it is fixed at the repo's creation, because it is baked into every manifest already written. A later config value is not applied — the init adopts the store's and logs a warning naming both — and moving it on an existing repo is an operator rewrite_manifests pass. It MUST be at least 1. Adoption governs the repository handle the run then writes through, not only the value it reports: an Icechunk config supplied when a repository is opened overrides the saved one, so an init whose config differs from the store reopens at the store's cut before any commit. The block and the manifests a run writes therefore always name the same cut.

At the defaults (split_order = commit_order = 6 at production) a shard-order manifest is 4^7 = 16,384 chunks — one order-6 cell, 64 leaves — which is ≈43 order-6 cells × 9 arrays ≈ 390 base manifests at California scale. The coarse levels add a few hundred (/13 at one order-2 cell per manifest, /12 at one order-1 cell, 12 per array at /11 and coarser), so the snapshot is ≈390 + a few hundred entries; at the full-globe worst case it is ≈442k + ~1k, not the ≈2.2M a per-level split_order cut would list. The global-scale setting is split_order: 4, commit_order: 3: 4^9 chunks per manifest, one order-4 cell (1,024 leaves), 3,072 manifests per array (27k across the arrays, a ≈2.7 MB snapshot), committed by the 768 order-3 nodes (768 commits) — the numbers §11.4's rationale derives.

(Informative.) Icechunk deduplicates a manifest's location strings only from min_num_chunks chunks upward (a configurable setting defaulting to 1,000, not a format constant). The default split clears it comfortably — one manifest per order-6 cell is 4^7 = 16,384 chunks per array, 64 leaves. The finest admissible split does not: split_order == shard_order is one whole leaf, 4^4 = 256 chunks per array at production, below the 1,000 default, so such a manifest carries no dictionary. The dictionary is worth having precisely because all refs of one leaf array carry the same location (the leaf's single shard object), so a hand-set split_order == shard_order trades it away.

11.6 What §11 does not cover (informative)

Windowed leaves ({id}_{window}.zarr, morton-hive/2) share a shard rank across windows, which is what the /2 row dimension is for: each window is a row (§11.2), so the array model holds them — and the writer now indexes them (issue #584 phase 2). A windowed hive store is in scope, and output.icechunk defaults ON there as it does for an unwindowed hive store:

• the run's init allocates one row per window label its dispatch units carry, plus the reserved all row when the store maintains the cross-window fold (pyramid.overview.all_time);
• each window leaf's refs land at its window's row. A leaf unit that names no window label is refused: all is the fold's own row, it IS allocated whenever the fold is maintained, so nothing downstream would catch a base ref written over it;
• the ladder's node units plan one overview object per row this run writes ({window}.zarr per window unit beneath the node, and all.zarr when the node closes), never per row the repo holds — which would re-plan and re-commit every window ever allocated, rewriting those rows' manifests on every append.

Raster hive products ((time, cells) arrays, never sharded — one row per acquisition, in append order) remain out of the writer's scope: the knob resolves off there and an explicit true is refused. They are tracked on issue #584 (its phase 4); the finalize and tags of §11.4 apply to them unchanged once they have a repo. The sweep-built §4 overviews are in scope since the ladder (§11.4): every declared overview level has its group in the store's one repo, and its refs. None of these change the leaf format, the t-digest storage or the moczarr reader.

Scale (informative). Snapshot size is proportional to the manifest count (≈100 B per entry) and manifest bytes to leaves per manifest (≈2.5 KB per leaf-array), which is what §11.5's split rule balances; a per-level squash is not available in one repo (a snapshot lists every manifest, rewrite_manifests is whole-repo, there is no branch merge), so finalize is one whole-repo pass per run. One repo per o3 region — o6 nodes committing into their region's repo, the hive manifest becoming a repo catalog, each region keeping the group-per-level layout — is the documented escape hatch should a single repo's snapshot ever be the bottleneck; it is never a default at any store size (espg, 2026-09-26): the split ratchet governs that dimension first, and the hatch trades "one zarr" for "one zarr per region". Whether it is ever needed is read off the first global run's per-node icechunk_rebases / icechunk_commit_s stage rows.

A skip-if-current legacy unit's lifecycle touch (#388) refreshes its objects in place and so moves their checksums (§11.3); the unit re-plans its refs from fresh HEADs — committed at once under commit: "leaf"; under the ladder it rewrites its sidecar and enters the same run's staged sweep as a dirt-only leaf, whose nodes re-gather and commit its refs without re-folding their overviews (§11.4). On the fleet the worker's response body carries icechunk_dirty, the dispatcher assembles the dirt-only set from those bodies, and each stage event carries its node slice as dirt_only ([[shard_key, window], …], absent when empty). A current unit therefore still writes no stats record, sidecar or sub-map, but may enter the sweep work set as dirt-only when its touch moved ref checksums and the repo is on. A versioned unit (§1.5) never enters this path: its touch reaches no version object, moves no ref checksum, and so is never dirt-only — its main refs stand as committed.