T3 §1 fold for ViT-Tiny at the BATCHED index — the op set 4c leg 4 renders #
ViTFoldG.lean folds the ten gradient nodes of the PER-EXAMPLE traversal
(ViTRender.vitBackAll at adam := true). This is its batched peer, at
ViTRenderB.vitBackAllB's constructors, and it exists because 4c leg 4 moves every committed ViT
artifact onto that traversal.
⭐ The bytes do not move and the denotation does. Measured 2026-09-07: all nineteen drop-free
ViT artifacts — vit_fwd, vitin_fwd and the seventeen AdamW/EMA train steps — re-render
byte-identically off vitBackAllB, because every batched form was built to emit its
per-example peer's text and tests/TestBatchedEmitTie.lean pins each one individually. So this
file is not about different bytes; it is about the AST those bytes are pretty of.
⭐⭐ And on one parameter the AST is genuinely better, which is the whole point of the leg.
The CLS token is ONE shared [192] vector, so its gradient is the sum of every example's CLS-row
cotangent. The per-example render emits denseBiasGradB (N := 1) — "sum one thing", correct there
because pretty B performed the batch lift OUTSIDE the AST — where the batched one emits
denseBiasGradB (N := vbB) and the sum is inside den. ViTRenderB.lean flags that line as
"THE ONE LINE WHERE N := 1 → N := vbB CHANGES THE FUNCTION", and clsGrad_denB below is the
statement the per-example clsGrad_den could not make. Same emitted text either way, which is
why the byte tie cannot see it and den_rowDenseBiasGradB_at_one exists to argue the point.
The op table of every committed ViT train step, after leg 4 #
| emitted node | lemma | per-example peer (fused ViTPoC op unless noted) |
|---|---|---|
veclnGammaGradB (25 LN γ: LN1/LN2 × 12 + final) | veclnGammaGradB_den | ViTPoC.veclnGammaSgd_den |
rowDenseBiasGradB (25 LN β) | rowDenseBiasGradB_den_lnbeta | ViTPoC.rowDenseBiasSgd_den_lnbeta |
rowDenseWeightGradB (Wq/Wk/Wv/Wo/Wfc1/Wfc2 × 12) | rowDenseWeightGradB_den | ViTPoC.rowDenseWeightSgd_den |
rowDenseBiasGradB (bq/bk/bv/bo/bfc1/bfc2 × 12) | rowDenseBiasGradB_den | ViTPoC.rowDenseBiasSgd_den |
patchEmbedWeightGradB / patchEmbedBiasGradB | patchEmbedWeightGradB_den / patchEmbedBiasGradB_den | ViTPoC.patchEmbedWeightSgd_den / patchEmbedBiasSgd_den |
posEmbedGradB | posEmbedGradB_den | ViTPoCG.posEmbedGrad_den (un-fused) |
denseBiasGradB at N = B (the CLS token) | clsGrad_denB | ViTPoCG.clsGrad_den (un-fused), at N = 1 |
weightGradB / biasGradB (the classifier) | headWGradB_den / headBGradB_den | ViTPoC.headW_den / headB_den |
⭐ No new mathematics: every proof is Finset.sum_congr rfl over the batch and then the
per-example bridge at batchSlice n. That is ResNet34FoldB.denseWGradB_den's shape, and
it is available because each batched den arm is literally ∑_batch of the per-example one — the
constructors were written that way (StableHLO.lean's own comment on veclnGammaGradB: "TWO-LEVEL:
the outer Σ_n is the batch, the inner Σ_r the rows within one example").
⭐ One lemma per op kind certifies every optimizer tail at once — AdamW, the wx/clip
variants, the EMA shadow and the 4× accumulation all consume the same *GradB node, and the
vitin_adamdp128x4wxclipdrop artifact whose accuracy the book quotes is one of them.
⭐ The LayerNorm form is the VECTOR one (γ β : Vec D), which is what the shipped
vitForwardKV runs; the scalar-affine spelling this cone was caught on three times is nowhere here.
Honest residual #
- ⚠
biasGradBis the IDENTITY on its operand and the classifier bias's batch reduce is in the emitted text, outside the AST — the constructor says so ("the channel sum happens in the emitted reduce") and it is the per-examplebiasGradcarve-out carried over unchanged, not a new one. SoheadBGradB_denis stated PER EXAMPLE, atbatchSlice n, which is the whole of what the node denotes. - Every lemma is
∀ cot. Pinning each to the emitted backward subgraph is the §1a tie:ViTTiePoCGB.vit_net_tiedGBat these nodes (4b.7); the per-exampleViTStepTie.leanstays at the SGD-inlinevit_train_step.mlir. - The
*bf16artifacts emitrowDenseWeightGradBBf16/patchEmbedWeightGradBBf16, their own kinds;Foundation/Bf16GradNodes.leanfolds them (the row-dense one keeps its f32 result). vitin_adamdp128x4*is four replicas: the all-reduce is emitted text outside the AST, so these lemmas are about the per-replica gradient node (4d).
Batched vector-LN γ GRADIENT denotes the certified Σ_n γ gradient. Two levels: the outer
sum is the batch, the inner one the tokens within one example. All 25 sites.
The SAME row-reduce op, certified against the vector-LN β forward. An LN β gradient and a dense bias gradient are the identical two-level reduce, so this constructor appears twice in the table against two different certified Jacobians — as it does per example. All 25 β sites.
Batched per-token dense weight GRADIENT denotes the certified Σ_n Σ_tokens x⊗dy. All six
denses in each of the 12 blocks. ⚠ The emitted dot_general contracts batch AND token in ONE
op; the two sums here are that contraction read apart.
Batched per-token dense bias GRADIENT denotes the certified Σ_n Σ_tokens dy.
bq/bk/bv/bo/bfc1/bfc2.
Batched patch-embed conv weight GRADIENT denotes the certified Σ_n patchify weight
gradient. ⚠ The emitted convolution contracts the batch axis itself, so the outer sum here
is inside one op rather than across N of them.
Batched patch-embed conv bias GRADIENT denotes the certified Σ_n bias gradient
(Σ_patches dy, the CLS row excluded — the inner p.succ).
Batched positional-embed GRADIENT denotes the certified Σ_n gradient — the summed
cotangent, since the positional table is added to every token and its Jacobian is the
identity. ⚠⚠ den_patchEmbedBiasGradB's neighbour warns that this batch sum is INVISIBLE at
N = 1: a render that dropped it type-checks and emits the same bytes.
The CLS-token GRADIENT, and this is the one statement the per-example fold could not make.
The render slices row 0 of every example's embed cotangent (clsSlice, lifted by batchOp)
and reduces the result as an [N, D] batch, so the op is denseBiasGradB at N = B and its
den sums over the batch — which is what a shared [192] parameter's gradient IS.
⭐ ViTPoCG.clsGrad_den is the same theorem at N = 1, where the batch lift lived in
pretty B outside the AST. The bytes are identical (biasGrad's emitted reduce takes the B
axis either way) and the functions are not; den_rowDenseBiasGradB_at_one is the general form
of the trap.
⚠ Stated at the committed ViT-Tiny dims rather than generically, for ViTPoCG.clsGrad_den's
reason: the CLS operand's type is Vec (N * (1 * D)), which reduces to Vec (N * D) only at a
literal D.
Batched classifier weight GRADIENT denotes the certified Σ_n outer product.
Batched classifier bias GRADIENT denotes the certified cotangent, PER EXAMPLE.
⚠ biasGradB is the identity on its operand — the reduce over the batch is in the emitted
text, outside the AST — so the statement this node supports is the per-example one at every
batchSlice n, and it is the per-example biasGrad carve-out carried over rather than a new
one. StableHLO.lean's constructor comment records the same thing on the emitter side.
A batched per-token dense weight gradient node, tied (rowDenseWeightGradB_den).
Equations
- One or more equations did not get rendered due to their size.
Instances For
A batched per-token dense bias gradient node, tied (rowDenseBiasGradB_den).
Equations
- One or more equations did not get rendered due to their size.
Instances For
A batched vector-LN γ gradient node, tied (veclnGammaGradB_den).
Equations
- One or more equations did not get rendered due to their size.
Instances For
A batched vector-LN β gradient node, tied (rowDenseBiasGradB_den_lnbeta).
Equations
- One or more equations did not get rendered due to their size.