The batch-statistics cifar8-BN backward chain IS the certified whole-net VJP #
Cifar8BbnStepTieB.cifar8Bbn_train_step_tiedB ties the thirty-eight gradient NODES of the
cifar8wb_bbn_* train steps: each denotes the layer's Jacobian contracted with the cotangent the
rendered backward chain hands it. That theorem takes the cotangents as the chain delivers them.
This file is the other half — that the chain delivers the right ones: the backward chain those
artifacts render, read in the BackLinks vocabulary the node tie uses (reluMaskB, bnBackB,
cInB, the per-example pool scatter, rowDenseBackFlat), IS the backward of the certified VJP of
the whole batched forward, relu ∘ dense ∘ relu ∘ dense ∘ dense ∘ pool ∘ (conv → BN → relu)² ∘ …,
at every batch size N. ResNet-50's r50InputGradB_eq_r34B_full_vjp is the statement this one
mirrors; what ResNet-50 does with sixteen opaque bottleneck witnesses, this net does with
seventeen concrete CertLayers, because every stage of it is a layer the suite already certifies.
How it is built #
maxPool2Layer— the batched 2×2/s2 max pool as aCertLayer:batchMapHasVJPAtover the per-examplemaxPoolFlatHasVJPAt', certified where every example's windows attain their max once (MaxPool2Smooth), its graph the render's.maxPoolBackB, faithful bybatchMapAux_eq_batchMapHasVJPAtwith each row's tierfl. The 2×2 peer ofHeadLayers' 3×3/s2 stem pool.cifar8BbnLayer— the net: eightcbReluLayers (conv → true BN → relu) and four pools in four stages, thendenseLayer,reluOut,denseLayer,reluOut,denseLayer.CertLayer.compdoes the composition proof; this file only names the layers.Cifar8BbnSmoothAt— the smooth-point hypotheses, one per kink: eight pre-relu activations off zero, four pools with unique window maxima, two head pre-activations off zero, each stated at a NAMED running activation (c8A1 … c8RA) so the.okdischarge stays onecomp_ok_ofper layer.cifar8BbnInputGradB— the chain, written out;cifar8BbnBHasVJPAt_backwardsays the layer's VJP backward is that chain,cifar8BbnInputGradB_correctreads it as∑ pdiv … * dy, andcifar8BbnLayer_fwdsays the layer's forward is the committed compositecifar8BbnForwardB.
Scope #
A smooth-point statement at one precision: the conv nodes are the f32 kinds, as in ResNet-50's
whole-net tie (the bf16 artifacts' gradient nodes are Cifar8BbnStepTieB's, on the renderers'
bf16 switch at the identity rounding). One device: cifar8wb_bbn has no data-parallel render.
It is about the INPUT cotangent chain; the thirty-eight parameter gradients are the node tie's,
and that each of them is the loss's derivative in its parameter is
Cifar8BbnParamGrad.lean
(cifar8Bbn_net_lossGrad).
Per-example smoothness of a batched pool input: every example's 2×2 windows attain their
maximum at exactly one cell (MaxPool2Smooth, read on that example's [C, 2h, 2w] slice).
Equations
- Proofs.StableHLO.MaxPool2SmoothB N c h w x = ∀ (r : Fin N), Proofs.MaxPool2Smooth (Proofs.Tensor3.unflatten (Proofs.Mat.unflatten x r))
Instances For
The batched 2×2 pool's VJP at a smooth point — batchMapHasVJPAt over each example's own
maxPoolFlatHasVJPAt'. Built field by field (no transport), so its .backward reduces.
Equations
- Proofs.StableHLO.maxPool2BHasVJPAt N c h w x hx = Proofs.batchMapHasVJPAt (Proofs.maxPoolFlat c h w) x (fun (r : Fin N) => Proofs.StableHLO.maxPoolFlatHasVJPAt' (Proofs.Mat.unflatten x r) ⋯) ⋯
Instances For
Its backward is the render's scatter: maxPoolBackFlat per example on that example's own
saved activation, which is den (.maxPoolBackB …).
Batched 2×2/s2 max pool as a CertLayer, certified where every example's windows are
smooth; its backward graph is the render's .maxPoolBackB at the saved pool input.
Equations
- One or more equations did not get rendered due to their size.
Instances For
comp's VJP backward, peeled: L₂'s then L₁'s. Definitional (vjpCompAt_backward); stated
so a whole-net chain unwinds by rw rather than by simp on the structure.
The conv → BN → relu stage's VJP backward, in the BackLinks vocabulary: the relu mask at the
pre-activation, the true-BN backward at the conv output, the conv input-VJP. The relu peer of
cbsB_back_eq.
cbReluLayer's VJP backward, in the BackLinks vocabulary.
maxPool2Layer's VJP backward is the per-example scatter.
reluOut's VJP backward is the relu mask.
The thirty-eight tensors of the eight-conv CIFAR net with BatchNorm, as one record: eight
(kernel, bias, γ, β) and the three dense layers. One ε is shared by every BatchNorm, as the
render bakes it.
- W₁ : Kernel4 c1 ic kH kW
- b₁ : Vec c1
- γ₁ : Vec c1
- β₁ : Vec c1
- W₂ : Kernel4 c1 c1 kH kW
- b₂ : Vec c1
- γ₂ : Vec c1
- β₂ : Vec c1
- W₃ : Kernel4 c2 c1 kH kW
- b₃ : Vec c2
- γ₃ : Vec c2
- β₃ : Vec c2
- W₄ : Kernel4 c2 c2 kH kW
- b₄ : Vec c2
- γ₄ : Vec c2
- β₄ : Vec c2
- W₅ : Kernel4 c3 c2 kH kW
- b₅ : Vec c3
- γ₅ : Vec c3
- β₅ : Vec c3
- W₆ : Kernel4 c3 c3 kH kW
- b₆ : Vec c3
- γ₆ : Vec c3
- β₆ : Vec c3
- W₇ : Kernel4 c4 c3 kH kW
- b₇ : Vec c4
- γ₇ : Vec c4
- β₇ : Vec c4
- W₈ : Kernel4 c4 c4 kH kW
- b₈ : Vec c4
- γ₈ : Vec c4
- β₈ : Vec c4
- b₉ : Vec d1
- Wa : Mat d1 d1
- ba : Vec d1
- Wb : Mat d1 nC
- bb : Vec nC
Instances For
The running activations, one def each, in forward order. Stage k runs at spatial
2^(5-k)·h, written as the explicit 2 * (…) nest the render's types use.
conv₁ → BN → relu, at 32².
Equations
- Proofs.c8A1 N p ε x = Proofs.StableHLO.cbReluB N p.W₁ p.b₁ ε p.γ₁ p.β₁ x
Instances For
conv₂ → BN → relu, at 32².
Equations
- Proofs.c8A2 N p ε x = Proofs.StableHLO.cbReluB N p.W₂ p.b₂ ε p.γ₂ p.β₂ (Proofs.c8A1 N p ε x)
Instances For
pool₁: 32² → 16².
Equations
- Proofs.c8P1 N p ε x = Proofs.StableHLO.batchMap N (Proofs.maxPoolFlat c1 (2 * (2 * (2 * h))) (2 * (2 * (2 * w)))) (Proofs.c8A2 N p ε x)
Instances For
conv₃ → BN → relu, at 16².
Equations
- Proofs.c8A3 N p ε x = Proofs.StableHLO.cbReluB N p.W₃ p.b₃ ε p.γ₃ p.β₃ (Proofs.c8P1 N p ε x)
Instances For
conv₄ → BN → relu, at 16².
Equations
- Proofs.c8A4 N p ε x = Proofs.StableHLO.cbReluB N p.W₄ p.b₄ ε p.γ₄ p.β₄ (Proofs.c8A3 N p ε x)
Instances For
pool₂: 16² → 8².
Equations
- Proofs.c8P2 N p ε x = Proofs.StableHLO.batchMap N (Proofs.maxPoolFlat c2 (2 * (2 * h)) (2 * (2 * w))) (Proofs.c8A4 N p ε x)
Instances For
conv₅ → BN → relu, at 8².
Equations
- Proofs.c8A5 N p ε x = Proofs.StableHLO.cbReluB N p.W₅ p.b₅ ε p.γ₅ p.β₅ (Proofs.c8P2 N p ε x)
Instances For
conv₆ → BN → relu, at 8².
Equations
- Proofs.c8A6 N p ε x = Proofs.StableHLO.cbReluB N p.W₆ p.b₆ ε p.γ₆ p.β₆ (Proofs.c8A5 N p ε x)
Instances For
pool₃: 8² → 4².
Equations
- Proofs.c8P3 N p ε x = Proofs.StableHLO.batchMap N (Proofs.maxPoolFlat c3 (2 * h) (2 * w)) (Proofs.c8A6 N p ε x)
Instances For
conv₇ → BN → relu, at 4².
Equations
- Proofs.c8A7 N p ε x = Proofs.StableHLO.cbReluB N p.W₇ p.b₇ ε p.γ₇ p.β₇ (Proofs.c8P3 N p ε x)
Instances For
conv₈ → BN → relu, at 4².
Equations
- Proofs.c8A8 N p ε x = Proofs.StableHLO.cbReluB N p.W₈ p.b₈ ε p.γ₈ p.β₈ (Proofs.c8A7 N p ε x)
Instances For
pool₄: 4² → 2², the flatten the head reads (c4·h·w wide).
Equations
- Proofs.c8P4 N p ε x = Proofs.StableHLO.batchMap N (Proofs.maxPoolFlat c4 h w) (Proofs.c8A8 N p ε x)
Instances For
dense₉ pre-activation.
Equations
- Proofs.c8H9 N p ε x = Proofs.StableHLO.batchMap N (Proofs.dense p.W₉ p.b₉) (Proofs.c8P4 N p ε x)
Instances For
denseₐ pre-activation.
Equations
- Proofs.c8HA N p ε x = Proofs.StableHLO.batchMap N (Proofs.dense p.Wa p.ba) (Proofs.c8R9 N p ε x)
Instances For
The batched eight-conv CIFAR forward with BatchNorm at batch statistics, to the logits:
the composite dense_b ∘ relu ∘ dense_a ∘ relu ∘ dense₉ ∘ pool ∘ (conv → BN → relu)² ∘ …
at batch N, through the named running activations above.
Equations
- Proofs.cifar8BbnForwardB N p ε x = Proofs.StableHLO.batchMap N (Proofs.dense p.Wb p.bb) (Proofs.c8RA N p ε x)
Instances For
The shape check: the composed layer's forward IS cifar8BbnForwardB.
Where the net is certified: each of the eight pre-relu activations off its kink, each of
the four pool inputs with unique window maxima per example, the two head pre-activations off
zero — at the running activations. Everything else in the net (conv, BN at 0 < ε, dense) is
smooth everywhere.
- q3 : StableHLO.MaxPool2SmoothB N c3 (2 * h) (2 * w) (c8A6 N p ε x)
- q4 : StableHLO.MaxPool2SmoothB N c4 h w (c8A8 N p ε x)
Instances For
Cifar8BbnSmoothAt is the layer's .ok: one comp_ok_of per layer, each naming its input.
The certified whole-net VJP of cifar8BbnForwardB at a smooth point.
Equations
- Proofs.cifar8BbnBHasVJPAt N p ε hε x hs = (Proofs.cifar8BbnLayer N p ε hε).vjp x ⋯
Instances For
The rendered backward chain, read in the BackLinks vocabulary, from the logits cotangent
dy to the input cotangent: three rowDenseBackFlats with the two head relu masks between
them, then per stage the pool scatter, and twice reluMaskB → bnBackB → cInB. Every step is
the den of the node the artifact emits (den_convBackBatched_eq_cInB, den_maxPoolBackB,
den_selectPosB, den of .denseRowBack), the BatchNorm link through bnInB_eq_bnBackB.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The chain IS the certified whole-net backward. The layer's VJP backward, peeled one
comp at a time, is each stage's backward in turn, and each of those is its BackLinks
form: cbReluB_back_eq at the eight conv stages, maxPool2BHasVJPAt_backward at the four
pools, and rfl at the dense and relu layers.
…and so is the certified whole-net VJP's.
The chain IS the pdiv-contracted Jacobian of the whole net — at every batch size, every
logits cotangent and every input coordinate, wherever the eight relus and the two head relus
are off their kinks and the four pools have unique window maxima (Cifar8BbnSmoothAt), with
0 < ε.