ResNet-34's T3 §1a TIE at TRUE BATCH-NORM — the un-fused, batched whole-net thread #
ResNet34FoldB.lean (4.1e) makes every parameter GRADIENT node of the batched ResNet-34
train step den-faithful for an arbitrary cotangent. This file removes the "arbitrary": each
cotangent is pinned to the one the emitted backward chain delivers, so the whole train step is
den-composed forward → loss → backward with no free activation and no symbolic cotangent.
It replaced the per-example ResNet34TiePoC.lean (deleted 2026-09-08: no committed artifact
exercised it once resnet34_train_step.mlir was retired), and four things about it are different in kind.
⭐⭐ The block cotangents are NOT derived here. The per-example close (ResNet34ChainClose.lean,
deleted 2026-09-08) spelled per-block cotangent vectors out by hand, because no whole-block VJP
existed when it was written. 4.1d's
r34IdB_has_vjp_at / r34DownB_has_vjp_at ARE the certified block backwards, so a block's input
cotangent is a .backward application — and r34{BasicBlock,DownBlock}BackBatchedGraph_faithful
already proves the emitted backward subgraph denotes exactly it. r34IdCotIn_eq_vjp and
r34DownCotIn_eq_vjp below are that statement in the vocabulary this file threads, and they are
what make the cross-block chain a composition of certified VJPs rather than a re-derivation.
⭐ The loss cotangent is the LABEL-SMOOTHED one, at a general target. ResNet34RenderB composes
the head cotangent from six kit ops — softmaxRow → subB → scaleB → addVB → shiftB → divConstB, α
baked at 0.1 and the ls0 variants at 0 — and the target arrives as the graph input %onehot,
which under mixup or cutmix is a soft vector drawn on the host. Foundation/SmoothedLossCot.lean
is that cotangent's lemma; the head fold below is stated at it, not at softmax − oneHot.
⭐ N is a binder. The capstone takes (N : Nat), exactly as efficientnet_net_tied does;
the artifacts at 32 (resnet34_sgd/adam_train_step) or 64 (resnet34in_momdp64) are instances.
T3 carries no numerals, so nothing here pins the batch.
⛔ The all-reduce, since 4d piece 2 (2026-09-07). In resnet34in_momdp64 each *GradB node
feeds allReduceMeanF — the collective as an AST node whose den is the replica MEAN of the
per-replica gradient nodes; until then emitGradAllReduce, emitted text and a declared carve-out
outside the SHlo AST. Every statement below is at the PER-REPLICA gradient node;
DataParallelNode.lean composes it with the mean and the tail. For the sync-BN data-parallel
render (2026-09-21) ResNet34SyncStepTieB.lean is the whole step: its r34_net_syncTiedB says
each all-reduced gradient IS this file's node at N := R·N.
⛔ The parameter census is 110, not the 146 the per-example tie named #
resnet34TrainStepFaithfulV and ResNet34RenderB both default to convBias := false: the conv
biases are gone from the signature (BatchNorm subsumes them, and He et al.'s .convBn has none),
bound instead to the zero constants zeroBiasPrelude emits. So resnet34_sgd_train_step.mlir
carries 110 SGD-updated tensors — stem 3 + 13 identity blocks × 6 + 3 downsample blocks × 9 +
dense 2 — and 146 is the census at convBias := true. (The per-example resnet34_train_step.mlir
carried the same 110 and was retired the same day by 4c leg 1.) The bias
conjuncts below are kept (they are one delegation each and they cover the flag), and they are about
ops the committed artifacts do not emit. Nothing about this weakens a theorem: every fold is
∀-quantified over op instances, and bias = 0 is one of them.
Conventions, stated because nothing here checks them #
| BatchNorm | batch (bnBatchLA, reduce [0,2,3], width N·h·w) |
| padding | symmetric at all seven stride-2 sites (convStrided*, not the XLA-SAME twins) |
| stem pool | 3×3/s2 (maxPool3s2Flat), not 2×2 |
| activation | relu, TWO kinks per block (the body's mid-relu and the post-residual outer one) |
| optimizer form | the RAW gradient (*GradB); the fused θ − lr·g appears in no batched r34 step |
| loss | label-smoothed softmax-CE at a general target, batch-meaned |
The relu backward mask — den (.selectPosB _ pre e) = fun i => if pre i > 0 then e i else 0.
r34 applies it twice per block (the body's mid-relu and the post-residual outer one) and once at
the stem.
Instances For
Batched STRIDED conv input-VJP (= den convStridedBackBatched; upsamples h → 2h). The
strided peer of EfficientNet's cInB. ⚠ SYMMETRIC padding — flatConvStride2, not the
XLA-SAME twin.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Batched true-BN input-cotangent, as the EMITTED backward computes it. Written as the den
of the backward op rather than as the certified VJP's .backward, because that is the form the
render's chain is in and den ignores the name strings — so every cotangent below is literally
what the artifact's bytes compute. ⚠ The render's node is .bnBatchBack, typed at
N·(oc·(h·w)); this is its network-layout twin, and bnInB_eq_den_bnBatchBack below says
the two denote one map up to reassocB. ⭐ It takes no β: the BatchNorm input-gradient does not
depend on the shift, which bnInB_eq_bnBackB records by holding for every β.
Equations
- Proofs.ResNet34TieB.bnInB N oc h w ε γ x dy = Proofs.StableHLO.den (Proofs.StableHLO.SHlo.bnBatchLABack "" "" "" ε γ x (Proofs.StableHLO.SHlo.operand "" dy))
Instances For
…and it IS the certified bnBatchLA VJP, for every β and every 0 < ε. This is
bnBatchLABack_faithful, and it is the only step in this file's cotangent chain that is not
rfl — everything else (the relu masks, the conv and strided-conv input-VJPs, the pool
backward) denotes its certified backward definitionally.
…and it IS the den of the node the render emits, .bnBatchBack at the N·(oc·(h·w))
index, read back through reassocB (EnetTiePoC.den_bnBatchLABack_eq_bnBatchBack).
Batched 3×3/s2 max-pool backward (= den maxPool3s2BackB): the select_and_scatter
denotation, per example on that example's own saved activation — which is why it is
batchMapAux and not batchMap.
Equations
- Proofs.ResNet34TieB.mpInB N c h w x dy = Proofs.StableHLO.batchMapAux N (Proofs.StableHLO.maxPool3s2BackFlat c h w) x dy
Instances For
ResNet34RenderB.idBackGradB, node for node, from the block-output cotangent dyOut:
%da = selectPosB(a) %dn2 = bnBatchBack(g2, c2) %dc2 = convBackBatched(W2)
%dr1 = selectPosB(n1) %dn1 = bnBatchBack(g1, c1) %dc1 = convBackBatched(W1)
%dx = addVB(%dc1, %da)
and the eight parameter nodes read W1,b1 ← %dn1, g1,bt1 ← %dr1, W2,b2 ← %dn2,
g2,bt2 ← %da. The four defs below are those four cotangents.
Cotangent at the block's pre-relu sum a — the outer relu's mask applied to dyOut. It feeds
bn₂'s γ/β directly AND (through the identity skip) the block-input fan-in.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at conv₁'s output — r34IdCotN1 through bn₁'s backward. Feeds W₁/b₁.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The block-INPUT cotangent: the residual fan-in addVB(convBack(dn1), da) the render emits —
the body branch plus the identity skip.
Equations
- Proofs.ResNet34TieB.r34IdCotIn N h w p xin dyOut i = Proofs.EnetTiePoC.cInB N p.W₁ p.b₁ (Proofs.ResNet34TieB.r34IdCotC1 N h w p xin dyOut) i + Proofs.ResNet34TieB.r34IdCotA N h w p xin dyOut i
Instances For
⭐⭐ The emitted fan-in IS the certified block VJP's backward. Not a re-derivation: the
render's seven-node backward subgraph denotes (r34IdB_has_vjp_at …).backward dyOut, which is
r34BasicBlockBackBatchedGraph_faithful read in this file's vocabulary. This is what makes the
cross-block thread a composition of certified VJPs.
ResNet34RenderB.downBackGradB adds one branch to the identity block's chain:
%da = selectPosB(a) %dn2 = bnBatchBack(g2,c2) %dc2 = convBackBatched(W2)
%dr1 = selectPosB(n1) %dn1 = bnBatchBack(g1,c1) %dc1 = convStridedBackBatched(W1)
%dnp = bnBatchBack(gp,cp) %dcp = convStridedBackBatched(Wp)
%dx = addVB(%dc1, %dcp)
so bnₚ's γ/β read the SAME %da that bn₂'s do — both feed the one addVB — and the projection's
Wp/bp read %dnp.
Cotangent at the pre-relu sum. Feeds bn₂'s AND bnₚ's γ/β — they share the one addVB.
Equations
- Proofs.ResNet34TieB.r34DownCotA N h w p xin dyOut = Proofs.ResNet34TieB.reluMaskB (N * (oc * h * w)) (Proofs.ResNet34TieB.r34DownPre N h w p xin) dyOut
Instances For
Cotangent at the STRIDED conv₁'s output. Feeds W₁/b₁.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the PROJECTION conv's output. Feeds Wp/bp; its BN's γ/β read r34DownCotA.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The block-INPUT cotangent: the projected-residual fan-in addVB(%dc1, %dcp). ⚠ Both
operands are real backward subgraphs here — unlike the identity block, where the skip passes
%da through verbatim.
Equations
- One or more equations did not get rendered due to their size.
Instances For
⭐⭐ The projected fan-in IS the certified downsample-block VJP's backward. The identity
block's r34IdCotIn_eq_vjp at the strided shape. ⚠ One add_comm: the render emits
addVB(body, projection) and r34DownBlockBackBatchedGraph builds addV(projection, body).
Same vector, and the emitted order is the one this file threads.
ResNet34RenderB, after the sixteen block backwards:
%dmp = maxPool3s2BackB(str) %dsr = selectPosB(stn) %dsn = bnBatchBack(sg, stc)
with sW,sb ← %dsn and sg,sbt ← %dsr.
Cotangent at the stem's post-relu, pre-pool activation — the 3×3/s2 pool's backward applied to the cotangent block 1 delivers at its input.
Equations
- Proofs.ResNet34TieB.r34StemCotP N h w Ws bs εs γs βs x cotPool = Proofs.ResNet34TieB.mpInB N oc h w (Proofs.StableHLO.cbReluStridedB N Ws bs εs γs βs x) cotPool
Instances For
Cotangent at the stem BN's output — the stem relu's mask. Feeds γs/βs.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the stem conv's output. Feeds sW/sb.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Each conjunct is ResNet34FoldB's ∀ cot fold instantiated at the cotangent the
render's chain delivers, so nothing here is a new proof: the bundles are the §1 fold with the
freedom removed. reassocB bridges the conv/relu index N·(c·h·w) to the BatchNorm parameter
ops' N·(c·(h·w)).
⚠ The two conv-BIAS conjuncts in each block are about conv{,Strided}BiasGradB, which the
committed artifacts do NOT emit — both renders run convBias := false and bind the bias operand to
zeroBiasPrelude's zero constant. They are kept because they cost one delegation each and they
cover the flag.
Identity basic block, tied. All eight parameter nodes — conv₁/conv₂ weight and bias, bn₁/bn₂
γ and β — denote the certified batched Σ_n gradient at the real forward activations and the
real backward-chain cotangent driven by dyOut.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Downsample basic block, tied. All twelve parameter nodes — the STRIDED conv₁, the stride-1
conv₂ and the 1×1/s2 option-B projection, each with bias and BatchNorm γ/β. ⚠ Both stride-2
sites are SYMMETRIC padding (convStrided*GradB, whose den is flatConvStride2_*), which is
ResNet's convention and NOT B0's or MobileNetV2's XLA-SAME.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stem, tied. The 7×7/s2 conv's weight and bias and its BatchNorm's γ/β, at the cotangent that reaches the stem through block 1's input fan-in and the 3×3/s2 pool's backward.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Vec (N·(1·K)) → Vec (N·K): the loss chain runs at one ROW per example (softmaxRow needs a
row index) and the dense parameter ops at the plain per-example width. The render writes one
SSA name for both, because 1 * K = K as an emitted shape; in Lean the two indices are
propositionally but not definitionally equal, so the cast is explicit.
Equations
- Proofs.ResNet34TieB.unrowB N K v i = v (Fin.cast ⋯ i)
Instances For
The head's block-side cotangent, as the CERTIFIED head backward delivers it. The head is
batchMap(dense) ∘ batchMap(GAP) — both smooth, both batchMap of a per-example op — so
r34HeadB_has_vjp is global and this needs no smoothness hypothesis, which is the one place in
the whole net where that is true.
Equations
- Proofs.ResNet34TieB.r34HeadCotBlk N h w Wd bd xin dy = (Proofs.r34HeadB_has_vjp N h w Wd bd).backward xin dy
Instances For
Head, tied. The classifier's weight and bias nodes denote the certified batched Σ_n
gradient at the real GAP output and the loss cotangent. ⚠ The bias conjunct's Jacobian witness
carries a zero activation rather than the real one: dense's derivative in b is the identity
whatever x is, so the statement is x-free and there is no per-example choice to make (the
same shape EfficientNetStepTie's bias conjuncts take).
Equations
- One or more equations did not get rendered due to their size.
Instances For
⭐⭐ The whole batch-BN ResNet-34 train step, tied. Threading resnet34ForwardB_full's own
prefixes as the block inputs and the label-smoothed loss cotangent down through the certified
head backward and the sixteen certified block backwards, every parameter GRADIENT node of the
net — stem 4, thirteen identity blocks × 8, three downsample blocks × 12, dense 2 — denotes the
certified batched Σ_n gradient. No free activation and no symbolic cotangent.
⭐ N is a binder and there is no smoothness hypothesis. The folds are ∀ cot statements
instantiated at explicitly-constructed cotangents, so the capstone needs neither 0 < ε nor a
relu-kink condition. Those enter only in r34IdCotIn_eq_vjp / r34DownCotIn_eq_vjp, which say
the constructed chain IS the certified whole-net backward — the two halves of the tie, kept
apart because they have different hypotheses.
⚠ Of the 146 conjunct slots, the committed artifacts exercise 110: both r34 renders run
convBias := false, so the 36 conv-bias nodes are not emitted (the biases are
zeroBiasPrelude's zero constants).
⛔ One replica. In resnet34in_momdp64 every gradient node feeds allReduceMeanF, an AST
node since 4d piece 2; ResNet34SyncTieB.r34_net_syncTiedB is the data-parallel step, and
its right-hand sides are this theorem's nodes at N := R·N.
⭐ And the cotangent the capstone threads is the smoothed loss's gradient. Row by row: the
g above is, at example n and class j, (1/B)·∂/∂logits of soft-target cross-entropy
against the SMOOTHED target (1−α)·t + α/K, at that example's real logits. The only hypothesis
is that the example's target sums to 1 — a one-hot, or mixup's convex combination of two.
Together with the capstone this closes the top of the chain: every parameter node denotes the
certified gradient at the cotangent of the loss the trainer actually minimises.