ResNet-34's whole-net input-VJP at TRUE BATCH-NORM #
ResNet34FullB.lean states the batch-BN forward and its typed graph. This file gives that forward
a certified HasVJPAt at the paper depth — the peer of what MobileNetV2FullBVJP.lean does
for MobileNetV2's seventeen bottlenecks.
No new mathematics, and one lemma one tier down #
Every block VJP is already proven at bnBatchLA: r34BasicBlockBHasVJPAt and
r34DownBlockBHasVJPAt (ResNet34BackB0.lean) are exactly the two block shapes
ResNet34FullB.lean's r34IdB / r34DownB unfold to. The bundle lemmas below are delegations
in the EfficientNetFullB0 style, and the whole net is those blocks' CertLayers composed.
The one lemma this needs below the net is batchMapHasVJPAt (Foundation/BatchMapVJPAt.lean):
r34's stem ends in batchMap N (maxPool3s2Flat 64 56 56) and a max-pool has no
derivative at a tie, so the GLOBAL batchMapHasVJP cannot lift it. With the per-example pool VJP
at a Vec point (maxPool3s2FlatHasVJPAtVec), the batched pool is two lines.
The hypothesis budget #
Pointwise (HasVJPAt), not global, and necessarily. relu is kinked. And r34 carries
two kink clauses per block, not one: the body's mid-relu AND the post-residual outer relu.
That outer relu is ResNet's structural difference from MobileNetV2/EfficientNet, whose residual
add IS the block output. Sixteen blocks therefore carry 32 clauses, plus the stem's relu and the
stem pool's condition (every window's maximum at one position, or the window all zero) — bundled per block into R34IdSmoothAt / R34DownSmoothAt, and
those 18 bundles into one R34SmoothAtB (the positivity bundles into R34PosB), so the apex
binds two hypotheses, exactly as MobileNetV2FullBVJP.lean's MNV2SmoothAtB / MNV2PosB.
The pool's condition is per example (∀ r : Fin N, MaxPool3s2Smooth … on that example's
row), because a tie is a property of one image's window, not of the batch. That is the shape
batchMapHasVJPAt consumes.
The running activations are named r34Pre0 … r34Pre16 so each bundle can be STATED at the
activation entering its block without a sixteen-deep nested application inline; r34Pre16 doubles
as the trunk, and resnet34ForwardBFull_eq_chain bridges it back to the committed
nested-application forward.
N is a variable throughout: the batch size is never pinned (the 224-px resolution and the
64…512 widths are literals).
Both relu sites of an identity basic block are away from the kink at v: the body's mid-relu
(at the first BN's output) and the OUTER relu (at the post-residual sum).
Instances For
Both relu sites of a downsample basic block are away from the kink at v. The mid-relu is
after the STRIDED conv's BN (already at h×w); the outer relu is at the projected residual.
- hmid (k : Fin (N * (oc * h * w))) : StableHLO.bnBatchLA N oc h w q.ε₁ q.γ₁ q.β₁ (StableHLO.batchMap N (flatConvStride2 q.W₁ q.b₁) v) k ≠ 0
Instances For
The stem's relu is away from the kink at the input x (the 7×7/s2 conv's BN output, at the
pre-pool 2h×2w grid).
Equations
- One or more equations did not get rendered due to their size.
Instances For
Identity basic block VJP — r34BasicBlockBHasVJPAt at the bundle's fields.
Equations
Instances For
Downsample basic block VJP — r34DownBlockBHasVJPAt at the bundle's fields.
Equations
Instances For
Stem VJP: the 7×7/s2 conv-bn-relu, then the batched 3×3/s2 pool — stemReluPoolLayer's,
certified as one layer so that a window of dead ReLUs is allowed.
Equations
- Proofs.r34StemBHasVJPAt N h w Ws bs εs hεs γs βs x hrelu hpool = (Proofs.stemReluPoolLayer N Ws bs εs hεs γs βs).vjp x ⋯
Instances For
The head is GLOBAL — GAP and dense are both smooth everywhere, and each is batchMap of a
per-example op, so batchMapHasVJP suffices and no smoothness hypothesis appears.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The stem, 7×7/s2 conv-bn-relu then the 3×3/s2 pool, as a CertLayer (stemReluPoolLayer).
Its ok is R34StemSmoothAt ∧ StemPoolSmoothAt at the conv's output.
Equations
- Proofs.r34StemLayer N h w Ws bs εs hεs γs βs = Proofs.stemReluPoolLayer N Ws bs εs hεs γs βs
Instances For
GAP then the classifier, as a CertLayer. Globally certified.
Equations
- Proofs.r34HeadLayer N h w Wd bd = (Proofs.StableHLO.gapLayer N).comp (Proofs.StableHLO.denseLayer N Wd bd)
Instances For
Equations
- Proofs.r34Pre1 N w = Proofs.r34IdB N 56 56 w.a0 ∘ Proofs.r34Pre0 N w
Instances For
Equations
- Proofs.r34Pre2 N w = Proofs.r34IdB N 56 56 w.a1 ∘ Proofs.r34Pre1 N w
Instances For
Equations
- Proofs.r34Pre3 N w = Proofs.r34IdB N 56 56 w.a2 ∘ Proofs.r34Pre2 N w
Instances For
Equations
- Proofs.r34Pre4 N w = Proofs.r34DownB N 28 28 w.d2 ∘ Proofs.r34Pre3 N w
Instances For
Equations
- Proofs.r34Pre5 N w = Proofs.r34IdB N 28 28 w.b0 ∘ Proofs.r34Pre4 N w
Instances For
Equations
- Proofs.r34Pre6 N w = Proofs.r34IdB N 28 28 w.b1 ∘ Proofs.r34Pre5 N w
Instances For
Equations
- Proofs.r34Pre7 N w = Proofs.r34IdB N 28 28 w.b2 ∘ Proofs.r34Pre6 N w
Instances For
Equations
- Proofs.r34Pre8 N w = Proofs.r34DownB N 14 14 w.d3 ∘ Proofs.r34Pre7 N w
Instances For
Equations
- Proofs.r34Pre9 N w = Proofs.r34IdB N 14 14 w.c0 ∘ Proofs.r34Pre8 N w
Instances For
Equations
- Proofs.r34Pre10 N w = Proofs.r34IdB N 14 14 w.c1 ∘ Proofs.r34Pre9 N w
Instances For
Equations
- Proofs.r34Pre11 N w = Proofs.r34IdB N 14 14 w.c2 ∘ Proofs.r34Pre10 N w
Instances For
Equations
- Proofs.r34Pre12 N w = Proofs.r34IdB N 14 14 w.c3 ∘ Proofs.r34Pre11 N w
Instances For
Equations
- Proofs.r34Pre13 N w = Proofs.r34IdB N 14 14 w.c4 ∘ Proofs.r34Pre12 N w
Instances For
Equations
- Proofs.r34Pre14 N w = Proofs.r34DownB N 7 7 w.d4 ∘ Proofs.r34Pre13 N w
Instances For
Equations
- Proofs.r34Pre15 N w = Proofs.r34IdB N 7 7 w.e0 ∘ Proofs.r34Pre14 N w
Instances For
Equations
- Proofs.r34Pre16 N w = Proofs.r34IdB N 7 7 w.e1 ∘ Proofs.r34Pre15 N w
Instances For
Every BatchNorm ε of the net is positive: the stem's and each block's bundle.
- d2 : R34DownPos w.d2
- d3 : R34DownPos w.d3
- d4 : R34DownPos w.d4
Instances For
Every relu is away from its kink and every stem-pool window is smooth or dead, each at the activation
its block actually sees: the stem's clauses at the image, block k's at r34Pre(k-1). The
head has none (GAP and dense are smooth).
- pool : StemPoolSmoothAt N 56 56 (StableHLO.cbReluStridedB N w.sW w.sb w.sε w.sγ w.sβ x)
- a0 : R34IdSmoothAt N 56 56 w.a0 (r34Pre0 N w x)
- a1 : R34IdSmoothAt N 56 56 w.a1 (r34Pre1 N w x)
- a2 : R34IdSmoothAt N 56 56 w.a2 (r34Pre2 N w x)
- d2 : R34DownSmoothAt N 28 28 w.d2 (r34Pre3 N w x)
- b0 : R34IdSmoothAt N 28 28 w.b0 (r34Pre4 N w x)
- b1 : R34IdSmoothAt N 28 28 w.b1 (r34Pre5 N w x)
- b2 : R34IdSmoothAt N 28 28 w.b2 (r34Pre6 N w x)
- d3 : R34DownSmoothAt N 14 14 w.d3 (r34Pre7 N w x)
- c0 : R34IdSmoothAt N 14 14 w.c0 (r34Pre8 N w x)
- c1 : R34IdSmoothAt N 14 14 w.c1 (r34Pre9 N w x)
- c2 : R34IdSmoothAt N 14 14 w.c2 (r34Pre10 N w x)
- c3 : R34IdSmoothAt N 14 14 w.c3 (r34Pre11 N w x)
- c4 : R34IdSmoothAt N 14 14 w.c4 (r34Pre12 N w x)
- d4 : R34DownSmoothAt N 7 7 w.d4 (r34Pre13 N w x)
- e0 : R34IdSmoothAt N 7 7 w.e0 (r34Pre14 N w x)
- e1 : R34IdSmoothAt N 7 7 w.e1 (r34Pre15 N w x)
Instances For
The committed nested-application forward IS the layered chain the VJP is stated on —
the r34 peer of mobilenetv2ForwardBFull_eq_chain, and what lets the VJP be about
resnet34ForwardBFull rather than about a re-spelling of it.
ResNet-34 as one certified layer, at every batch size. Its .faithful is a whole-net
backward graph, stem pool included, proven to denote the VJP.
Equations
- One or more equations did not get rendered due to their size.
Instances For
R34SmoothAtB is the layer's .ok.
ResNet-34 at TRUE BATCH-NORM has a certified input-VJP at a smooth point — all sixteen
basic blocks. r34NetLayer's .vjp, read at the layered chain, under two hypotheses: R34PosB (every ε > 0) and R34SmoothAtB (every relu clause and the pool's
no-tie, each at its block's own input).
Pointwise, and necessarily: relu is kinked. Each block contributes TWO clauses — the body's mid-relu and the post-residual OUTER relu — where MobileNetV2's bottleneck contributes two relu6 clauses and EfficientNet's MBConv contributes none.
The head takes no hypothesis at all (GAP and dense are smooth, and each is batchMap of a
per-example op), and N is a variable: the batch size is never pinned (the 224-px resolution
and the 64…512 widths are literals).
Equations
- Proofs.resnet34ForwardBFullHasVJPAt N w hq x hx = Proofs.HasVJPAt.congr ⋯ ((Proofs.r34NetLayer N w hq).vjp x ⋯)
Instances For
Public correctness theorem: the sixteen-block batch-BN backward equals the
pdiv-contracted Jacobian of resnet34ForwardBFull ITSELF — the committed
nested-application forward ResNet34FullB.lean defines and
resnet34FwdGraphBFull_faithful proves the typed graph denotes — not of the layered chain
the VJP is assembled on. Tied back through resnet34ForwardBFull_eq_chain.
The committed forward is differentiable at every smooth point — the layer's .diff. What
the seal's seal_differentiableAt needs.