MobileNetV2's whole-net input-VJP at batch BatchNorm #
MobileNetV2FullB.lean states the batch-BN forward and its typed graph. This file gives that
forward a certified HasVJPAt at the paper depth — the MobileNetV2 peer of
ResNet34FullBVJP.lean.
Built from the block VJPs #
Every block VJP is already proven at bnBatchLA: mnv2BodyBHasVJPAt and
mnv2DownBodyBHasVJPAt (MobileNetV2BackB0.lean) are exactly the two body shapes
mnv2ExpOnlyB / mnv2StridedB unfold to, and residualHasVJPAt wraps the first for the ten
skip blocks. The bundle lemmas below are delegations in the EfficientNetFullB0 style.
No pointwise batchMap lemma is needed. ResNet-34's stem ends in
batchMap N (maxPool3s2Flat …) and a max-pool has no derivative at a tie, so it needs
batchMapHasVJPAt. MobileNetV2 has NO stem pool — the stem is conv-BN-relu6 and downsamples
once — and its head's GAP and dense are smooth, so batchMapHasVJP (the global one) covers
every batchMap in the net.
Three shapes are bnRelu6StageHasVJPAt at a different inner op, and that lemma is already
generic in it: the stem is that stage at flatConvStride2Xla, the head's first stage is cbrB,
and the expand/depthwise stages are cbrB / dwbrB / dwbrBstrided. The t = 1 block
projB ∘ dwbrB (b1) has no mnv2*BodyB peer; it is dwbrLayer ; projLayer.
The hypothesis budget #
Pointwise (HasVJPAt), not global. relu6 is kinked on both sides, so each site carries
≠ 0 ∧ ≠ 6 and a global HasVJP through it is false.
Two kink clauses per bottleneck, and the second is not r34's. ResNet-34's blocks carry the
body's mid-relu AND a post-residual OUTER relu. MobileNetV2's carry the EXPAND relu6 and the
DEPTHWISE relu6, both inside the body — the linear bottleneck has no activation after project,
so the residual add IS the block output and contributes nothing. Sixteen expand-bearing blocks give
32 clauses, plus b1's single depthwise clause, plus the stem's and the head's: 35 relu6 sites,
bundled into 19 binders.
The positivity bundles are IVPos / IVNoExpPos (MobileNetV2FullPaper.lean): 0 < ε at
each BatchNorm site. The smoothness bundles are declared here, because a kink condition names the
activation, which is bnBatchLA's output.
The running activations are named mnv2PreB0 … mnv2PreB17 so each bundle can be STATED at the
activation entering its block without a seventeen-deep nested application inline; mnv2PreB17
doubles as the trunk, and mobilenetv2ForwardBFull_eq_chain bridges it back to the committed
nested-application forward.
N is a variable throughout.
Both relu6 sites of a stride-1 bottleneck are away from BOTH kinks at v, at batch BN: the
expand-BN output and the depthwise-BN output each avoid 0 and 6 in every coordinate.
- hd (k : Fin (N * (mid * h * w))) : StableHLO.bnBatchLA N mid h w q.dε q.dγ q.dβ (StableHLO.batchMap N (depthwiseFlat q.dW q.db) (StableHLO.cbrB N q.eW q.eb q.eε q.eγ q.eβ v)) k ≠ 0 ∧ StableHLO.bnBatchLA N mid h w q.dε q.dγ q.dβ (StableHLO.batchMap N (depthwiseFlat q.dW q.db) (StableHLO.cbrB N q.eW q.eb q.eε q.eγ q.eβ v)) k ≠ 6
Instances For
Both relu6 sites of a stride-2 bottleneck are away from both kinks at v, at batch BN (the
expand runs at the pre-downsample 2h x 2w grid, the XLA-SAME depthwise at h x w).
- hd (k : Fin (N * (mid * h * w))) : StableHLO.bnBatchLA N mid h w q.dε q.dγ q.dβ (StableHLO.batchMap N (depthwiseStride2FlatXla q.dW q.db) (StableHLO.cbrB N q.eW q.eb q.eε q.eγ q.eβ v)) k ≠ 0 ∧ StableHLO.bnBatchLA N mid h w q.dε q.dγ q.dβ (StableHLO.batchMap N (depthwiseStride2FlatXla q.dW q.db) (StableHLO.cbrB N q.eW q.eb q.eε q.eγ q.eβ v)) k ≠ 6
Instances For
The single relu6 site of the t = 1 bottleneck (depthwise-BN output) is away from both
kinks at batch BN.
- hd (k : Fin (N * (ic * h * w))) : StableHLO.bnBatchLA N ic h w q.dε q.dγ q.dβ (StableHLO.batchMap N (depthwiseFlat q.dW q.db) v) k ≠ 0 ∧ StableHLO.bnBatchLA N ic h w q.dε q.dγ q.dβ (StableHLO.batchMap N (depthwiseFlat q.dW q.db) v) k ≠ 6
Instances For
The stem's relu6 is away from both kinks at the input x, at batch BN.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The head's relu6 is away from both kinks at the trunk output v, at batch BN.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stem VJP: bnRelu6StageHasVJPAt at the XLA-SAME strided conv. That lemma takes the inner
op as a parameter, so the stride-2 stem is the same construction as every stride-1 stage.
Equations
- Proofs.mnv2StemBHasVJPAt N h w Ws bs εs hεs γs βs x hs = Proofs.StableHLO.bnRelu6StageHasVJPAt N (Proofs.flatConvStride2Xla Ws bs) ⋯ (Proofs.flatConvStride2XlaHasVJP Ws bs) εs hεs γs βs x hs
Instances For
t = 1 bottleneck VJP (b1): projB ∘ dwbrB, the VJP of dwbrLayer ; projLayer. The one
block shape with no mnv2*BodyB lemma to delegate to.
Equations
Instances For
Stride-1 no-skip bottleneck VJP (b11, b17) — mnv2BodyBHasVJPAt at the bundle's fields.
Equations
Instances For
Stride-1 skip bottleneck VJP — the body VJP under residualHasVJPAt. The identity arm is
smooth everywhere, so the skip adds no hypothesis.
Equations
- Proofs.mnv2ResidBHasVJPAt N h w p hq v hs = Proofs.residualHasVJPAt (Proofs.mnv2ExpOnlyB N h w p) v ⋯ (Proofs.mnv2ExpOnlyBHasVJPAt N h w p hq v hs)
Instances For
Stride-2 downsampling bottleneck VJP — mnv2DownBodyBHasVJPAt at the bundle's fields.
Equations
Instances For
Head VJP: the 1x1 conv-bn-relu6 stage (cbrB, pointwise), then GAP and dense — both smooth,
both batchMap of a per-example op, so both lift with the GLOBAL batchMapHasVJP.
Unlike r34's, this head is not hypothesis-free: MobileNetV2 puts a relu6 in front of the
pool, so the head carries the net's 35th kink site.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Equations
- Proofs.mnv2PreB1 N w = Proofs.mnv2NoExpB N 112 112 w.b1 ∘ Proofs.mnv2PreB0 N w
Instances For
Equations
- Proofs.mnv2PreB2 N w = Proofs.mnv2StridedB N 56 56 w.b2 ∘ Proofs.mnv2PreB1 N w
Instances For
Equations
- Proofs.mnv2PreB3 N w = Proofs.mnv2ResidB N 56 56 w.b3 ∘ Proofs.mnv2PreB2 N w
Instances For
Equations
- Proofs.mnv2PreB4 N w = Proofs.mnv2StridedB N 28 28 w.b4 ∘ Proofs.mnv2PreB3 N w
Instances For
Equations
- Proofs.mnv2PreB5 N w = Proofs.mnv2ResidB N 28 28 w.b5 ∘ Proofs.mnv2PreB4 N w
Instances For
Equations
- Proofs.mnv2PreB6 N w = Proofs.mnv2ResidB N 28 28 w.b6 ∘ Proofs.mnv2PreB5 N w
Instances For
Equations
- Proofs.mnv2PreB7 N w = Proofs.mnv2StridedB N 14 14 w.b7 ∘ Proofs.mnv2PreB6 N w
Instances For
Equations
- Proofs.mnv2PreB8 N w = Proofs.mnv2ResidB N 14 14 w.b8 ∘ Proofs.mnv2PreB7 N w
Instances For
Equations
- Proofs.mnv2PreB9 N w = Proofs.mnv2ResidB N 14 14 w.b9 ∘ Proofs.mnv2PreB8 N w
Instances For
Equations
- Proofs.mnv2PreB10 N w = Proofs.mnv2ResidB N 14 14 w.b10 ∘ Proofs.mnv2PreB9 N w
Instances For
Equations
- Proofs.mnv2PreB11 N w = Proofs.mnv2ExpOnlyB N 14 14 w.b11 ∘ Proofs.mnv2PreB10 N w
Instances For
Equations
- Proofs.mnv2PreB12 N w = Proofs.mnv2ResidB N 14 14 w.b12 ∘ Proofs.mnv2PreB11 N w
Instances For
Equations
- Proofs.mnv2PreB13 N w = Proofs.mnv2ResidB N 14 14 w.b13 ∘ Proofs.mnv2PreB12 N w
Instances For
Equations
- Proofs.mnv2PreB14 N w = Proofs.mnv2StridedB N 7 7 w.b14 ∘ Proofs.mnv2PreB13 N w
Instances For
Equations
- Proofs.mnv2PreB15 N w = Proofs.mnv2ResidB N 7 7 w.b15 ∘ Proofs.mnv2PreB14 N w
Instances For
Equations
- Proofs.mnv2PreB16 N w = Proofs.mnv2ResidB N 7 7 w.b16 ∘ Proofs.mnv2PreB15 N w
Instances For
Equations
- Proofs.mnv2PreB17 N w = Proofs.mnv2ExpOnlyB N 7 7 w.b17 ∘ Proofs.mnv2PreB16 N w
Instances For
Every BatchNorm ε of the net is positive: the stem's, the head's, and each block's bundle.
- b1 : IVNoExpPos w.b1
Instances For
Every relu6 is away from its kinks, each at the activation its block actually sees: the
stem's clauses at the image, block k's at mnv2PreB(k-1), the head's at the trunk's output.
- b1 : IVNoExpSmoothAtB N 112 112 w.b1 (mnv2PreB0 N w x)
- b2 : IVStridedSmoothAtB N 56 56 w.b2 (mnv2PreB1 N w x)
- b3 : IVSmoothAtB N 56 56 w.b3 (mnv2PreB2 N w x)
- b4 : IVStridedSmoothAtB N 28 28 w.b4 (mnv2PreB3 N w x)
- b5 : IVSmoothAtB N 28 28 w.b5 (mnv2PreB4 N w x)
- b6 : IVSmoothAtB N 28 28 w.b6 (mnv2PreB5 N w x)
- b7 : IVStridedSmoothAtB N 14 14 w.b7 (mnv2PreB6 N w x)
- b8 : IVSmoothAtB N 14 14 w.b8 (mnv2PreB7 N w x)
- b9 : IVSmoothAtB N 14 14 w.b9 (mnv2PreB8 N w x)
- b10 : IVSmoothAtB N 14 14 w.b10 (mnv2PreB9 N w x)
- b11 : IVSmoothAtB N 14 14 w.b11 (mnv2PreB10 N w x)
- b12 : IVSmoothAtB N 14 14 w.b12 (mnv2PreB11 N w x)
- b13 : IVSmoothAtB N 14 14 w.b13 (mnv2PreB12 N w x)
- b14 : IVStridedSmoothAtB N 7 7 w.b14 (mnv2PreB13 N w x)
- b15 : IVSmoothAtB N 7 7 w.b15 (mnv2PreB14 N w x)
- b16 : IVSmoothAtB N 7 7 w.b16 (mnv2PreB15 N w x)
- b17 : IVSmoothAtB N 7 7 w.b17 (mnv2PreB16 N w x)
- head : MNV2HeadSmoothAtB N 7 7 w.hW w.hb w.hε w.hγ w.hβ (mnv2PreB17 N w x)
Instances For
MobileNetV2 at batch BatchNorm has a certified input-VJP at a smooth point — all
seventeen bottlenecks. Chains stem → the [t,c,n,s] ladder → head with vjpCompDiffAt
under two hypotheses: MNV2PosB (every ε > 0) and MNV2SmoothAtB (every relu6 clause, each
at its block's own input).
Pointwise, because relu6 is kinked on both sides. Each expand-bearing block
contributes two clauses — the expand relu6 and the depthwise relu6, both INSIDE the body —
where ResNet-34's basic block contributes a mid-relu and a post-residual OUTER relu. The
linear bottleneck has no activation after project, so MobileNetV2's residual add is the
block output and adds nothing.
Unlike r34's, the head is not hypothesis-free: its 1x1 conv-BN is followed by a relu6.
N is a variable.
Equations
- Proofs.mobilenetv2ForwardBFullHasVJPAt N w hq x hx = (Proofs.mnv2ChainB✝ N w hq x hx).fst
Instances For
The committed nested-application forward is the layered chain the VJP is stated on —
what lets the VJP be about mobilenetv2ForwardBFull rather than about a re-spelling of it.
The .correct field of mobilenetv2ForwardBFullHasVJPAt, restated at the nested forward
mobilenetv2ForwardBFull (the one mobilenetv2FwdGraphBFull_faithful proves the typed graph
denotes) through mobilenetv2ForwardBFull_eq_chain: for weights with MNV2PosB and an input
with MNV2SmoothAtB, its backward is the pdiv-contracted Jacobian. The tie of the
hand-written input-gradient chain to this backward is
mnv2InputGradB_eq_mobilenetv2B_full_vjp.
The committed forward is differentiable at every smooth point — the chain's .snd, read
back through mobilenetv2ForwardBFull_eq_chain. What the seal's seal_differentiableAt needs.