MobileNetV2's T3 §1a TIE at TRUE BATCH-NORM — the un-fused, batched whole-net thread #
MobileNetV2FoldPaperG.lean (4b.4) makes every parameter GRADIENT node of the batched
MobileNetV2 train step den-faithful for an arbitrary cotangent. This file removes the
"arbitrary": each cotangent is pinned to the one the emitted backward chain delivers, so the whole
train step is den-composed forward → loss → backward with no free activation and no symbolic
cotangent. With 4.2b it completes MobileNetV2's T3, and it is ResNet34StepTieB.lean's peer.
⭐⭐ The block cotangents are NOT derived here. 4.2b's mnv2{ExpOnly,Resid,Strided,NoExp}B_has_vjp_at
ARE the certified block backwards, and mnv2{Body,DownBody,ResidBlock}BackBatchedGraph_faithful
(MobileNetV2BackB0.lean) already prove the emitted backward subgraphs denote exactly them. The
four *CotIn_eq_vjp lemmas below are those statements in this file's vocabulary, and they are what
make the cross-block thread a composition of certified VJPs rather than a re-derivation.
⭐ One parameter-tie bundle covers twelve of the seventeen blocks. A skip block and a stride-1
widening have the SAME parameter cotangents — the identity skip changes only the dx handed to the
previous block, which is why MobileNetV2RenderB's irBackStride1GradB is one function with a
skip flag rather than two near-copies. So mnv2Stride1TiedB is stated once and instantiated at
b3, b5, b6, b8–b13, b15, b16 (skip) and b11, b17 (no skip).
⭐ The loss cotangent is the LABEL-SMOOTHED one, at a general target, shared with ResNet-34:
Foundation/SmoothedLossCot.lean. MobileNetV2RenderB composes it from the same six kit ops
(softmaxRow → subB → scaleB → addVB → shiftB → divConstB) with α at 0.1 and the target arriving
as the graph input %onehot — a soft vector under mixup or cutmix.
⭐ N is a binder. The artifacts at 32 (mobilenetv2_adam_train_step) or 64
(mobilenetv2in_rmsdp64) are instances. T3 carries no numerals.
⛔ The all-reduce, since 4d piece 2 (2026-09-07). In mobilenetv2in_rmsdp64 each *GradB
node feeds allReduceMeanF — the collective as an AST node whose den is the replica MEAN of the
per-replica gradient nodes; until then emitGradAllReduce, emitted text and a declared carve-out
outside the SHlo AST. Every statement below is at the PER-REPLICA gradient node;
DataParallelNode.lean composes it with the mean and the tail.
⚠ bnInB and bnInB_eq_bnBackB are ResNet-34's, imported rather than copied. They are the
batched BatchNorm input-cotangent written as the den of the emitted backward op, and its identity
with the certified bnBatchLA VJP — both net-agnostic, and they happen to live in the file that
first needed them. What MobileNetV2 adds is the TWO-SIDED relu6 mask (relu6MaskB, where r34
threads the one-sided reluMaskB), the depthwise input-VJP and its XLA-SAME strided peer.
The parameter census is 158, and this file states 210 slots #
MobileNetV2RenderB defaults to convBias := false: every conv, depthwise and project bias is
folded into the BatchNorm that follows it and bound to zeroBiasPrelude's shared zero constant. So
mobilenetv2_adam_train_step.mlir carries 158 updated parameter tensors — stem 3 + b1 6 +
16 blocks × 9 + head 3 + dense 2 — and 210 is the census at convBias := true. The bias conjuncts
below are kept (one delegation each, and they cover the flag) and are about ops the committed
artifacts do not emit. Nothing here weakens a theorem: every fold is ∀-quantified over op
instances, and bias = 0 is one of them.
Conventions, stated because nothing here checks them #
| BatchNorm | batch (bnBatchLA, reduce [0,2,3], width N·h·w), 52 sites |
| padding | XLA-SAME at all five stride-2 sites (the stem conv, the four strided depthwises) |
| activation | relu6, TWO kinks per site (≠ 0 and ≠ 6), 35 sites, none after a project |
| optimizer form | the RAW gradient (*GradB); this net's renders emit no fused θ − lr·g at all |
| loss | label-smoothed softmax-CE at a general target, batch-meaned |
The relu6 backward mask — den (.selectMidB _ pre e) = fun i => if 0 < pre i ∧ pre i < 6 then e i else 0. TWO-sided, where ResNet-34's reluMaskB tests pre i > 0 only. MobileNetV2
applies it at 35 sites: two per expand-bearing block, one in b1, one at the stem, one in the
head.
Instances For
Batched XLA-SAME STRIDED depthwise input-VJP (= den depthwiseStridedXlaBackBatched;
upsamples h → 2h). ⚠ NOT EnetTiePoC.dStridedInB, which is the SYMMETRIC
depthwiseStride2Flat — B0's strided depthwise and MobileNetV2's have identical types and
different certificates, and this is the one place that distinction is recorded on the backward
side.
Equations
- One or more equations did not get rendered due to their size.
Instances For
MobileNetV2RenderB.irBackNoExpGradB, node for node, from the block-output cotangent dyOut:
%dpc = bnBatchBack(pg, pc) %ddr = convBackBatched(pW) %ddm = selectMidB(dn)
%ddn = bnBatchBack(dg, dc) %dxb = depthwiseBackBatched(dW)
with dW,db ← %ddn, dg,dbt ← %ddm, pW,pb ← %dpc, pg,pbt ← %dy.
Cotangent at the project conv's output. Feeds pW/pb.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the depthwise BN's output — the project conv's input-VJP masked by the depthwise
relu6. Feeds dg/dbt.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the depthwise conv's output. Feeds dW/db.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The block-INPUT cotangent: the depthwise backward, directly — no expand conv and no skip.
Equations
- Proofs.MobileNetV2TieB.mnv2NoExpCotIn N h w p xin dyOut = Proofs.EnetTiePoC.dInB N p.dW p.db (Proofs.MobileNetV2TieB.mnv2NoExpCotDc N h w p xin dyOut)
Instances For
The t = 1 block's backward graph: the two stage graphs chained at their forward activations,
dwbrLayer ; projLayer's graph. ⚠ It lives here rather than in MobileNetV2BackB0.lean because
mnv2NoExpB is a wrapper of MobileNetV2FullB.lean's, one tier above that file's vocabulary.
Equations
- One or more equations did not get rendered due to their size.
Instances For
⭐⭐ The emitted t = 1 chain IS the certified block VJP's backward.
MobileNetV2RenderB.irBackStride1GradB, node for node, from the block-output cotangent dyOut:
%dpc = bnBatchBack(pg, pc) %ddr = convBackBatched(pW) %ddm = selectMidB(dn)
%ddn = bnBatchBack(dg, dc) %der = depthwiseBackBatched(dW) %dem = selectMidB(en)
%den = bnBatchBack(eg, ec) %dxb = convBackBatched(eW)
%dx = if skip then addVB(%dxb, %dy) else %dxb
with eW,eb ← %den, eg,ebt ← %dem, dW,db ← %ddn, dg,dbt ← %ddm, pW,pb ← %dpc,
pg,pbt ← %dy. ⭐ The skip flag touches ONLY %dx, so all twelve parameter cotangents below are
shared between the skip blocks and the two stage-first widenings (b11, b17).
The expand stage's output — the depthwise's input.
Equations
- Proofs.MobileNetV2TieB.mnv2XE N h w p xin = Proofs.StableHLO.cbrB N p.eW p.eb p.eε p.eγ p.eβ xin
Instances For
Cotangent at the depthwise BN's output — the project conv's input-VJP masked by the depthwise
relu6. Feeds dg/dbt.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the depthwise conv's output. Feeds dW/db.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the expand BN's output — the depthwise's input-VJP masked by the expand relu6.
Feeds eg/ebt.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The BODY's input cotangent — the expand conv's backward. This is the whole block-input
cotangent for a widening (b11, b17); a skip block adds dyOut to it.
Equations
- Proofs.MobileNetV2TieB.mnv2CotInBody N h w p xin dyOut = Proofs.EnetTiePoC.cInB N p.eW p.eb (Proofs.MobileNetV2TieB.mnv2CotEc N h w p xin dyOut)
Instances For
⭐⭐ The emitted stride-1 chain IS the certified body VJP's backward — the widening blocks'
_eq_vjp, straight from mnv2BodyBackBatchedGraph_faithful.
The SKIP block's input cotangent: the body branch plus the identity skip, the addVB
fan-in the render emits.
Equations
- Proofs.MobileNetV2TieB.mnv2ResidCotIn N h w p xin dyOut i = Proofs.MobileNetV2TieB.mnv2CotInBody N h w p xin dyOut i + dyOut i
Instances For
⭐⭐ The emitted residual fan-in IS the certified skip-block VJP's backward, from
mnv2ResidBlockBackBatchedGraph_faithful. ⭐ No add_comm is needed here, unlike ResNet-34's
downsample block: the render emits addVB(body, %dy) and residualBackGraph builds the fan-in
in the same order.
MobileNetV2RenderB.irBackStridedGradB: the stride-1 chain with the depthwise replaced by its
XLA-SAME strided peer, so the expand half runs at the 2h × 2w input grid and %der upsamples.
There is no skip, so %dx is %dxb directly.
The expand stage's output at the pre-downsample grid — the strided depthwise's input.
Equations
- Proofs.MobileNetV2TieB.mnv2XES N h w p xin = Proofs.StableHLO.cbrB N p.eW p.eb p.eε p.eγ p.eβ xin
Instances For
Cotangent at the project conv's output. Feeds pW/pb.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the strided depthwise BN's output. Feeds dg/dbt.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the strided depthwise conv's output. Feeds dW/db.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the expand BN's output, at the 2h × 2w grid — the STRIDED depthwise's input-VJP
(which upsamples) masked by the expand relu6. Feeds eg/ebt.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the expand conv's output. Feeds eW/eb.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The block-INPUT cotangent: the expand conv's backward. No skip — the downsample changes both spatial and channels.
Equations
- Proofs.MobileNetV2TieB.mnv2StridedCotIn N h w p xin dyOut = Proofs.EnetTiePoC.cInB N p.eW p.eb (Proofs.MobileNetV2TieB.mnv2SCotEc N h w p xin dyOut)
Instances For
⭐⭐ The emitted stride-2 chain IS the certified downsample-body VJP's backward.
MobileNetV2RenderB, after the seventeen block backwards:
%dsm = selectMidB(stn) %dsn = bnBatchBack(sg, stc)
with sW,sb ← %dsn and sg,sbt ← %dsm. ⭐ There is NO conv-back past %x, and no pool: the stem
is one XLA-SAME strided conv, its BatchNorm and one relu6.
Cotangent at the stem BN's output — the stem relu6's mask. Feeds sg/sbt.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the stem conv's output. Feeds sW/sb.
Equations
- One or more equations did not get rendered due to their size.
Instances For
MobileNetV2RenderB, from the loss cotangent g:
%dgi = denseRowBack(Wd) %dgp = gapBackBatched %dhm = selectMidB(hn)
%dhn = bnBatchBack(hg, hc) %dhx = convBackBatched(hW)
with Wd,bd ← %dy, hW,hb ← %dhn, hg,hbt ← %dhm, and %dhx the cotangent handed to b17.
⚠ Unlike ResNet-34's, this head is NOT batchMap of a smooth per-example map — MobileNetV2 puts a
1x1 conv-BN-relu6 in front of the pool — so the chain is spelled here as dens, exactly as
EfficientNetStepTie.enetHeadTied spells B0's identically-shaped head.
Cotangent at the GAP output — the classifier's input-VJP.
Equations
- Proofs.MobileNetV2TieB.mnv2HeadCotGapIn N Wd g = Proofs.StableHLO.rowDenseBackFlat N oc nCls Wd g
Instances For
Cotangent at the head relu6's output — the GAP backward.
Equations
- Proofs.MobileNetV2TieB.mnv2HeadCotHr N h w Wd g = Proofs.EnetTiePoC.gapInB N oc h w (Proofs.MobileNetV2TieB.mnv2HeadCotGapIn N Wd g)
Instances For
Cotangent at the head BN's output — the head relu6's mask. Feeds hg/hbt.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the head conv's output. Feeds hW/hb.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The cotangent the head hands to b17 — the head conv's backward.
Equations
- Proofs.MobileNetV2TieB.mnv2HeadCotBlk N h w Wh bh εh γh βh Wd xin g = Proofs.EnetTiePoC.cInB N Wh bh (Proofs.MobileNetV2TieB.mnv2HeadCotHc N h w Wh bh εh γh βh Wd xin g)
Instances For
Each conjunct is MobileNetV2FoldPaperG's ∀ cot fold instantiated at the cotangent the
render's chain delivers, so nothing here is a new proof: the bundles are the §1 fold with the
freedom removed. reassocB bridges the conv/relu6 index N·(c·h·w) to the BatchNorm parameter
ops' N·(c·(h·w)).
⚠ The BIAS conjuncts are about conv{,StridedXla}BiasGradB and depthwise{,StridedXla}BiasGradB,
which the committed artifacts do NOT emit — MobileNetV2RenderB runs convBias := false and binds
every bias operand to zeroBiasPrelude's zero constant. They are kept because they cost one
delegation each and they cover the flag.
Stem, tied. The 3x3/s2 XLA-SAME conv's weight and bias and its BatchNorm's γ/β, at the
cotangent that reaches the stem through block 1's input fan-in. ⚠ convStridedXla*, not r34's
symmetric convStrided*: identical types, identical emitted shapes, different certificates.
Equations
- One or more equations did not get rendered due to their size.
Instances For
t = 1 block (b1), tied. All eight parameter nodes — the stride-1 depthwise's weight and
bias and its BatchNorm's γ/β, then the project 1x1's weight and bias and its BatchNorm's γ/β.
⭐ The project BatchNorm's γ/β read dyOut itself: the linear bottleneck has no activation
after project, so the block-output cotangent IS that BatchNorm's output cotangent.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stride-1 inverted-residual block, tied — all twelve parameter nodes. ⭐ ONE statement for
twelve of the seventeen blocks: the ten identity-skip ones and the two stage-first widenings
(b11, b17). A skip changes only the dx handed to the previous block, never a parameter
cotangent, which is why MobileNetV2RenderB.irBackStride1GradB is one function with a flag.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stride-2 downsampling block, tied — all twelve parameter nodes (b2, b4, b7, b14).
Identical to the stride-1 profile except that the expand half runs at the 2h x 2w input grid
and the depthwise is the XLA-SAME strided one. ⚠ depthwiseStridedXla*GradB, NOT B0's
symmetric depthwiseStrided*GradB: the two have identical types and identical emitted shapes,
and only the certificate says which correlation the weight gradient runs.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Head, tied. The 1x1 conv's weight and bias, its BatchNorm's γ/β, and the classifier's
weight and bias, at the loss cotangent g and the chain it drives. ⚠ The dense-bias conjunct's
Jacobian witness carries a zero activation: dense's derivative in b is the identity whatever
x is, so the statement is x-free (the shape EfficientNetStepTie's bias conjuncts take).
Equations
- One or more equations did not get rendered due to their size.
Instances For
⭐⭐ The whole batch-BN MobileNetV2 train step, tied. Threading
mobilenetv2ForwardB_full's own prefixes as the block inputs and the label-smoothed loss
cotangent down through the head chain and the seventeen certified block backwards, every
parameter GRADIENT node of the net — stem 4, b1 8, sixteen blocks x 12, head 4, dense 2 —
denotes the certified batched Σ_n gradient. No free activation and no symbolic cotangent.
With 4.2b this is MobileNetV2's T3 complete at batch BatchNorm.
⭐ N is a binder and there is no smoothness hypothesis. The folds are ∀ cot statements
instantiated at explicitly-constructed cotangents, so the capstone needs neither 0 < ε nor a
relu6-kink condition. Those enter only in the four *CotIn_eq_vjp lemmas, which say the
constructed chain IS the certified whole-net backward — the two halves of the tie, kept apart
because they have different hypotheses.
⚠ Of the 210 conjunct slots, the committed artifacts exercise 158: MobileNetV2RenderB
runs convBias := false, so the 52 bias nodes are not emitted (each bias is folded into the
BatchNorm after it and bound to zeroBiasPrelude's zero constant).
⛔ One replica. In mobilenetv2in_rmsdp64 every gradient node feeds allReduceMeanF, an AST
node since 4d piece 2; MobileNetV2SyncTieB.mnv2_net_syncTiedB is the data-parallel step, and
its right-hand sides are this theorem's nodes at N := R·N.
⭐ And the cotangent the capstone threads is the smoothed loss's gradient. Row by row: the
g above is, at example n and class j, (1/B)·∂/∂logits of soft-target cross-entropy
against the SMOOTHED target (1−α)·t + α/K, at that example's real logits. The only hypothesis
is that the example's target sums to 1 — a one-hot, or mixup's convex combination of two.
Together with the capstone this closes the top of the chain: every parameter node denotes the
certified gradient at the cotangent of the loss the trainer actually minimises. Shared with
ResNet-34 through Foundation/SmoothedLossCot.lean, at a general target.