The train-step tie for MobileNetV4-Conv-M — every gradient node at its chain cotangent #
Every parameter gradient node MNv4's batched train step emits denotes the certified gradient
for an arbitrary cotangent, by a leaf lemma it shares with ResNet-34 or EfficientNet-B0 (the
table below). This file removes that freedom: each node is stated at the cotangent the render's own
backward chain delivers, driven by a loss cotangent g at the logits.
No accuracy is quoted for this net. The statements are about timm's mobilenetv4_conv_medium;
the gates that pin the artifacts to it are scripts/parity/mnv4_timm_parity.py,
scripts/parity/mnv4_forward_tie.py and scripts/parity/grad_tie.py --net mnv4.
The UIB bottleneck is linear, and that makes MNv4's chain shorter than ResNet's #
uibFwdSkipB emits addVB (project-BN out) (block input) with no activation after the add and
none after the project's BatchNorm. So the block-output cotangent dyOut reaches the project BN's
γ and β directly — where ResNet-50's r50IdCotA must first pass through the post-residual relu's
mask, and where its skip branch carries the masked cotangent. Here the skip fan-in is
addVB (body dx) dyOut, unmasked.
And every block's *CotIn_eq_vjp is its block layer's .faithful, not a new derivation. The
UIB bodies, the fused stage and the head are CertLayers, so den (graph x e) = vjp.backward (den e) is already a theorem one tier down — the fact the r34, mnv2 and R50 ties re-derive per
block. mnv4BodyCotIn_eq_vjp, mnv4SBodyCotIn_eq_vjp, mnv4SkipCotIn_eq_vjp,
mnv4FusedCotIn_eq_vjp and mnv4HeadCotIn_eq_vjp state it for each constructed cotangent the
capstone threads: each is the layer's .faithful plus a definitional match. The stem has none:
no render emits a gradient into %x.
No new fp32 op-kind lemmas — MNv4's nine kinds are ResNet-34's and EfficientNet-B0's #
| op kind | sites | certificate |
|---|---|---|
bnGammaGradB / bnBetaGradB | 77 BN layers | GradNodeB.bnGammaGradB_den / bnBetaGradB_den |
convWeightGradB | expands, projects, both head convs, the fused project | GradNodeB.convWGradB_den |
convStridedWeightGradB (SYMMETRIC) | the stem's and the fused stage's 3×3/s2 | GradNodeB.convStridedWGradB_den |
depthwiseWeightGradB | every stride-1 depthwise | GradNodeB.depthwiseWGradB_den |
depthwiseStridedWeightGradB | rows 1, 3, 11's post-DW (timm's dw_mid) | GradNodeB.depthwiseStridedWGradB_den |
denseWeightGradB / denseBiasGradB | the classifier | GradNodeB.denseWGradB_den / denseBGradB_den |
Every strided site pads symmetrically (timm), so the stem and the fused stage share one certificate.
MNv4 emits no conv bias gradient at all. MobileNetV4RenderB has no convBias flag — every
bias is folded into its BatchNorm and bound to %zb{c} — so convBiasGradB and its strided peers
are never emitted and there is nothing to state. Same situation as ResNet-50.
Precision is a flag on the statement. The bf16 ImageNet artifacts
(mnv4in_emaacc{,dp}8x128wxdowd005bf16, mnv4in_acc{,dp}8x128wxdropdowd01bf16) emit the bf16
conv and depthwise weight-gradient kinds (convWeightGradBBf16, depthwiseWeightGradBBf16, …;
Foundation.Bf16GradNodes). Every conv and depthwise weight node below is stated on the renderers'
switch — ConvWTiedBAt bf16 / ConvStridedWTiedBAt bf16 / DepthwiseWTiedBAt bf16 /
DepthwiseStridedWTiedBAt bf16 (Foundation.GradNodesBAt): bf16 := false is the f32 node,
bf16 := true the bf16 one — and the capstone mnv4_net_tiedB takes bf16, so it reaches the
bf16 artifacts' gradient nodes read over ℝ exactly as it reads the f32 ones (Bf16Erasure: at the
identity rounding the bf16 kind denotes what its f32 peer does). BatchNorm's and the classifier's
nodes carry no flag because no render switches them.
Everything here is generic in the row #
Every definition and theorem below takes a UibSpec binder s and reads its widths off it, so
s.ic, s.h and s.ic * s.expand are variables. That is not a convenience: MNv4's resolutions
are literals, and letting den and width-indexed rfls run at 224/112/56/28/14/7 ends in kernel
timeouts (MobileNetV4FullB.lean). Stated at a row binder they stay stuck; the capstone then
instantiates at the 21 concrete rows.
The chain, node for node from uibBackSkipGradB #
| cotangent | at | feeds |
|---|---|---|
dyOut | the block output — and the project BN's output, the add being linear | %u{p}pg, %u{p}pbt |
CotPc | project conv's output (bnBatchBack) | %u{p}pW |
CotDn | post-DW BN's output (convBackBatched, then the post-DW relu's selectPos mask) | %u{p}dg, %u{p}dbt |
CotDc | post-DW conv's output | %u{p}dW |
CotEn | expand BN's output (depthwiseBackBatched, then the expand relu's mask) | %u{p}eg, %u{p}ebt |
CotEc | expand conv's output | %u{p}eW |
CotQn | pre-DW BN's output (convBackBatched; no mask — the pre-DW is BN only) | %u{p}qg, %u{p}qbt |
CotQc | pre-DW conv's output | %u{p}qW |
CotIn | the block input — addVB (depthwiseBackBatched dQc) dyOut | the previous block |
A cotangent one step off is a silently wrong gradient, not a type error: %u{p}eg reads the
cotangent at the expand BN's OUTPUT and %u{p}eW the one at the expand CONV's output, and both
have the same type.
Data parallel. In mnv4in_adamdp64* every node named here feeds allReduceMeanF and
BatchNorm is synchronised across replicas, so a replica's gradient node is not this file's node at
its own N. This file describes a data-parallel artifact only at N := R·N: the per-replica
statement is MobileNetV4SyncTieB.mnv4_net_syncTiedB, whose right-hand side is mnv4_net_tiedB's
node at the global batch.
Cotangent at the project CONV's output — dyOut through the project BN's backward.
dyOut itself is the cotangent at the project BN's output: the bottleneck is linear, so
nothing masks it. Feeds %u{p}pW.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the post-DW BN's output — the project conv's input-VJP, masked by the post-DW
relu. Feeds %u{p}dg and %u{p}dbt.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the post-DW CONV's output — through the post-DW BN's backward. Feeds %u{p}dW.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the expand BN's output — masked by the expand relu. Feeds %u{p}eg/%u{p}ebt.
This is where the chain dispatches on the table, exactly as mnv4PostDWSlot does and
off the same row: with a post-depthwise the incoming cotangent is that depthwise's input-VJP;
without one (postDWk = 0, the ConvNeXt-like and FFN rows) the project conv's input-VJP
arrives here directly, because the render emits no post-DW nodes at all. One chain, three
stride-1 profiles, and the if reduces at every concrete row.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the expand CONV's output — through the expand BN's backward. Feeds %u{p}eW.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the pre-DW BN's output — the expand conv's input-VJP, UNMASKED: the pre-DW is
BN only (timm's dw_start). Feeds %u{p}qg and %u{p}qbt.
Equations
- Proofs.Mnv4TieB.mnv4CotQn N s p xin dyOut = Proofs.BackLinks.cInB N p.We p.be (Proofs.Mnv4TieB.mnv4CotEc N s p xin dyOut)
Instances For
Cotangent at the pre-DW CONV's output — through the pre-DW BN's backward. Feeds %u{p}qW.
Equations
- One or more equations did not get rendered due to their size.
Instances For
A near-copy of the stride-1 chain, and it has to be. The post-DW carries the stride
(timm's dw_mid), so the block input, the BN-only pre-DW and the expand sit at 2h and the post-DW
and the project at h — a change that runs through every type in the chain.
And there is no skip: all three stride-2 rows change channels (ic ≠ oc), so the block IS
the body and there is no addVB fan-in.
Cotangent at the project CONV's output. dyOut itself is the cotangent at the project BN's
output: the bottleneck is linear. Feeds %u{p}pW.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the STRIDED post-DW BN's output — the project conv's input-VJP, masked by the
post-DW relu. Feeds %u{p}dg and %u{p}dbt.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the STRIDED post-DW CONV's output — through its BN's backward. Feeds %u{p}dW.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the expand BN's output, at 2h — the strided post-DW's input-VJP, masked by the
expand relu. Feeds %u{p}eg/%u{p}ebt.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the expand CONV's output, at 2h. Feeds %u{p}eW.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the pre-DW BN's output, at 2h — the expand conv's input-VJP, unmasked (the
pre-DW is BN only). Feeds %u{p}qg and %u{p}qbt.
Equations
- Proofs.Mnv4TieB.mnv4SCotQn N s p xin dyOut = Proofs.BackLinks.cInB N p.We p.be (Proofs.Mnv4TieB.mnv4SCotEc N s p xin dyOut)
Instances For
Cotangent at the pre-DW CONV's output, at 2h. Feeds %u{p}qW.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The strided block's input cotangent, at 2h: the pre-DW's input-VJP when there is one, the
expand conv's otherwise. No fan-in: ic ≠ oc, so the block has no skip.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The BODY's input cotangent — what the render's dx carries before the skip fan-in.
Dispatches on s.preDWk the way mnv4PreDWSlot does: with a pre-depthwise the body's dx
is that depthwise's input-VJP, without one it is the expand conv's.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The skip fan-in itself: body dx + dyOut, at the block-input shape.
Split from mnv4BodyCotIn for the same reason mnv4SkipGraphB is split from the body graph
builders: the add needs s.oc and s.ic to be the SAME type, which they are at every stride-1
row and are not at a row binder. The body's cotangent is row-generic; the add is applied at the
concrete row, where s.oc = s.ic is rfl.
Equations
- Proofs.Mnv4TieB.mnv4SkipCotIn bodyDx dyOut i = bodyDx i + dyOut i
Instances For
ExtraDW block, tied. All twelve parameter nodes — four conv/depthwise weights and four
BatchNorm γ/β pairs — denote the certified batched Σ_n gradient at the real forward
activations and the real backward-chain cotangent driven by dyOut.
Each BatchNorm's γ/β reads the cotangent at THAT BatchNorm's output (CotQn, CotEn,
CotDn, and dyOut itself for the project) while its conv reads the one at the conv's output
(CotQc, CotEc, CotDc, CotPc). Off by one and the gradient is silently wrong — the two
have the same type. The project BN's pair reads dyOut unmasked: the bottleneck is linear.
There are no conv-bias conjuncts: MobileNetV4RenderB has no convBias flag, so those ops
are never emitted and every slot here is exercised by the artifact.
Equations
- One or more equations did not get rendered due to their size.
Instances For
And it holds — twelve instantiations of the shared ∀ cot leaf folds with
the freedom removed: the cotangents are the chain's, not free.
Equations
- One or more equations did not get rendered due to their size.
Instances For
And it holds — nine instantiations of the shared ∀ cot leaf folds at the chain's cotangents.
And it holds — six instantiations of the shared ∀ cot leaf folds at the chain's cotangents.
Strided block, tied — rows 1, 3 and 11. All twelve parameter nodes denote the certified
batched Σ_n gradient at the real forward activations and the backward-chain cotangent driven
by dyOut.
The strided node is the post-DW's depthwiseStridedWeightGradB (timm's dw_mid,
symmetric), reading its input at 2h; the pre-DW and the expand nodes read theirs at 2h, the
project's at h.
No skip and no addVB: ic ≠ oc at all three rows, so the block IS its body.
Equations
- One or more equations did not get rendered due to their size.
Instances For
And it holds — twelve instantiations (one at the strided post-DW) of the shared
∀ cot leaf folds with the freedom removed.
All three are generic in their widths, for the reason MobileNetV4FullB.lean's stem-graph
docstring records: pinning MNv4's literal resolutions here lets den and the
width-indexed rfls actually run, and the kernel gives up. The capstone instantiates.
Cotangent at the stem BN's output — the fused stage's dx, masked by the stem relu.
Feeds %sg and %sbt.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the stem CONV's output — through the stem BN's backward. Feeds %sW.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stem, tied. Its three nodes at the chain's cotangents.
And the chain stops here. No render emits a gradient into %x, so the artifact's
backward ends at this weight gradient. That is why the
stem sits outside MobileNetV4FullB.lean's CertLayer trunk, and it is B0's situation
exactly.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the fused stage's project CONV output. dyF reaches the project BN's γ/β
unmasked — the fused stage ends in a BatchNorm with no activation. Feeds %f0pW.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the fused BN's output — the project conv's input-VJP, masked by the fused relu.
Feeds %f0cg and %f0cbt.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the fused CONV's output — through the fused BN's backward. Feeds %f0cW.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The fused stage's input cotangent — the SYMMETRIC strided conv's input-VJP, landing at 2h.
Equations
- Proofs.Mnv4TieB.mnv4FusedCotIn N h w Wc bc εc γc βc Wp bp εp γp βp xin dyF = Proofs.BackLinks.cStridedInB N Wc bc (Proofs.Mnv4TieB.mnv4FusedCotC N h w Wc bc εc γc βc Wp bp εp γp βp xin dyF)
Instances For
Fused stage, tied — its six parameter nodes. %f0cW is convStridedWeightGradB,
symmetric padding, like the stem's.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The head, timm's order: cn_960 conv-bn-relu at h×w, GAP, conv_head conv-bn-relu on the
pooled [N, mid, 1, 1], dense. The two 1×1 relabellings (mnv4_pool11) are no-ops in the render
and Fin.cast reads here.
The pooled head features at [N, mid, 1, 1]: GAP of the first head stage, relabelled.
Equations
- Proofs.Mnv4TieB.mnv4HeadPool N h w W1 b1 ε1 γ1 β1 xin = Proofs.Mnv4TieB.mnv4To11 (Proofs.StableHLO.batchMap N (Proofs.globalAvgPoolFlat mid h w) (Proofs.StableHLO.cbReluB N W1 b1 ε1 γ1 β1 xin))
Instances For
The dense input: conv_head-bn-relu of the pooled features, relabelled to [N, oc].
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at conv_head's BN output ([N, oc, 1, 1]) — the dense input-VJP (rowDenseBackFlat,
the den of the render's denseRowBack), relabelled,
masked by that stage's relu. Feeds %hg and %hbt.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at conv_head's CONV output. Feeds %hW.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the FIRST head BN's output — conv_head's input-VJP, relabelled, spread by the
GAP backward, masked by the first relu. Feeds %h1g and %h1bt.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the first head CONV's output. Feeds %h1W.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The head's input cotangent — what block 21 receives as its dyOut.
Equations
- Proofs.Mnv4TieB.mnv4HeadCotIn N h w W1 b1 ε1 γ1 β1 W2 b2 ε2 γ2 β2 Wd bd xin g = Proofs.BackLinks.cInB N W1 b1 (Proofs.Mnv4TieB.mnv4HeadCotH1c N h w W1 b1 ε1 γ1 β1 W2 b2 ε2 γ2 β2 Wd bd xin g)
Instances For
Head, tied — all EIGHT nodes: two conv weights, two BatchNorm γ/β pairs, and the
classifier's weight and bias. conv_head's nodes (%hW, %hg, %hbt) read the POOLED
features at h = w = 1; the classifier reads conv_head's relu output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The capstone below threads dy21 … dyStem through mnv4HeadCotIn, mnv4BodyCotIn +
mnv4SkipCotIn, mnv4SBodyCotIn and mnv4FusedCotIn. Each is stated here to be the backward
of the certified CertLayer VJP it stands for, at a block input where that layer is certified
(its .ok). Every proof is the layer's .faithful at e := .operand "" dyOut, then a
definitional match with the constructed chain.
A stride-1 UIB body's input cotangent is its certified VJP's backward. The chain the
render emits (mnv4BodyCotIn, through mnv4CotQc … mnv4CotPc) is
(mnv4BodyOfRow N s p).vjp's backward at the block input, for every row: the four
preDWk/postDWk cases (ExtraDW, ConvNeXt-like, FFN, and the IB shape no Conv-M row uses)
each reduce to the body layer's .faithful.
A strided UIB body's input cotangent is its certified VJP's backward — rows 1, 3, 11,
at 2h.
The skip fan-in is the residual layer's backward: body dx + dyOut, with body dx the
body's certified backward, is CertLayer.residual's. Generic in the body, so at a stride-1 row
it composes with mnv4BodyCotIn_eq_vjp (the add needs s.oc = s.ic, which is rfl at every
concrete row and not at a row binder).
The fused stage's input cotangent is its certified VJP's backward.
The head's input cotangent — block 21's dyOut — is its certified VJP's backward, at
any loss cotangent g.
The per-BLOCK forward prefixes: mnv4Blk0 is the fused stage's output — block 1's input —
and mnv4Blk{k} is the activation entering block k+1.
MobileNetV4FullB.lean's mnv4Pre0 … mnv4Pre6 are the resolution-group prefixes, which is
the granularity the whole-net forward, graph and VJP need; the tie needs one per BLOCK, so these 22 name the finer chain.
mnv4Blk0 is definitionally mnv4Pre1.
Equations
- Proofs.Mnv4TieB.mnv4Blk0 N w x = Proofs.StableHLO.mnv4Pre1 N w x
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
The whole batch-BN MobileNetV4-Conv-M train step, tied. Threading the net's own forward
prefixes as the block inputs and an arbitrary loss cotangent g down through the head, the 21
UIB blocks and the fused stage — each dy is that layer's certified VJP backward wherever the
layer is certified (mnv4HeadCotIn_eq_vjp, mnv4BodyCotIn_eq_vjp with
mnv4SkipCotIn_eq_vjp, mnv4SBodyCotIn_eq_vjp, mnv4FusedCotIn_eq_vjp) — every parameter
GRADIENT node of the net — stem 3, fused 6, thirteen ExtraDW blocks × 12, four
ConvNeXt-like × 9, four FFN × 6, head 8 — denotes the certified batched Σ_n gradient. That is
233, the render's own census and mnv4_fwd.mlir's signature minus %x. No free activation
and no symbolic cotangent below the loss.
g is a binder. The loss chain is not part of this statement; mnv4_lossCot_is_smoothedCE_grad
instantiates it at the label-smoothed softmax cotangent the artifacts actually emit.
No smoothness hypothesis and no 0 < ε, and N and nCls are both binders. The folds
are ∀ cot statements at explicitly constructed cotangents; the kink and positivity conditions
live in the CertLayers whose .ok MobileNetV4FullBVJP.lean binds. The statement is at
224×224; unlike ResNet-50 there is no resolution binder q. One device, at either precision:
bf16 selects the conv and depthwise weight nodes' kind — false the f32 artifacts', true
the bf16 ones', read over ℝ (Bf16Erasure); the right-hand side is the same certified
gradient at either value.
On the data-parallel (sync-BN) artifacts this theorem applies at N := R·N: the per-replica
statement is MobileNetV4SyncTieB.mnv4_net_syncTiedB, whose right-hand sides are this
theorem's nodes at the global batch. The AdamW update that follows is not stated here.
And the loss cotangent g is instantiated: MNv4's is the label-smoothed softmax chain.
Row by row, the six-op chain MobileNetV4RenderB emits (softmaxRow → subB → scaleB → addVB → shiftB → divConstB, α = 0.1) is (1/B)·∂/∂logits of soft-target cross-entropy against the
SMOOTHED target, at that example's real logits.
softmaxRow at m := 1 — the rowB/unrowB spelling ResNet-34 and ResNet-50 use, NOT
ConvNeXt's and ViT's expe → softmaxDiv at the plain N·K width. Read off the render's own
lines rather than assumed: those two take different lemmas (smoothedLossCotGraph here,
smoothedLossCotGraphDiv there) and nothing in the types tells them apart.
MNv4 ships one loss — there is no BCE twin to state, where ResNet-50 needed both. The only hypothesis is that the example's target sums to 1: a one-hot, or mixup's convex combination.