ConvNeXt-T's whole-net backward tie — the stage fold and the even-kernel repair #
The per-example whole-net backward tie for ConvNeXt-T: convnextInputGrad, with every slot
pinned to the certified per-op backward at its own saved activation, is
(convNextForwardTChHasVJP …).backward x. The batched form is
ConvNeXtWholeBackCertifiedTieB.lean.
Even kernels. conv2d pads by pH = (kH-1)/2, so the reversed-kernel forward conv
(convFlatBack) is the adjoint only when kH - 1 - pH = pH, i.e. only for odd kH. ConvNeXt's
4×4/s4 patchify stem and its three 2×2/s2 downsamples are even-kernel sites; at kH = 4 the
symmetric-pad backward is the adjoint of a conv shifted one pixel. StableHLO.Basic's
.convStridedBack / .convStridedBackBatched pad asymmetrically, [[kH-1-pH, pH]], and their
den is the certified VJP; BackwardMaps.lean's flatConvStride2Back / flatConvStride4Back
are convFlatBack ∘ scatter at the symmetric pad. The repair is padOdd
(EvenKernelConvBack.lean): an even-kernel conv is an odd-kernel conv on the kernel zero-extended
at (+1,+1), which is the emitter's asymmetric pad written in the vocabulary BackwardMaps.lean
already has, so the existing odd-kernel leaf tie applies.
What is here.
cnxDownChBack_eq_vjp— the stage-boundary downsample tie,lnB ∘ flatConvStride2Back (padOdd W)against(cnxDownChWHasVJP …).backward.padOddis required:p.Wis2×2.cnxStageChKBack_eq_vjp— the depth-kstage fold.HasVJPforconvNextStageChKis built head-first (block0runs first), so its backward composes the block backwards in the opposite order, each at its own saved activation, and the tail's saved input is block0's forward output. The induction step is one rewrite of the block tie (cnxBlockChBack_eq_vjp) and one of the inductive hypothesis.cnxSavedA0 … cnxSavedA10—convNextForwardTCh's eleven stage inputs, named as functions, so that the same constants are both the activations the backward's slots are saved at and thefargument of eachvjpComp.convnextInputGrad_eq_convNextForwardTCh_vjp— the apex.convNextForwardTChHasVJPis a globalHasVJP, because GELU, LayerNorm, convolution and the layer scale are all smooth. Its only hypotheses are the 23 LayerNorm positivities, so, like ViT's (vitInputGradK_eq_vitForwardKV_vjp) and EfficientNet-B0's (efficientnetInputGradBFull_eq_efficientnetForwardB_full_vjp), and unlike the ReLU nets' (ResNet-34/50, MobileNetV2, MobileNetV4), it carries no smoothness side-condition.convNextForwardTCh_eq_chain(proved byrw) says the chain the apex instantiates is the committed forward.
The downsample backward tie. cnxDownBack (padOdd p.W) lnB — the strided-conv backward at
the ZERO-EXTENDED kernel, then the channel-LN back at the input resolution — is
(cnxDownChWHasVJP h w p hε).backward v.
padOdd is required: p.W is 2×2, so cnxDownBack p.W reverses a conv shifted one pixel
(EvenKernelConvBack.lean). The downsamples are three of ConvNeXt-T's four even-kernel sites;
the stem is the fourth.
One channel-LN ConvNeXt block's backward at a saved input v — exactly the left-hand side of
cnxBlockChBack_eq_vjp, named so the stage recursion can be written down.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The depth-k stage backward, at a saved stage input v.
Head-first, like the forward it reverses. convNextStageChK (k+1) ps = convNextStageChK k (ps ∘ succ) ∘ cnxBlockChW (ps 0) applies block 0 FIRST, so the backward
applies block 0's reverse LAST — cnxBlockChBackAt (ps 0) v ∘ (the rest). And the saved
activation threads forward through the recursion: the tail's saved input is
cnxBlockChW (ps 0) v, block 0's output.
Equations
- Proofs.cnxStageChKBack gf 0 x_3 x✝ = id
- Proofs.cnxStageChKBack gf k.succ ps x✝ = Proofs.cnxBlockChBackAt gf (ps 0) x✝ ∘ Proofs.cnxStageChKBack gf k (fun (i : Fin k) => ps i.succ) (Proofs.cnxBlockChW gf (ps 0) x✝)
Instances For
The stage-fold tie. The hand-composed depth-k stage backward is
(convNextStageChKHasVJP k ps hε).backward. Induction on k: the base case is
identityHasVJP's fun _ dy => dy, and the step is one rewrite of the block tie
(cnxBlockChBack_eq_vjp) and one of the inductive hypothesis at the shifted saved
activation.
rowLNVecFlatHasVJP_backward_eq at the FUNCTION level — the direction and shape a whole-net
rw needs. The committed lemma is pointwise in dy and oriented certified-to-hand; a chain
rewrite wants hand-to-certified with dy abstracted.
Every stage whose declared type carries a computed dimension (cnxDownChW's
Vec (cin * (2 * h) * (2 * w)), the head LayerNorm's Vec (1 * 768)) gets a wrapper here, its
type ascribed in the chain's spelling, with Differentiable/HasVJP peers ascribed the same way
(see the proof-shape notes after the module docstring).
The wrappers, their Differentiable/HasVJP peers, the stem's
cnxSavedA0_differentiable/cnxV0 and the four normalised leaf ties below are public:
ConvNeXtWholeBackCertifiedTieB.lean lifts the same twelve stages over a batch and needs them at
exactly these spellings.
Downsample 1 at the chain's dimension spelling.
Equations
- Proofs.cnxDn1 w = Proofs.cnxDownChW 28 28 w.d1
Instances For
Equations
- Proofs.cnxDn1Vjp w hd1 = Proofs.cnxDownChWHasVJP 28 28 w.d1 hd1
Instances For
Downsample 2 at the chain's dimension spelling.
Equations
- Proofs.cnxDn2 w = Proofs.cnxDownChW 14 14 w.d2
Instances For
Downsample 3 at the chain's dimension spelling.
Equations
- Proofs.cnxDn3 w = Proofs.cnxDownChW 7 7 w.d3
Instances For
Equations
- Proofs.cnxDn3Vjp w hd3 = Proofs.cnxDownChWHasVJP 7 7 w.d3 hd3
Instances For
The head LayerNorm at Vec 768, not Vec (1 * 768) — the same normalisation, at the one
site where the computed dimension is a 1 * rather than a 2 *.
Equations
- Proofs.cnxLNh w = Proofs.rowLNVecFlat 1 768 w.hε w.hγ w.hβ
Instances For
Equations
- Proofs.cnxLNhVjp w hhε = Proofs.rowLNVecFlatHasVJP 1 768 w.hε w.hγ w.hβ hhε
Instances For
convNextForwardTCh's eleven stage inputs, named as functions, so that the same constants
are both the saved activations the backward's slots are indexed by and the f argument of each
vjpComp; every link below is then a one-step iota with syntactically identical sides.
The stem conv's output — the stem LayerNorm's saved input.
Equations
Instances For
Equations
Instances For
Stage 1's saved input.
Equations
- Proofs.cnxSavedA1 w = Proofs.chanLNTensor3 96 56 56 w.sε w.sγ w.sβ ∘ Proofs.cnxSavedA0 w
Instances For
Downsample 1's saved input.
Equations
- Proofs.cnxSavedA2 gf w = Proofs.convNextStageChK gf 3 w.s1 ∘ Proofs.cnxSavedA1 w
Instances For
Stage 2's saved input.
Equations
- Proofs.cnxSavedA3 gf w = Proofs.cnxDn1 w ∘ Proofs.cnxSavedA2 gf w
Instances For
Downsample 2's saved input.
Equations
- Proofs.cnxSavedA4 gf w = Proofs.convNextStageChK gf 3 w.s2 ∘ Proofs.cnxSavedA3 gf w
Instances For
Stage 3's saved input.
Equations
- Proofs.cnxSavedA5 gf w = Proofs.cnxDn2 w ∘ Proofs.cnxSavedA4 gf w
Instances For
Downsample 3's saved input.
Equations
- Proofs.cnxSavedA6 gf w = Proofs.convNextStageChK gf 9 w.s3 ∘ Proofs.cnxSavedA5 gf w
Instances For
Stage 4's saved input.
Equations
- Proofs.cnxSavedA7 gf w = Proofs.cnxDn3 w ∘ Proofs.cnxSavedA6 gf w
Instances For
The head LayerNorm's saved input.
Equations
- Proofs.cnxSavedA9 gf w = Proofs.globalAvgPoolFlat 768 7 7 ∘ Proofs.cnxSavedA8✝ gf w
Instances For
The classifier's saved input.
Equations
- Proofs.cnxSavedA10 gf w = Proofs.cnxLNh w ∘ Proofs.cnxSavedA9 gf w
Instances For
convNextForwardTChHasVJP as a TERM-mode vjpComp chain.
Equations
- Proofs.convNextForwardTChVjpChain gf w hsε h1 hd1 h2 hd2 h3 hd3 h4 hhε = Proofs.cnxV11✝ gf w hsε h1 hd1 h2 hd2 h3 hd3 h4 hhε
Instances For
convnextInputGrad is the certified whole-net ConvNeXt-T gradient, at one image x,
every nC, under the 23 LayerNorm positivities; the forward is the drop-free
convNextForwardTCh.