Documentation

LeanMlir.Proofs.Nets.ConvNeXt.ConvNeXtFoldGB

T3 §1 fold for ConvNeXt-T at the BATCHED index — the op set every ImageNet artifact renders #

ConvNeXtFoldG.lean folds the fourteen gradient nodes of the PER-EXAMPLE traversal (ConvNeXtRender.convNextBackAll at adam := true). This is its batched peer, at ConvNeXtRenderB.convNextBackAllB's constructors — and unlike ViT's (ViTFoldGB.lean), it was owed BEFORE any renderer swap: every convnextin_* train step, every *drop* variant and the ConvNeXt-S/B artifacts have rendered from the batched chain since they existed, so the artifact behind this net's quoted ImageNet accuracy (convnextin_adamdpwxclipdrop) had a fold at the per-example constructors that no committed byte of it is pretty of. 4b's "one lemma per op kind certifies every optimizer tail" was, for ConvNeXt, a statement about the Imagenette pair only.

The bytes are the same on the forward and differ on 78 backward lines. The batched convBackBatched emits the conv input-VJP's transpose/reverse in the other order from the per-example convBack — commuting ops on disjoint axes, one kernel — and tests/TestConvNeXtFwdBTie.lean allows exactly that pair and nothing else. 4c leg 3 moves the drop-free writers onto this chain; this file lands first, per leg 1's ordering rule, so that no committed artifact is ever pretty of an AST without a fold.

The op table of convnext_adam_train_step.mlir and every convnextin_* train step #

emitted nodelemmaper-example peer it batches
layerScaleChGammaGradB (18 block γ)layerScaleChGammaGradB_denCnxPoCG.layerScaleChGammaGrad_den
convWeightGradB / convBiasGradB (18 expand + 18 project 1×1, + the stem bias)convWGradB_den / convBGradB_denResNet34PoCB's, verbatim
depthwiseWeightGradB / depthwiseBiasGradB (18 × 7×7)depthwiseWGradB_den / depthwiseBGradB_denEnetPoCG / Mnv2PaperPoCG
convStridedWeightGradB / convStridedBiasGradB (3 × 2×2/s2 downsample)convStridedWGradB_den / convStridedBGradB_denResNet34PoCB's, verbatim
convStride4WeightGradB (patchify stem)psWGradB_denflatConvStride4_weight_grad_has_vjp, per example
veclnGammaGradB / rowDenseBiasGradB at R = h·w (22 spatial LN sites)chanLnGammaGradB_den / chanLnBetaGradB_denCnxPoCG.chanLnGammaGrad_den / chanLnBetaGrad_den
veclnGammaGradB / rowDenseBiasGradB at R = 1 (the head LN, after GAP)headLnGammaGradB_den / headLnBetaGradB_denViTPoCGB's two-level LN lemmas
weightGradB / biasGradB (the classifier)headWGradB_den / headBGradB_denViTPoCGB.headWGradB_den / headBGradB_den
convWeightGradBBf16 / depthwiseWeightGradBBf16 / convStridedWeightGradBBf16 / convStride4WeightGradBBf16 (the bf16 artifacts)Bf16PoC.convWGradBBf16_den and its siblings, Foundation/Bf16GradNodes.leannone — a bf16 node is its own op kind

No new mathematics. Every proof is Finset.sum_congr rfl over the batch and then the per-example bridge at batchSlice nResNet34FoldB.denseWGradB_den's shape — because each batched den arm is literally the per-example one under a batch sum. The channel-LN sites add one step: the batched render hands the LN ops batchMap N (chanLNRows c h w) of the saved input and of the cotangent (the [h·w, c] transposed views, lifted per example), and batchSlice_batchMap peels the lift so ConvNeXtChannelLN's permutation argument applies at each slice.

The bf16 artifacts (convnextin_adamwxclipdropbf16, the S/B twins) emit *GradBBf16 constructors, not these nodes: their den rounds the operands and the result once, outside the batch sum. Those are their own op kinds, folded once for every net in Foundation/Bf16GradNodes.lean (first stated in this file, 2026-09-07).

One lemma per op kind certifies every optimizer tail at once — AdamW, the wx/clip variants, the EMA shadow, drop-path and the data-parallel twins all consume the same *GradB node, and convnextin_adamdpwxclipdrop, whose accuracy the book quotes, is one of them.

Honest residual #

theorem Proofs.CnxPoCGB.layerScaleChGammaGradB_den {N c h w : } (xN cotN : String) (x : Vec (N * (c * h * w))) (γ : Vec c) (dy : Vec (N * (c * h * w))) (cc : Fin c) :
StableHLO.den (StableHLO.SHlo.layerScaleChGammaGradB xN x (StableHLO.SHlo.operand cotN dy)) cc = n : Fin N, j : Fin (c * h * w), pdiv (fun (γ' : Vec c) => layerScale (fun (k : Fin (c * h * w)) => γ' (StableHLO.chanIdx c h w k)) (StableHLO.batchSlice N (c * h * w) x n)) γ cc j * StableHLO.batchSlice N (c * h * w) dy n j

Batched per-channel layer-scale γ GRADIENT denotes the certified Σ_n gradient. The emitted reduce contracts batch and spatial in one op; den reads it as the batch sum of the per-example dγ_c = Σ_{k : chanIdx k = c} x_k·dy_k. All 18 blocks.

theorem Proofs.CnxPoCGB.convWGradB_den {N ic oc h w kH kW : } (xN cotN : String) (b : Vec oc) (x : Vec (N * (ic * h * w))) (W : Kernel4 oc ic kH kW) (cot : Vec (N * (oc * h * w))) (idx : Fin (oc * ic * kH * kW)) :
StableHLO.den (StableHLO.SHlo.convWeightGradB xN b x W (StableHLO.SHlo.operand cotN cot)) idx = n : Fin N, j : Fin (oc * h * w), pdiv (fun (v' : Vec (oc * ic * kH * kW)) => (conv2d (Kernel4.unflatten v') b (Tensor3.unflatten (StableHLO.batchSlice N (ic * h * w) x n))).flatten) W.flatten idx j * StableHLO.batchSlice N (oc * h * w) cot n j

Batched conv weight GRADIENT denotes the certified Σ_n weight gradient. Kernel-generic: every 1×1 expand and project. ResNet34PoCB.convWGradB_den verbatim.

theorem Proofs.CnxPoCGB.convBGradB_den {N ic oc h w kH kW : } (cotN : String) (W : Kernel4 oc ic kH kW) (x : Vec (N * (ic * h * w))) (b : Vec oc) (cot : Vec (N * (oc * h * w))) (o : Fin oc) :
StableHLO.den (StableHLO.SHlo.convBiasGradB W x b (StableHLO.SHlo.operand cotN cot)) o = n : Fin N, j : Fin (oc * h * w), pdiv (fun (b' : Vec oc) => (conv2d W b' (Tensor3.unflatten (StableHLO.batchSlice N (ic * h * w) x n))).flatten) b o j * StableHLO.batchSlice N (oc * h * w) cot n j

Batched conv bias GRADIENT denotes the certified Σ_n bias gradient. The expand/project biases and the patchify stem's psb (at 4×4).

theorem Proofs.CnxPoCGB.depthwiseWGradB_den {N c h w kH kW : } (xN cotN : String) (b : Vec c) (x : Vec (N * (c * h * w))) (W : DepthwiseKernel c kH kW) (cot : Vec (N * (c * h * w))) (idx : Fin (c * kH * kW)) :
StableHLO.den (StableHLO.SHlo.depthwiseWeightGradB xN b x W (StableHLO.SHlo.operand cotN cot)) idx = n : Fin N, j : Fin (c * h * w), pdiv (fun (v' : Vec (c * kH * kW)) => (depthwiseConv2d (Tensor3.unflatten v') b (Tensor3.unflatten (StableHLO.batchSlice N (c * h * w) x n))).flatten) (Tensor3.flatten W) idx j * StableHLO.batchSlice N (c * h * w) cot n j

Batched depthwise weight GRADIENT denotes the certified Σ_n weight gradient. All 18 blocks; the kernel size is a variable, so 7×7 is an instance. EnetPoCG.depthwiseWGradB_den.

theorem Proofs.CnxPoCGB.depthwiseBGradB_den {N c h w kH kW : } (cotN : String) (W : DepthwiseKernel c kH kW) (x : Vec (N * (c * h * w))) (b : Vec c) (cot : Vec (N * (c * h * w))) (o : Fin c) :
StableHLO.den (StableHLO.SHlo.depthwiseBiasGradB W x b (StableHLO.SHlo.operand cotN cot)) o = n : Fin N, j : Fin (c * h * w), pdiv (fun (b' : Vec c) => (depthwiseConv2d W b' (Tensor3.unflatten (StableHLO.batchSlice N (c * h * w) x n))).flatten) b o j * StableHLO.batchSlice N (c * h * w) cot n j

Batched depthwise bias GRADIENT denotes the certified Σ_n bias gradient. ConvNeXt's depthwises carry a bias (no BatchNorm follows them), which is Mnv2PaperPoCG's lemma.

theorem Proofs.CnxPoCGB.convStridedWGradB_den {N ic oc h w kH kW : } (xN cotN : String) (b : Vec oc) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (W : Kernel4 oc ic kH kW) (cot : Vec (N * (oc * h * w))) (idx : Fin (oc * ic * kH * kW)) :
StableHLO.den (StableHLO.SHlo.convStridedWeightGradB xN b x W (StableHLO.SHlo.operand cotN cot)) idx = n : Fin N, j : Fin (oc * h * w), pdiv (fun (v' : Vec (oc * ic * kH * kW)) => flatConvStride2 (Kernel4.unflatten v') b (StableHLO.batchSlice N (ic * (2 * h) * (2 * w)) x n)) W.flatten idx j * StableHLO.batchSlice N (oc * h * w) cot n j

Batched strided conv weight GRADIENT denotes the certified Σ_n weight gradient. The three 2×2/s2 downsamples; kernel-generic, ResNet34PoCB.convStridedWGradB_den verbatim.

theorem Proofs.CnxPoCGB.convStridedBGradB_den {N ic oc h w kH kW : } (cotN : String) (W : Kernel4 oc ic kH kW) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (b : Vec oc) (cot : Vec (N * (oc * h * w))) (o : Fin oc) :
StableHLO.den (StableHLO.SHlo.convStridedBiasGradB W x b (StableHLO.SHlo.operand cotN cot)) o = n : Fin N, j : Fin (oc * h * w), pdiv (fun (b' : Vec oc) => flatConvStride2 W b' (StableHLO.batchSlice N (ic * (2 * h) * (2 * w)) x n)) b o j * StableHLO.batchSlice N (oc * h * w) cot n j

Batched strided conv bias GRADIENT denotes the certified Σ_n bias gradient.

theorem Proofs.CnxPoCGB.psWGradB_den {N ic oc h w kH kW : } (xN cotN : String) (b : Vec oc) (x : Vec (N * (ic * (2 * (2 * h)) * (2 * (2 * w))))) (W : Kernel4 oc ic kH kW) (cot : Vec (N * (oc * h * w))) (idx : Fin (oc * ic * kH * kW)) :
StableHLO.den (StableHLO.SHlo.convStride4WeightGradB xN b x W (StableHLO.SHlo.operand cotN cot)) idx = n : Fin N, j : Fin (oc * h * w), pdiv (fun (v' : Vec (oc * ic * kH * kW)) => flatConvStride4 (Kernel4.unflatten v') b (StableHLO.batchSlice N (ic * (2 * (2 * h)) * (2 * (2 * w))) x n)) W.flatten idx j * StableHLO.batchSlice N (oc * h * w) cot n j

Batched patchify-stem weight GRADIENT denotes the certified Σ_n weight gradient. ⚠ The emitted convolution contracts the batch axis itself (the transpose trick), so the outer sum is inside one op rather than across N of them — same as the strided ops.

theorem Proofs.CnxPoCGB.chanLnGammaGradB_den {N c h w : } (xN epsStr cotN : String) (ε : ) (β : Vec c) (x : Vec (N * (c * h * w))) (γ : Vec c) (cot : Vec (N * (c * h * w))) (k : Fin c) :
StableHLO.den (StableHLO.SHlo.veclnGammaGradB xN epsStr ε (StableHLO.batchMap N (chanLNRows c h w) x) (StableHLO.SHlo.operand cotN (StableHLO.batchMap N (chanLNRows c h w) cot))) k = n : Fin N, j : Fin (c * h * w), pdiv (fun (γ' : Vec c) => chanLNTensor3 c h w ε γ' β (StableHLO.batchSlice N (c * h * w) x n)) γ k j * StableHLO.batchSlice N (c * h * w) cot n j

Batched channel-LN γ GRADIENT denotes the certified Σ_n γ gradient. All 22 spatial sites (1 stem + 18 block + 3 downsample). Two levels: the outer sum is the batch, the inner the h·w rows within one example.

theorem Proofs.CnxPoCGB.chanLnBetaGradB_den {N c h w : } (cotN : String) (ε : ) (γ : Vec c) (x : Vec (N * (c * h * w))) (β : Vec c) (cot : Vec (N * (c * h * w))) (k : Fin c) :
StableHLO.den (StableHLO.SHlo.operand cotN (StableHLO.batchMap N (chanLNRows c h w) cot)).rowDenseBiasGradB k = n : Fin N, j : Fin (c * h * w), pdiv (fun (β' : Vec c) => chanLNTensor3 c h w ε γ β' (StableHLO.batchSlice N (c * h * w) x n)) β k j * StableHLO.batchSlice N (c * h * w) cot n j

Batched channel-LN β GRADIENT denotes the certified Σ_n β gradient. The β gradient is the plain two-level row reduce, so the render uses the same rowDenseBiasGradB op ViT's LN β does.

theorem Proofs.CnxPoCGB.headLnGammaGradB_den {N R D : } (xN epsStr cotN : String) (ε : ) (βv : Vec D) (x : Vec (N * (R * D))) (γ : Vec D) (dy : Vec (N * (R * D))) (k : Fin D) :
StableHLO.den (StableHLO.SHlo.veclnGammaGradB xN epsStr ε x (StableHLO.SHlo.operand cotN dy)) k = n : Fin N, o : Fin (R * D), pdiv (fun (gv : Vec D) => Mat.flatten fun (r : Fin R) => layerNormVec D ε gv βv (Mat.unflatten (StableHLO.batchSlice N (R * D) x n) r)) γ k o * StableHLO.batchSlice N (R * D) dy n o

Batched head-LN γ GRADIENT denotes the certified Σ_n γ gradient. ViT's two-level vector LayerNorm lemma; the head LN runs after GAP, so the render instantiates it at R = 1.

theorem Proofs.CnxPoCGB.headLnBetaGradB_den {N R D : } (cotN : String) (ε : ) (γv : Vec D) (X : Fin NMat R D) (β : Vec D) (dy : Vec (N * (R * D))) (i : Fin D) :
StableHLO.den (StableHLO.SHlo.operand cotN dy).rowDenseBiasGradB i = n : Fin N, o : Fin (R * D), pdiv (fun (bv : Vec D) => Mat.flatten fun (r : Fin R) => layerNormVec D ε γv bv (X n r)) β i o * StableHLO.batchSlice N (R * D) dy n o

Batched head-LN β GRADIENT denotes the certified Σ_n β gradient.

theorem Proofs.CnxPoCGB.headWGradB_den {N D nC : } (aN cotN : String) (a : Vec (N * D)) (Wc : Mat D nC) (bc : Vec nC) (cot : Vec (N * nC)) (i : Fin D) (j : Fin nC) :
StableHLO.den (StableHLO.SHlo.weightGradB aN a (StableHLO.SHlo.operand cotN cot)) (finProdFinEquiv (i, j)) = n : Fin N, k : Fin nC, pdiv (fun (v : Vec (D * nC)) => dense (Mat.unflatten v) bc (StableHLO.batchSlice N D a n)) Wc.flatten (finProdFinEquiv (i, j)) k * StableHLO.batchSlice N nC cot n k

Batched classifier weight GRADIENT denotes the certified Σ_n outer product.

theorem Proofs.CnxPoCGB.headBGradB_den {N D nC : } (cotN : String) (Wc : Mat D nC) (a : Vec D) (bc : Vec nC) (cot : Vec (N * nC)) (n : Fin N) (i : Fin nC) :
StableHLO.batchSlice N nC (StableHLO.den (StableHLO.SHlo.operand cotN cot).biasGradB) n i = j : Fin nC, pdiv (fun (b' : Vec nC) => dense Wc b' a) bc i j * StableHLO.batchSlice N nC cot n j

Batched classifier bias GRADIENT denotes the certified cotangent, PER EXAMPLE. biasGradB is the identity on its operand — the batch reduce is emitted text outside the AST — so the statement is at every batchSlice n, the per-example carve-out carried over.

def Proofs.CnxPoCGB.ChanLNGammaTiedB (N h w : ) {c : } (xN epsStr cotN : String) (ε : ) (β : Vec c) (x : Vec (N * (c * h * w))) (γ : Vec c) (cot : Vec (N * (c * h * w))) :

A batched channel-LN γ gradient node, tied (chanLnGammaGradB_den).

Equations
  • One or more equations did not get rendered due to their size.
Instances For
    def Proofs.CnxPoCGB.ChanLNBetaTiedB (N h w : ) {c : } (cotN : String) (ε : ) (γ : Vec c) (x : Vec (N * (c * h * w))) (β : Vec c) (cot : Vec (N * (c * h * w))) :

    A batched channel-LN β gradient node, tied (chanLnBetaGradB_den).

    Equations
    • One or more equations did not get rendered due to their size.
    Instances For