Documentation

LeanMlir.Proofs.Nets.ConvNeXt.ConvNeXtWholeBackCertifiedTie

ConvNeXt-T's whole-net backward tie — the stage fold and the even-kernel repair #

The per-example whole-net backward tie for ConvNeXt-T: convnextInputGrad, with every slot pinned to the certified per-op backward at its own saved activation, is (convNextForwardTChHasVJP …).backward x. The batched form is ConvNeXtWholeBackCertifiedTieB.lean.

Even kernels. conv2d pads by pH = (kH-1)/2, so the reversed-kernel forward conv (convFlatBack) is the adjoint only when kH - 1 - pH = pH, i.e. only for odd kH. ConvNeXt's 4×4/s4 patchify stem and its three 2×2/s2 downsamples are even-kernel sites; at kH = 4 the symmetric-pad backward is the adjoint of a conv shifted one pixel. StableHLO.Basic's .convStridedBack / .convStridedBackBatched pad asymmetrically, [[kH-1-pH, pH]], and their den is the certified VJP; BackwardMaps.lean's flatConvStride2Back / flatConvStride4Back are convFlatBack ∘ scatter at the symmetric pad. The repair is padOdd (EvenKernelConvBack.lean): an even-kernel conv is an odd-kernel conv on the kernel zero-extended at (+1,+1), which is the emitter's asymmetric pad written in the vocabulary BackwardMaps.lean already has, so the existing odd-kernel leaf tie applies.

What is here.

  1. cnxDownChBack_eq_vjp — the stage-boundary downsample tie, lnB ∘ flatConvStride2Back (padOdd W) against (cnxDownChWHasVJP …).backward. padOdd is required: p.W is 2×2.
  2. cnxStageChKBack_eq_vjp — the depth-k stage fold. HasVJP for convNextStageChK is built head-first (block 0 runs first), so its backward composes the block backwards in the opposite order, each at its own saved activation, and the tail's saved input is block 0's forward output. The induction step is one rewrite of the block tie (cnxBlockChBack_eq_vjp) and one of the inductive hypothesis.
  3. cnxSavedA0 … cnxSavedA10 — convNextForwardTCh's eleven stage inputs, named as functions, so that the same constants are both the activations the backward's slots are saved at and the f argument of each vjpComp.
  4. convnextInputGrad_eq_convNextForwardTCh_vjp — the apex. convNextForwardTChHasVJP is a global HasVJP, because GELU, LayerNorm, convolution and the layer scale are all smooth. Its only hypotheses are the 23 LayerNorm positivities, so, like ViT's (vitInputGradK_eq_vitForwardKV_vjp) and EfficientNet-B0's (efficientnetInputGradBFull_eq_efficientnetForwardB_full_vjp), and unlike the ReLU nets' (ResNet-34/50, MobileNetV2, MobileNetV4), it carries no smoothness side-condition. convNextForwardTCh_eq_chain (proved by rw) says the chain the apex instantiates is the committed forward.
theorem Proofs.cnxDownChBack_eq_vjp {cin cout h w : ℕ} (p : CnxDownParamsCh cin cout) (hε : 0 < p.ε) (v : Vec (cin * (2 * h) * (2 * w))) :
cnxDownBack (padOdd p.W) (chanLNTensor3Back cin (2 * h) (2 * w) p.ε p.γ v) = (cnxDownChWHasVJP h w p hε).backward v

The downsample backward tie. cnxDownBack (padOdd p.W) lnB — the strided-conv backward at the ZERO-EXTENDED kernel, then the channel-LN back at the input resolution — is (cnxDownChWHasVJP h w p hε).backward v.

padOdd is required: p.W is 2×2, so cnxDownBack p.W reverses a conv shifted one pixel (EvenKernelConvBack.lean). The downsamples are three of ConvNeXt-T's four even-kernel sites; the stem is the fourth.

noncomputable def Proofs.cnxBlockChBackAt (gf : GeluForm) {c cExp h w kHd kWd : ℕ} (p : CnxBlockParamsCh c cExp h w kHd kWd) (v : Vec (c * h * w)) :
Vec (c * h * w) → Vec (c * h * w)

One channel-LN ConvNeXt block's backward at a saved input v — exactly the left-hand side of cnxBlockChBack_eq_vjp, named so the stage recursion can be written down.

Equations
  • One or more equations did not get rendered due to their size.
Instances For
    noncomputable def Proofs.cnxStageChKBack (gf : GeluForm) {c cExp h w kH kW : ℕ} (k : ℕ) (ps : Fin k → CnxBlockParamsCh c cExp h w kH kW) :
    Vec (c * h * w) → Vec (c * h * w) → Vec (c * h * w)

    The depth-k stage backward, at a saved stage input v.

    Head-first, like the forward it reverses. convNextStageChK (k+1) ps = convNextStageChK k (ps ∘ succ) ∘ cnxBlockChW (ps 0) applies block 0 FIRST, so the backward applies block 0's reverse LAST — cnxBlockChBackAt (ps 0) v ∘ (the rest). And the saved activation threads forward through the recursion: the tail's saved input is cnxBlockChW (ps 0) v, block 0's output.

    Equations
    Instances For
      theorem Proofs.cnxStageChKBack_eq_vjp {gf : GeluForm} {c cExp h w kHd kWd : ℕ} (hkHd : 2 * ((kHd - 1) / 2) + 1 = kHd) (hkWd : 2 * ((kWd - 1) / 2) + 1 = kWd) (k : ℕ) (ps : Fin k → CnxBlockParamsCh c cExp h w kHd kWd) (hε : ∀ (i : Fin k), 0 < (ps i).εn) (v : Vec (c * h * w)) :

      The stage-fold tie. The hand-composed depth-k stage backward is (convNextStageChKHasVJP k ps hε).backward. Induction on k: the base case is identityHasVJP's fun _ dy => dy, and the step is one rewrite of the block tie (cnxBlockChBack_eq_vjp) and one of the inductive hypothesis at the shifted saved activation.

      theorem Proofs.rowLNVecFlatHasVJP_backward_eq_fun {s c : ℕ} (ε : ℝ) (hε : 0 < ε) (γ β : Vec c) (X : Vec (s * c)) :
      rowLNVecFlatBack s c ε γ X = (rowLNVecFlatHasVJP s c ε γ β hε).backward X

      rowLNVecFlatHasVJP_backward_eq at the FUNCTION level — the direction and shape a whole-net rw needs. The committed lemma is pointwise in dy and oriented certified-to-hand; a chain rewrite wants hand-to-certified with dy abstracted.

      Every stage whose declared type carries a computed dimension (cnxDownChW's Vec (cin * (2 * h) * (2 * w)), the head LayerNorm's Vec (1 * 768)) gets a wrapper here, its type ascribed in the chain's spelling, with Differentiable/HasVJP peers ascribed the same way (see the proof-shape notes after the module docstring).

      The wrappers, their Differentiable/HasVJP peers, the stem's cnxSavedA0_differentiable/cnxV0 and the four normalised leaf ties below are public: ConvNeXtWholeBackCertifiedTieB.lean lifts the same twelve stages over a batch and needs them at exactly these spellings.

      noncomputable def Proofs.cnxDn1 {nC : ℕ} (w : CnxTWeightsCh nC) :
      Vec (96 * 56 * 56) → Vec (192 * 28 * 28)

      Downsample 1 at the chain's dimension spelling.

      Equations
      Instances For
        noncomputable def Proofs.cnxDn1Vjp {nC : ℕ} (w : CnxTWeightsCh nC) (hd1 : 0 < w.d1.ε) :
        Equations
        Instances For
          noncomputable def Proofs.cnxDn2 {nC : ℕ} (w : CnxTWeightsCh nC) :
          Vec (192 * 28 * 28) → Vec (384 * 14 * 14)

          Downsample 2 at the chain's dimension spelling.

          Equations
          Instances For
            noncomputable def Proofs.cnxDn3 {nC : ℕ} (w : CnxTWeightsCh nC) :
            Vec (384 * 14 * 14) → Vec (768 * 7 * 7)

            Downsample 3 at the chain's dimension spelling.

            Equations
            Instances For
              noncomputable def Proofs.cnxDn3Vjp {nC : ℕ} (w : CnxTWeightsCh nC) (hd3 : 0 < w.d3.ε) :
              Equations
              Instances For
                noncomputable def Proofs.cnxLNh {nC : ℕ} (w : CnxTWeightsCh nC) :
                Vec 768 → Vec 768

                The head LayerNorm at Vec 768, not Vec (1 * 768) — the same normalisation, at the one site where the computed dimension is a 1 * rather than a 2 *.

                Equations
                Instances For
                  noncomputable def Proofs.cnxLNhVjp {nC : ℕ} (w : CnxTWeightsCh nC) (hhε : 0 < w.hε) :
                  Equations
                  Instances For

                    convNextForwardTCh's eleven stage inputs, named as functions, so that the same constants are both the saved activations the backward's slots are indexed by and the f argument of each vjpComp; every link below is then a one-step iota with syntactically identical sides.

                    noncomputable def Proofs.cnxSavedA0 {nC : ℕ} (w : CnxTWeightsCh nC) :
                    Vec (3 * 224 * 224) → Vec (96 * 56 * 56)

                    The stem conv's output — the stem LayerNorm's saved input.

                    Equations
                    Instances For
                      noncomputable def Proofs.cnxV0 {nC : ℕ} (w : CnxTWeightsCh nC) :
                      Equations
                      Instances For
                        noncomputable def Proofs.cnxSavedA1 {nC : ℕ} (w : CnxTWeightsCh nC) :
                        Vec (3 * 224 * 224) → Vec (96 * 56 * 56)

                        Stage 1's saved input.

                        Equations
                        Instances For
                          noncomputable def Proofs.cnxSavedA2 (gf : GeluForm) {nC : ℕ} (w : CnxTWeightsCh nC) :
                          Vec (3 * 224 * 224) → Vec (96 * 56 * 56)

                          Downsample 1's saved input.

                          Equations
                          Instances For
                            noncomputable def Proofs.cnxSavedA3 (gf : GeluForm) {nC : ℕ} (w : CnxTWeightsCh nC) :
                            Vec (3 * 224 * 224) → Vec (192 * 28 * 28)

                            Stage 2's saved input.

                            Equations
                            Instances For
                              noncomputable def Proofs.cnxSavedA4 (gf : GeluForm) {nC : ℕ} (w : CnxTWeightsCh nC) :
                              Vec (3 * 224 * 224) → Vec (192 * 28 * 28)

                              Downsample 2's saved input.

                              Equations
                              Instances For
                                noncomputable def Proofs.cnxSavedA5 (gf : GeluForm) {nC : ℕ} (w : CnxTWeightsCh nC) :
                                Vec (3 * 224 * 224) → Vec (384 * 14 * 14)

                                Stage 3's saved input.

                                Equations
                                Instances For
                                  noncomputable def Proofs.cnxSavedA6 (gf : GeluForm) {nC : ℕ} (w : CnxTWeightsCh nC) :
                                  Vec (3 * 224 * 224) → Vec (384 * 14 * 14)

                                  Downsample 3's saved input.

                                  Equations
                                  Instances For
                                    noncomputable def Proofs.cnxSavedA7 (gf : GeluForm) {nC : ℕ} (w : CnxTWeightsCh nC) :
                                    Vec (3 * 224 * 224) → Vec (768 * 7 * 7)

                                    Stage 4's saved input.

                                    Equations
                                    Instances For
                                      noncomputable def Proofs.cnxSavedA9 (gf : GeluForm) {nC : ℕ} (w : CnxTWeightsCh nC) :
                                      Vec (3 * 224 * 224) → Vec 768

                                      The head LayerNorm's saved input.

                                      Equations
                                      Instances For
                                        noncomputable def Proofs.cnxSavedA10 (gf : GeluForm) {nC : ℕ} (w : CnxTWeightsCh nC) :
                                        Vec (3 * 224 * 224) → Vec 768

                                        The classifier's saved input.

                                        Equations
                                        Instances For
                                          theorem Proofs.cnxDn1Back_eq_vjp {nC : ℕ} (w : CnxTWeightsCh nC) (hd1 : 0 < w.d1.ε) (v : Vec (96 * 56 * 56)) :
                                          cnxDownBack (padOdd w.d1.W) (chanLNTensor3Back 96 56 56 w.d1.ε w.d1.γ v) = (cnxDn1Vjp w hd1).backward v
                                          theorem Proofs.cnxDn3Back_eq_vjp {nC : ℕ} (w : CnxTWeightsCh nC) (hd3 : 0 < w.d3.ε) (v : Vec (384 * 14 * 14)) :
                                          cnxDownBack (padOdd w.d3.W) (chanLNTensor3Back 384 14 14 w.d3.ε w.d3.γ v) = (cnxDn3Vjp w hd3).backward v
                                          theorem Proofs.cnxLNhBack_eq_vjp {nC : ℕ} (w : CnxTWeightsCh nC) (hhε : 0 < w.hε) (v : Vec (1 * 768)) :
                                          rowLNVecFlatBack 1 768 w.hε w.hγ v = (cnxLNhVjp w hhε).backward v
                                          noncomputable def Proofs.convNextForwardTChVjpChain (gf : GeluForm) {nC : ℕ} (w : CnxTWeightsCh nC) (hsε : 0 < w.sε) (h1 : ∀ (i : Fin 3), 0 < (w.s1 i).εn) (hd1 : 0 < w.d1.ε) (h2 : ∀ (i : Fin 3), 0 < (w.s2 i).εn) (hd2 : 0 < w.d2.ε) (h3 : ∀ (i : Fin 9), 0 < (w.s3 i).εn) (hd3 : 0 < w.d3.ε) (h4 : ∀ (i : Fin 3), 0 < (w.s4 i).εn) (hhε : 0 < w.hε) :

                                          convNextForwardTChHasVJP as a TERM-mode vjpComp chain.

                                          Equations
                                          Instances For
                                            theorem Proofs.convnextInputGrad_eq_convNextForwardTCh_vjp {gf : GeluForm} {nC : ℕ} (w : CnxTWeightsCh nC) (hsε : 0 < w.sε) (h1 : ∀ (i : Fin 3), 0 < (w.s1 i).εn) (hd1 : 0 < w.d1.ε) (h2 : ∀ (i : Fin 3), 0 < (w.s2 i).εn) (hd2 : 0 < w.d2.ε) (h3 : ∀ (i : Fin 9), 0 < (w.s3 i).εn) (hd3 : 0 < w.d3.ε) (h4 : ∀ (i : Fin 3), 0 < (w.s4 i).εn) (hhε : 0 < w.hε) (x : Vec (3 * 224 * 224)) :
                                            convnextInputGrad w.Wd (padOdd w.sW) (chanLNTensor3Back 96 56 56 w.sε w.sγ (cnxSavedA0 w x)) (rowLNVecFlatBack 1 768 w.hε w.hγ (cnxSavedA9 gf w x)) (cnxStageChKBack gf 3 w.s1 (cnxSavedA1 w x)) (cnxDownBack (padOdd w.d1.W) (chanLNTensor3Back 96 56 56 w.d1.ε w.d1.γ (cnxSavedA2 gf w x))) (cnxStageChKBack gf 3 w.s2 (cnxSavedA3 gf w x)) (cnxDownBack (padOdd w.d2.W) (chanLNTensor3Back 192 28 28 w.d2.ε w.d2.γ (cnxSavedA4 gf w x))) (cnxStageChKBack gf 9 w.s3 (cnxSavedA5 gf w x)) (cnxDownBack (padOdd w.d3.W) (chanLNTensor3Back 384 14 14 w.d3.ε w.d3.γ (cnxSavedA6 gf w x))) (cnxStageChKBack gf 3 w.s4 (cnxSavedA7 gf w x)) = (convNextForwardTChHasVJP gf w hsε h1 hd1 h2 hd2 h3 hd3 h4 hhε).backward x

                                            convnextInputGrad is the certified whole-net ConvNeXt-T gradient, at one image x, every nC, under the 23 LayerNorm positivities; the forward is the drop-free convNextForwardTCh.