Documentation

LeanMlir.Proofs.Nets.ConvNeXt.ConvNeXtWholeBackCertifiedTie

ConvNeXt-T's whole-net backward tie — the stage fold, and ⛔ what the tie FOUND #

Read EvenKernelConvBack.lean first; the finding is the deliverable. This file was started as the ConvNeXt peer of ResNet-34's whole-net tie (today r34InputGradB_eq_r34B_full_vjp) — planning/archive/float_budget_numbers_log.md §3.18, done BEFORE the number rather than after it, because §3.10's tie found r34 reversing the wrong pool and moved a committed number 4×. It paid out the same way at the first leaf it touched.

⛔⛔ WHAT THE TIE FOUND: convFlatBack IS NOT THE ADJOINT AT AN EVEN KERNEL, and ConvNeXt is the only net in the repo that has one. conv2d pads by pH = (kH-1)/2, so the reversed-kernel forward conv is the adjoint only when kH - 1 - pH = pH, i.e. only for odd kH. ConvNeXt's 4×4/s4 patchify stem and its three 2×2/s2 downsamples are the four sites where that fails; at kH = 4 the hand-written backward is the adjoint of a conv shifted one pixel. Every other net is all-odd (R34 7×7/3×3/1×1, MobileNetV2 and EfficientNet-B0 1×1/3×3/5×5), and ViT's 16×16 patch embed never routes through conv2d at all.

Nothing trained is affected, and the codegen tier already knew. StableHLO.lean's .convStridedBack pads ASYMMETRICALLY, [[kH-1-pH, pH]], in both the per-example (:6120) and the batched (:8248) arms, and its den is the certified VJP; the batched comment names the same quantity — "the symmetric [[p,p],[p,p]] … AGREES at every odd kernel and is WRONG at even ones (kH=2 ⇒ [[0,0]] where the VJP needs [[1,0]])". The fix landed on TWO tiers and never reached the third: BackwardMaps.lean's flatConvStride2Back / flatConvStride4Back, which are convFlatBack ∘ scatter at the SYMMETRIC pad. ⭐ That is imagenet_specs_drift_from_twins in its "a fix landed on one tier and its twin kept the old spelling" form, for the third time (§3.10's pool and §3.16's head LayerNorm were the first two).

What is here. The repair is padOdd (EvenKernelConvBack.lean): an even-kernel conv is an odd-kernel conv on the kernel zero-extended at (+1,+1), which is the emitter's asymmetric pad written in the vocabulary BackwardMaps.lean already has, so the existing odd-kernel leaf tie does all the work and no new conv machinery is needed. On top of it:

  1. cnxDownChBack_eq_vjp — the stage-boundary downsample tie, lnB ∘ flatConvStride2Back (padOdd W) against (cnxDownChW_has_vjp …).backward. ⛔ padOdd is load-bearing: p.W is 2×2. Two existing ties composed.
  2. cnxStageChKBack_eq_vjp — ⭐ the depth-k stage fold, §3.18's "one real proof". HasVJP for convNextStageChK is built head-first (block 0 runs first), so its backward composes the block backwards in the OPPOSITE order, each at its own saved activation, and the tail's saved input is block 0's forward OUTPUT. The induction step is one rewrite of the block tie (cnxBlockChBack_eq_vjp) and one of the inductive hypothesis.
  3. cnxSavedA0cnxSavedA10convNextForwardTCh's eleven stage inputs, named as FUNCTIONS, so that the same twelve constants are both the activations the backward's slots are saved at and the f argument of each vjp_comp.
  4. ⭐⭐ convnextInputGrad_eq_convNextForwardTCh_vjp — THE APEX. convnextInputGrad, with every slot pinned to the certified per-op backward at its own saved activation, IS (convNextForwardTCh_has_vjp …).backward x. The ConvNeXt peer of r34InputGradB_eq_r34B_full_vjp, and stronger: convNextForwardTCh_has_vjp is HasVJP — everywhere — not the smooth-point HasVJPAt that one is, because GELU, LayerNorm, convolution and the layer scale are all smooth and ConvNeXt has no kink anywhere. Its only hypotheses are the 23 LayerNorm positivities, so unlike every other whole-net backward tie in this repo it carries no smoothness side-condition. ⭐⭐ And ConvNeXt has its shape check too (planning/archive/float_budget_numbers_log.md §3.14): convNextForwardTCh_eq_chain, the rfl saying the chain the apex instantiates IS the committed forward, written before anyone needed it.

⭐⭐ WHAT THE ASSEMBLY COST, AND IT IS ONE RULE: never hand the unifier two spellings of the same thing in an APPLIED position. Every expensive step in this file was an instance, and each is free once the spelling is normalised at a definition:

planning/archive/float_budget_numbers_log.md §3.7(d) records this trap in its other guise, where the computed dimension meets a metavariable (2 * ?h = 112) and the unification is higher-order; there the fix is to pin the implicit. Here h is given explicitly and it still costs — two CLOSED spellings of one numeral are enough. ⛔ And it is invisible in an unapplied position: convNextForwardTCh_vjp_chain's ascription compares the whole twelve-factor composition against the committed one and is free, because no x is in sight to evaluate.

The other half of the shape is the term-mode chain, and it is not a preference. convNextForwardTCh_has_vjp is a tactic proof, so its eleven haves are letFun and its .backward does not reduce; the whole-net rfl against it returned no result at maxHeartbeats 8000000, twice, ~8 min each. HasVJP.backward_unique transfers through .correct instead, which costs nothing, and the term-mode peer must be top-level defs rather than a let chain — a let used twice per level zeta-expands to 2^11 copies of the prefix.

⚠ ResNet-34's shape check is resnet34ForwardB_full_eq_slots (ResNet34BackCertifiedTieB.lean), and it is the net the hole first bit.

theorem Proofs.cnxDownChBack_eq_vjp {cin cout h w : } (p : CnxDownParamsCh cin cout) ( : 0 < p.ε) (v : Vec (cin * (2 * h) * (2 * w))) :
cnxDownBack (padOdd p.W) (chanLNTensor3Back cin (2 * h) (2 * w) p.ε p.γ v) = (cnxDownChW_has_vjp h w p ).backward v

The downsample backward tie. cnxDownBack (padOdd p.W) lnB — the strided-conv backward at the ZERO-EXTENDED kernel, then the channel-LN back at the input resolution — is (cnxDownChW_has_vjp h w p hε).backward v.

padOdd is load-bearing and not cosmetic: p.W is 2×2, so cnxDownBack p.W reverses a conv shifted one pixel (EvenKernelConvBack.lean). This is one of the four sites the whole-net tie found.

noncomputable def Proofs.cnxBlockChBackAt {c cExp h w kHd kWd : } (p : CnxBlockParamsCh c cExp h w kHd kWd) (v : Vec (c * h * w)) :
Vec (c * h * w)Vec (c * h * w)

One channel-LN ConvNeXt block's backward at a saved input v — exactly the left-hand side of cnxBlockChBack_eq_vjp, named so the stage recursion can be written down.

Equations
  • One or more equations did not get rendered due to their size.
Instances For
    noncomputable def Proofs.cnxStageChKBack {c cExp h w kH kW : } (k : ) (ps : Fin kCnxBlockParamsCh c cExp h w kH kW) :
    Vec (c * h * w)Vec (c * h * w)Vec (c * h * w)

    The depth-k stage backward, at a saved stage input v.

    Head-first, like the forward it reverses. convNextStageChK (k+1) ps = convNextStageChK k (ps ∘ succ) ∘ cnxBlockChW (ps 0) applies block 0 FIRST, so the backward applies block 0's reverse LAST — cnxBlockChBackAt (ps 0) v ∘ (the rest). And the saved activation threads forward through the recursion: the tail's saved input is cnxBlockChW (ps 0) v, block 0's OUTPUT. Getting either of those backwards is the §3.3-lesson-2 trap (the stage fold once associated the other way), and it is the DEFINITION that decides, never the analogy.

    Equations
    Instances For
      theorem Proofs.cnxStageChKBack_eq_vjp {c cExp h w kHd kWd : } (hkHd : 2 * ((kHd - 1) / 2) + 1 = kHd) (hkWd : 2 * ((kWd - 1) / 2) + 1 = kWd) (k : ) (ps : Fin kCnxBlockParamsCh c cExp h w kHd kWd) ( : ∀ (i : Fin k), 0 < (ps i).εn) (v : Vec (c * h * w)) :

      ⭐⭐ THE STAGE-FOLD TIE. The hand-composed depth-k stage backward IS (convNextStageChK_has_vjp k ps hε).backward. Induction on k: the base case is identity_has_vjp's fun _ dy => dy, and the step is one rewrite of the block tie (cnxBlockChBack_eq_vjp) and one of the inductive hypothesis at the shifted saved activation.

      theorem Proofs.rowLNVecFlat_has_vjp_backward_eq_fun {s c : } (ε : ) ( : 0 < ε) (γ β : Vec c) (X : Vec (s * c)) :
      rowLNVecFlatBack s c ε γ X = (rowLNVecFlat_has_vjp s c ε γ β ).backward X

      rowLNVecFlat_has_vjp_backward_eq at the FUNCTION level — the direction and shape a whole-net rw needs. The committed lemma is pointwise in dy and oriented certified-to-hand; a chain rewrite wants hand-to-certified with dy abstracted.

      ⭐⭐ Every stage whose declared type carries a COMPUTED dimension gets a wrapper here, and that is the whole reason this section closes in seconds. cnxDownChW h w p is declared over Vec (cin * (2 * h) * (2 * w)); the chain spells the same type Vec (96 * 56 * 56). Both are closed terms and they are equal, but in an APPLIED position the unifier does not reduce 2 * 28 to 56 — it descends into the semantics of both sides instead, and the diagnostics name what it reaches: conv2d_input_grad_formula, Finset.sum, Mat.unflatten, cnxBlockChW. Measured, one tie at a time: the three downsample links cost 3 s, 15 s and then do not finish, while every stage, LayerNorm, GAP and dense link is free. With the wrappers below — a one-line def per offending stage, its type ascribed in the chain's spelling, and its Differentiable/HasVJP peers ascribed the same way — the twelve chain defs and all eleven links together cost 2.9 s, of which the links are ~0.3 s.

      planning/archive/float_budget_numbers_log.md §3.7(d) records this trap in its other guise, where the computed dimension meets a metavariable (2 * ?h = 112) and the unification is higher-order. There the fix is to pin the implicit. Here h is already given explicitly and it still costs: two CLOSED spellings of one numeral are enough. ⛔ And it is invisible in an unapplied position — convNextForwardTCh_vjp_chain's ascription below compares the whole twelve-factor composition against the committed one and is free, because no x is in sight to evaluate.

      ⭐ The wrappers, their Differentiable/HasVJP peers, the stem's cnxD0/cnxV0 and the four normalised leaf ties below are PUBLIC: ConvNeXtWholeBackCertifiedTieB.lean lifts the same twelve stages over a batch and needs them at exactly these spellings.

      noncomputable def Proofs.cnxDn1 {nC : } (w : CnxTWeightsCh nC) :
      Vec (96 * 56 * 56)Vec (192 * 28 * 28)

      Downsample 1 at the chain's dimension spelling.

      Equations
      Instances For
        theorem Proofs.cnxDn1Diff {nC : } (w : CnxTWeightsCh nC) (hd1 : 0 < w.d1.ε) :
        noncomputable def Proofs.cnxDn1Vjp {nC : } (w : CnxTWeightsCh nC) (hd1 : 0 < w.d1.ε) :
        Equations
        Instances For
          noncomputable def Proofs.cnxDn2 {nC : } (w : CnxTWeightsCh nC) :
          Vec (192 * 28 * 28)Vec (384 * 14 * 14)

          Downsample 2 at the chain's dimension spelling.

          Equations
          Instances For
            theorem Proofs.cnxDn2Diff {nC : } (w : CnxTWeightsCh nC) (hd2 : 0 < w.d2.ε) :
            noncomputable def Proofs.cnxDn2Vjp {nC : } (w : CnxTWeightsCh nC) (hd2 : 0 < w.d2.ε) :
            Equations
            Instances For
              noncomputable def Proofs.cnxDn3 {nC : } (w : CnxTWeightsCh nC) :
              Vec (384 * 14 * 14)Vec (768 * 7 * 7)

              Downsample 3 at the chain's dimension spelling.

              Equations
              Instances For
                theorem Proofs.cnxDn3Diff {nC : } (w : CnxTWeightsCh nC) (hd3 : 0 < w.d3.ε) :
                noncomputable def Proofs.cnxDn3Vjp {nC : } (w : CnxTWeightsCh nC) (hd3 : 0 < w.d3.ε) :
                Equations
                Instances For
                  noncomputable def Proofs.cnxLNh {nC : } (w : CnxTWeightsCh nC) :
                  Vec 768Vec 768

                  The head LayerNorm at Vec 768, not Vec (1 * 768) — the same normalisation, at the one site where the computed dimension is a 1 * rather than a 2 *.

                  Equations
                  Instances For
                    theorem Proofs.cnxLNhDiff {nC : } (w : CnxTWeightsCh nC) (hhε : 0 < w.) :
                    noncomputable def Proofs.cnxLNhVjp {nC : } (w : CnxTWeightsCh nC) (hhε : 0 < w.) :
                    Equations
                    Instances For

                      convNextForwardTCh's eleven stage inputs, named — and named as FUNCTIONS, so that the same twelve constants are both the saved activations the backward's slots are indexed by and the f argument of each vjp_comp. That is what makes every link below a one-step iota with syntactically identical sides: the alternative — an applied cnxSavedA k w x on one side and the chain's own f x on the other — is defeq, and identifying the two costs 2 s at depth one and does not finish at depth two.

                      noncomputable def Proofs.cnxSavedA0 {nC : } (w : CnxTWeightsCh nC) :
                      Vec (3 * 224 * 224)Vec (96 * 56 * 56)

                      The stem conv's output — the stem LayerNorm's saved input.

                      Equations
                      Instances For
                        noncomputable def Proofs.cnxV0 {nC : } (w : CnxTWeightsCh nC) :
                        Equations
                        Instances For
                          noncomputable def Proofs.cnxSavedA1 {nC : } (w : CnxTWeightsCh nC) :
                          Vec (3 * 224 * 224)Vec (96 * 56 * 56)

                          Stage 1's saved input.

                          Equations
                          Instances For
                            noncomputable def Proofs.cnxSavedA2 {nC : } (w : CnxTWeightsCh nC) :
                            Vec (3 * 224 * 224)Vec (96 * 56 * 56)

                            Downsample 1's saved input.

                            Equations
                            Instances For
                              noncomputable def Proofs.cnxSavedA3 {nC : } (w : CnxTWeightsCh nC) :
                              Vec (3 * 224 * 224)Vec (192 * 28 * 28)

                              Stage 2's saved input.

                              Equations
                              Instances For
                                noncomputable def Proofs.cnxSavedA4 {nC : } (w : CnxTWeightsCh nC) :
                                Vec (3 * 224 * 224)Vec (192 * 28 * 28)

                                Downsample 2's saved input.

                                Equations
                                Instances For
                                  noncomputable def Proofs.cnxSavedA5 {nC : } (w : CnxTWeightsCh nC) :
                                  Vec (3 * 224 * 224)Vec (384 * 14 * 14)

                                  Stage 3's saved input.

                                  Equations
                                  Instances For
                                    noncomputable def Proofs.cnxSavedA6 {nC : } (w : CnxTWeightsCh nC) :
                                    Vec (3 * 224 * 224)Vec (384 * 14 * 14)

                                    Downsample 3's saved input.

                                    Equations
                                    Instances For
                                      noncomputable def Proofs.cnxSavedA7 {nC : } (w : CnxTWeightsCh nC) :
                                      Vec (3 * 224 * 224)Vec (768 * 7 * 7)

                                      Stage 4's saved input.

                                      Equations
                                      Instances For
                                        noncomputable def Proofs.cnxSavedA8 {nC : } (w : CnxTWeightsCh nC) :
                                        Vec (3 * 224 * 224)Vec (768 * 7 * 7)

                                        GAP's saved input.

                                        Equations
                                        Instances For
                                          noncomputable def Proofs.cnxSavedA9 {nC : } (w : CnxTWeightsCh nC) :
                                          Vec (3 * 224 * 224)Vec 768

                                          The head LayerNorm's saved input.

                                          Equations
                                          Instances For
                                            noncomputable def Proofs.cnxSavedA10 {nC : } (w : CnxTWeightsCh nC) :
                                            Vec (3 * 224 * 224)Vec 768

                                            The classifier's saved input.

                                            Equations
                                            Instances For
                                              theorem Proofs.cnxDn1Back_eq_vjp {nC : } (w : CnxTWeightsCh nC) (hd1 : 0 < w.d1.ε) (v : Vec (96 * 56 * 56)) :
                                              cnxDownBack (padOdd w.d1.W) (chanLNTensor3Back 96 56 56 w.d1.ε w.d1.γ v) = (cnxDn1Vjp w hd1).backward v
                                              theorem Proofs.cnxDn2Back_eq_vjp {nC : } (w : CnxTWeightsCh nC) (hd2 : 0 < w.d2.ε) (v : Vec (192 * 28 * 28)) :
                                              cnxDownBack (padOdd w.d2.W) (chanLNTensor3Back 192 28 28 w.d2.ε w.d2.γ v) = (cnxDn2Vjp w hd2).backward v
                                              theorem Proofs.cnxDn3Back_eq_vjp {nC : } (w : CnxTWeightsCh nC) (hd3 : 0 < w.d3.ε) (v : Vec (384 * 14 * 14)) :
                                              cnxDownBack (padOdd w.d3.W) (chanLNTensor3Back 384 14 14 w.d3.ε w.d3.γ v) = (cnxDn3Vjp w hd3).backward v
                                              theorem Proofs.cnxLNhBack_eq_vjp {nC : } (w : CnxTWeightsCh nC) (hhε : 0 < w.) (v : Vec (1 * 768)) :
                                              rowLNVecFlatBack 1 768 w. w. v = (cnxLNhVjp w hhε).backward v
                                              noncomputable def Proofs.convNextForwardTCh_vjp_chain {nC : } (w : CnxTWeightsCh nC) (hsε : 0 < w.) (h1 : ∀ (i : Fin 3), 0 < (w.s1 i).εn) (hd1 : 0 < w.d1.ε) (h2 : ∀ (i : Fin 3), 0 < (w.s2 i).εn) (hd2 : 0 < w.d2.ε) (h3 : ∀ (i : Fin 9), 0 < (w.s3 i).εn) (hd3 : 0 < w.d3.ε) (h4 : ∀ (i : Fin 3), 0 < (w.s4 i).εn) (hhε : 0 < w.) :

                                              convNextForwardTCh_has_vjp as a TERM-mode vjp_comp chain.

                                              Equations
                                              Instances For
                                                theorem Proofs.convnextInputGrad_eq_convNextForwardTCh_vjp {nC : } (w : CnxTWeightsCh nC) (hsε : 0 < w.) (h1 : ∀ (i : Fin 3), 0 < (w.s1 i).εn) (hd1 : 0 < w.d1.ε) (h2 : ∀ (i : Fin 3), 0 < (w.s2 i).εn) (hd2 : 0 < w.d2.ε) (h3 : ∀ (i : Fin 9), 0 < (w.s3 i).εn) (hd3 : 0 < w.d3.ε) (h4 : ∀ (i : Fin 3), 0 < (w.s4 i).εn) (hhε : 0 < w.) (x : Vec (3 * 224 * 224)) :
                                                convnextInputGrad w.Wd (padOdd w.sW) (chanLNTensor3Back 96 56 56 w. w. (cnxSavedA0 w x)) (rowLNVecFlatBack 1 768 w. w. (cnxSavedA9 w x)) (cnxStageChKBack 3 w.s1 (cnxSavedA1 w x)) (cnxDownBack (padOdd w.d1.W) (chanLNTensor3Back 96 56 56 w.d1.ε w.d1.γ (cnxSavedA2 w x))) (cnxStageChKBack 3 w.s2 (cnxSavedA3 w x)) (cnxDownBack (padOdd w.d2.W) (chanLNTensor3Back 192 28 28 w.d2.ε w.d2.γ (cnxSavedA4 w x))) (cnxStageChKBack 9 w.s3 (cnxSavedA5 w x)) (cnxDownBack (padOdd w.d3.W) (chanLNTensor3Back 384 14 14 w.d3.ε w.d3.γ (cnxSavedA6 w x))) (cnxStageChKBack 3 w.s4 (cnxSavedA7 w x)) = (convNextForwardTCh_has_vjp w hsε h1 hd1 h2 hd2 h3 hd3 h4 hhε).backward x

                                                ⭐⭐ convnextInputGrad IS the certified whole-net ConvNeXt-T gradient.