Documentation

LeanMlir.Proofs.Nets.MobileNet.MobileNetV2FullBVJP

MobileNetV2's whole-net input-VJP at batch BatchNorm #

MobileNetV2FullB.lean states the batch-BN forward and its typed graph. This file gives that forward a certified HasVJPAt at the paper depth — the MobileNetV2 peer of ResNet34FullBVJP.lean.

Built from the block VJPs #

Every block VJP is already proven at bnBatchLA: mnv2BodyBHasVJPAt and mnv2DownBodyBHasVJPAt (MobileNetV2BackB0.lean) are exactly the two body shapes mnv2ExpOnlyB / mnv2StridedB unfold to, and residualHasVJPAt wraps the first for the ten skip blocks. The bundle lemmas below are delegations in the EfficientNetFullB0 style.

No pointwise batchMap lemma is needed. ResNet-34's stem ends in batchMap N (maxPool3s2Flat …) and a max-pool has no derivative at a tie, so it needs batchMapHasVJPAt. MobileNetV2 has NO stem pool — the stem is conv-BN-relu6 and downsamples once — and its head's GAP and dense are smooth, so batchMapHasVJP (the global one) covers every batchMap in the net.

Three shapes are bnRelu6StageHasVJPAt at a different inner op, and that lemma is already generic in it: the stem is that stage at flatConvStride2Xla, the head's first stage is cbrB, and the expand/depthwise stages are cbrB / dwbrB / dwbrBstrided. The t = 1 block projB ∘ dwbrB (b1) has no mnv2*BodyB peer; it is dwbrLayer ; projLayer.

The hypothesis budget #

Pointwise (HasVJPAt), not global. relu6 is kinked on both sides, so each site carries ≠ 0 ∧ ≠ 6 and a global HasVJP through it is false.

Two kink clauses per bottleneck, and the second is not r34's. ResNet-34's blocks carry the body's mid-relu AND a post-residual OUTER relu. MobileNetV2's carry the EXPAND relu6 and the DEPTHWISE relu6, both inside the body — the linear bottleneck has no activation after project, so the residual add IS the block output and contributes nothing. Sixteen expand-bearing blocks give 32 clauses, plus b1's single depthwise clause, plus the stem's and the head's: 35 relu6 sites, bundled into 19 binders.

The positivity bundles are IVPos / IVNoExpPos (MobileNetV2FullPaper.lean): 0 < ε at each BatchNorm site. The smoothness bundles are declared here, because a kink condition names the activation, which is bnBatchLA's output.

The running activations are named mnv2PreB0 … mnv2PreB17 so each bundle can be STATED at the activation entering its block without a seventeen-deep nested application inline; mnv2PreB17 doubles as the trunk, and mobilenetv2ForwardBFull_eq_chain bridges it back to the committed nested-application forward.

N is a variable throughout.

structure Proofs.IVSmoothAtB (N h w : ℕ) {ic mid oc : ℕ} (q : IVW ic mid oc) (v : Vec (N * (ic * h * w))) :

Both relu6 sites of a stride-1 bottleneck are away from BOTH kinks at v, at batch BN: the expand-BN output and the depthwise-BN output each avoid 0 and 6 in every coordinate.

Instances For
    structure Proofs.IVStridedSmoothAtB (N h w : ℕ) {ic mid oc : ℕ} (q : IVW ic mid oc) (v : Vec (N * (ic * (2 * h) * (2 * w)))) :

    Both relu6 sites of a stride-2 bottleneck are away from both kinks at v, at batch BN (the expand runs at the pre-downsample 2h x 2w grid, the XLA-SAME depthwise at h x w).

    Instances For
      structure Proofs.IVNoExpSmoothAtB (N h w : ℕ) {ic oc : ℕ} (q : IVWNoExp ic oc) (v : Vec (N * (ic * h * w))) :

      The single relu6 site of the t = 1 bottleneck (depthwise-BN output) is away from both kinks at batch BN.

      Instances For
        def Proofs.MNV2StemSmoothAtB (N h w : ℕ) {ic oc kH kW : ℕ} (Ws : Kernel4 oc ic kH kW) (bs : Vec oc) (εs : ℝ) (γs βs : Vec oc) (x : Vec (N * (ic * (2 * h) * (2 * w)))) :

        The stem's relu6 is away from both kinks at the input x, at batch BN.

        Equations
        • One or more equations did not get rendered due to their size.
        Instances For
          def Proofs.MNV2HeadSmoothAtB (N h w : ℕ) {ic oc : ℕ} (Wh : Kernel4 oc ic 1 1) (bh : Vec oc) (εh : ℝ) (γh βh : Vec oc) (v : Vec (N * (ic * h * w))) :

          The head's relu6 is away from both kinks at the trunk output v, at batch BN.

          Equations
          • One or more equations did not get rendered due to their size.
          Instances For
            noncomputable def Proofs.mnv2StemBHasVJPAt (N h w : ℕ) {ic oc kH kW : ℕ} (Ws : Kernel4 oc ic kH kW) (bs : Vec oc) (εs : ℝ) (hεs : 0 < εs) (γs βs : Vec oc) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (hs : MNV2StemSmoothAtB N h w Ws bs εs γs βs x) :
            HasVJPAt (mnv2StemB N h w Ws bs εs γs βs) x

            Stem VJP: bnRelu6StageHasVJPAt at the XLA-SAME strided conv. That lemma takes the inner op as a parameter, so the stride-2 stem is the same construction as every stride-1 stage.

            Equations
            Instances For
              theorem Proofs.mnv2StemB_differentiableAt (N h w : ℕ) {ic oc kH kW : ℕ} (Ws : Kernel4 oc ic kH kW) (bs : Vec oc) (εs : ℝ) (hεs : 0 < εs) (γs βs : Vec oc) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (hs : MNV2StemSmoothAtB N h w Ws bs εs γs βs x) :
              DifferentiableAt ℝ (mnv2StemB N h w Ws bs εs γs βs) x
              noncomputable def Proofs.mnv2NoExpBHasVJPAt (N h w : ℕ) {ic oc : ℕ} (p : IVWNoExp ic oc) (hq : IVNoExpPos p) (v : Vec (N * (ic * h * w))) (hs : IVNoExpSmoothAtB N h w p v) :
              HasVJPAt (mnv2NoExpB N h w p) v

              t = 1 bottleneck VJP (b1): projB ∘ dwbrB, the VJP of dwbrLayer ; projLayer. The one block shape with no mnv2*BodyB lemma to delegate to.

              Equations
              Instances For
                noncomputable def Proofs.mnv2ExpOnlyBHasVJPAt (N h w : ℕ) {ic mid oc : ℕ} (p : IVW ic mid oc) (hq : IVPos p) (v : Vec (N * (ic * h * w))) (hs : IVSmoothAtB N h w p v) :

                Stride-1 no-skip bottleneck VJP (b11, b17) — mnv2BodyBHasVJPAt at the bundle's fields.

                Equations
                Instances For
                  noncomputable def Proofs.mnv2ResidBHasVJPAt (N h w : ℕ) {c mid : ℕ} (p : IVW c mid c) (hq : IVPos p) (v : Vec (N * (c * h * w))) (hs : IVSmoothAtB N h w p v) :
                  HasVJPAt (mnv2ResidB N h w p) v

                  Stride-1 skip bottleneck VJP — the body VJP under residualHasVJPAt. The identity arm is smooth everywhere, so the skip adds no hypothesis.

                  Equations
                  Instances For
                    noncomputable def Proofs.mnv2StridedBHasVJPAt (N h w : ℕ) {ic mid oc : ℕ} (p : IVW ic mid oc) (hq : IVPos p) (v : Vec (N * (ic * (2 * h) * (2 * w)))) (hs : IVStridedSmoothAtB N h w p v) :

                    Stride-2 downsampling bottleneck VJP — mnv2DownBodyBHasVJPAt at the bundle's fields.

                    Equations
                    Instances For
                      noncomputable def Proofs.mnv2HeadBHasVJPAt (N h w : ℕ) {ic oc nCls : ℕ} (Wh : Kernel4 oc ic 1 1) (bh : Vec oc) (εh : ℝ) (hεh : 0 < εh) (γh βh : Vec oc) (Wd : Mat oc nCls) (bd : Vec nCls) (v : Vec (N * (ic * h * w))) (hs : MNV2HeadSmoothAtB N h w Wh bh εh γh βh v) :
                      HasVJPAt (mnv2HeadB N h w Wh bh εh γh βh Wd bd) v

                      Head VJP: the 1x1 conv-bn-relu6 stage (cbrB, pointwise), then GAP and dense — both smooth, both batchMap of a per-example op, so both lift with the GLOBAL batchMapHasVJP. Unlike r34's, this head is not hypothesis-free: MobileNetV2 puts a relu6 in front of the pool, so the head carries the net's 35th kink site.

                      Equations
                      • One or more equations did not get rendered due to their size.
                      Instances For
                        noncomputable def Proofs.mnv2PreB0 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                        Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (32 * 112 * 112))
                        Equations
                        Instances For
                          noncomputable def Proofs.mnv2PreB1 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                          Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (16 * 112 * 112))
                          Equations
                          Instances For
                            noncomputable def Proofs.mnv2PreB2 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                            Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (24 * 56 * 56))
                            Equations
                            Instances For
                              noncomputable def Proofs.mnv2PreB3 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                              Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (24 * 56 * 56))
                              Equations
                              Instances For
                                noncomputable def Proofs.mnv2PreB4 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                                Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (32 * 28 * 28))
                                Equations
                                Instances For
                                  noncomputable def Proofs.mnv2PreB5 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                                  Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (32 * 28 * 28))
                                  Equations
                                  Instances For
                                    noncomputable def Proofs.mnv2PreB6 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                                    Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (32 * 28 * 28))
                                    Equations
                                    Instances For
                                      noncomputable def Proofs.mnv2PreB7 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                                      Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (64 * 14 * 14))
                                      Equations
                                      Instances For
                                        noncomputable def Proofs.mnv2PreB8 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                                        Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (64 * 14 * 14))
                                        Equations
                                        Instances For
                                          noncomputable def Proofs.mnv2PreB9 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                                          Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (64 * 14 * 14))
                                          Equations
                                          Instances For
                                            noncomputable def Proofs.mnv2PreB10 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                                            Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (64 * 14 * 14))
                                            Equations
                                            Instances For
                                              noncomputable def Proofs.mnv2PreB11 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                                              Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (96 * 14 * 14))
                                              Equations
                                              Instances For
                                                noncomputable def Proofs.mnv2PreB12 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                                                Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (96 * 14 * 14))
                                                Equations
                                                Instances For
                                                  noncomputable def Proofs.mnv2PreB13 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                                                  Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (96 * 14 * 14))
                                                  Equations
                                                  Instances For
                                                    noncomputable def Proofs.mnv2PreB14 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                                                    Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (160 * 7 * 7))
                                                    Equations
                                                    Instances For
                                                      noncomputable def Proofs.mnv2PreB15 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                                                      Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (160 * 7 * 7))
                                                      Equations
                                                      Instances For
                                                        noncomputable def Proofs.mnv2PreB16 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                                                        Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (160 * 7 * 7))
                                                        Equations
                                                        Instances For
                                                          noncomputable def Proofs.mnv2PreB17 (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) :
                                                          Vec (N * (3 * (2 * 112) * (2 * 112))) → Vec (N * (320 * 7 * 7))
                                                          Equations
                                                          Instances For
                                                            structure Proofs.MNV2PosB {nCls : ℕ} (w : MNV2BWeights nCls) :

                                                            Every BatchNorm ε of the net is positive: the stem's, the head's, and each block's bundle.

                                                            Instances For
                                                              structure Proofs.MNV2SmoothAtB (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :

                                                              Every relu6 is away from its kinks, each at the activation its block actually sees: the stem's clauses at the image, block k's at mnv2PreB(k-1), the head's at the trunk's output.

                                                              Instances For
                                                                noncomputable def Proofs.mobilenetv2ForwardBFullHasVJPAt (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (hq : MNV2PosB w) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) (hx : MNV2SmoothAtB N w x) :
                                                                HasVJPAt (mnv2HeadB N 7 7 w.hW w.hb w.hε w.hγ w.hβ w.fcW w.fcb ∘ mnv2PreB17 N w) x

                                                                MobileNetV2 at batch BatchNorm has a certified input-VJP at a smooth point — all seventeen bottlenecks. Chains stem → the [t,c,n,s] ladder → head with vjpCompDiffAt under two hypotheses: MNV2PosB (every ε > 0) and MNV2SmoothAtB (every relu6 clause, each at its block's own input).

                                                                Pointwise, because relu6 is kinked on both sides. Each expand-bearing block contributes two clauses — the expand relu6 and the depthwise relu6, both INSIDE the body — where ResNet-34's basic block contributes a mid-relu and a post-residual OUTER relu. The linear bottleneck has no activation after project, so MobileNetV2's residual add is the block output and adds nothing.

                                                                Unlike r34's, the head is not hypothesis-free: its 1x1 conv-BN is followed by a relu6. N is a variable.

                                                                Equations
                                                                Instances For
                                                                  theorem Proofs.mnv2PreB0_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB0 N w x = mnv2StemB N 112 112 w.sW w.sb w.sε w.sγ w.sβ x
                                                                  theorem Proofs.mnv2PreB1_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB1 N w x = mnv2NoExpB N 112 112 w.b1 (mnv2PreB0 N w x)
                                                                  theorem Proofs.mnv2PreB2_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB2 N w x = mnv2StridedB N 56 56 w.b2 (mnv2PreB1 N w x)
                                                                  theorem Proofs.mnv2PreB3_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB3 N w x = mnv2ResidB N 56 56 w.b3 (mnv2PreB2 N w x)
                                                                  theorem Proofs.mnv2PreB4_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB4 N w x = mnv2StridedB N 28 28 w.b4 (mnv2PreB3 N w x)
                                                                  theorem Proofs.mnv2PreB5_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB5 N w x = mnv2ResidB N 28 28 w.b5 (mnv2PreB4 N w x)
                                                                  theorem Proofs.mnv2PreB6_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB6 N w x = mnv2ResidB N 28 28 w.b6 (mnv2PreB5 N w x)
                                                                  theorem Proofs.mnv2PreB7_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB7 N w x = mnv2StridedB N 14 14 w.b7 (mnv2PreB6 N w x)
                                                                  theorem Proofs.mnv2PreB8_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB8 N w x = mnv2ResidB N 14 14 w.b8 (mnv2PreB7 N w x)
                                                                  theorem Proofs.mnv2PreB9_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB9 N w x = mnv2ResidB N 14 14 w.b9 (mnv2PreB8 N w x)
                                                                  theorem Proofs.mnv2PreB10_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB10 N w x = mnv2ResidB N 14 14 w.b10 (mnv2PreB9 N w x)
                                                                  theorem Proofs.mnv2PreB11_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB11 N w x = mnv2ExpOnlyB N 14 14 w.b11 (mnv2PreB10 N w x)
                                                                  theorem Proofs.mnv2PreB12_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB12 N w x = mnv2ResidB N 14 14 w.b12 (mnv2PreB11 N w x)
                                                                  theorem Proofs.mnv2PreB13_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB13 N w x = mnv2ResidB N 14 14 w.b13 (mnv2PreB12 N w x)
                                                                  theorem Proofs.mnv2PreB14_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB14 N w x = mnv2StridedB N 7 7 w.b14 (mnv2PreB13 N w x)
                                                                  theorem Proofs.mnv2PreB15_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB15 N w x = mnv2ResidB N 7 7 w.b15 (mnv2PreB14 N w x)
                                                                  theorem Proofs.mnv2PreB16_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB16 N w x = mnv2ResidB N 7 7 w.b16 (mnv2PreB15 N w x)
                                                                  theorem Proofs.mnv2PreB17_apply (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mnv2PreB17 N w x = mnv2ExpOnlyB N 7 7 w.b17 (mnv2PreB16 N w x)
                                                                  theorem Proofs.mobilenetv2ForwardBFull_eq_chain (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  mobilenetv2ForwardBFull N w x = (mnv2HeadB N 7 7 w.hW w.hb w.hε w.hγ w.hβ w.fcW w.fcb ∘ mnv2PreB17 N w) x

                                                                  The committed nested-application forward is the layered chain the VJP is stated on — what lets the VJP be about mobilenetv2ForwardBFull rather than about a re-spelling of it.

                                                                  theorem Proofs.mobilenetv2ForwardBFullHasVJPAt_correct (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (hq : MNV2PosB w) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) (hx : MNV2SmoothAtB N w x) (dy : Vec (N * nCls)) (i : Fin (N * (3 * (2 * 112) * (2 * 112)))) :
                                                                  (mobilenetv2ForwardBFullHasVJPAt N w hq x hx).backward dy i = ∑ j : Fin (N * nCls), pdiv (mobilenetv2ForwardBFull N w) x i j * dy j

                                                                  The .correct field of mobilenetv2ForwardBFullHasVJPAt, restated at the nested forward mobilenetv2ForwardBFull (the one mobilenetv2FwdGraphBFull_faithful proves the typed graph denotes) through mobilenetv2ForwardBFull_eq_chain: for weights with MNV2PosB and an input with MNV2SmoothAtB, its backward is the pdiv-contracted Jacobian. The tie of the hand-written input-gradient chain to this backward is mnv2InputGradB_eq_mobilenetv2B_full_vjp.

                                                                  theorem Proofs.mobilenetv2ForwardBFull_differentiableAt (N : ℕ) {nCls : ℕ} (w : MNV2BWeights nCls) (hq : MNV2PosB w) (x : Vec (N * (3 * (2 * 112) * (2 * 112)))) (hx : MNV2SmoothAtB N w x) :

                                                                  The committed forward is differentiable at every smooth point — the chain's .snd, read back through mobilenetv2ForwardBFull_eq_chain. What the seal's seal_differentiableAt needs.