Documentation

LeanMlir.Proofs.Nets.ResNet.ResNet34FullBVJP

ResNet-34's whole-net input-VJP at TRUE BATCH-NORM #

ResNet34FullB.lean states the batch-BN forward and its typed graph. This file gives that forward a certified HasVJPAt at the paper depth — the peer of what MobileNetV2FullBVJP.lean does for MobileNetV2's seventeen bottlenecks.

No new mathematics, and one lemma one tier down #

Every block VJP is already proven at bnBatchLA: r34BasicBlockBHasVJPAt and r34DownBlockBHasVJPAt (ResNet34BackB0.lean) are exactly the two block shapes ResNet34FullB.lean's r34IdB / r34DownB unfold to. The bundle lemmas below are delegations in the EfficientNetFullB0 style, and the whole net is those blocks' CertLayers composed.

The one lemma this needs below the net is batchMapHasVJPAt (Foundation/BatchMapVJPAt.lean): r34's stem ends in batchMap N (maxPool3s2Flat 64 56 56) and a max-pool has no derivative at a tie, so the GLOBAL batchMapHasVJP cannot lift it. With the per-example pool VJP at a Vec point (maxPool3s2FlatHasVJPAtVec), the batched pool is two lines.

The hypothesis budget #

Pointwise (HasVJPAt), not global, and necessarily. relu is kinked. And r34 carries two kink clauses per block, not one: the body's mid-relu AND the post-residual outer relu. That outer relu is ResNet's structural difference from MobileNetV2/EfficientNet, whose residual add IS the block output. Sixteen blocks therefore carry 32 clauses, plus the stem's relu and the stem pool's condition (every window's maximum at one position, or the window all zero) — bundled per block into R34IdSmoothAt / R34DownSmoothAt, and those 18 bundles into one R34SmoothAtB (the positivity bundles into R34PosB), so the apex binds two hypotheses, exactly as MobileNetV2FullBVJP.lean's MNV2SmoothAtB / MNV2PosB.

The pool's condition is per example (∀ r : Fin N, MaxPool3s2Smooth … on that example's row), because a tie is a property of one image's window, not of the batch. That is the shape batchMapHasVJPAt consumes.

The running activations are named r34Pre0 … r34Pre16 so each bundle can be STATED at the activation entering its block without a sixteen-deep nested application inline; r34Pre16 doubles as the trunk, and resnet34ForwardBFull_eq_chain bridges it back to the committed nested-application forward.

N is a variable throughout: the batch size is never pinned (the 224-px resolution and the 64…512 widths are literals).

structure Proofs.R34IdPos {c : ℕ} (q : R34IdW c) :

Both BN epsilons of an identity basic block are positive.

Instances For
    structure Proofs.R34DownPos {ic oc : ℕ} (q : R34DownW ic oc) :

    All three BN epsilons of a downsample basic block are positive (body twice, projection once).

    Instances For
      structure Proofs.R34IdSmoothAt (N h w : ℕ) {c : ℕ} (q : R34IdW c) (v : Vec (N * (c * h * w))) :

      Both relu sites of an identity basic block are away from the kink at v: the body's mid-relu (at the first BN's output) and the OUTER relu (at the post-residual sum).

      Instances For
        structure Proofs.R34DownSmoothAt (N h w : ℕ) {ic oc : ℕ} (q : R34DownW ic oc) (v : Vec (N * (ic * (2 * h) * (2 * w)))) :

        Both relu sites of a downsample basic block are away from the kink at v. The mid-relu is after the STRIDED conv's BN (already at h×w); the outer relu is at the projected residual.

        Instances For
          def Proofs.R34StemSmoothAt (N h w : ℕ) {ic oc : ℕ} (Ws : Kernel4 oc ic 7 7) (bs : Vec oc) (εs : ℝ) (γs βs : Vec oc) (x : Vec (N * (ic * (2 * (2 * h)) * (2 * (2 * w))))) :

          The stem's relu is away from the kink at the input x (the 7×7/s2 conv's BN output, at the pre-pool 2h×2w grid).

          Equations
          • One or more equations did not get rendered due to their size.
          Instances For
            noncomputable def Proofs.r34IdBHasVJPAt (N h w : ℕ) {c : ℕ} (p : R34IdW c) (hq : R34IdPos p) (v : Vec (N * (c * h * w))) (hs : R34IdSmoothAt N h w p v) :
            HasVJPAt (r34IdB N h w p) v

            Identity basic block VJP — r34BasicBlockBHasVJPAt at the bundle's fields.

            Equations
            Instances For
              noncomputable def Proofs.r34DownBHasVJPAt (N h w : ℕ) {ic oc : ℕ} (p : R34DownW ic oc) (hq : R34DownPos p) (v : Vec (N * (ic * (2 * h) * (2 * w)))) (hs : R34DownSmoothAt N h w p v) :
              HasVJPAt (r34DownB N h w p) v

              Downsample basic block VJP — r34DownBlockBHasVJPAt at the bundle's fields.

              Equations
              Instances For
                noncomputable def Proofs.r34StemBHasVJPAt (N h w : ℕ) {ic oc : ℕ} (Ws : Kernel4 oc ic 7 7) (bs : Vec oc) (εs : ℝ) (hεs : 0 < εs) (γs βs : Vec oc) (x : Vec (N * (ic * (2 * (2 * h)) * (2 * (2 * w))))) (hrelu : R34StemSmoothAt N h w Ws bs εs γs βs x) (hpool : StemPoolSmoothAt N h w (StableHLO.cbReluStridedB N Ws bs εs γs βs x)) :
                HasVJPAt (r34StemB N h w Ws bs εs γs βs) x

                Stem VJP: the 7×7/s2 conv-bn-relu, then the batched 3×3/s2 pool — stemReluPoolLayer's, certified as one layer so that a window of dead ReLUs is allowed.

                Equations
                Instances For
                  theorem Proofs.r34StemB_differentiableAt (N h w : ℕ) {ic oc : ℕ} (Ws : Kernel4 oc ic 7 7) (bs : Vec oc) (εs : ℝ) (hεs : 0 < εs) (γs βs : Vec oc) (x : Vec (N * (ic * (2 * (2 * h)) * (2 * (2 * w))))) (hrelu : R34StemSmoothAt N h w Ws bs εs γs βs x) (hpool : StemPoolSmoothAt N h w (StableHLO.cbReluStridedB N Ws bs εs γs βs x)) :
                  DifferentiableAt ℝ (r34StemB N h w Ws bs εs γs βs) x
                  noncomputable def Proofs.r34HeadBHasVJP (N h w : ℕ) {c nCls : ℕ} (Wd : Mat c nCls) (bd : Vec nCls) :
                  HasVJP (r34HeadB N h w Wd bd)

                  The head is GLOBAL — GAP and dense are both smooth everywhere, and each is batchMap of a per-example op, so batchMapHasVJP suffices and no smoothness hypothesis appears.

                  Equations
                  • One or more equations did not get rendered due to their size.
                  Instances For
                    theorem Proofs.r34HeadB_differentiable (N h w : ℕ) {c nCls : ℕ} (Wd : Mat c nCls) (bd : Vec nCls) :
                    noncomputable def Proofs.r34StemLayer (N h w : ℕ) {ic oc : ℕ} (Ws : Kernel4 oc ic 7 7) (bs : Vec oc) (εs : ℝ) (hεs : 0 < εs) (γs βs : Vec oc) :
                    StableHLO.CertLayer (N * (ic * (2 * (2 * h)) * (2 * (2 * w)))) (N * (oc * h * w))

                    The stem, 7×7/s2 conv-bn-relu then the 3×3/s2 pool, as a CertLayer (stemReluPoolLayer). Its ok is R34StemSmoothAt ∧ StemPoolSmoothAt at the conv's output.

                    Equations
                    Instances For
                      theorem Proofs.r34StemLayer_fwd (N h w : ℕ) {ic oc : ℕ} (Ws : Kernel4 oc ic 7 7) (bs : Vec oc) (εs : ℝ) (hεs : 0 < εs) (γs βs : Vec oc) :
                      (r34StemLayer N h w Ws bs εs hεs γs βs).fwd = r34StemB N h w Ws bs εs γs βs
                      noncomputable def Proofs.r34HeadLayer (N h w : ℕ) {c nCls : ℕ} (Wd : Mat c nCls) (bd : Vec nCls) :
                      StableHLO.CertLayer (N * (c * h * w)) (N * nCls)

                      GAP then the classifier, as a CertLayer. Globally certified.

                      Equations
                      Instances For
                        theorem Proofs.r34HeadLayer_fwd (N h w : ℕ) {c nCls : ℕ} (Wd : Mat c nCls) (bd : Vec nCls) :
                        (r34HeadLayer N h w Wd bd).fwd = r34HeadB N h w Wd bd
                        noncomputable def Proofs.r34Pre0 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                        Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (64 * 56 * 56))
                        Equations
                        Instances For
                          noncomputable def Proofs.r34Pre1 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                          Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (64 * 56 * 56))
                          Equations
                          Instances For
                            noncomputable def Proofs.r34Pre2 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                            Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (64 * 56 * 56))
                            Equations
                            Instances For
                              noncomputable def Proofs.r34Pre3 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                              Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (64 * 56 * 56))
                              Equations
                              Instances For
                                noncomputable def Proofs.r34Pre4 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                                Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (128 * 28 * 28))
                                Equations
                                Instances For
                                  noncomputable def Proofs.r34Pre5 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                                  Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (128 * 28 * 28))
                                  Equations
                                  Instances For
                                    noncomputable def Proofs.r34Pre6 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                                    Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (128 * 28 * 28))
                                    Equations
                                    Instances For
                                      noncomputable def Proofs.r34Pre7 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                                      Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (128 * 28 * 28))
                                      Equations
                                      Instances For
                                        noncomputable def Proofs.r34Pre8 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                                        Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (256 * 14 * 14))
                                        Equations
                                        Instances For
                                          noncomputable def Proofs.r34Pre9 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                                          Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (256 * 14 * 14))
                                          Equations
                                          Instances For
                                            noncomputable def Proofs.r34Pre10 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                                            Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (256 * 14 * 14))
                                            Equations
                                            Instances For
                                              noncomputable def Proofs.r34Pre11 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                                              Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (256 * 14 * 14))
                                              Equations
                                              Instances For
                                                noncomputable def Proofs.r34Pre12 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                                                Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (256 * 14 * 14))
                                                Equations
                                                Instances For
                                                  noncomputable def Proofs.r34Pre13 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                                                  Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (256 * 14 * 14))
                                                  Equations
                                                  Instances For
                                                    noncomputable def Proofs.r34Pre14 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                                                    Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (512 * 7 * 7))
                                                    Equations
                                                    Instances For
                                                      noncomputable def Proofs.r34Pre15 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                                                      Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (512 * 7 * 7))
                                                      Equations
                                                      Instances For
                                                        noncomputable def Proofs.r34Pre16 (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) :
                                                        Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) → Vec (N * (512 * 7 * 7))
                                                        Equations
                                                        Instances For
                                                          structure Proofs.R34PosB {nCls : ℕ} (w : R34BWeights nCls) :

                                                          Every BatchNorm ε of the net is positive: the stem's and each block's bundle.

                                                          Instances For
                                                            structure Proofs.R34SmoothAtB (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :

                                                            Every relu is away from its kink and every stem-pool window is smooth or dead, each at the activation its block actually sees: the stem's clauses at the image, block k's at r34Pre(k-1). The head has none (GAP and dense are smooth).

                                                            Instances For
                                                              theorem Proofs.r34Pre0_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre0 N w x = r34StemB N 56 56 w.sW w.sb w.sε w.sγ w.sβ x
                                                              theorem Proofs.r34Pre1_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre1 N w x = r34IdB N 56 56 w.a0 (r34Pre0 N w x)
                                                              theorem Proofs.r34Pre2_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre2 N w x = r34IdB N 56 56 w.a1 (r34Pre1 N w x)
                                                              theorem Proofs.r34Pre3_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre3 N w x = r34IdB N 56 56 w.a2 (r34Pre2 N w x)
                                                              theorem Proofs.r34Pre4_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre4 N w x = r34DownB N 28 28 w.d2 (r34Pre3 N w x)
                                                              theorem Proofs.r34Pre5_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre5 N w x = r34IdB N 28 28 w.b0 (r34Pre4 N w x)
                                                              theorem Proofs.r34Pre6_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre6 N w x = r34IdB N 28 28 w.b1 (r34Pre5 N w x)
                                                              theorem Proofs.r34Pre7_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre7 N w x = r34IdB N 28 28 w.b2 (r34Pre6 N w x)
                                                              theorem Proofs.r34Pre8_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre8 N w x = r34DownB N 14 14 w.d3 (r34Pre7 N w x)
                                                              theorem Proofs.r34Pre9_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre9 N w x = r34IdB N 14 14 w.c0 (r34Pre8 N w x)
                                                              theorem Proofs.r34Pre10_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre10 N w x = r34IdB N 14 14 w.c1 (r34Pre9 N w x)
                                                              theorem Proofs.r34Pre11_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre11 N w x = r34IdB N 14 14 w.c2 (r34Pre10 N w x)
                                                              theorem Proofs.r34Pre12_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre12 N w x = r34IdB N 14 14 w.c3 (r34Pre11 N w x)
                                                              theorem Proofs.r34Pre13_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre13 N w x = r34IdB N 14 14 w.c4 (r34Pre12 N w x)
                                                              theorem Proofs.r34Pre14_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre14 N w x = r34DownB N 7 7 w.d4 (r34Pre13 N w x)
                                                              theorem Proofs.r34Pre15_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre15 N w x = r34IdB N 7 7 w.e0 (r34Pre14 N w x)
                                                              theorem Proofs.r34Pre16_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              r34Pre16 N w x = r34IdB N 7 7 w.e1 (r34Pre15 N w x)
                                                              theorem Proofs.resnet34ForwardBFull_eq_chain (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                              resnet34ForwardBFull N w x = (r34HeadB N 7 7 w.Wd w.bd ∘ r34Pre16 N w) x

                                                              The committed nested-application forward IS the layered chain the VJP is stated on — the r34 peer of mobilenetv2ForwardBFull_eq_chain, and what lets the VJP be about resnet34ForwardBFull rather than about a re-spelling of it.

                                                              noncomputable def Proofs.r34NetLayer (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (hq : R34PosB w) :
                                                              StableHLO.CertLayer (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56)))) (N * nCls)

                                                              ResNet-34 as one certified layer, at every batch size. Its .faithful is a whole-net backward graph, stem pool included, proven to denote the VJP.

                                                              Equations
                                                              • One or more equations did not get rendered due to their size.
                                                              Instances For
                                                                theorem Proofs.r34NetLayer_fwd_apply (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (hq : R34PosB w) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :

                                                                The composed layer's forward IS the committed nested-application forward: peeled one comp at a time, then each layer's forward by its rfl lemma.

                                                                theorem Proofs.r34SmoothAtB_ok (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (hq : R34PosB w) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) (hx : R34SmoothAtB N w x) :
                                                                (r34NetLayer N w hq).ok x

                                                                R34SmoothAtB is the layer's .ok.

                                                                noncomputable def Proofs.resnet34ForwardBFullHasVJPAt (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (hq : R34PosB w) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) (hx : R34SmoothAtB N w x) :
                                                                HasVJPAt (r34HeadB N 7 7 w.Wd w.bd ∘ r34Pre16 N w) x

                                                                ResNet-34 at TRUE BATCH-NORM has a certified input-VJP at a smooth point — all sixteen basic blocks. r34NetLayer's .vjp, read at the layered chain, under two hypotheses: R34PosB (every ε > 0) and R34SmoothAtB (every relu clause and the pool's no-tie, each at its block's own input).

                                                                Pointwise, and necessarily: relu is kinked. Each block contributes TWO clauses — the body's mid-relu and the post-residual OUTER relu — where MobileNetV2's bottleneck contributes two relu6 clauses and EfficientNet's MBConv contributes none.

                                                                The head takes no hypothesis at all (GAP and dense are smooth, and each is batchMap of a per-example op), and N is a variable: the batch size is never pinned (the 224-px resolution and the 64…512 widths are literals).

                                                                Equations
                                                                Instances For
                                                                  theorem Proofs.resnet34ForwardBFullHasVJPAt_correct (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (hq : R34PosB w) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) (hx : R34SmoothAtB N w x) (dy : Vec (N * nCls)) (i : Fin (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) :
                                                                  (resnet34ForwardBFullHasVJPAt N w hq x hx).backward dy i = ∑ j : Fin (N * nCls), pdiv (resnet34ForwardBFull N w) x i j * dy j

                                                                  Public correctness theorem: the sixteen-block batch-BN backward equals the pdiv-contracted Jacobian of resnet34ForwardBFull ITSELF — the committed nested-application forward ResNet34FullB.lean defines and resnet34FwdGraphBFull_faithful proves the typed graph denotes — not of the layered chain the VJP is assembled on. Tied back through resnet34ForwardBFull_eq_chain.

                                                                  theorem Proofs.resnet34ForwardBFull_differentiableAt (N : ℕ) {nCls : ℕ} (w : R34BWeights nCls) (hq : R34PosB w) (x : Vec (N * (3 * (2 * (2 * 56)) * (2 * (2 * 56))))) (hx : R34SmoothAtB N w x) :

                                                                  The committed forward is differentiable at every smooth point — the layer's .diff. What the seal's seal_differentiableAt needs.