Documentation

LeanMlir.Proofs.Nets.EfficientNet.EfficientNetFullB0Drop

EfficientNet-B0 with stochastic depth and classifier dropout — forward, graph, faithfulness #

EfficientNetFullB0 and EfficientNetFullB0Eval state the sixteen-block net without its two regularisers; the *drop* / *do* artifacts render them. This file states both forwards WITH them, as the renderer (enetFwdChain) emits them, and proves the typed graphs denote them:

Each is Optional, as the renderer's sd / cd flags are: none renders no node, so one statement covers efficientnet_drop_fwd (sd only), efficientnet_do_fwd (cd only) and efficientnetin_dropdo_fwd (both), and their _eval twins at inference BatchNorm.

The residual-block-with-drop and the dropout head are text-guarded against the renderer in Codegen/FwdGraphTextTies at training BatchNorm (the eval graphs carry the three-block eval graph's SSA names, as EfficientNetFullB0Eval records). These artifacts are f32. The train steps' backward through the drop sites is outside this statement, as it is outside EfficientNetStepTieG.

noncomputable def Proofs.dropPathOpt (N n : ℕ) :
Option (Vec N) → Vec (N * n) → Vec (N * n)

Drop-path at a site that may be absent: none is the identity.

Equations
Instances For
    noncomputable def Proofs.dropoutOpt {m : ℕ} :
    Option (Vec m) → Vec m → Vec m

    Dropout at a site that may be absent: none is the identity.

    Equations
    Instances For
      theorem Proofs.dropPathOpt_ones (N n : ℕ) :
      dropPathOpt N n (some fun (x : Fin N) => 1) = id

      At the all-ones scale drop-path is the identity.

      theorem Proofs.dropoutOpt_ones {m : ℕ} :
      dropoutOpt (some fun (x : Fin m) => 1) = id

      At the all-ones mask dropout is the identity.

      noncomputable def Proofs.mbResidDropW (N h w : ℕ) {c mid kh kw r : ℕ} (p : MBW c mid c r kh kw) (s : Option (Vec N)) :
      Vec (N * (c * h * w)) → Vec (N * (c * h * w))

      A residual MBConv6 block with its drop site: the scale on the branch, then the skip add (eFwd's placement).

      Equations
      • One or more equations did not get rendered due to their size.
      Instances For
        theorem Proofs.mbResidDropW_none (N h w : ℕ) {c mid kh kw r : ℕ} (p : MBW c mid c r kh kw) :
        mbResidDropW N h w p none = mbResidW N h w p
        theorem Proofs.mbResidDropW_ones (N h w : ℕ) {c mid kh kw r : ℕ} (p : MBW c mid c r kh kw) :
        mbResidDropW N h w p (some fun (x : Fin N) => 1) = mbResidW N h w p
        noncomputable def Proofs.headDoFwdB (N : ℕ) {c oc h w nC : ℕ} (Wh : Kernel4 oc c 1 1) (bh : Vec oc) (εh : ℝ) (γh βh : Vec oc) (Wfc : Mat oc nC) (bfc : Vec nC) (m : Option (Vec (N * oc))) :
        Vec (N * (c * h * w)) → Vec (N * nC)

        The head with classifier dropout: 1×1 conv-bn-swish → GAP → dropout → dense.

        Equations
        • One or more equations did not get rendered due to their size.
        Instances For
          theorem Proofs.headDoFwdB_none (N : ℕ) {c oc h w nC : ℕ} (Wh : Kernel4 oc c 1 1) (bh : Vec oc) (εh : ℝ) (γh βh : Vec oc) (Wfc : Mat oc nC) (bfc : Vec nC) :
          headDoFwdB N Wh bh εh γh βh Wfc bfc none = headFwdB N Wh bh εh γh βh Wfc bfc
          theorem Proofs.headDoFwdB_ones (N : ℕ) {c oc h w nC : ℕ} (Wh : Kernel4 oc c 1 1) (bh : Vec oc) (εh : ℝ) (γh βh : Vec oc) (Wfc : Mat oc nC) (bfc : Vec nC) :
          headDoFwdB N Wh bh εh γh βh Wfc bfc (some fun (x : Fin (N * oc)) => 1) = headFwdB N Wh bh εh γh βh Wfc bfc
          noncomputable def Proofs.mbResidEvalDropW (N h w : ℕ) (ε : ℝ) {c mid kh kw r : ℕ} (p : MBWEval c mid c r kh kw) (s : Option (Vec N)) :
          Vec (N * (c * h * w)) → Vec (N * (c * h * w))

          mbResidDropW at inference BatchNorm.

          Equations
          • One or more equations did not get rendered due to their size.
          Instances For
            theorem Proofs.mbResidEvalDropW_none (N h w : ℕ) (ε : ℝ) {c mid kh kw r : ℕ} (p : MBWEval c mid c r kh kw) :
            mbResidEvalDropW N h w ε p none = mbResidEvalW N h w ε p
            theorem Proofs.mbResidEvalDropW_ones (N h w : ℕ) (ε : ℝ) {c mid kh kw r : ℕ} (p : MBWEval c mid c r kh kw) :
            mbResidEvalDropW N h w ε p (some fun (x : Fin N) => 1) = mbResidEvalW N h w ε p
            noncomputable def Proofs.headDoFwdBEval (N : ℕ) {c oc h w nC : ℕ} (ε : ℝ) (Wh : Kernel4 oc c 1 1) (bh γh βh μh vh : Vec oc) (Wfc : Mat oc nC) (bfc : Vec nC) (m : Option (Vec (N * oc))) :
            Vec (N * (c * h * w)) → Vec (N * nC)

            headDoFwdB at inference BatchNorm.

            Equations
            • One or more equations did not get rendered due to their size.
            Instances For
              theorem Proofs.headDoFwdBEval_none (N : ℕ) {c oc h w nC : ℕ} (ε : ℝ) (Wh : Kernel4 oc c 1 1) (bh γh βh μh vh : Vec oc) (Wfc : Mat oc nC) (bfc : Vec nC) :
              headDoFwdBEval N ε Wh bh γh βh μh vh Wfc bfc none = headFwdBEval N ε Wh bh γh βh μh vh Wfc bfc
              theorem Proofs.headDoFwdBEval_ones (N : ℕ) {c oc h w nC : ℕ} (ε : ℝ) (Wh : Kernel4 oc c 1 1) (bh γh βh μh vh : Vec oc) (Wfc : Mat oc nC) (bfc : Vec nC) :
              headDoFwdBEval N ε Wh bh γh βh μh vh Wfc bfc (some fun (x : Fin (N * oc)) => 1) = headFwdBEval N ε Wh bh γh βh μh vh Wfc bfc
              noncomputable def Proofs.efficientnetForwardBFullDrop (N : ℕ) {nCls : ℕ} (w : B0Weights nCls) (sd : Option (Fin 9 → Vec N)) (cd : Option (Vec (N * 1280))) (x : Vec (N * (3 * 224 * 224))) :
              Vec (N * nCls)

              EfficientNet-B0 with stochastic depth and classifier dropout at training BatchNorm: efficientnetForwardBFull with sd's nine per-example scales on the skip blocks' branches (site k = the k-th of b3 b5 b7 b8 b10 b11 b13 b14 b15) and cd's mask before the classifier.

              Equations
              • One or more equations did not get rendered due to their size.
              Instances For

                With neither site rendered, the forward is efficientnetForwardBFull.

                theorem Proofs.efficientnetForwardBFullDrop_ones (N : ℕ) {nCls : ℕ} (w : B0Weights nCls) (x : Vec (N * (3 * 224 * 224))) :
                efficientnetForwardBFullDrop N w (some fun (x : Fin 9) (x_1 : Fin N) => 1) (some fun (x : Fin (N * 1280)) => 1) x = efficientnetForwardBFull N w x

                At the all-ones masks the forward is efficientnetForwardBFull, exactly — the masks the driver passes to the forward artifacts.

                noncomputable def Proofs.efficientnetForwardBFullEvalDrop (N : ℕ) (ε : ℝ) {nCls : ℕ} (w : B0WeightsEval nCls) (sd : Option (Fin 9 → Vec N)) (cd : Option (Vec (N * 1280))) (x : Vec (N * (3 * 224 * 224))) :
                Vec (N * nCls)

                efficientnetForwardBFullDrop at inference BatchNorm.

                Equations
                • One or more equations did not get rendered due to their size.
                Instances For
                  theorem Proofs.efficientnetForwardBFullEvalDrop_ones (N : ℕ) (ε : ℝ) {nCls : ℕ} (w : B0WeightsEval nCls) (x : Vec (N * (3 * 224 * 224))) :
                  efficientnetForwardBFullEvalDrop N ε w (some fun (x : Fin 9) (x_1 : Fin N) => 1) (some fun (x : Fin (N * 1280)) => 1) x = efficientnetForwardBFullEval N ε w x

                  At the all-ones masks the inference forward is efficientnetForwardBFullEval, exactly.

                  def Proofs.StableHLO.dropPathOptG (mN : String) {N n : ℕ} :
                  Option (Vec N) → SHlo (N * n) → SHlo (N * n)

                  A dropPathB node when the site is rendered, nothing otherwise.

                  Equations
                  Instances For
                    theorem Proofs.StableHLO.den_dropPathOptG (mN : String) {N n : ℕ} (s : Option (Vec N)) (e : SHlo (N * n)) :
                    den (dropPathOptG mN s e) = dropPathOpt N n s (den e)
                    def Proofs.StableHLO.dropoutOptG (mN : String) {N n : ℕ} :
                    Option (Vec (N * n)) → SHlo (N * n) → SHlo (N * n)

                    A dropoutB node when the site is rendered, nothing otherwise.

                    Equations
                    Instances For
                      theorem Proofs.StableHLO.den_dropoutOptG (mN : String) {N n : ℕ} (m : Option (Vec (N * n))) (e : SHlo (N * n)) :
                      den (dropoutOptG mN m e) = dropoutOpt m (den e)
                      def Proofs.StableHLO.mbResidDropGraphB (p epsStr mN : String) {N c mid h w kHd kWd r : ℕ} (We : Kernel4 mid c 1 1) (be : Vec mid) (εe : ℝ) (γe βe : Vec mid) (Wd : DepthwiseKernel mid kHd kWd) (bd : Vec mid) (εd : ℝ) (γd βd : Vec mid) (Wz₁ : Mat mid r) (bz₁ : Vec r) (Wz₂ : Mat r mid) (bz₂ : Vec mid) (Wp : Kernel4 c mid 1 1) (bp : Vec c) (εp : ℝ) (γp βp : Vec c) (s : Option (Vec N)) (e : SHlo (N * (c * h * w))) :
                      SHlo (N * (c * h * w))

                      Residual MBConv6 with its drop site: mbResidGraphB with dropPathOptG on the branch before the addVB — the node sequence eFwd … (drop := some i) emits.

                      Equations
                      • One or more equations did not get rendered due to their size.
                      Instances For
                        theorem Proofs.StableHLO.mbResidDropGraphB_none (p epsStr mN : String) {N c mid h w kHd kWd r : ℕ} (We : Kernel4 mid c 1 1) (be : Vec mid) (εe : ℝ) (γe βe : Vec mid) (Wd : DepthwiseKernel mid kHd kWd) (bd : Vec mid) (εd : ℝ) (γd βd : Vec mid) (Wz₁ : Mat mid r) (bz₁ : Vec r) (Wz₂ : Mat r mid) (bz₂ : Vec mid) (Wp : Kernel4 c mid 1 1) (bp : Vec c) (εp : ℝ) (γp βp : Vec c) (e : SHlo (N * (c * h * w))) :
                        mbResidDropGraphB p epsStr mN We be εe γe βe Wd bd εd γd βd Wz₁ bz₁ Wz₂ bz₂ Wp bp εp γp βp none e = mbResidGraphB p epsStr We be εe γe βe Wd bd εd γd βd Wz₁ bz₁ Wz₂ bz₂ Wp bp εp γp βp e

                        With no drop site it is mbResidGraphB.

                        def Proofs.StableHLO.mbResidDropGraphW (pfx epsStr mN : String) (N h w : ℕ) {c mid kh kw r : ℕ} (p : MBW c mid c r kh kw) (s : Option (Vec N)) (e : SHlo (N * (c * h * w))) :
                        SHlo (N * (c * h * w))

                        The graph with its drop site as a weight-bundle wrapper.

                        Equations
                        • One or more equations did not get rendered due to their size.
                        Instances For
                          theorem Proofs.StableHLO.mbResidDropGraphW_faithful (pfx epsStr mN : String) (N h w : ℕ) {c mid kh kw r : ℕ} (p : MBW c mid c r kh kw) (s : Option (Vec N)) (e : SHlo (N * (c * h * w))) :
                          den (mbResidDropGraphW pfx epsStr mN N h w p s e) = mbResidDropW N h w p s (den e)
                          def Proofs.StableHLO.headGraphBDo (epsStr mN : String) {N c oc h w nC : ℕ} (Wh : Kernel4 oc c 1 1) (bh : Vec oc) (εh : ℝ) (γh βh : Vec oc) (Wfc : Mat oc nC) (bfc : Vec nC) (m : Option (Vec (N * oc))) (e : SHlo (N * (c * h * w))) :
                          SHlo (N * nC)

                          Head with classifier dropout: headGraphB with dropoutOptG between the GAP and the dense, its mask the input mN (the renderer's is doName).

                          Equations
                          • One or more equations did not get rendered due to their size.
                          Instances For
                            theorem Proofs.StableHLO.headGraphBDo_faithful (epsStr mN : String) {N c oc h w nC : ℕ} (Wh : Kernel4 oc c 1 1) (bh : Vec oc) (εh : ℝ) (γh βh : Vec oc) (Wfc : Mat oc nC) (bfc : Vec nC) (m : Option (Vec (N * oc))) (e : SHlo (N * (c * h * w))) :
                            den (headGraphBDo epsStr mN Wh bh εh γh βh Wfc bfc m e) = headDoFwdB N Wh bh εh γh βh Wfc bfc m (den e)
                            def Proofs.StableHLO.mbResidDropGraphBEval (p epsStr mN : String) {N c mid h w kHd kWd r : ℕ} (ε : ℝ) (We : Kernel4 mid c 1 1) (be γe βe μe ve : Vec mid) (Wd : DepthwiseKernel mid kHd kWd) (bd γd βd μd vd : Vec mid) (Wz₁ : Mat mid r) (bz₁ : Vec r) (Wz₂ : Mat r mid) (bz₂ : Vec mid) (Wp : Kernel4 c mid 1 1) (bp γp βp μp vp : Vec c) (s : Option (Vec N)) (e : SHlo (N * (c * h * w))) :
                            SHlo (N * (c * h * w))

                            mbResidDropGraphB at inference BatchNorm (mbResidGraphBEval's nodes).

                            Equations
                            • One or more equations did not get rendered due to their size.
                            Instances For
                              def Proofs.StableHLO.mbResidDropGraphEvalW (pfx epsStr mN : String) (N h w : ℕ) (ε : ℝ) {c mid kh kw r : ℕ} (p : MBWEval c mid c r kh kw) (s : Option (Vec N)) (e : SHlo (N * (c * h * w))) :
                              SHlo (N * (c * h * w))

                              The inference graph with its drop site as a weight-bundle wrapper.

                              Equations
                              • One or more equations did not get rendered due to their size.
                              Instances For
                                theorem Proofs.StableHLO.mbResidDropGraphEvalW_faithful (pfx epsStr mN : String) (N h w : ℕ) (ε : ℝ) {c mid kh kw r : ℕ} (p : MBWEval c mid c r kh kw) (s : Option (Vec N)) (e : SHlo (N * (c * h * w))) :
                                den (mbResidDropGraphEvalW pfx epsStr mN N h w ε p s e) = mbResidEvalDropW N h w ε p s (den e)
                                def Proofs.StableHLO.headGraphBEvalDo (epsStr mN : String) {N c oc h w nC : ℕ} (ε : ℝ) (Wh : Kernel4 oc c 1 1) (bh γh βh μh vh : Vec oc) (Wfc : Mat oc nC) (bfc : Vec nC) (m : Option (Vec (N * oc))) (e : SHlo (N * (c * h * w))) :
                                SHlo (N * nC)

                                headGraphBDo at inference BatchNorm (headGraphBEval's nodes).

                                Equations
                                • One or more equations did not get rendered due to their size.
                                Instances For
                                  theorem Proofs.StableHLO.headGraphBEvalDo_faithful (epsStr mN : String) {N c oc h w nC : ℕ} (ε : ℝ) (Wh : Kernel4 oc c 1 1) (bh γh βh μh vh : Vec oc) (Wfc : Mat oc nC) (bfc : Vec nC) (m : Option (Vec (N * oc))) (e : SHlo (N * (c * h * w))) :
                                  den (headGraphBEvalDo epsStr mN ε Wh bh γh βh μh vh Wfc bfc m e) = headDoFwdBEval N ε Wh bh γh βh μh vh Wfc bfc m (den e)
                                  def Proofs.StableHLO.efficientnetFwdGraphBFullDrop (N : ℕ) (epsStr : String) {nCls : ℕ} (w : B0Weights nCls) (sd : Option (Fin 9 → Vec N)) (cd : Option (Vec (N * 1280))) (x : Vec (N * (3 * 224 * 224))) :
                                  SHlo (N * nCls)

                                  The B0 forward graph with stochastic depth and classifier dropout — the typed form of efficientnet_drop_fwd / efficientnet_do_fwd / efficientnetin_dropdo_fwd: each skip block's drop site reads %dp<i> at its BLOCK index i, the classifier dropout %do.

                                  Equations
                                  • One or more equations did not get rendered due to their size.
                                  Instances For
                                    theorem Proofs.StableHLO.efficientnetFwdGraphBFullDrop_faithful (N : ℕ) (epsStr : String) {nCls : ℕ} (w : B0Weights nCls) (sd : Option (Fin 9 → Vec N)) (cd : Option (Vec (N * 1280))) (x : Vec (N * (3 * 224 * 224))) :

                                    The graph denotes the forward, at every mask — one rw per block, then rfl.

                                    def Proofs.StableHLO.efficientnetFwdGraphBFullEvalDrop (N : ℕ) (epsStr : String) (ε : ℝ) {nCls : ℕ} (w : B0WeightsEval nCls) (sd : Option (Fin 9 → Vec N)) (cd : Option (Vec (N * 1280))) (x : Vec (N * (3 * 224 * 224))) :
                                    SHlo (N * nCls)

                                    The inference twin — the typed form of efficientnet_drop_fwd_eval / efficientnet_do_fwd_eval.

                                    Equations
                                    • One or more equations did not get rendered due to their size.
                                    Instances For
                                      theorem Proofs.StableHLO.efficientnetFwdGraphBFullEvalDrop_faithful (N : ℕ) (epsStr : String) (ε : ℝ) {nCls : ℕ} (w : B0WeightsEval nCls) (sd : Option (Fin 9 → Vec N)) (cd : Option (Vec (N * 1280))) (x : Vec (N * (3 * 224 * 224))) :