Documentation

LeanMlir.Proofs.Nets.EfficientNet.EfficientNetStepTieG

EfficientNet-B0's T3 §1a TIE at the UN-FUSED gradient and the SMOOTHED loss #

EfficientNetStepTie.lean ties all 262 parameters of the SGD-inline efficientnet_train_step.mlir: each fused theta - lr * g op dens to the certified step at the cotangent the emitted backward chain delivers. This file is that statement re-pointed along the two axes 4b left open (planning/archive/proofs_tier_to_paper_nets.md, "What is NOT done, and is the honest boundary").

Axis 1 — the OPTIMIZER FORM. Every conjunct is at the RAW gradient node (*GradB), which is what efficientnet_adam_train_step.mlir and every ImageNet artifact emit; the fused op appears only in the SGD-inline file. One statement therefore covers AdamW, RMSProp, EMA, the clipped and drop-path variants and their data-parallel and bf16 twins, because they all consume this node. 4b.1's EfficientNetFoldG.lean is the fold each conjunct delegates to.

Axis 2 — the LOSS. The capstone's top-of-chain cotangent is Foundation/SmoothedLossCot.lean's, at a GENERAL target: the six-op chain softmaxRow → subB → scaleB → addVB → shiftB → divConstB the batched renders emit, with the target arriving as the graph input %onehot — a soft vector under mixup or cutmix. The fused file pins it to softmax − oneHot, the gradient of plain cross-entropy at a hard label, which no ImageNet artifact computes. unrowB / rowB are ResNet-34's casts between the loss chain's one-row-per- example index and the dense ops' plain per-example width.

What is NOT new, and why the file is a transformation rather than a proof #

Every cotangent chain, every forward activation and every Jacobian witness is EfficientNetStepTie.lean's, unchanged. The fusion is rfl*SgdB_eq_grad says the fused op IS theta - lr * applied to the un-fused one — so each conjunct's proof is the fused file's with the wrapper peeling dropped, exactly as 4b's folds were. The lr, wN, bN, gN and lrStr binders disappear with the wrapper.

The head takes g as a PARAMETER here. The fused enetHeadTied computes g := rowSoftmax(logits) − onehot internally, which is what pinned that file to the hard label. Making it a binder is the whole of axis 2: the per-block ties are forall cot statements and were already loss-agnostic, so only the head and the capstone had to move.

Conventions carried unchanged from the fused file: batch BatchNorm (bnBatchLA), XLA-SAME at the 3x3/s2 stem and SYMMETRIC at the strided depthwises, swish (no kink, so no smoothness hypothesis anywhere), and the SE gate's fan-in folded into the block VJPs. ⛔ ONE REPLICA: in efficientnetin_emarmsdp64dropdo every gradient node feeds allReduceMeanF — the collective as an AST node since 4d piece 2 (2026-09-07), until then emitted text and a declared carve-out. Every statement here is at the per-replica node; DataParallelNode.lean composes it with the replica mean and the tail (adamW_at_allReduceMeanF). For the sync-BN data-parallel render (2026-09-21) EfficientNetSyncStepTieG.lean is the whole step: its efficientnet_net_syncTiedG says each all-reduced gradient IS this file's node at N := R·N (without drop-path and dropout).

def Proofs.EnetTiePoCG.enetExpTiedG {N ic mid oc h w r kHd kWd : } (xN vN epsStr cotN : String) (εe : ) (hεe : 0 < εe) (εd : ) (hεd : 0 < εd) (εp : ) (hεp : 0 < εp) (We : Kernel4 mid ic 1 1) (be γe βe : Vec mid) (Wd : DepthwiseKernel mid kHd kWd) (bd γd βd : Vec mid) (Wz1 : Mat mid r) (bz1 : Vec r) (Wz2 : Mat r mid) (bz2 : Vec mid) (Wp : Kernel4 oc mid 1 1) (bp γp βp : Vec oc) (xin : Vec (N * (ic * h * w))) (dyOut : Vec (N * (oc * h * w))) :
Equations
  • One or more equations did not get rendered due to their size.
Instances For
    theorem Proofs.EnetTiePoCG.enet_exp_tiedG {N ic mid oc h w r kHd kWd : } (xN vN epsStr cotN : String) (εe : ) (hεe : 0 < εe) (εd : ) (hεd : 0 < εd) (εp : ) (hεp : 0 < εp) (We : Kernel4 mid ic 1 1) (be γe βe : Vec mid) (Wd : DepthwiseKernel mid kHd kWd) (bd γd βd : Vec mid) (Wz1 : Mat mid r) (bz1 : Vec r) (Wz2 : Mat r mid) (bz2 : Vec mid) (Wp : Kernel4 oc mid 1 1) (bp γp βp : Vec oc) (xin : Vec (N * (ic * h * w))) (dyOut : Vec (N * (oc * h * w))) :
    enetExpTiedG xN vN epsStr cotN εe hεe εd hεd εp hεp We be γe βe Wd bd γd βd Wz1 bz1 Wz2 bz2 Wp bp γp βp xin dyOut

    Strided downsampling MBConv block — all 16 params tied (b2/b4/b6/b12) #

    Same as the expand block EXCEPT the expand stage lives at the block-input grid 2h×2w and the depthwise is strided (depthwiseStridedWeightSgdB, the expand-side cotangent cotEr upsamples h→2h via dStridedInB). No skip (spatial+channels change).

    def Proofs.EnetTiePoCG.enetStridedTiedG {N ic mid oc h w r kHd kWd : } (xN vN epsStr cotN : String) (εe : ) (hεe : 0 < εe) (εd : ) (hεd : 0 < εd) (εp : ) (hεp : 0 < εp) (We : Kernel4 mid ic 1 1) (be γe βe : Vec mid) (Wd : DepthwiseKernel mid kHd kWd) (bd γd βd : Vec mid) (Wz1 : Mat mid r) (bz1 : Vec r) (Wz2 : Mat r mid) (bz2 : Vec mid) (Wp : Kernel4 oc mid 1 1) (bp γp βp : Vec oc) (xin : Vec (N * (ic * (2 * h) * (2 * w)))) (dyOut : Vec (N * (oc * h * w))) :

    Strided downsampling MBConv block, tied. All 16 params at the real forward (expand at 2h×2w, strided depthwise 2h→h) + the chain cotangents driven by dyOut.

    Equations
    • One or more equations did not get rendered due to their size.
    Instances For
      theorem Proofs.EnetTiePoCG.enet_strided_tiedG {N ic mid oc h w r kHd kWd : } (xN vN epsStr cotN : String) (εe : ) (hεe : 0 < εe) (εd : ) (hεd : 0 < εd) (εp : ) (hεp : 0 < εp) (We : Kernel4 mid ic 1 1) (be γe βe : Vec mid) (Wd : DepthwiseKernel mid kHd kWd) (bd γd βd : Vec mid) (Wz1 : Mat mid r) (bz1 : Vec r) (Wz2 : Mat r mid) (bz2 : Vec mid) (Wp : Kernel4 oc mid 1 1) (bp γp βp : Vec oc) (xin : Vec (N * (ic * (2 * h) * (2 * w)))) (dyOut : Vec (N * (oc * h * w))) :
      enetStridedTiedG xN vN epsStr cotN εe hεe εd hεd εp hεp We be γe βe Wd bd γd βd Wz1 bz1 Wz2 bz2 Wp bp γp βp xin dyOut

      No-expand MBConv block (b1, t=1) — all 12 params tied (depthwise on ic → SE → project) #

      NO expand conv: the depthwise runs directly on the block input (ic channels). 12 params (4 depthwise+BN, 4 SE, 4 project). The SE squeeze/excite is on ic channels (ic → r → ic).

      def Proofs.EnetTiePoCG.enetNoExpTiedG {N ic oc h w r kHd kWd : } (xN vN epsStr cotN : String) (εd : ) (hεd : 0 < εd) (εp : ) (hεp : 0 < εp) (Wd : DepthwiseKernel ic kHd kWd) (bd γd βd : Vec ic) (Wz1 : Mat ic r) (bz1 : Vec r) (Wz2 : Mat r ic) (bz2 : Vec ic) (Wp : Kernel4 oc ic 1 1) (bp γp βp : Vec oc) (xin : Vec (N * (ic * h * w))) (dyOut : Vec (N * (oc * h * w))) :

      No-expand MBConv block, tied. All 12 params at the real forward + chain cotangents.

      Equations
      • One or more equations did not get rendered due to their size.
      Instances For
        theorem Proofs.EnetTiePoCG.enet_noexp_tiedG {N ic oc h w r kHd kWd : } (xN vN epsStr cotN : String) (εd : ) (hεd : 0 < εd) (εp : ) (hεp : 0 < εp) (Wd : DepthwiseKernel ic kHd kWd) (bd γd βd : Vec ic) (Wz1 : Mat ic r) (bz1 : Vec r) (Wz2 : Mat r ic) (bz2 : Vec ic) (Wp : Kernel4 oc ic 1 1) (bp γp βp : Vec oc) (xin : Vec (N * (ic * h * w))) (dyOut : Vec (N * (oc * h * w))) :
        enetNoExpTiedG xN vN epsStr cotN εd hεd εp hεp Wd bd γd βd Wz1 bz1 Wz2 bz2 Wp bp γp βp xin dyOut

        Stem — the 3×3/s2 conv-bn-swish (4 params), feeding block 1 #

        swish(bn(convStride2Xla Ws bs x)), 3→32 at 224→112, at the XLA-SAME phase the shipped stem uses. The cotangent block 1 delivers at the stem swish output (dyStem) lifts through swish-back

        def Proofs.EnetTiePoCG.enetStemTiedG {N ic oc h w kHs kWs : } (xN vN epsStr cotN : String) (εs : ) (hεs : 0 < εs) (Ws : Kernel4 oc ic kHs kWs) (bs γs βs : Vec oc) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (dyStem : Vec (N * (oc * h * w))) :

        Stem, tied. The 3×3/s2 conv (Ws/bs) + its true-BN (γs/βs) at the real stem forward + the cotangent through the stem swish (no maxpool, no conv-back).

        Equations
        • One or more equations did not get rendered due to their size.
        Instances For
          theorem Proofs.EnetTiePoCG.enet_stem_tiedG {N ic oc h w kHs kWs : } (xN vN epsStr cotN : String) (εs : ) (hεs : 0 < εs) (Ws : Kernel4 oc ic kHs kWs) (bs γs βs : Vec oc) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (dyStem : Vec (N * (oc * h * w))) :
          enetStemTiedG xN vN epsStr cotN εs hεs Ws bs γs βs x dyStem

          Head — the 1×1 conv-bn-swish (4 params) → GAP → dense (Wfc/bfc), + the loss cotangent #

          dense(GAP(swish(bn(conv Wh bh))))) (320→1280 conv, GAP, 1280→nClasses dense), then the batched per-row softmax-CE gradient g = rowSoftmax(logits) − onehot. The head conv params tie at the chain cotangent (loss → dense-back → GAP-back → swish/BN-back); the dense Wfc/bfc tie at the loss cotangent g directly.

          def Proofs.EnetTiePoCG.enetHeadTiedG {N c oc h w nC : } (xN vN epsStr cotN dN : String) (εh : ) (hεh : 0 < εh) (Wh : Kernel4 oc c 1 1) (bh γh βh : Vec oc) (Wfc : Mat oc nC) (bfc : Vec nC) (xhead : Vec (N * (c * h * w))) (g : Vec (N * nC)) :

          Head, tied. The 4 head conv-bn params + the 2 dense params (Wfc/bfc) denote the certified step at the real head forward + the loss-driven cotangent g = rowSoftmax(logits) − onehot.

          Equations
          • One or more equations did not get rendered due to their size.
          Instances For
            theorem Proofs.EnetTiePoCG.enet_head_tiedG {N c oc h w nC : } (xN vN epsStr cotN dN : String) (εh : ) (hεh : 0 < εh) (Wh : Kernel4 oc c 1 1) (bh γh βh : Vec oc) (Wfc : Mat oc nC) (bfc : Vec nC) (xhead : Vec (N * (c * h * w))) (g : Vec (N * nC)) :
            enetHeadTiedG xN vN epsStr cotN dN εh hεh Wh bh γh βh Wfc bfc xhead g

            @[irreducible] bundle-taking *TiedAt wrappers — one per block type, for the whole-net thread #

            Each takes the B0Weights block bundle (MBW/MBWNoExp) + its ε-positivity + the block input + the downstream cotangent dyOut, and delegates to the per-block-type tie. @[irreducible] keeps the 16-deep capstone thread opaque to the elaborator (the r34/mnv2 heartbeat lesson).

            @[irreducible]
            def Proofs.EnetTiePoCG.enetExpTiedGAt (xN vN epsStr cotN : String) {N ic mid oc r kh kw : } (h w : ) (p : MBW ic mid oc r kh kw) (he : 0 < p.) (hd : 0 < p.) (hp : 0 < p.) (xin : Vec (N * (ic * h * w))) (dyOut : Vec (N * (oc * h * w))) :
            Equations
            • One or more equations did not get rendered due to their size.
            Instances For
              theorem Proofs.EnetTiePoCG.enet_exp_tiedGAt (xN vN epsStr cotN : String) {N ic mid oc r kh kw : } (h w : ) (p : MBW ic mid oc r kh kw) (he : 0 < p.) (hd : 0 < p.) (hp : 0 < p.) (xin : Vec (N * (ic * h * w))) (dyOut : Vec (N * (oc * h * w))) :
              enetExpTiedGAt xN vN epsStr cotN h w p he hd hp xin dyOut
              @[irreducible]
              def Proofs.EnetTiePoCG.enetStridedTiedGAt (xN vN epsStr cotN : String) {N ic mid oc r kh kw : } (h w : ) (p : MBW ic mid oc r kh kw) (he : 0 < p.) (hd : 0 < p.) (hp : 0 < p.) (xin : Vec (N * (ic * (2 * h) * (2 * w)))) (dyOut : Vec (N * (oc * h * w))) :
              Equations
              • One or more equations did not get rendered due to their size.
              Instances For
                theorem Proofs.EnetTiePoCG.enet_strided_tiedGAt (xN vN epsStr cotN : String) {N ic mid oc r kh kw : } (h w : ) (p : MBW ic mid oc r kh kw) (he : 0 < p.) (hd : 0 < p.) (hp : 0 < p.) (xin : Vec (N * (ic * (2 * h) * (2 * w)))) (dyOut : Vec (N * (oc * h * w))) :
                enetStridedTiedGAt xN vN epsStr cotN h w p he hd hp xin dyOut
                @[irreducible]
                def Proofs.EnetTiePoCG.enetNoExpTiedGAt (xN vN epsStr cotN : String) {N ic oc r kh kw : } (h w : ) (p : MBWNoExp ic oc r kh kw) (hd : 0 < p.) (hp : 0 < p.) (xin : Vec (N * (ic * h * w))) (dyOut : Vec (N * (oc * h * w))) :
                Equations
                • One or more equations did not get rendered due to their size.
                Instances For
                  theorem Proofs.EnetTiePoCG.enet_noexp_tiedGAt (xN vN epsStr cotN : String) {N ic oc r kh kw : } (h w : ) (p : MBWNoExp ic oc r kh kw) (hd : 0 < p.) (hp : 0 < p.) (xin : Vec (N * (ic * h * w))) (dyOut : Vec (N * (oc * h * w))) :
                  enetNoExpTiedGAt xN vN epsStr cotN h w p hd hp xin dyOut
                  theorem Proofs.EnetTiePoCG.efficientnet_net_tiedG (xN vN epsStr cotN dN : String) (N : ) (w : B0Weights) (hsε : 0 < w.) (hb1d : 0 < w.b1.) (hb1p : 0 < w.b1.) (hb2e : 0 < w.b2.) (hb2d : 0 < w.b2.) (hb2p : 0 < w.b2.) (hb3e : 0 < w.b3.) (hb3d : 0 < w.b3.) (hb3p : 0 < w.b3.) (hb4e : 0 < w.b4.) (hb4d : 0 < w.b4.) (hb4p : 0 < w.b4.) (hb5e : 0 < w.b5.) (hb5d : 0 < w.b5.) (hb5p : 0 < w.b5.) (hb6e : 0 < w.b6.) (hb6d : 0 < w.b6.) (hb6p : 0 < w.b6.) (hb7e : 0 < w.b7.) (hb7d : 0 < w.b7.) (hb7p : 0 < w.b7.) (hb8e : 0 < w.b8.) (hb8d : 0 < w.b8.) (hb8p : 0 < w.b8.) (hb9e : 0 < w.b9.) (hb9d : 0 < w.b9.) (hb9p : 0 < w.b9.) (hb10e : 0 < w.b10.) (hb10d : 0 < w.b10.) (hb10p : 0 < w.b10.) (hb11e : 0 < w.b11.) (hb11d : 0 < w.b11.) (hb11p : 0 < w.b11.) (hb12e : 0 < w.b12.) (hb12d : 0 < w.b12.) (hb12p : 0 < w.b12.) (hb13e : 0 < w.b13.) (hb13d : 0 < w.b13.) (hb13p : 0 < w.b13.) (hb14e : 0 < w.b14.) (hb14d : 0 < w.b14.) (hb14p : 0 < w.b14.) (hb15e : 0 < w.b15.) (hb15d : 0 < w.b15.) (hb15p : 0 < w.b15.) (hb16e : 0 < w.b16.) (hb16d : 0 < w.b16.) (hb16p : 0 < w.b16.) (hhε : 0 < w.) (aStr negAK bStr logN ohN : String) (α B : ) (x : Vec (N * (3 * 224 * 224))) (t : Vec (N * (1 * 10))) :
                  have a0 := stemB N w.sW w.sb w. w. w. x; have a1 := mbNoExpW N 112 112 w.b1 a0; have a2 := mbStridedW N 56 56 w.b2 a1; have a3 := mbResidW N 56 56 w.b3 a2; have a4 := mbStridedW N 28 28 w.b4 a3; have a5 := mbResidW N 28 28 w.b5 a4; have a6 := mbStridedW N 14 14 w.b6 a5; have a7 := mbResidW N 14 14 w.b7 a6; have a8 := mbResidW N 14 14 w.b8 a7; have a9 := mbExpW N 14 14 w.b9 a8; have a10 := mbResidW N 14 14 w.b10 a9; have a11 := mbResidW N 14 14 w.b11 a10; have a12 := mbStridedW N 7 7 w.b12 a11; have a13 := mbResidW N 7 7 w.b13 a12; have a14 := mbResidW N 7 7 w.b14 a13; have a15 := mbResidW N 7 7 w.b15 a14; have a16 := mbExpW N 7 7 w.b16 a15; have g := ResNet34TieB.unrowB N 10 (StableHLO.den (smoothedLossCotGraph N 10 α B aStr negAK bStr logN ohN (ResNet34TieB.rowB N 10 (headFwdB N w.hW w.hb w. w. w. w.fcW w.fcb a16)) t)); have dy16 := (headFwdB_has_vjp N w.hW w.hb w. hhε w. w. w.fcW w.fcb).backward a16 g; have dy15 := (mbExpW_has_vjp N 7 7 w.b16 hb16e hb16d hb16p).backward a15 dy16; have dy14 := (mbResidW_has_vjp N 7 7 w.b15 hb15e hb15d hb15p).backward a14 dy15; have dy13 := (mbResidW_has_vjp N 7 7 w.b14 hb14e hb14d hb14p).backward a13 dy14; have dy12 := (mbResidW_has_vjp N 7 7 w.b13 hb13e hb13d hb13p).backward a12 dy13; have dy11 := (mbStridedW_has_vjp N 7 7 w.b12 hb12e hb12d hb12p).backward a11 dy12; have dy10 := (mbResidW_has_vjp N 14 14 w.b11 hb11e hb11d hb11p).backward a10 dy11; have dy9 := (mbResidW_has_vjp N 14 14 w.b10 hb10e hb10d hb10p).backward a9 dy10; have dy8 := (mbExpW_has_vjp N 14 14 w.b9 hb9e hb9d hb9p).backward a8 dy9; have dy7 := (mbResidW_has_vjp N 14 14 w.b8 hb8e hb8d hb8p).backward a7 dy8; have dy6 := (mbResidW_has_vjp N 14 14 w.b7 hb7e hb7d hb7p).backward a6 dy7; have dy5 := (mbStridedW_has_vjp N 14 14 w.b6 hb6e hb6d hb6p).backward a5 dy6; have dy4 := (mbResidW_has_vjp N 28 28 w.b5 hb5e hb5d hb5p).backward a4 dy5; have dy3 := (mbStridedW_has_vjp N 28 28 w.b4 hb4e hb4d hb4p).backward a3 dy4; have dy2 := (mbResidW_has_vjp N 56 56 w.b3 hb3e hb3d hb3p).backward a2 dy3; have dy1 := (mbStridedW_has_vjp N 56 56 w.b2 hb2e hb2d hb2p).backward a1 dy2; have dy0 := (mbNoExpW_has_vjp N 112 112 w.b1 hb1d hb1p).backward a0 dy1; enetStemTiedG xN vN epsStr cotN w. hsε w.sW w.sb w. w. x dy0 enetNoExpTiedGAt xN vN epsStr cotN 112 112 w.b1 hb1d hb1p a0 dy1 enetStridedTiedGAt xN vN epsStr cotN 56 56 w.b2 hb2e hb2d hb2p a1 dy2 enetExpTiedGAt xN vN epsStr cotN 56 56 w.b3 hb3e hb3d hb3p a2 dy3 enetStridedTiedGAt xN vN epsStr cotN 28 28 w.b4 hb4e hb4d hb4p a3 dy4 enetExpTiedGAt xN vN epsStr cotN 28 28 w.b5 hb5e hb5d hb5p a4 dy5 enetStridedTiedGAt xN vN epsStr cotN 14 14 w.b6 hb6e hb6d hb6p a5 dy6 enetExpTiedGAt xN vN epsStr cotN 14 14 w.b7 hb7e hb7d hb7p a6 dy7 enetExpTiedGAt xN vN epsStr cotN 14 14 w.b8 hb8e hb8d hb8p a7 dy8 enetExpTiedGAt xN vN epsStr cotN 14 14 w.b9 hb9e hb9d hb9p a8 dy9 enetExpTiedGAt xN vN epsStr cotN 14 14 w.b10 hb10e hb10d hb10p a9 dy10 enetExpTiedGAt xN vN epsStr cotN 14 14 w.b11 hb11e hb11d hb11p a10 dy11 enetStridedTiedGAt xN vN epsStr cotN 7 7 w.b12 hb12e hb12d hb12p a11 dy12 enetExpTiedGAt xN vN epsStr cotN 7 7 w.b13 hb13e hb13d hb13p a12 dy13 enetExpTiedGAt xN vN epsStr cotN 7 7 w.b14 hb14e hb14d hb14p a13 dy14

                  The whole 16-MBConv EfficientNet-B0 train step, tied at the GRADIENT nodes and the SMOOTHED loss. Threading the real batched (true-BN + SE) forward efficientnetForwardB_full and the backward cotangent chain (swish masks, the SE gate fan-in, true-BN backs, the residual fan-in folded into the block VJPs), the stem, all 16 MBConv blocks, the conv-bn-swish head, and the dense head all denote the certified batched Σ_n loss-descent step.