Documentation

LeanMlir.Proofs.Nets.EfficientNet.EfficientNetFullWholeBackCertifiedTie

efficientnetInputGradBFull is the certified whole-net paper EfficientNet-B0 gradient #

The whole-net backward tie for efficientnetForwardBFull, all sixteen MBConv blocks, against the concrete efficientnetForwardBFullHasVJP and, through efficientnetForwardBFullHasVJP_correct, against the Jacobian of the committed nested-application forward itself.

The stem and head endpoint ties (stemBBack_eq_vjp_backward, headFwdBBack_eq_vjp_backward) are EfficientNetWholeBackCertifiedTie.lean's, used at the paper widths (head 320→1280 at 7×7); the sixteen blocks stay opaque in the tie, entering as HasVJP witnesses whose .backward is what the reverse chain's block slots are pinned to, so the composition is checked between variables. Unlike MobileNetV2's tie there is no smooth point: swish and the SE sigmoid are differentiable everywhere, and the only hypothesis is w.EpsPos. The forward has no drop-path and no classifier dropout, and is in exact arithmetic.

The three pieces #

  1. efficientnetBFullHasVJP — the generic eighteen-stage apex head ∘ b16 ∘ … ∘ b1 ∘ stem, seventeen vjpComps and nothing else, and opaqueA0 … A16, one prefix def per running activation (the MobileNetV2 paper file's answer to the quadratic writing of nested applications; plain defs so the closing rfl sees through them).
  2. efficientnetInputGradBFull_eq_efficientnetB_full_vjp — the tie with the stem and head concrete and the sixteen blocks opaque: unfold, two rws, rfl.
  3. efficientnetInputGradBFull_eq_efficientnetForwardB_full_vjp — the tie instantiated at the sixteen concrete blocks (mbNoExpW, mbStridedW, mbResidW, mbExpW at B0Weights's widths) and carried to efficientnetForwardBFullHasVJP by HasVJP.backward_unique: two witnesses for one map have one backward, so the tactic-built whole-net witness never has to be unfolded. efficientnetInputGradBFull_correct then reads the result through efficientnetForwardBFullHasVJP_correct, whose proof IS the shape check efficientnetForwardBFull_eq_chain: the hand-written chain is the pdiv-contracted Jacobian of efficientnetForwardBFull at every input, every cotangent and every pixel.

General N. bnBatchLAHasVJP exists at every batch size, so the tie does.

noncomputable def Proofs.efficientnetBFullHasVJP {s0 s1 s2 s3 s4 s5 s6 s7 s8 s9 s10 s11 s12 s13 s14 s15 s16 s17 s18 : ℕ} (stem : Vec s0 → Vec s1) (b1 : Vec s1 → Vec s2) (b2 : Vec s2 → Vec s3) (b3 : Vec s3 → Vec s4) (b4 : Vec s4 → Vec s5) (b5 : Vec s5 → Vec s6) (b6 : Vec s6 → Vec s7) (b7 : Vec s7 → Vec s8) (b8 : Vec s8 → Vec s9) (b9 : Vec s9 → Vec s10) (b10 : Vec s10 → Vec s11) (b11 : Vec s11 → Vec s12) (b12 : Vec s12 → Vec s13) (b13 : Vec s13 → Vec s14) (b14 : Vec s14 → Vec s15) (b15 : Vec s15 → Vec s16) (b16 : Vec s16 → Vec s17) (head : Vec s17 → Vec s18) (dstem : Differentiable ℝ stem) (db1 : Differentiable ℝ b1) (db2 : Differentiable ℝ b2) (db3 : Differentiable ℝ b3) (db4 : Differentiable ℝ b4) (db5 : Differentiable ℝ b5) (db6 : Differentiable ℝ b6) (db7 : Differentiable ℝ b7) (db8 : Differentiable ℝ b8) (db9 : Differentiable ℝ b9) (db10 : Differentiable ℝ b10) (db11 : Differentiable ℝ b11) (db12 : Differentiable ℝ b12) (db13 : Differentiable ℝ b13) (db14 : Differentiable ℝ b14) (db15 : Differentiable ℝ b15) (db16 : Differentiable ℝ b16) (dhead : Differentiable ℝ head) (hstem : HasVJP stem) (hb1 : HasVJP b1) (hb2 : HasVJP b2) (hb3 : HasVJP b3) (hb4 : HasVJP b4) (hb5 : HasVJP b5) (hb6 : HasVJP b6) (hb7 : HasVJP b7) (hb8 : HasVJP b8) (hb9 : HasVJP b9) (hb10 : HasVJP b10) (hb11 : HasVJP b11) (hb12 : HasVJP b12) (hb13 : HasVJP b13) (hb14 : HasVJP b14) (hb15 : HasVJP b15) (hb16 : HasVJP b16) (hhead : HasVJP head) :
HasVJP (head ∘ b16 ∘ b15 ∘ b14 ∘ b13 ∘ b12 ∘ b11 ∘ b10 ∘ b9 ∘ b8 ∘ b7 ∘ b6 ∘ b5 ∘ b4 ∘ b3 ∘ b2 ∘ b1 ∘ stem)

Whole-network paper EfficientNet-B0 VJP. The VJP of the eighteen-stage chain head ∘ b16 ∘ … ∘ b1 ∘ stem — seventeen vjpComps, dimension-generic and parametric in every component, HasVJP everywhere (no smooth point: every activation on this path is differentiable on all of ℝ).

Equations
  • One or more equations did not get rendered due to their size.
Instances For
    theorem Proofs.efficientnetInputGradBFull_eq_efficientnetB_full_vjp (N : ℕ) {nCls : ℕ} (Ws : Kernel4 32 3 3 3) (bs : Vec 32) (εs : ℝ) (hεs : 0 < εs) (γs βs : Vec 32) (Wh : Kernel4 1280 320 1 1) (bh : Vec 1280) (εh : ℝ) (hεh : 0 < εh) (γh βh : Vec 1280) (Wfc : Mat 1280 nCls) (bfc : Vec nCls) (b1 : Vec (N * (32 * 112 * 112)) → Vec (N * (16 * 112 * 112))) (b2 : Vec (N * (16 * 112 * 112)) → Vec (N * (24 * 56 * 56))) (b3 : Vec (N * (24 * 56 * 56)) → Vec (N * (24 * 56 * 56))) (b4 : Vec (N * (24 * 56 * 56)) → Vec (N * (40 * 28 * 28))) (b5 : Vec (N * (40 * 28 * 28)) → Vec (N * (40 * 28 * 28))) (b6 : Vec (N * (40 * 28 * 28)) → Vec (N * (80 * 14 * 14))) (b7 b8 : Vec (N * (80 * 14 * 14)) → Vec (N * (80 * 14 * 14))) (b9 : Vec (N * (80 * 14 * 14)) → Vec (N * (112 * 14 * 14))) (b10 b11 : Vec (N * (112 * 14 * 14)) → Vec (N * (112 * 14 * 14))) (b12 : Vec (N * (112 * 14 * 14)) → Vec (N * (192 * 7 * 7))) (b13 b14 b15 : Vec (N * (192 * 7 * 7)) → Vec (N * (192 * 7 * 7))) (b16 : Vec (N * (192 * 7 * 7)) → Vec (N * (320 * 7 * 7))) (db1 : Differentiable ℝ b1) (db2 : Differentiable ℝ b2) (db3 : Differentiable ℝ b3) (db4 : Differentiable ℝ b4) (db5 : Differentiable ℝ b5) (db6 : Differentiable ℝ b6) (db7 : Differentiable ℝ b7) (db8 : Differentiable ℝ b8) (db9 : Differentiable ℝ b9) (db10 : Differentiable ℝ b10) (db11 : Differentiable ℝ b11) (db12 : Differentiable ℝ b12) (db13 : Differentiable ℝ b13) (db14 : Differentiable ℝ b14) (db15 : Differentiable ℝ b15) (db16 : Differentiable ℝ b16) (hb1 : HasVJP b1) (hb2 : HasVJP b2) (hb3 : HasVJP b3) (hb4 : HasVJP b4) (hb5 : HasVJP b5) (hb6 : HasVJP b6) (hb7 : HasVJP b7) (hb8 : HasVJP b8) (hb9 : HasVJP b9) (hb10 : HasVJP b10) (hb11 : HasVJP b11) (hb12 : HasVJP b12) (hb13 : HasVJP b13) (hb14 : HasVJP b14) (hb15 : HasVJP b15) (hb16 : HasVJP b16) (x : Vec (N * (3 * 224 * 224))) :
    efficientnetInputGradBFull N Ws Wh Wfc ((bnBatchLAHasVJP N 32 112 112 εs hεs γs βs).backward (StableHLO.batchMap N (flatConvStride2Xla Ws bs) x)) ((swishHasVJP (N * (32 * 112 * 112))).backward (StableHLO.bnBatchLA N 32 112 112 εs γs βs (StableHLO.batchMap N (flatConvStride2Xla Ws bs) x))) ((bnBatchLAHasVJP N 1280 7 7 εh hεh γh βh).backward (StableHLO.batchMap N (flatConv Wh bh) (opaqueA16 (stemB N Ws bs εs γs βs) b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 b11 b12 b13 b14 b15 b16 x))) ((swishHasVJP (N * (1280 * 7 * 7))).backward (StableHLO.bnBatchLA N 1280 7 7 εh γh βh (StableHLO.batchMap N (flatConv Wh bh) (opaqueA16 (stemB N Ws bs εs γs βs) b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 b11 b12 b13 b14 b15 b16 x)))) (hb1.backward (opaqueA0 (stemB N Ws bs εs γs βs) x)) (hb2.backward (opaqueA1 (stemB N Ws bs εs γs βs) b1 x)) (hb3.backward (opaqueA2 (stemB N Ws bs εs γs βs) b1 b2 x)) (hb4.backward (opaqueA3 (stemB N Ws bs εs γs βs) b1 b2 b3 x)) (hb5.backward (opaqueA4 (stemB N Ws bs εs γs βs) b1 b2 b3 b4 x)) (hb6.backward (opaqueA5 (stemB N Ws bs εs γs βs) b1 b2 b3 b4 b5 x)) (hb7.backward (opaqueA6 (stemB N Ws bs εs γs βs) b1 b2 b3 b4 b5 b6 x)) (hb8.backward (opaqueA7 (stemB N Ws bs εs γs βs) b1 b2 b3 b4 b5 b6 b7 x)) (hb9.backward (opaqueA8 (stemB N Ws bs εs γs βs) b1 b2 b3 b4 b5 b6 b7 b8 x)) (hb10.backward (opaqueA9 (stemB N Ws bs εs γs βs) b1 b2 b3 b4 b5 b6 b7 b8 b9 x)) (hb11.backward (opaqueA10 (stemB N Ws bs εs γs βs) b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 x)) (hb12.backward (opaqueA11 (stemB N Ws bs εs γs βs) b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 b11 x)) (hb13.backward (opaqueA12 (stemB N Ws bs εs γs βs) b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 b11 b12 x)) (hb14.backward (opaqueA13 (stemB N Ws bs εs γs βs) b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 b11 b12 b13 x)) (hb15.backward (opaqueA14 (stemB N Ws bs εs γs βs) b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 b11 b12 b13 b14 x)) (hb16.backward (opaqueA15 (stemB N Ws bs εs γs βs) b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 b11 b12 b13 b14 b15 x)) = (efficientnetBFullHasVJP (stemB N Ws bs εs γs βs) b1 b2 b3 b4 b5 b6 b7 b8 b9 b10 b11 b12 b13 b14 b15 b16 (headFwdB N Wh bh εh γh βh Wfc bfc) ⋯ db1 db2 db3 db4 db5 db6 db7 db8 db9 db10 db11 db12 db13 db14 db15 db16 ⋯ (stemBHasVJP N Ws bs εs hεs γs βs) hb1 hb2 hb3 hb4 hb5 hb6 hb7 hb8 hb9 hb10 hb11 hb12 hb13 hb14 hb15 hb16 (headFwdBHasVJP N Wh bh εh hεh γh βh Wfc bfc)).backward x

    The tie — efficientnetInputGradBFull is the certified whole-net paper-B0 gradient. The committed backward chain, with its stem/head BatchNorm and swish slots filled by the certified per-op backwards and its sixteen MBConv blocks left opaque, equals the backward of efficientnetBFullHasVJP at those eighteen stages. unfold, two rws, rfl.

    theorem Proofs.efficientnetInputGradBFull_eq_efficientnetForwardB_full_vjp (N : ℕ) {nCls : ℕ} (w : B0Weights nCls) (hεw : w.EpsPos) (x : Vec (N * (3 * 224 * 224))) :
    efficientnetInputGradBFull N w.sW w.hW w.fcW ((bnBatchLAHasVJP N 32 112 112 w.sε ⋯ w.sγ w.sβ).backward (StableHLO.batchMap N (flatConvStride2Xla w.sW w.sb) x)) ((swishHasVJP (N * (32 * 112 * 112))).backward (StableHLO.bnBatchLA N 32 112 112 w.sε w.sγ w.sβ (StableHLO.batchMap N (flatConvStride2Xla w.sW w.sb) x))) ((bnBatchLAHasVJP N 1280 7 7 w.hε ⋯ w.hγ w.hβ).backward (StableHLO.batchMap N (flatConv w.hW w.hb) (opaqueA16 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) (mbResidW N 14 14 w.b11) (mbStridedW N 7 7 w.b12) (mbResidW N 7 7 w.b13) (mbResidW N 7 7 w.b14) (mbResidW N 7 7 w.b15) (mbExpW N 7 7 w.b16) x))) ((swishHasVJP (N * (1280 * 7 * 7))).backward (StableHLO.bnBatchLA N 1280 7 7 w.hε w.hγ w.hβ (StableHLO.batchMap N (flatConv w.hW w.hb) (opaqueA16 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) (mbResidW N 14 14 w.b11) (mbStridedW N 7 7 w.b12) (mbResidW N 7 7 w.b13) (mbResidW N 7 7 w.b14) (mbResidW N 7 7 w.b15) (mbExpW N 7 7 w.b16) x)))) ((mbNoExpWHasVJP N 112 112 w.b1 ⋯ ⋯).backward (opaqueA0 (stemB N w.sW w.sb w.sε w.sγ w.sβ) x)) ((mbStridedWHasVJP N 56 56 w.b2 ⋯ ⋯ ⋯).backward (opaqueA1 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) x)) ((mbResidWHasVJP N 56 56 w.b3 ⋯ ⋯ ⋯).backward (opaqueA2 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) x)) ((mbStridedWHasVJP N 28 28 w.b4 ⋯ ⋯ ⋯).backward (opaqueA3 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) x)) ((mbResidWHasVJP N 28 28 w.b5 ⋯ ⋯ ⋯).backward (opaqueA4 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) x)) ((mbStridedWHasVJP N 14 14 w.b6 ⋯ ⋯ ⋯).backward (opaqueA5 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) x)) ((mbResidWHasVJP N 14 14 w.b7 ⋯ ⋯ ⋯).backward (opaqueA6 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) x)) ((mbResidWHasVJP N 14 14 w.b8 ⋯ ⋯ ⋯).backward (opaqueA7 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) x)) ((mbExpWHasVJP N 14 14 w.b9 ⋯ ⋯ ⋯).backward (opaqueA8 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) x)) ((mbResidWHasVJP N 14 14 w.b10 ⋯ ⋯ ⋯).backward (opaqueA9 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) x)) ((mbResidWHasVJP N 14 14 w.b11 ⋯ ⋯ ⋯).backward (opaqueA10 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) x)) ((mbStridedWHasVJP N 7 7 w.b12 ⋯ ⋯ ⋯).backward (opaqueA11 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) (mbResidW N 14 14 w.b11) x)) ((mbResidWHasVJP N 7 7 w.b13 ⋯ ⋯ ⋯).backward (opaqueA12 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) (mbResidW N 14 14 w.b11) (mbStridedW N 7 7 w.b12) x)) ((mbResidWHasVJP N 7 7 w.b14 ⋯ ⋯ ⋯).backward (opaqueA13 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) (mbResidW N 14 14 w.b11) (mbStridedW N 7 7 w.b12) (mbResidW N 7 7 w.b13) x)) ((mbResidWHasVJP N 7 7 w.b15 ⋯ ⋯ ⋯).backward (opaqueA14 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) (mbResidW N 14 14 w.b11) (mbStridedW N 7 7 w.b12) (mbResidW N 7 7 w.b13) (mbResidW N 7 7 w.b14) x)) ((mbExpWHasVJP N 7 7 w.b16 ⋯ ⋯ ⋯).backward (opaqueA15 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) (mbResidW N 14 14 w.b11) (mbStridedW N 7 7 w.b12) (mbResidW N 7 7 w.b13) (mbResidW N 7 7 w.b14) (mbResidW N 7 7 w.b15) x)) = (efficientnetForwardBFullHasVJP N w hεw).backward x

    The chain, at the sixteen concrete MBConv blocks, is efficientnetForwardBFullHasVJP's backward. The tie above instantiated at mbNoExpW/mbStridedW/mbResidW/mbExpW at B0Weights's widths, then HasVJP.backward_unique between the generic apex and the tactic-built whole-net witness — both are VJPs of the same eighteen-stage composition, so they have the same backward, and neither is unfolded.

    theorem Proofs.efficientnetInputGradBFull_correct (N : ℕ) {nCls : ℕ} (w : B0Weights nCls) (hεw : w.EpsPos) (x : Vec (N * (3 * 224 * 224))) (dy : Vec (N * nCls)) (i : Fin (N * (3 * 224 * 224))) :
    efficientnetInputGradBFull N w.sW w.hW w.fcW ((bnBatchLAHasVJP N 32 112 112 w.sε ⋯ w.sγ w.sβ).backward (StableHLO.batchMap N (flatConvStride2Xla w.sW w.sb) x)) ((swishHasVJP (N * (32 * 112 * 112))).backward (StableHLO.bnBatchLA N 32 112 112 w.sε w.sγ w.sβ (StableHLO.batchMap N (flatConvStride2Xla w.sW w.sb) x))) ((bnBatchLAHasVJP N 1280 7 7 w.hε ⋯ w.hγ w.hβ).backward (StableHLO.batchMap N (flatConv w.hW w.hb) (opaqueA16 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) (mbResidW N 14 14 w.b11) (mbStridedW N 7 7 w.b12) (mbResidW N 7 7 w.b13) (mbResidW N 7 7 w.b14) (mbResidW N 7 7 w.b15) (mbExpW N 7 7 w.b16) x))) ((swishHasVJP (N * (1280 * 7 * 7))).backward (StableHLO.bnBatchLA N 1280 7 7 w.hε w.hγ w.hβ (StableHLO.batchMap N (flatConv w.hW w.hb) (opaqueA16 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) (mbResidW N 14 14 w.b11) (mbStridedW N 7 7 w.b12) (mbResidW N 7 7 w.b13) (mbResidW N 7 7 w.b14) (mbResidW N 7 7 w.b15) (mbExpW N 7 7 w.b16) x)))) ((mbNoExpWHasVJP N 112 112 w.b1 ⋯ ⋯).backward (opaqueA0 (stemB N w.sW w.sb w.sε w.sγ w.sβ) x)) ((mbStridedWHasVJP N 56 56 w.b2 ⋯ ⋯ ⋯).backward (opaqueA1 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) x)) ((mbResidWHasVJP N 56 56 w.b3 ⋯ ⋯ ⋯).backward (opaqueA2 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) x)) ((mbStridedWHasVJP N 28 28 w.b4 ⋯ ⋯ ⋯).backward (opaqueA3 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) x)) ((mbResidWHasVJP N 28 28 w.b5 ⋯ ⋯ ⋯).backward (opaqueA4 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) x)) ((mbStridedWHasVJP N 14 14 w.b6 ⋯ ⋯ ⋯).backward (opaqueA5 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) x)) ((mbResidWHasVJP N 14 14 w.b7 ⋯ ⋯ ⋯).backward (opaqueA6 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) x)) ((mbResidWHasVJP N 14 14 w.b8 ⋯ ⋯ ⋯).backward (opaqueA7 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) x)) ((mbExpWHasVJP N 14 14 w.b9 ⋯ ⋯ ⋯).backward (opaqueA8 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) x)) ((mbResidWHasVJP N 14 14 w.b10 ⋯ ⋯ ⋯).backward (opaqueA9 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) x)) ((mbResidWHasVJP N 14 14 w.b11 ⋯ ⋯ ⋯).backward (opaqueA10 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) x)) ((mbStridedWHasVJP N 7 7 w.b12 ⋯ ⋯ ⋯).backward (opaqueA11 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) (mbResidW N 14 14 w.b11) x)) ((mbResidWHasVJP N 7 7 w.b13 ⋯ ⋯ ⋯).backward (opaqueA12 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) (mbResidW N 14 14 w.b11) (mbStridedW N 7 7 w.b12) x)) ((mbResidWHasVJP N 7 7 w.b14 ⋯ ⋯ ⋯).backward (opaqueA13 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) (mbResidW N 14 14 w.b11) (mbStridedW N 7 7 w.b12) (mbResidW N 7 7 w.b13) x)) ((mbResidWHasVJP N 7 7 w.b15 ⋯ ⋯ ⋯).backward (opaqueA14 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) (mbResidW N 14 14 w.b11) (mbStridedW N 7 7 w.b12) (mbResidW N 7 7 w.b13) (mbResidW N 7 7 w.b14) x)) ((mbExpWHasVJP N 7 7 w.b16 ⋯ ⋯ ⋯).backward (opaqueA15 (stemB N w.sW w.sb w.sε w.sγ w.sβ) (mbNoExpW N 112 112 w.b1) (mbStridedW N 56 56 w.b2) (mbResidW N 56 56 w.b3) (mbStridedW N 28 28 w.b4) (mbResidW N 28 28 w.b5) (mbStridedW N 14 14 w.b6) (mbResidW N 14 14 w.b7) (mbResidW N 14 14 w.b8) (mbExpW N 14 14 w.b9) (mbResidW N 14 14 w.b10) (mbResidW N 14 14 w.b11) (mbStridedW N 7 7 w.b12) (mbResidW N 7 7 w.b13) (mbResidW N 7 7 w.b14) (mbResidW N 7 7 w.b15) x)) dy i = ∑ j : Fin (N * nCls), pdiv (efficientnetForwardBFull N w) x i j * dy j

    The hand-written sixteen-block chain is the Jacobian-transpose of the committed efficientnetForwardBFull — at every input, every loss cotangent and every input pixel. The tie above read through efficientnetForwardBFullHasVJP_correct, whose proof is the shape check efficientnetForwardBFull_eq_chain: the nested-application forward the render denotes and the ∘-chain the VJP was assembled on are one function.