The FULL EfficientNet-B0 — all 16 MBConv blocks, batched forward graph + faithfulness #
Scales EfficientNetStagesPC's representative (stem + 3 MBConv + head) to the real B0
[t,c,n,s,k] spec — 16 MBConv layers — at the batched index N·(c·h·w). Pure enumeration + chaining
of the generic per-block machinery; the only genuinely-new piece is the 4th block shape
(mbExp: expand + stride-1 + no residual, used by stage-5/stage-7 first blocks where ic ≠ oc).
B0 stage spec [t,c,n,s,k]:
s1 (1,16,1,1,3) s2 (6,24,2,2,3) s3 (6,40,2,2,5) s4 (6,80,3,2,3)
s5 (6,112,3,1,5) s6 (6,192,4,2,5) s7 (6,320,1,1,3); stem 3×3-s2 (3→32), head 1×1 (320→1280)→GAP→dense.
Per-block (ic, mid=t·ic, oc, r=⌈ic/4⌉, k, spatial, kind):
b1 32→16 mid32 r8 k3 @112 noExp b9 80→112 mid480 r20 k5 @14 exp(no-resid)
b2 16→24 mid96 r4 k3 112→56 strided b10 112→112 mid672 r28 k5 @14 resid
b3 24→24 mid144 r6 k3 @56 resid b11 112→112 mid672 r28 k5 @14 resid
b4 24→40 mid144 r6 k5 56→28 strided b12 112→192 mid672 r28 k5 14→7 strided
b5 40→40 mid240 r10 k5 @28 resid b13 192→192 mid1152 r48 k5 @7 resid
b6 40→80 mid240 r10 k3 28→14 strided b14 192→192 mid1152 r48 k5 @7 resid
b7 80→80 mid480 r20 k3 @14 resid b15 192→192 mid1152 r48 k5 @7 resid
b8 80→80 mid480 r20 k3 @14 resid b16 192→320 mid1152 r48 k3 @7 exp(no-resid)
Padding: the 3×3/s2 stem AND the four strided depthwises (b2, b4, b6, b12) pad at the XLA-SAME
phase — TF's SAME at every strided conv, so the spec is tf_efficientnet_b0's. (Until D3,
2026-10-12, the four depthwises padded symmetrically, a hybrid neither timm net computes;
scripts/parity/enet_timm_parity.py --pad hybrid still measures that distance.)
Precision is a flag on the graph. The block builders (EfficientNetStagesPC, mbExpGraphB
here) choose every conv and depthwise constructor through the renderers' switch
(StableHLO.PrecisionSwitch), so one graph describes the f32 artifacts (bf16 := false) and the
bf16 ones (bf16 := true, the kinds efficientnetin_emarmsdp64dropdowxeps0001bf16 emits); the
squeeze-excite denses, BatchNorm, swish, GAP and the classifier carry no flag because no render
switches them. efficientnetFwdGraphBFull_faithful holds at either value through Bf16Erasure
(the bf16 kind at the identity rounding is its f32 peer); the rounding itself is not modelled, as
it is not at f32: den is over ℝ at both values.
References #
- Hu, Shen, Sun 2018, Squeeze-and-Excitation Networks. https://arxiv.org/abs/1709.01507
- Tan & Le 2019, EfficientNet: Rethinking Model Scaling for CNNs. https://arxiv.org/abs/1905.11946
The tail every MBConv block shares — depthwise BatchNorm → swish → squeeze-excite →
1×1 project → project BatchNorm. MBW and MBWNoExp both extend it, so the three block kinds
share one tail chain through toEnTail.
- dε : ℝ
- dγ : Vec mid
- dβ : Vec mid
- z1 : Mat mid rd
- zb1 : Vec rd
- z2 : Mat rd mid
- zb2 : Vec mid
- pW : Kernel4 oc mid 1 1
- pb : Vec oc
- pε : ℝ
- pγ : Vec oc
- pβ : Vec oc
Instances For
Weights of the MBConv1 block (t=1, no expand; depthwise on ic, SE ic→r→ic, project ic→oc):
the depthwise kernel, then the shared tail.
- dW : DepthwiseKernel ic kh kw
- db : Vec ic
Instances For
All 262 EfficientNet-B0 parameters: stem (3×3-s2 3→32) + 16 MBConv blocks (the real [t,c,n,s,k]
spec) + head (1×1 320→1280) + dense (1280→nCls).
- sW : Kernel4 32 3 3 3
- sb : Vec 32
- sε : ℝ
- sγ : Vec 32
- sβ : Vec 32
- b1 : MBWNoExp 32 16 8 3 3
- b2 : MBW 16 96 24 4 3 3
- b3 : MBW 24 144 24 6 3 3
- b4 : MBW 24 144 40 6 5 5
- b5 : MBW 40 240 40 10 5 5
- b6 : MBW 40 240 80 10 3 3
- b7 : MBW 80 480 80 20 3 3
- b8 : MBW 80 480 80 20 3 3
- b9 : MBW 80 480 112 20 5 5
- b10 : MBW 112 672 112 28 5 5
- b11 : MBW 112 672 112 28 5 5
- b12 : MBW 112 672 192 28 5 5
- b13 : MBW 192 1152 192 48 5 5
- b14 : MBW 192 1152 192 48 5 5
- b15 : MBW 192 1152 192 48 5 5
- b16 : MBW 192 1152 320 48 3 3
- hW : Kernel4 1280 320 1 1
- hb : Vec 1280
- hε : ℝ
- hγ : Vec 1280
- hβ : Vec 1280
- fcW : Mat 1280 nCls
- fcb : Vec nCls
Instances For
All 49 BatchNorm εs of EfficientNet-B0 are positive: stem, the 16 blocks, head.
Instances For
MBConv6 expand + stride-1 + no-residual graph (the mbResidGraphB body without the addV skip).
Equations
- One or more equations did not get rendered due to their size.
Instances For
Equations
Instances For
Equations
Instances For
Equations
Instances For
The B0 forward every B0 capstone is stated against: the stem 3×3/s2 at the XLA-SAME phase,
the sixteen MBConv blocks with their four strided depthwises at that phase too (TF SAME), SE
width ic/4, batch BatchNorm, then the head. No drop-path or dropout; the drop-carrying
forward is EfficientNetFullB0Drop's.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The full batched EfficientNet-B0 forward graph at the batched index N·(c·h·w): stem → 16
MBConv blocks (the real [t,c,n,s,k] spec, 3×3 and 5×5 depthwise, true batch-norm, squeeze-excite,
4 stride-2 downsamples, identity residuals where s=1 ∧ ic=oc) → head → GAP → dense. bf16
selects the conv and depthwise kinds at every site (.convAt / .convStridedXlaAt /
.depthwiseAt / .depthwiseStridedXlaAt, StableHLO.PrecisionSwitch): false is the f32
artifacts' graph, true the bf16 ones'; the squeeze-excite and the dense stay f32 in both.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Full batched EfficientNet-B0 forward faithfulness. The full 16-MBConv batched graph (true
batch-norm + SE) denotes efficientnetForwardBFull. Chained from the per-block *GraphW_faithful
lemmas (one rw per block, outermost→innermost), then a structural rfl (the forward is
nested-application form, blocks opaque).
The full EfficientNet-B0 has a (correct) VJP. Chained from the per-block gradients (stem → 16
MBConv blocks → head) via vjpComp. Stated on the ∘-composition of the blocks (= the full
forward by construction; keeps the blocks opaque so the chain closes structurally). The full-depth,
batched, true-batch-norm + SE analogue of efficientnetHasVJP.
Equations
- One or more equations did not get rendered due to their size.
Instances For
efficientnetForwardBFull = the ∘-chain of the VJP's statement — the
kernel-checked bridge between the nested-application and composition forms,
closing the form-gap this file shipped with. PROOF-SHAPE MATTERS (the ConvNeXt-T
convNextForwardTCh_eq_chain lesson): equation-lemma rw + 17 comp_apply
rewrites close syntactically; a simp/rfl proof of the same statement makes
the kernel reduce the block bodies (no reducibility, no defeq cache) and
deterministically time out.
The .correct field of efficientnetForwardBFullHasVJP, restated at the nested forward
efficientnetForwardBFull through efficientnetForwardBFull_eq_chain: its backward is the
pdiv-contracted Jacobian at every input. The tie of the hand-written input-gradient chain to
this backward is efficientnetInputGradBFull_eq_efficientnetForwardB_full_vjp.