EfficientNet-B0 with stochastic depth and classifier dropout — forward, graph, faithfulness #

EfficientNetFullB0 and EfficientNetFullB0Eval state the sixteen-block net without its two regularisers; the *drop* / *do* artifacts render them. This file states both forwards WITH them, as the renderer (enetFwdChain) emits them, and proves the typed graphs denote them:

• stochastic depth (sd, the %dp<i> inputs) — a per-example scale dropPath on the residual BRANCH, before the skip add, at the nine blocks that carry a skip (b3 b5 b7 b8 b10 b11 b13 b14 b15, the renderer's enetDropIdxs [2, 4, 6, 7, 9, 10, 12, 13, 14], block-indexed);
• classifier dropout (cd, the %do input) — a per-element dropout between the GAP and the dense, width 1280 at every class count.

Each is Optional, as the renderer's sd / cd flags are: none renders no node, so one statement covers efficientnet_drop_fwd (sd only), efficientnet_do_fwd (cd only) and efficientnetin_dropdo_fwd (both), and their _eval twins at inference BatchNorm.

• efficientnetFwdGraphBFullDrop_faithful / efficientnetFwdGraphBFullEvalDrop_faithful — the graph denotes the forward, at every mask;
• efficientnetForwardBFullDrop_none / efficientnetForwardBFullDrop_ones — with no site, or at the all-ones masks the driver passes at eval, the forward IS efficientnetForwardBFull (likewise the eval twins, efficientnetForwardBFullEvalDrop_none / efficientnetForwardBFullEvalDrop_ones). The identity is exact: the keep probability is folded into the mask (Training/DropPath). The Option sites, their graph nodes and their VJPs are Proofs.Foundation.DropSites', shared with the ties that thread the same binder; the block's and the head's VJPs with the sites (mbResidDropWHasVJP, headDoFwdBHasVJP) are here, the forward's peers.

The residual-block-with-drop and the dropout head are text-guarded against the renderer in Codegen/FwdGraphTextTies at training BatchNorm; the eval graphs use the render's names too, and their drop-free blocks are guarded at inference BatchNorm. The training graph takes the renderers' bf16 flag through the same switch as EfficientNetFullB0 (StableHLO.PrecisionSwitch), so efficientnetFwdGraphBFullDrop … true is the forward of the *bf16 train steps — efficientnetin_emarmsdp64dropdowxeps0001bf16, the run the book reports, among them — read over ℝ as the f32 one is; the eval graphs stay f32, as the eval artifacts are. The train steps' backward through the drop sites is EfficientNetStepTieG's, at the same sd / cd binders.

References #

• Huang et al. 2016, Deep Networks with Stochastic Depth. https://arxiv.org/abs/1603.09382
• Srivastava et al. 2014, Dropout: A Simple Way to Prevent Neural Networks from Overfitting. https://jmlr.org/papers/v15/srivastava14a.html