MobileNetV4-Conv-M with stochastic depth and classifier dropout — forward, graph, faithfulness #

MobileNetV4FullB states the net and its classifier-dropout form (mnv4FwdGraphBFullDo). The paper-tier train steps mnv4in_acc{,dp}8x128wxdropdowd01bf16 also carry stochastic depth (sd, the %dp<k> inputs): a per-example scale dropPath on the residual BRANCH, after the project BN and before the skip add, at each of the eighteen skip rows. Site k is the k-th skip row in table order (the renderer's mnv4DropSites):

The three strided rows (1, 3, 11) have no skip and no site. This file states that forward with both regularisers, as the renderer's uibFwdSkipB … (drop := some k) and mnv4HeadFwdB … (cd := true) emit them, and proves the typed graph denotes it:

• mnv4FwdGraphBFullDrop_faithful — the graph denotes mobilenetv4ForwardBFullDrop, at every pair of masks;
• mobilenetv4ForwardBFullDrop_sdOnes — at the all-ones drop masks the forward is the dropout forward mobilenetv4ForwardBFullDo; mobilenetv4ForwardBFullDrop_ones — at all-ones masks everywhere it is mobilenetv4ForwardBFull, exactly (the keep probability is folded into the mask, Training/DropPath).

Those two artifacts are bf16; this is their f32 form, as mnv4FwdGraphBFullDo is for the classifier-dropout ones. The train steps' backward through the drop sites is outside this statement, as it is outside MobileNetV4StepTieB.

Why a separate file, and why rw. MobileNetV4FullB's group graphs are private and their whole-net proof is the one whose simp only spelling dies in the kernel, so the drop-carrying groups are new definitions here rather than edits there. Every group and whole-net step below is an outside-in rw with a faithfulness lemma of the shape den <subgraph> = <sub>.fwd (den ·), so den never meets a literal-width term it could start evaluating.

References #

• Huang et al. 2016, Deep Networks with Stochastic Depth. https://arxiv.org/abs/1603.09382
• Qin et al. 2024, MobileNetV4: Universal Models for the Mobile Ecosystem. https://arxiv.org/abs/2404.10518

At the all-ones masks each group is its CertLayer's forward. Stated against the *_fwd_apply expansions, where the terms are variables.