The batch-statistics cifar8-BN net — every gradient node IS the loss's derivative #

Cifar8BbnStepTieB.cifar8Bbn_train_step_tiedB ties the 38 gradient nodes of the cifar8wb_bbn_* steps, each at the cotangent the rendered chain threads to it; Cifar8BbnWholeBackB says that chain is the certified whole-net backward. This file is the rung above both: at those cotangents, the loss of the WHOLE net with one parameter varied is differentiable in it and the node is its gradient (HasGradAt), for any loss L of the logits with gradient g there (cifar8Bbn_net_lossGrad). The batched peer of Cifar8BnParamGrad.cifar8Bn_net_lossGrad (the per-image net) and of ResNet34ParamGrad.r34_net_lossGrad.

The pools #

A 2×2 max pool has no derivative at a window whose maximum is attained twice, and after a ReLU every dead window is such a tie. So, as the per-image rung and ResNet-34's stem do, the pools are frozen at a selection σ (one cell per window per example): the pool is then the gather at σ, and PoolSelB.maxPoolB_relu_eventuallyEq_sel says the pooled ReLU IS that gather along any parameter, near a point where every window is dead or tied only between twins — cells equal at every value of the weights upstream of that pool (C8PoolTwin1 … C8PoolTwin4). The gradient is then of the gather model, which is smooth, and the chain's pool cotangent is the scatter along σ (selScatter). At σ = argmax that scatter is the render's maxPoolBackB (PoolSelB.batchMapAux_maxPoolBackFlat_eq_selScatter), which is how cifar8Bbn_net_lossGrad's chain is the node tie's.

How it is built #

• c8Z1 … c8Z8 — the eight pre-ReLU activations (c8Aₖ = relu c8Zₖ); C8PoolTwinⱼ — twins of pool j, stated on c8Z₂ⱼ at every record; Cifar8BbnLossSmoothAt — the hypotheses.
• c8S₁ … c8S₈ — the loss read from stage k's pre-activation through the REAL net; c8G₁ … c8G₈ — the same through the gather model. c8_germ_stageₖ: along any parameter path through stage k's weights, the two agree near the point.
• cifar8Bbn_net_lossGrad — the head by hasGradAt_denseB/hasGradAt_relu, the body by the gather model pulled back one op at a time (hasGradAt_gatherReluB, hasGradAt_bnBatchLA, hasGradAt_conv, hasGradAt_relu), each node by its ParamGradNodes lemma, and each transported to the real net by its stage's germ.

Scope #

One device; the conv weight nodes on the renderers' bf16 switch (convWAt_hasGradAt), the others carry no flag. The smoothed-CE instantiation at the artifacts' own cotangent is cifar8Bbn_net_lossGrad_CE.