The cifar8-bn (Chapter 4 deeper, 8-conv per-channel BN) TIE #
cifar8's tie (Cifar8Tie.cifar8_train_step_tied_certified) + a BN-back at every conv. The backward
chain alternates BN-output cotangent dyBnᵢ (relu-masked — fed to the γ/β ops) and conv-output
cotangent cotCᵢ (bnPerChannelTensor3GradInput of dyBnᵢ — fed to the conv W/b ops), repeated
over 4 conv→conv→pool stages, crossing each pool as conv-back then maxpool-back.
Zero new ops/bridges/constructors. Conv ties reuse SgdNode.convW_den/convB_den; BN ties reuse
SgdNode.bnGamma_den/bnBeta_den; the dense head reuses SgdNode.denseW_den/denseB_den, the
loss cotangent cifar8's. All 38 parameter tensors (8 conv W/b, 8 BN γ/β, 3 dense W/b) are stated in
cifar8Bn_train_step_tied_certified, each with the generics.
Not the trained artifact. The committed cifar8_bn_* / cifar8w_bn_* arms are this
renderer with a separate optimizer, feeding the same chain to *Grad nodes; they are tied in
Cifar8BnTieG.cifar8Bn_train_step_tiedG.
Scope (same as the rest of the suite) #
- Below the output layer the cotangents are the rendered chain (
cotCᵢ,dyBnᵢincifar8Bn_train_step_tied_certified, andmlpCotOut1/mlpCotOut0in the head). The loss-gradient statement isCifar8BnTieG.cifar8Bn_net_lossGrad, at the trained arms'*Gradnodes and with each pool's cotangent routed to one maximal cell, as the renderedselect_and_scatterdoes; this chain routes it to the first maximal cell (maxPool2Argmax), the op's own choice, so each pool step is the capstone's scatter at that selection (SmallParamGrad.maxpool_flatDenote_eq_selScatter). - Conv/BN backward rendered hand-written (cotangent SSA ↔ chain-cot per-op trust); per-op
prettylexing; ℝ → Float32.
Whole cifar8-bn train step, tied. All 38 parameter tensors (8 conv W+b, 8 BN γ+β,
3 dense W+b; 30 conjuncts, one BnSgdPairTied per γ/β pair), at the real cifar8-bn forward,
denote θ − lr·(certified per-layer Jacobian · c) with c the rendered backward-chain
cotangent driven by the composed softmax-CE cotangent g. The conv ops are fed the BN-back
cotangents cotC1–8; the BN ops the relu-masked cotangents dyBn1–8; both are the rendered
cifar8-bn backward chain (cifar8's chain + a BN-back at every conv). The dense head is fed
mlpCotOut0, mlpCotOut1 and g. The loss-gradient form is
Cifar8BnTieG.cifar8Bn_net_lossGrad (see the module's Scope).