Row-wise lifting of a per-row HasVJP family. Each row r gets its own map g r (with its own VJP); the matrix backward runs (g r).backward on row r's cotangent. The per-row peer of rowwise_has_vjp_mat.

Per-channel BatchNorm (matrix view): BN each row (= channel-slice of m = h·w spatial cells) with its own (γ_c, β_c). The real-ResNet BN that bnForward's global scalar version approximates.

Per-channel BN VJP (block-diagonal). Each channel runs its own bn_has_vjp; the cross-channel Jacobian blocks vanish (pdivMat_rowIndep_perRow_at).

Per-channel BN as a flat-vector op Vec (oc·m) → Vec (oc·m) (row-major, channel c = the m-wide slab at flat positions finProdFinEquiv (c, ·)).

Per-channel BN flat VJP — the block-diagonal matrix VJP bridged to Vec.

Per-channel consolidated BN input-gradient — the renderable closed form: run the per-example three-term bn_grad_input on each channel-slice (m = h·w spatial cells), reusing that channel's γ_c. This is exactly what a bnPerChannelBack SHlo op / renderLNBack-per-channel emits; the abstract bnPerChannelFlat_has_vjp.backward is the spec it must match.

Re-association index Fin (oc*(h*w)) → Fin (oc*h*w): a Mat-split flat index (c, s) with s ↔ (hi, wi) maps to the Tensor3 flat index ((c, hi), wi). A pure product re-association — no arithmetic.

Re-association index Fin (oc*h*w) → Fin (oc*(h*w)): the inverse direction, Tensor3 ((c, hi), wi) ↦ Mat-split (c, (hi, wi)).

Tensor3 → Mat-split reindex: read the ((c,hi),wi) cell at Mat position (c, (hi,wi)). A reindexCLM, hence continuous-linear / differentiable.

Mat-split → Tensor3 reindex (the inverse relabeling).

Inference BN on one channel's m activations: yᵢ = γ · (xᵢ − μ) · (var + ε)^(−1/2) + β, with μ/var supplied rather than computed from x. The bnForward peer — note it takes x pointwise, where bnForward reduces over all of x to get its own μ/σ².

Per-channel inference BN (Mat layout) — row c gets channel c's frozen stats.

Per-channel inference BN (flat layout) — the bnPerChannelFlat peer.

Per-channel inference BN (Tensor3 layout) — the bnPerChannelTensor3 peer, through the same reassoc bridge. This is what SHlo.bnPerChannelEvalF denotes.

The rendered per-channel γ gradient: dγ_c = Σ_{s} dy_(c,s) · x̂_(c,s) (the reduce over batch/spatial of dy·x̂ that the bnGammaSgd op emits). x̂ is recomputed from the saved BN input v (the conv output). Lives here (not PerChannelBNGrad) so the bnGammaSgd SHlo op's den can reference it.

The rendered per-channel β gradient: dβ_c = Σ_{s} dy_(c,s).

VJP of the forward reindex — the scatter pdiv_reindex gives. Mirrors decimateFlat_has_vjp.

Per-channel BatchNorm on the Tensor3 (oc*h)*w activation layout. Conjugate the Mat-split bnPerChannelFlat by the layout bridge: relabel to Mat-split, normalize each channel over its h·w spatial cells, relabel back. This is the op B9 wires into the ResNet-34 trainer (its den target).

Per-channel BN (Tensor3 layout) VJP — block-diagonal across channels, lifted through the layout bridge by vjp_comp (twice).

Renderable per-channel BN backward on the Tensor3 (oc*h)*w layout — relabel to Mat-split, run the per-channel consolidated three-term bnPerChannel_grad_input, relabel back. This is exactly what the bnPerChannelBack SHlo op emits (per-channel renderLNBack, reducing over the spatial axis); its faithfulness spec is below.

Transpose-reindex Fin (oc*(N*(h*w))) → Fin (N*(oc*(h*w))): a per-channel Mat index (c, (n, s)) maps to the network [N,C,H,W] flat index (n, (c, s)) — swap the batch and channel axes (s ↔ (hi,wi) carried along). A permutation.

Transpose-reindex Fin (N*(oc*(h*w))) → Fin (oc*(N*(h*w))): the inverse, network (n, (c, s)) ↦ per-channel Mat (c, (n, s)).

[N,C,H,W] → [C,N·H·W] reindex (gather the network cell at the Mat position).

[C,N·H·W] → [N,C,H,W] reindex (the inverse relabeling).

Batch-norm per channel on the network's [N,C,H,W] layout. Conjugate the Mat-split bnPerChannelFlat (with m = N·h·w, the whole batch's cells per channel) by the transpose bridge: relabel [N,C,H,W] → [C, N·H·W], normalize each channel over ALL its batch+spatial cells, relabel back. The EfficientNet normalization.

⭐⭐ Synchronised batch-norm on the [N,C,H,W] layout — statistics HANDED IN.

bnBatchTensor4's peer, conjugated by the same [N,C,H,W] → [C, N·H·W] bridge, but the per-channel normalisation reads μ and the second moment m2 from its arguments instead of reducing x for them. Under data parallelism those arguments are the ALL-REDUCED global statistics — which is how one replica normalises over a batch it cannot see.

⚠ It takes the SECOND MOMENT and forms the variance itself as m2 − μ². That is not a convenience: E[x²] of a union of equal shards is the mean of the shards' E[x²], so it survives an allReduceMeanF, whereas the variance of a union is not the mean of the shards' variances and does not.

Batch-norm (network layout) VJP — block-diagonal across channels (now coupling the whole batch within each channel), lifted through the transpose bridge.

Renderable batch-norm backward on the [N,C,H,W] layout — relabel to the per-channel Mat, run the consolidated three-term bnPerChannel_grad_input over the whole batch (m = N·h·w), relabel back. Exactly what the batched batch-norm backward StableHLO fragment emits (reduce over [0,2,3] per channel).

Per-channel SYNC backward (flat layout) — bnPerChannel_grad_input's peer, with each channel's μ, E[x²] and two reduction means supplied rather than reduced out of x/dy. Under data parallelism those are the all-reduced global ones.

⭐⭐ The per-channel γ gradient with x̂ at HANDED-IN statistics — bnPerChannel_grad_gamma's peer. Under sync-BN the forward normalised with the all-reduced global μ/E[x²], so ∂L/∂γ_c = Σ dy·x̂ must use the SAME x̂; bnPerChannel_grad_gamma rebuilds it from the shard's own statistics (bnXhat), which is a different function once R > 1. β's gradient reads no statistic and needs no peer.

⭐⭐ The SYNC batch-norm input-VJP on [N,C,H,W] — bnBatchTensor4_grad_input's peer, through the same bnchwFwd/bnchwBack bridge. What a replica emits for its shard of the backward, given the four all-reduced per-channel statistic vectors.