The cifar8 BN net at BATCH statistics — cifar8wb_bbn #
The eight-conv CIFAR net with per-channel BatchNorm, rendered on the batched op family with every
BN site reducing over [0,2,3] — Ioffe & Szegedy's layer, the one Chapter 5's ResNet-34 trains —
instead of the per-example [2,3] the cifar8w_bn / cifar8wb_bn artifacts normalise by. Same
38 parameters, same layer list; what moves is the statistic each BN divides by, and with it
everything a batch-statistics layer needs that a per-image one does not:
- the train step hands back each BN's batch μ/var (
bnBatchMeanB/bnBatchVarB, 16[c]outputs after%loss/%bc1/%bc2) and takes 16 passthrough stat inputs before%onehot, exactly the[θ|m|v]+ stats protocolResNet34RenderBuses andVerifiedNet.trainAdamScheddrives (bnChannelsnon-empty ⇒ EMA the returned stats); @cifar8wb_bbn_fwd_eval— the inference forward with frozen running statistics at every BN site (bnEvalSite→bnPerChannelEvalF), the artifact the driver scores through;@cifar8wb_bbn_fwd— the batch-BN training forward, emitted from the SAME chain the train step differentiates (cifar8BbnFwdChainB), so it is a byte prefix of the train step and theLEAN_MLIR_EVAL_BATCHSTATSdiagnostic reads the net that trains.
Three BN emit sites — forward, input-cotangent, γ-gradient — are the shared
SyncBnSites
ones every batch-BN renderer uses, so at replicas > 1 this render is sync-BN for free; no
data-parallel artifact is written.
Faithful by construction like every render here: the AST is built only from verified constructors,
so pretty(provenGraph) needs no new proof. Hand-written text: the signatures, the constants, the
report-only %loss and the %bc1/%bc2 passthroughs — none of it feeds a parameter or a statistic.
The per-example ties (Cifar8BnStepTie, Cifar8BnStepTieG) are stated at bnPerChannel*; the
batch-statistics step tie at these nodes is Cifar8BbnStepTieB (planned in
planning/cifar-batch-bn-rerender.md). The convolutions, pools, dense head and optimizer tails
are byte-for-byte the nodes cifar8BnTrainStepBText emits — only the BN sites and the stat
plumbing differ — so bf16 reaches the same 23 convolutions it reaches there, and BN stays f32.
The #eval writers are in CnnArtifacts.lean.
The 38-parameter signature of the cifar8 BN net, cifar8BnVerified.toSpecs order:
(conv W, conv b, BN γ, BN β) ×8, then the three dense (W, b). ONE source for the order and the
shapes — the arg signature, the packed m/v blocks and the return types all read it.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The 16 running-stat slots — 8 BN layers × (μ, var), each [c], in BN-forward order
n1 … n8, μ before var per layer (bnStatSlots). This is exactly the order
cifar8wbBbnVerified.bnChannels lists (#[16,16,16,16,32,32,32,32]), which is how the driver
packs runningBnStats. The train step takes them as passthrough inputs (%n1mui, …) and returns
the batch statistics in the same order; @cifar8wb_bbn_fwd_eval reads them as %n1mu, ….
Equations
- One or more equations did not get rendered due to their size.
Instances For
One conv→BN→relu layer at batch statistics, batched op family: batchOp (convAt bf16 …)
(the bf16 twin when bf16), then bnFwdSite (bnBatchF at one replica, the sync-BN
composition above it), then batchOp relu. Parameters %W{i}/%cb{i}/%g{i}/%bt{i};
the sync-BN collectives are tagged n{i}, which is no parameter's name.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Saved forward SSA names the train step's backward and statistics read.
- code : String
- l1 : C8BbnLayer
- l2 : C8BbnLayer
- l3 : C8BbnLayer
- l4 : C8BbnLayer
- l5 : C8BbnLayer
- l6 : C8BbnLayer
- l7 : C8BbnLayer
- l8 : C8BbnLayer
- pool1 : String
- pool2 : String
- pool3 : String
- pool4 : String
- h9 : String
- a9 : String
- ha : String
- aa : String
- log : String
Instances For
The batch-BN training forward, batched op family, at N := B — the ONE chain both
@cifar8wb_bbn_fwd and every cifar8wb_bbn_*_train_step emit, so the forward artifact is a
byte prefix of the train step by construction. (conv→BN→relu)×2→pool, ×4, then
(dense→relu)×2→dense. h,w are the final pooled spatials; stage spatials double from s4 = 2h back up to s1 (the input).
Equations
- One or more equations did not get rendered due to their size.
Instances For
The parameter block of a signature: %{nm}{suf}: {ty} for each entry of cifar8BnSigList.
Equations
- One or more equations did not get rendered due to their size.
Instances For
@<slug>_fwd — the batch-BN TRAINING forward (BN over the batch's own statistics),
cifar8BbnFwdChainB and a return. Not what the driver scores through — that is
@<slug>_fwd_eval — but what LEAN_MLIR_EVAL_BATCHSTATS reads and what the BN-world check
in trainAdamSched compares against the train step (dimensions = [0, 2, 3] in both). A byte
prefix of every <slug>_*_train_step body from %v0 on.
Equations
- One or more equations did not get rendered due to their size.
Instances For
@<slug>_fwd_eval — the inference forward with FROZEN running statistics. The
per-example chain (flatConvF / maxPoolF / denseF / reluF under pretty B) with every
BN site bnEvalSite (bnPerChannelEvalF: affine BN at the %n{i}mu/%n{i}var graph inputs,
no reduction), so this forward is class-batch-independent. Inputs: %x, the 38 parameters,
then the 16 stat slots of cifar8BbnStatSigList — the layout trainAdamSched packs
(paramShapes ++ bnStatShapes). Returns logits [B, nClasses]. f32 only: BN is f32 in every
arm, and the bf16 train steps score through this same artifact, as the ImageNet nets do.
Equations
- One or more equations did not get rendered due to their size.
Instances For
One layer's backward at the BN: the relu mask on the BN output (selectPosB), then
bnBackSite (bnBatchBack at one replica, the sync-BN backward above it) reading the layer's
conv output and packed statistics. Returns (code, dBn, dHc): the cotangent at the BN output
(which the γ/β gradients read) and at the conv output (which the conv gradients read).
Equations
- One or more equations did not get rendered due to their size.
Instances For
One layer's four parameter gradients and optimizer tails, in cifar8BnSigList order
(W, conv bias, γ, β): the batched conv weight gradient (bf16 twin when bf16) and bias
gradient at the layer's INPUT xIn, bnGammaSite and bnBetaGradB at the BN-output cotangent.
Returns (code, θ' names, m' names, v' names).
Equations
- One or more equations did not get rendered due to their size.
Instances For
One BN layer's returned batch statistics — μ then var, each [oc] — from that layer's BN
INPUT (bnBatchMeanB / bnBatchVarB: the SAME bnMean/bnVar the forward normalised by).
At replicas > 1 read off the all-reduced packed vector (bnStatsMeanB / bnStatsVarB), so
the host EMAs the GLOBAL batch statistics.
Equations
- One or more equations did not get rendered due to their size.
Instances For
cifar8BbnTrainStepBText — the cifar8 BN train step at BATCH statistics.
cifar8BnTrainStepBText with its eight bnPerChannelF/bnPerChannelBack/bnGammaGrad sites
replaced by the batch-coupled bnFwdSite/bnBackSite/bnGammaSite (bnBatchF /
bnBatchBack / bnGammaGradB, reduce over [0,2,3]) and bnBetaGradB, plus the
running-statistics protocol: 16 passthrough stat inputs after %bc2 and 16 batch-statistic
outputs after %bc2 (cifar8BbnStatSigList order). Signature otherwise the packed
[θ|m|v] one every cifar8 optimizer render shares — %x, 38 θ, 38 m, 38 v, %lr/%bc1/%bc2,
16 stats, %onehot → 38 θ', 38 m', 38 v', %loss/%bc1/%bc2, 16 stats — so
VerifiedNet.trainAdamSched drives it unchanged, as it drives ResNet-34's.
Three optimizers from one body (optTail); bf16 routes the 23 convolutions through their
bf16 twins and leaves BN in f32, as cifar8BnTrainStepBText and every ImageNet render do.
Entry @{slug}_{variant}_train_step, the driver's convention.
Equations
- One or more equations did not get rendered due to their size.