Documentation

LeanMlir.Proofs.Codegen.CnnBbnRender

The cifar8 BN net at BATCH statistics — cifar8wb_bbn #

The eight-conv CIFAR net with per-channel BatchNorm, rendered on the batched op family with every BN site reducing over [0,2,3] — Ioffe & Szegedy's layer, the one Chapter 5's ResNet-34 trains — instead of the per-example [2,3] the cifar8w_bn / cifar8wb_bn artifacts normalise by. Same 38 parameters, same layer list; what moves is the statistic each BN divides by, and with it everything a batch-statistics layer needs that a per-image one does not:

Three BN emit sites — forward, input-cotangent, γ-gradient — are the shared SyncBnSites ones every batch-BN renderer uses, so at replicas > 1 this render is sync-BN for free; no data-parallel artifact is written.

Faithful by construction like every render here: the AST is built only from verified constructors, so pretty(provenGraph) needs no new proof. Hand-written text: the signatures, the constants, the report-only %loss and the %bc1/%bc2 passthroughs — none of it feeds a parameter or a statistic. The per-example ties (Cifar8BnStepTie, Cifar8BnStepTieG) are stated at bnPerChannel*; the batch-statistics step tie at these nodes is Cifar8BbnStepTieB (planned in planning/cifar-batch-bn-rerender.md). The convolutions, pools, dense head and optimizer tails are byte-for-byte the nodes cifar8BnTrainStepBText emits — only the BN sites and the stat plumbing differ — so bf16 reaches the same 23 convolutions it reaches there, and BN stays f32.

The #eval writers are in CnnArtifacts.lean.

def Proofs.StableHLO.cifar8BnSigList (ic c1 c2 c3 c4 flat d1 nClasses kH kW : ℕ) :

The 38-parameter signature of the cifar8 BN net, cifar8BnVerified.toSpecs order: (conv W, conv b, BN γ, BN β) ×8, then the three dense (W, b). ONE source for the order and the shapes — the arg signature, the packed m/v blocks and the return types all read it.

Equations
  • One or more equations did not get rendered due to their size.
Instances For

    The 16 running-stat slots — 8 BN layers × (μ, var), each [c], in BN-forward order n1 … n8, μ before var per layer (bnStatSlots). This is exactly the order cifar8wbBbnVerified.bnChannels lists (#[16,16,16,16,32,32,32,32]), which is how the driver packs runningBnStats. The train step takes them as passthrough inputs (%n1mui, …) and returns the batch statistics in the same order; @cifar8wb_bbn_fwd_eval reads them as %n1mu, ….

    Equations
    • One or more equations did not get rendered due to their size.
    Instances For

      One conv→BN→relu layer's saved SSA names: conv out (= BN input), BN out, the BN's packed sync statistics ("" at one replica), relu out.

      Instances For
        def Proofs.StableHLO.c8bbnConvBnRelu (B ic oc hh ww kH kW : ℕ) (bf16 sync : Bool) (replicas : ℕ) (epsStr : String) (i : ℕ) (xIn : String) :

        One conv→BN→relu layer at batch statistics, batched op family: batchOp (convAt bf16 …) (the bf16 twin when bf16), then bnFwdSite (bnBatchF at one replica, the sync-BN composition above it), then batchOp relu. Parameters %W{i}/%cb{i}/%g{i}/%bt{i}; the sync-BN collectives are tagged n{i}, which is no parameter's name.

        Equations
        • One or more equations did not get rendered due to their size.
        Instances For

          Saved forward SSA names the train step's backward and statistics read.

          Instances For
            def Proofs.StableHLO.cifar8BbnFwdChainB (B ic c1 c2 c3 c4 h w d1 nClasses kH kW : ℕ) (epsStr : String) (bf16 : Bool := false) (replicas : ℕ := 1) :

            The batch-BN training forward, batched op family, at N := B — the ONE chain both @cifar8wb_bbn_fwd and every cifar8wb_bbn_*_train_step emit, so the forward artifact is a byte prefix of the train step by construction. (conv→BN→relu)×2→pool, ×4, then (dense→relu)×2→dense. h,w are the final pooled spatials; stage spatials double from s4 = 2h back up to s1 (the input).

            Equations
            • One or more equations did not get rendered due to their size.
            Instances For

              The parameter block of a signature: %{nm}{suf}: {ty} for each entry of cifar8BnSigList.

              Equations
              • One or more equations did not get rendered due to their size.
              Instances For
                def Proofs.StableHLO.cifar8BbnFwdText (B ic c1 c2 c3 c4 h w d1 nClasses kH kW : ℕ) (epsStr slug : String) (bf16 : Bool := false) :

                @<slug>_fwd — the batch-BN TRAINING forward (BN over the batch's own statistics), cifar8BbnFwdChainB and a return. Not what the driver scores through — that is @<slug>_fwd_eval — but what LEAN_MLIR_EVAL_BATCHSTATS reads and what the BN-world check in trainAdamSched compares against the train step (dimensions = [0, 2, 3] in both). A byte prefix of every <slug>_*_train_step body from %v0 on.

                Equations
                • One or more equations did not get rendered due to their size.
                Instances For
                  def Proofs.StableHLO.cifar8BbnFwdEvalText (B ic c1 c2 c3 c4 h w d1 nClasses kH kW : ℕ) (epsStr slug : String) :

                  @<slug>_fwd_eval — the inference forward with FROZEN running statistics. The per-example chain (flatConvF / maxPoolF / denseF / reluF under pretty B) with every BN site bnEvalSite (bnPerChannelEvalF: affine BN at the %n{i}mu/%n{i}var graph inputs, no reduction), so this forward is class-batch-independent. Inputs: %x, the 38 parameters, then the 16 stat slots of cifar8BbnStatSigList — the layout trainAdamSched packs (paramShapes ++ bnStatShapes). Returns logits [B, nClasses]. f32 only: BN is f32 in every arm, and the bf16 train steps score through this same artifact, as the ImageNet nets do.

                  Equations
                  • One or more equations did not get rendered due to their size.
                  Instances For
                    def Proofs.StableHLO.c8bbnLayerBack (B oc hh ww : ℕ) (sync : Bool) (replicas : ℕ) (epsStr : String) (i : ℕ) (L : C8BbnLayer) (dAc : String) :

                    One layer's backward at the BN: the relu mask on the BN output (selectPosB), then bnBackSite (bnBatchBack at one replica, the sync-BN backward above it) reading the layer's conv output and packed statistics. Returns (code, dBn, dHc): the cotangent at the BN output (which the γ/β gradients read) and at the conv output (which the conv gradients read).

                    Equations
                    • One or more equations did not get rendered due to their size.
                    Instances For
                      def Proofs.StableHLO.c8bbnLayerGrads (B ic oc hh ww kH kW : ℕ) (bf16 sync : Bool) (replicas : ℕ) (opt : CifarOpt) (epsStr : String) (i : ℕ) (xIn : String) (L : C8BbnLayer) (nDbn nDhc : String) :

                      One layer's four parameter gradients and optimizer tails, in cifar8BnSigList order (W, conv bias, γ, β): the batched conv weight gradient (bf16 twin when bf16) and bias gradient at the layer's INPUT xIn, bnGammaSite and bnBetaGradB at the BN-output cotangent. Returns (code, θ' names, m' names, v' names).

                      Equations
                      • One or more equations did not get rendered due to their size.
                      Instances For

                        One BN layer's returned batch statistics — μ then var, each [oc] — from that layer's BN INPUT (bnBatchMeanB / bnBatchVarB: the SAME bnMean/bnVar the forward normalised by). At replicas > 1 read off the all-reduced packed vector (bnStatsMeanB / bnStatsVarB), so the host EMAs the GLOBAL batch statistics.

                        Equations
                        • One or more equations did not get rendered due to their size.
                        Instances For
                          def Proofs.StableHLO.cifar8BbnTrainStepBText (B ic c1 c2 c3 c4 h w d1 nClasses kH kW : ℕ) (bnEpsStr invBStr b1Str ob1Str b2Str ob2Str aEpsStr wdStr slug variant : String) (replicas : ℕ := 1) (opt : CifarOpt := CifarOpt.adamw) (bf16 : Bool := false) :

                          cifar8BbnTrainStepBText — the cifar8 BN train step at BATCH statistics.

                          cifar8BnTrainStepBText with its eight bnPerChannelF/bnPerChannelBack/bnGammaGrad sites replaced by the batch-coupled bnFwdSite/bnBackSite/bnGammaSite (bnBatchF / bnBatchBack / bnGammaGradB, reduce over [0,2,3]) and bnBetaGradB, plus the running-statistics protocol: 16 passthrough stat inputs after %bc2 and 16 batch-statistic outputs after %bc2 (cifar8BbnStatSigList order). Signature otherwise the packed [θ|m|v] one every cifar8 optimizer render shares — %x, 38 θ, 38 m, 38 v, %lr/%bc1/%bc2, 16 stats, %onehot → 38 θ', 38 m', 38 v', %loss/%bc1/%bc2, 16 stats — so VerifiedNet.trainAdamSched drives it unchanged, as it drives ResNet-34's.

                          Three optimizers from one body (optTail); bf16 routes the 23 convolutions through their bf16 twins and leaves BN in f32, as cifar8BnTrainStepBText and every ImageNet render do. Entry @{slug}_{variant}_train_step, the driver's convention.

                          Equations
                          • One or more equations did not get rendered due to their size.
                          Instances For