Documentation

LeanMlir.Proofs.Foundation.GradNodesB

The batched f32 gradient nodes — one *GradB_den per op kind, shared by every conv net #

Every batched train step the suite renders emits the RAW gradient (*GradB) and hands it to an optimizer tail (SGD, heavy-ball, Adam/AdamW, RMSProp, EMA, the data-parallel all-reduce), so one lemma per op kind — "this node denotes the certified Σ_n gradient" — certifies every optimizer variant of every net at once. Each proof is Finset.sum_congr rfl over the batch and then the per-example VJP at batchSlice n. The bf16 kinds (*GradBBf16) are a different real number and are folded in Bf16GradNodes; the fused *SgdB ops are these through StableHLO.Basic's *SgdB_eq_grad family (rfl).

Every lemma is in namespace GradNodeB:

op kindsemitted by
conv / strided-conv (symmetric) W and b, BN γ/β, dense W and b (the bias at a rectangular Mat a c, which the SE squeeze needs), the *TiedB clause Propsevery conv net
XLA-SAME strided conv W, depthwise W, symmetric strided depthwise WEfficientNet-B0, MobileNetV2/V4, ConvNeXt
XLA-SAME strided conv b, depthwise b, XLA-SAME strided depthwise W and bMobileNetV2, ConvNeXt
stride-4 patchify conv WConvNeXt
vector-LN γ / β and their *TiedB clauses, the classifier head W and bViT, ConvNeXt

The other ViT nodes (per-token dense, patch embed, position, CLS) live in ViTFoldGB; ConvNeXt's channel-LN and layer-scale nodes in ConvNeXtFoldGB.

Note: Padding is invisible in the types: the symmetric and XLA-SAME strided kinds have identical types and identical emitted shapes, and only the certificate tells them apart.

Every lemma is ∀ cot; pinning each cotangent to the emitted backward subgraph is each net's *StepTie* file.

theorem Proofs.GradNodeB.convWGradB_den {N ic oc h w kH kW : ℕ} (xN cotN : String) (b : Vec oc) (x : Vec (N * (ic * h * w))) (W : Kernel4 oc ic kH kW) (cot : Vec (N * (oc * h * w))) (idx : Fin (oc * ic * kH * kW)) :
StableHLO.den (StableHLO.SHlo.convWeightGradB xN b x W (StableHLO.SHlo.operand cotN cot)) idx = ∑ n : Fin N, ∑ j : Fin (oc * h * w), pdiv (fun (v' : Vec (oc * ic * kH * kW)) => (conv2d (Kernel4.unflatten v') b (Tensor3.unflatten (StableHLO.batchSlice N (ic * h * w) x n))).flatten) W.flatten idx j * StableHLO.batchSlice N (oc * h * w) cot n j

Batched stride-1 conv weight GRADIENT denotes the certified Σ_n weight gradient. The un-fused peer of EnetFold.convWB_den: same Σ_n of conv_weight_grad_bridge, with no θ − lr· wrapper because the batched r34 render hands this node to an optimizer tail.

theorem Proofs.GradNodeB.convBGradB_den {N ic oc h w kH kW : ℕ} (cotN : String) (W : Kernel4 oc ic kH kW) (x : Vec (N * (ic * h * w))) (b : Vec oc) (cot : Vec (N * (oc * h * w))) (o : Fin oc) :
StableHLO.den (StableHLO.SHlo.convBiasGradB W x b (StableHLO.SHlo.operand cotN cot)) o = ∑ n : Fin N, ∑ j : Fin (oc * h * w), pdiv (fun (b' : Vec oc) => (conv2d W b' (Tensor3.unflatten (StableHLO.batchSlice N (ic * h * w) x n))).flatten) b o j * StableHLO.batchSlice N (oc * h * w) cot n j

Batched stride-1 conv bias GRADIENT denotes the certified Σ_n bias gradient.

theorem Proofs.GradNodeB.convStridedWGradB_den {N ic oc h w kH kW : ℕ} (xN cotN : String) (b : Vec oc) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (W : Kernel4 oc ic kH kW) (cot : Vec (N * (oc * h * w))) (idx : Fin (oc * ic * kH * kW)) :
StableHLO.den (StableHLO.SHlo.convStridedWeightGradB xN b x W (StableHLO.SHlo.operand cotN cot)) idx = ∑ n : Fin N, ∑ j : Fin (oc * h * w), pdiv (fun (v' : Vec (oc * ic * kH * kW)) => flatConvStride2 (Kernel4.unflatten v') b (StableHLO.batchSlice N (ic * (2 * h) * (2 * w)) x n)) W.flatten idx j * StableHLO.batchSlice N (oc * h * w) cot n j

Batched strided conv weight GRADIENT denotes the certified Σ_n weight gradient. Generic in the kernel size, so the one lemma certifies the 7x7 stem AND every 3x3 downsample W1 AND every 1x1 projection Wp.

theorem Proofs.GradNodeB.convStridedBGradB_den {N ic oc h w kH kW : ℕ} (cotN : String) (W : Kernel4 oc ic kH kW) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (b : Vec oc) (cot : Vec (N * (oc * h * w))) (o : Fin oc) :
StableHLO.den (StableHLO.SHlo.convStridedBiasGradB W x b (StableHLO.SHlo.operand cotN cot)) o = ∑ n : Fin N, ∑ j : Fin (oc * h * w), pdiv (fun (b' : Vec oc) => flatConvStride2 W b' (StableHLO.batchSlice N (ic * (2 * h) * (2 * w)) x n)) b o j * StableHLO.batchSlice N (oc * h * w) cot n j

Batched strided conv bias GRADIENT denotes the certified Σ_n bias gradient.

theorem Proofs.GradNodeB.bnGammaGradB_den {N oc h w : ℕ} (vN epsStr cotN : String) (ε : ℝ) (γ β : Vec oc) (v cot : Vec (N * (oc * (h * w)))) (c : Fin oc) :
StableHLO.den (StableHLO.SHlo.bnGammaGradB vN epsStr ε v (StableHLO.SHlo.operand cotN cot)) c = ∑ j : Fin (oc * (N * (h * w))), pdiv (fun (γ' : Vec oc) => bnPerChannelFlat oc (N * (h * w)) ε γ' β (bnchwFwd N oc h w v)) γ c j * bnchwFwd N oc h w cot j

Batched BN γ GRADIENT denotes the certified per-channel γ gradient over the merged batch+spatial axis m = N·(h·w). γ enters affinely, so there is no batch coupling in the PARAM gradient and this is bnPerChannelGradGamma_correct at that width, through the network→oc-major reindex bnchwFwd. Generic in the free β.

theorem Proofs.GradNodeB.bnBetaGradB_den {N oc h w : ℕ} (cotN : String) (ε : ℝ) (γ β : Vec oc) (v : Vec (oc * (N * (h * w)))) (cot : Vec (N * (oc * (h * w)))) (c : Fin oc) :
StableHLO.den (StableHLO.SHlo.operand cotN cot).bnBetaGradB c = ∑ j : Fin (oc * (N * (h * w))), pdiv (fun (β' : Vec oc) => bnPerChannelFlat oc (N * (h * w)) ε γ β' v) β c j * bnchwFwd N oc h w cot j

Batched BN β GRADIENT denotes the certified per-channel β gradient Σ_{batch,spatial} cot at m = N·(h·w). Carries a free v/γ — β's gradient is the channel sum and depends on neither.

def Proofs.GradNodeB.BnPairTiedB (N oc h w : ℕ) (vN epsStr cotN : String) (ε : ℝ) (γ β : Vec oc) (v cot : Vec (N * (oc * (h * w)))) :

One batched BN layer's γ and β gradient nodes, tied — the pair every step tie states per BatchNorm: the emitted bnGammaGradB / bnBetaGradB denote the certified per-channel γ and β gradients over the merged batch+spatial axis, at the layer's pre-BN activation v and its output cotangent cot (both in the network layout).

Equations
  • One or more equations did not get rendered due to their size.
Instances For
    theorem Proofs.GradNodeB.bnPairTiedB_holds {N oc h w : ℕ} {vN epsStr cotN : String} {ε : ℝ} {γ β : Vec oc} {v cot : Vec (N * (oc * (h * w)))} :
    BnPairTiedB N oc h w vN epsStr cotN ε γ β v cot
    theorem Proofs.GradNodeB.denseWGradB_den {N a c : ℕ} (xN cotN : String) (x : Vec (N * a)) (W : Mat a c) (b : Vec c) (cot : Vec (N * c)) (i : Fin a) (j : Fin c) :

    Batched dense weight GRADIENT denotes the certified Σ_n outer product.

    def Proofs.GradNodeB.ConvWTiedB (N h w : ℕ) {ic oc kH kW : ℕ} (xN cotN : String) (b : Vec oc) (x : Vec (N * (ic * h * w))) (W : Kernel4 oc ic kH kW) (cot : Vec (N * (oc * h * w))) :

    A stride-1 conv weight gradient node, tied (convWGradB_den).

    Equations
    • One or more equations did not get rendered due to their size.
    Instances For
      def Proofs.GradNodeB.ConvBTiedB (N h w : ℕ) {ic oc kH kW : ℕ} (cotN : String) (W : Kernel4 oc ic kH kW) (x : Vec (N * (ic * h * w))) (b : Vec oc) (cot : Vec (N * (oc * h * w))) :

      A stride-1 conv bias gradient node, tied (convBGradB_den).

      Equations
      • One or more equations did not get rendered due to their size.
      Instances For
        def Proofs.GradNodeB.ConvStridedWTiedB (N h w : ℕ) {ic oc kH kW : ℕ} (xN cotN : String) (b : Vec oc) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (W : Kernel4 oc ic kH kW) (cot : Vec (N * (oc * h * w))) :

        A stride-2 (symmetric-pad) conv weight gradient node, tied (convStridedWGradB_den).

        Equations
        • One or more equations did not get rendered due to their size.
        Instances For
          def Proofs.GradNodeB.ConvStridedBTiedB (N h w : ℕ) {ic oc kH kW : ℕ} (cotN : String) (W : Kernel4 oc ic kH kW) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (b : Vec oc) (cot : Vec (N * (oc * h * w))) :

          A stride-2 (symmetric-pad) conv bias gradient node, tied (convStridedBGradB_den).

          Equations
          • One or more equations did not get rendered due to their size.
          Instances For
            def Proofs.GradNodeB.ConvStridedXlaWTiedB (N h w : ℕ) {ic oc kH kW : ℕ} (xN cotN : String) (b : Vec oc) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (W : Kernel4 oc ic kH kW) (cot : Vec (N * (oc * h * w))) :

            A stride-2 XLA-SAME conv weight gradient node, tied (GradNodeB.convStridedXlaWGradB_den).

            Equations
            • One or more equations did not get rendered due to their size.
            Instances For
              def Proofs.GradNodeB.DepthwiseWTiedB (N h w : ℕ) {c kH kW : ℕ} (xN cotN : String) (b : Vec c) (x : Vec (N * (c * h * w))) (W : DepthwiseKernel c kH kW) (cot : Vec (N * (c * h * w))) :

              A stride-1 depthwise weight gradient node, tied (GradNodeB.depthwiseWGradB_den).

              Equations
              • One or more equations did not get rendered due to their size.
              Instances For
                def Proofs.GradNodeB.DepthwiseBTiedB (N h w : ℕ) {c kH kW : ℕ} (cotN : String) (W : DepthwiseKernel c kH kW) (x : Vec (N * (c * h * w))) (b : Vec c) (cot : Vec (N * (c * h * w))) :

                A stride-1 depthwise bias gradient node, tied (GradNodeB.depthwiseBGradB_den).

                Equations
                • One or more equations did not get rendered due to their size.
                Instances For
                  def Proofs.GradNodeB.DepthwiseStridedWTiedB (N h w : ℕ) {c kH kW : ℕ} (xN cotN : String) (b : Vec c) (x : Vec (N * (c * (2 * h) * (2 * w)))) (W : DepthwiseKernel c kH kW) (cot : Vec (N * (c * h * w))) :

                  A stride-2 depthwise weight gradient node, tied (GradNodeB.depthwiseStridedWGradB_den).

                  Equations
                  • One or more equations did not get rendered due to their size.
                  Instances For
                    def Proofs.GradNodeB.DenseWTiedB (N : ℕ) {a c : ℕ} (xN cotN : String) (x : Vec (N * a)) (W : Mat a c) (b : Vec c) (cot : Vec (N * c)) :

                    A dense weight gradient node, tied (denseWGradB_den).

                    Equations
                    • One or more equations did not get rendered due to their size.
                    Instances For
                      def Proofs.GradNodeB.DenseBTiedB (N : ℕ) {a c : ℕ} (cotN : String) (W : Mat a c) (x : Vec a) (b : Vec c) (cot : Vec (N * c)) :

                      A dense bias gradient node, tied — free in W and x, which b's gradient ignores.

                      Equations
                      • One or more equations did not get rendered due to their size.
                      Instances For
                        theorem Proofs.GradNodeB.denseBGradB_den {N a c : ℕ} (cotN : String) (W : Mat a c) (x : Vec a) (b : Vec c) (cot : Vec (N * c)) (j : Fin c) :
                        StableHLO.den (StableHLO.SHlo.operand cotN cot).denseBiasGradB j = ∑ n : Fin N, ∑ k : Fin c, pdiv (fun (b' : Vec c) => dense W b' x) b j k * StableHLO.batchSlice N c cot n k

                        Batched dense bias GRADIENT denotes the certified Σ_n cotangent sum. r34's peer at a RECTANGULAR witness Mat a c, which the SE's c → r squeeze needs; the gradient is the channel sum and depends on neither W nor x, so the widening is free.

                        theorem Proofs.GradNodeB.convStridedXlaWGradB_den {N ic oc h w kH kW : ℕ} (xN cotN : String) (b : Vec oc) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (W : Kernel4 oc ic kH kW) (cot : Vec (N * (oc * h * w))) (idx : Fin (oc * ic * kH * kW)) :
                        StableHLO.den (StableHLO.SHlo.convStridedXlaWeightGradB xN b x W (StableHLO.SHlo.operand cotN cot)) idx = ∑ n : Fin N, ∑ j : Fin (oc * h * w), pdiv (fun (v' : Vec (oc * ic * kH * kW)) => flatConvStride2Xla (Kernel4.unflatten v') b (StableHLO.batchSlice N (ic * (2 * h) * (2 * w)) x n)) W.flatten idx j * StableHLO.batchSlice N (oc * h * w) cot n j

                        Batched XLA-SAME strided conv weight GRADIENT denotes the certified Σ_n weight gradient. B0's 3×3/s2 stem, the net's one XLA-phase site. Σ_n of flatConvStride2XlaWeightGradHasVJP.correct — the odd-phase weight VJP, so the certified gradient is the gradient of the net that ships.

                        theorem Proofs.GradNodeB.depthwiseWGradB_den {N c h w kH kW : ℕ} (xN cotN : String) (b : Vec c) (x : Vec (N * (c * h * w))) (W : DepthwiseKernel c kH kW) (cot : Vec (N * (c * h * w))) (idx : Fin (c * kH * kW)) :
                        StableHLO.den (StableHLO.SHlo.depthwiseWeightGradB xN b x W (StableHLO.SHlo.operand cotN cot)) idx = ∑ n : Fin N, ∑ j : Fin (c * h * w), pdiv (fun (v' : Vec (c * kH * kW)) => (depthwiseConv2d (Tensor3.unflatten v') b (Tensor3.unflatten (StableHLO.batchSlice N (c * h * w) x n))).flatten) (Tensor3.flatten W) idx j * StableHLO.batchSlice N (c * h * w) cot n j

                        Batched stride-1 depthwise weight GRADIENT denotes the certified Σ_n weight gradient. Σ_n of the flattened depthwiseWeightGradHasVJP3.correct. Generic in the kernel size, so the one lemma covers every 3×3 and every 5×5 depthwise.

                        theorem Proofs.GradNodeB.depthwiseStridedWGradB_den {N c h w kH kW : ℕ} (xN cotN : String) (b : Vec c) (x : Vec (N * (c * (2 * h) * (2 * w)))) (W : DepthwiseKernel c kH kW) (cot : Vec (N * (c * h * w))) (idx : Fin (c * kH * kW)) :
                        StableHLO.den (StableHLO.SHlo.depthwiseStridedWeightGradB xN b x W (StableHLO.SHlo.operand cotN cot)) idx = ∑ n : Fin N, ∑ j : Fin (c * h * w), pdiv (fun (v' : Vec (c * kH * kW)) => depthwiseStride2Flat (Tensor3.unflatten v') b (StableHLO.batchSlice N (c * (2 * h) * (2 * w)) x n)) (Tensor3.flatten W) idx j * StableHLO.batchSlice N (c * h * w) cot n j

                        Batched strided depthwise weight GRADIENT denotes the certified Σ_n weight gradient. The strided VJP is already flat, so this is Σ_n of depthwiseStride2WeightGradHasVJP.correct.

                        theorem Proofs.GradNodeB.convStridedXlaBGradB_den {N ic oc h w kH kW : ℕ} (cotN : String) (W : Kernel4 oc ic kH kW) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (b : Vec oc) (cot : Vec (N * (oc * h * w))) (o : Fin oc) :
                        StableHLO.den (StableHLO.SHlo.convStridedXlaBiasGradB W x b (StableHLO.SHlo.operand cotN cot)) o = ∑ n : Fin N, ∑ j : Fin (oc * h * w), pdiv (fun (b' : Vec oc) => flatConvStride2Xla W b' (StableHLO.batchSlice N (ic * (2 * h) * (2 * w)) x n)) b o j * StableHLO.batchSlice N (oc * h * w) cot n j

                        Batched XLA-SAME strided conv bias GRADIENT denotes the certified Σ_n bias gradient. The stem's bias slot, at convBias := true. Same reduce text as the stride-1 bias grad; the den is the odd-phase bias VJP.

                        theorem Proofs.GradNodeB.depthwiseBGradB_den {N c h w kH kW : ℕ} (cotN : String) (W : DepthwiseKernel c kH kW) (x : Vec (N * (c * h * w))) (b : Vec c) (cot : Vec (N * (c * h * w))) (o : Fin c) :
                        StableHLO.den (StableHLO.SHlo.depthwiseBiasGradB W x b (StableHLO.SHlo.operand cotN cot)) o = ∑ n : Fin N, ∑ j : Fin (c * h * w), pdiv (fun (b' : Vec c) => (depthwiseConv2d W b' (Tensor3.unflatten (StableHLO.batchSlice N (c * h * w) x n))).flatten) b o j * StableHLO.batchSlice N (c * h * w) cot n j

                        Batched stride-1 depthwise bias GRADIENT denotes the certified Σ_n bias gradient. The depthwise bias slots at convBias := true. EfficientNet has no instance of this op — its depthwise convs are followed by BatchNorm, so their bias is always folded.

                        theorem Proofs.GradNodeB.depthwiseStridedXlaWGradB_den {N c h w kH kW : ℕ} (xN cotN : String) (b : Vec c) (x : Vec (N * (c * (2 * h) * (2 * w)))) (W : DepthwiseKernel c kH kW) (cot : Vec (N * (c * h * w))) (idx : Fin (c * kH * kW)) :
                        StableHLO.den (StableHLO.SHlo.depthwiseStridedXlaWeightGradB xN b x W (StableHLO.SHlo.operand cotN cot)) idx = ∑ n : Fin N, ∑ j : Fin (c * h * w), pdiv (fun (v' : Vec (c * kH * kW)) => depthwiseStride2FlatXla (Tensor3.unflatten v') b (StableHLO.batchSlice N (c * (2 * h) * (2 * w)) x n)) (Tensor3.flatten W) idx j * StableHLO.batchSlice N (c * h * w) cot n j

                        Batched XLA-SAME strided depthwise weight GRADIENT denotes the certified Σ_n weight gradient. The four stride-2 depthwises (b2/b4/b7/b14). Note: This is the Xla op — its weight-grad correlation keeps the [p−1, p+1] pad, the opposite asymmetry from the input-grad, and that asymmetry is the whole content of the variant. B0's strided depthwise is the SYMMETRIC op, so the two nets do not share this certificate.

                        theorem Proofs.GradNodeB.depthwiseStridedXlaBGradB_den {N c h w kH kW : ℕ} (cotN : String) (W : DepthwiseKernel c kH kW) (x : Vec (N * (c * (2 * h) * (2 * w)))) (b : Vec c) (cot : Vec (N * (c * h * w))) (o : Fin c) :
                        StableHLO.den (StableHLO.SHlo.depthwiseStridedXlaBiasGradB W x b (StableHLO.SHlo.operand cotN cot)) o = ∑ n : Fin N, ∑ j : Fin (c * h * w), pdiv (fun (b' : Vec c) => depthwiseStride2FlatXla W b' (StableHLO.batchSlice N (c * (2 * h) * (2 * w)) x n)) b o j * StableHLO.batchSlice N (c * h * w) cot n j

                        Batched XLA-SAME strided depthwise bias GRADIENT denotes the certified Σ_n bias gradient. At convBias := true.

                        theorem Proofs.GradNodeB.psWGradB_den {N ic oc h w kH kW : ℕ} (xN cotN : String) (b : Vec oc) (x : Vec (N * (ic * (2 * (2 * h)) * (2 * (2 * w))))) (W : Kernel4 oc ic kH kW) (cot : Vec (N * (oc * h * w))) (idx : Fin (oc * ic * kH * kW)) :
                        StableHLO.den (StableHLO.SHlo.convStride4WeightGradB xN b x W (StableHLO.SHlo.operand cotN cot)) idx = ∑ n : Fin N, ∑ j : Fin (oc * h * w), pdiv (fun (v' : Vec (oc * ic * kH * kW)) => flatConvStride4 (Kernel4.unflatten v') b (StableHLO.batchSlice N (ic * (2 * (2 * h)) * (2 * (2 * w))) x n)) W.flatten idx j * StableHLO.batchSlice N (oc * h * w) cot n j

                        Batched patchify-stem weight GRADIENT denotes the certified Σ_n weight gradient. Note: The emitted convolution contracts the batch axis itself (the transpose trick), so the outer sum is inside one op rather than across N of them — same as the strided ops.

                        Each tie clause holds #

                        One lemma per clause above, every argument implicit: a step tie's conjunction of clauses is then an anonymous constructor of these, its arguments read off the goal.

                        theorem Proofs.GradNodeB.convWTiedB_holds {N h w ic oc kH kW : ℕ} {xN cotN : String} {b : Vec oc} {x : Vec (N * (ic * h * w))} {W : Kernel4 oc ic kH kW} {cot : Vec (N * (oc * h * w))} :
                        ConvWTiedB N h w xN cotN b x W cot
                        theorem Proofs.GradNodeB.convBTiedB_holds {N h w ic oc kH kW : ℕ} {cotN : String} {W : Kernel4 oc ic kH kW} {x : Vec (N * (ic * h * w))} {b : Vec oc} {cot : Vec (N * (oc * h * w))} :
                        ConvBTiedB N h w cotN W x b cot
                        theorem Proofs.GradNodeB.convStridedWTiedB_holds {N h w ic oc kH kW : ℕ} {xN cotN : String} {b : Vec oc} {x : Vec (N * (ic * (2 * h) * (2 * w)))} {W : Kernel4 oc ic kH kW} {cot : Vec (N * (oc * h * w))} :
                        ConvStridedWTiedB N h w xN cotN b x W cot
                        theorem Proofs.GradNodeB.convStridedBTiedB_holds {N h w ic oc kH kW : ℕ} {cotN : String} {W : Kernel4 oc ic kH kW} {x : Vec (N * (ic * (2 * h) * (2 * w)))} {b : Vec oc} {cot : Vec (N * (oc * h * w))} :
                        ConvStridedBTiedB N h w cotN W x b cot
                        theorem Proofs.GradNodeB.convStridedXlaWTiedB_holds {N h w ic oc kH kW : ℕ} {xN cotN : String} {b : Vec oc} {x : Vec (N * (ic * (2 * h) * (2 * w)))} {W : Kernel4 oc ic kH kW} {cot : Vec (N * (oc * h * w))} :
                        ConvStridedXlaWTiedB N h w xN cotN b x W cot
                        theorem Proofs.GradNodeB.depthwiseWTiedB_holds {N h w c kH kW : ℕ} {xN cotN : String} {b : Vec c} {x : Vec (N * (c * h * w))} {W : DepthwiseKernel c kH kW} {cot : Vec (N * (c * h * w))} :
                        DepthwiseWTiedB N h w xN cotN b x W cot
                        theorem Proofs.GradNodeB.depthwiseBTiedB_holds {N h w c kH kW : ℕ} {cotN : String} {W : DepthwiseKernel c kH kW} {x : Vec (N * (c * h * w))} {b : Vec c} {cot : Vec (N * (c * h * w))} :
                        DepthwiseBTiedB N h w cotN W x b cot
                        theorem Proofs.GradNodeB.depthwiseStridedWTiedB_holds {N h w c kH kW : ℕ} {xN cotN : String} {b : Vec c} {x : Vec (N * (c * (2 * h) * (2 * w)))} {W : DepthwiseKernel c kH kW} {cot : Vec (N * (c * h * w))} :
                        DepthwiseStridedWTiedB N h w xN cotN b x W cot
                        theorem Proofs.GradNodeB.denseWTiedB_holds {N a c : ℕ} {xN cotN : String} {x : Vec (N * a)} {W : Mat a c} {b : Vec c} {cot : Vec (N * c)} :
                        DenseWTiedB N xN cotN x W b cot
                        theorem Proofs.GradNodeB.denseBTiedB_holds {N a c : ℕ} {cotN : String} {W : Mat a c} {x : Vec a} {b : Vec c} {cot : Vec (N * c)} :
                        DenseBTiedB N cotN W x b cot

                        The batched vector-LayerNorm γ / β nodes, their tie clauses, and the classifier head's weight and bias nodes. An LN β gradient and a per-token dense bias gradient are the same two-level reduce, so rowDenseBiasGradB appears here against the LN forward.

                        theorem Proofs.GradNodeB.veclnGammaGradB_den {N R D : ℕ} (xN epsStr cotN : String) (ε : ℝ) (βv : Vec D) (x : Vec (N * (R * D))) (γ : Vec D) (dy : Vec (N * (R * D))) (k : Fin D) :
                        StableHLO.den (StableHLO.SHlo.veclnGammaGradB xN epsStr ε x (StableHLO.SHlo.operand cotN dy)) k = ∑ n : Fin N, ∑ o : Fin (R * D), pdiv (fun (gv : Vec D) => Mat.flatten fun (r : Fin R) => layerNormVec D ε gv βv (Mat.unflatten (StableHLO.batchSlice N (R * D) x n) r)) γ k o * StableHLO.batchSlice N (R * D) dy n o

                        Batched vector-LN γ GRADIENT denotes the certified Σ_n γ gradient. Two levels: the outer sum is the batch, the inner one the tokens within one example. All 25 sites.

                        theorem Proofs.GradNodeB.rowDenseBiasGradB_den_lnbeta {N R D : ℕ} (cotN : String) (ε : ℝ) (γv : Vec D) (X : Fin N → Mat R D) (β : Vec D) (dy : Vec (N * (R * D))) (i : Fin D) :
                        StableHLO.den (StableHLO.SHlo.operand cotN dy).rowDenseBiasGradB i = ∑ n : Fin N, ∑ o : Fin (R * D), pdiv (fun (bv : Vec D) => Mat.flatten fun (r : Fin R) => layerNormVec D ε γv bv (X n r)) β i o * StableHLO.batchSlice N (R * D) dy n o

                        The SAME row-reduce op, certified against the vector-LN β forward. An LN β gradient and a dense bias gradient are the identical two-level reduce, so this constructor appears twice in the table against two different certified Jacobians — as it does per example. All 25 β sites.

                        theorem Proofs.GradNodeB.headWGradB_den {N D nC : ℕ} (aN cotN : String) (a : Vec (N * D)) (Wc : Mat D nC) (bc : Vec nC) (cot : Vec (N * nC)) (i : Fin D) (j : Fin nC) :
                        StableHLO.den (StableHLO.SHlo.weightGradB aN a (StableHLO.SHlo.operand cotN cot)) (finProdFinEquiv (i, j)) = ∑ n : Fin N, ∑ k : Fin nC, pdiv (fun (v : Vec (D * nC)) => dense (Mat.unflatten v) bc (StableHLO.batchSlice N D a n)) Wc.flatten (finProdFinEquiv (i, j)) k * StableHLO.batchSlice N nC cot n k

                        Batched classifier weight GRADIENT denotes the certified Σ_n outer product.

                        theorem Proofs.GradNodeB.headBGradB_den {N D nC : ℕ} (cotN : String) (Wc : Mat D nC) (a : Fin N → Vec D) (bc : Vec nC) (cot : Vec (N * nC)) (i : Fin nC) :
                        StableHLO.den (StableHLO.SHlo.operand cotN cot).biasGradB i = ∑ n : Fin N, ∑ j : Fin nC, pdiv (fun (b' : Vec nC) => dense Wc b' (a n)) bc i j * StableHLO.batchSlice N nC cot n j

                        Batched classifier bias GRADIENT denotes the certified cotangent, summed over the batch. The node's den carries the batch reduce its text prints, so this is the whole statement the classifier bias's update needs, as headWGradB_den is for the weight. a n is example n's head input (the bias gradient does not depend on it).

                        def Proofs.GradNodeB.VecLNGammaTiedB (N R : ℕ) {D : ℕ} (xN epsStr cotN : String) (ε : ℝ) (βv : Vec D) (x : Vec (N * (R * D))) (γ : Vec D) (dy : Vec (N * (R * D))) :

                        A batched vector-LN γ gradient node, tied (veclnGammaGradB_den).

                        Equations
                        • One or more equations did not get rendered due to their size.
                        Instances For
                          def Proofs.GradNodeB.VecLNBetaTiedB (N R : ℕ) {D : ℕ} (cotN : String) (ε : ℝ) (γv : Vec D) (x : Vec (N * (R * D))) (β : Vec D) (dy : Vec (N * (R * D))) :

                          A batched vector-LN β gradient node, tied (rowDenseBiasGradB_den_lnbeta).

                          Equations
                          • One or more equations did not get rendered due to their size.
                          Instances For
                            theorem Proofs.GradNodeB.vecLNGammaTiedB_holds {N R D : ℕ} {xN epsStr cotN : String} {ε : ℝ} {βv : Vec D} {x : Vec (N * (R * D))} {γ : Vec D} {dy : Vec (N * (R * D))} :
                            VecLNGammaTiedB N R xN epsStr cotN ε βv x γ dy
                            theorem Proofs.GradNodeB.vecLNBetaTiedB_holds {N R D : ℕ} {cotN : String} {ε : ℝ} {γv : Vec D} {x : Vec (N * (R * D))} {β : Vec D} {dy : Vec (N * (R * D))} :
                            VecLNBetaTiedB N R cotN ε γv x β dy