Documentation

LeanMlir.Proofs.Nets.ResNet.ResNet34FoldB

T3 §1 fold for ResNet-34 at TRUE BATCH-NORM — the UN-FUSED gradient ops #

ResNet34Fold.lean makes every parameter output of the per-example SGD train step den-faithful. This is its batched peer, and one thing about it is different in kind.

r34's batched render emits *GradB, not *SgdB. Every batched ResNet-34 train step — resnet34_sgd_train_step, the Adam family, resnet34in_mom256 and its data-parallel peers — emits the RAW gradient and hands it to an optimizer tail (adamMNextF/adamVNextF, heavy-ball, plain SGD). The fused θ − lr·∂Loss/∂θ op only appears in renders whose optimizer is SGD-inline, which EfficientNet's is and r34's batched one is not. Every den = certified lemma in the repo before this file is stated at the fused form, so none of them applies here.

That makes this tier better, not worse. A statement about the gradient covers every optimizer variant at once: sgd, mom, momdp64, adam and adamdp128 all consume the same *GradB node, so one lemma per op kind certifies the whole family. ⚠ The bf16 twins do NOT: a bf16 render emits *GradBBf16, its own kind, folded in Foundation/Bf16GradNodes.lean. It is also the form ConvNeXt's psW carve-out already had to take for a different reason (a hand-written SGD wrap).

And no new mathematics: the *SgdB peers were already proven, and the fusion is rfl. StableHLO.lean's *SgdB_eq_grad family (convWeightSgdB_eq_grad, …) says each fused op IS θ − lr· applied to the un-fused one, all by rfl, and its own docstring says it exists to "unblock a batched resnet34_adam_train_step rendered from Proofs/ — the blocker was the fusion, never Adam." So the eight lemmas below are the per-example VJP bridge under Σ_n with no θ − lr· wrapper, and EfficientNetFold.lean's fused lemmas are them through *SgdB_eq_grad.

Symmetric padding, not XLA-SAME. The strided lemmas here are about convStridedWeightGradB / convStridedBiasGradB, whose den is flatConvStride2_*; B0's peers are about the convStridedXla* ops and flatConvStride2Xla_*. The two op families have identical types and identical emitted shapes, so nothing but the certificate distinguishes them — and r34 is the PyTorch-origin net, so symmetric is the shipped phase.

Honest residual (the boundary every fold carries) #

theorem Proofs.ResNet34PoCB.convWGradB_den {N ic oc h w kH kW : } (xN cotN : String) (b : Vec oc) (x : Vec (N * (ic * h * w))) (W : Kernel4 oc ic kH kW) (cot : Vec (N * (oc * h * w))) (idx : Fin (oc * ic * kH * kW)) :
StableHLO.den (StableHLO.SHlo.convWeightGradB xN b x W (StableHLO.SHlo.operand cotN cot)) idx = n : Fin N, j : Fin (oc * h * w), pdiv (fun (v' : Vec (oc * ic * kH * kW)) => (conv2d (Kernel4.unflatten v') b (Tensor3.unflatten (StableHLO.batchSlice N (ic * h * w) x n))).flatten) W.flatten idx j * StableHLO.batchSlice N (oc * h * w) cot n j

Batched stride-1 conv weight GRADIENT denotes the certified Σ_n weight gradient. The un-fused peer of EnetPoC.convWB_den: same Σ_n of conv_weight_grad_bridge, with no θ − lr· wrapper because the batched r34 render hands this node to an optimizer tail.

theorem Proofs.ResNet34PoCB.convBGradB_den {N ic oc h w kH kW : } (cotN : String) (W : Kernel4 oc ic kH kW) (x : Vec (N * (ic * h * w))) (b : Vec oc) (cot : Vec (N * (oc * h * w))) (o : Fin oc) :
StableHLO.den (StableHLO.SHlo.convBiasGradB W x b (StableHLO.SHlo.operand cotN cot)) o = n : Fin N, j : Fin (oc * h * w), pdiv (fun (b' : Vec oc) => (conv2d W b' (Tensor3.unflatten (StableHLO.batchSlice N (ic * h * w) x n))).flatten) b o j * StableHLO.batchSlice N (oc * h * w) cot n j

Batched stride-1 conv bias GRADIENT denotes the certified Σ_n bias gradient.

theorem Proofs.ResNet34PoCB.convStridedWGradB_den {N ic oc h w kH kW : } (xN cotN : String) (b : Vec oc) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (W : Kernel4 oc ic kH kW) (cot : Vec (N * (oc * h * w))) (idx : Fin (oc * ic * kH * kW)) :
StableHLO.den (StableHLO.SHlo.convStridedWeightGradB xN b x W (StableHLO.SHlo.operand cotN cot)) idx = n : Fin N, j : Fin (oc * h * w), pdiv (fun (v' : Vec (oc * ic * kH * kW)) => flatConvStride2 (Kernel4.unflatten v') b (StableHLO.batchSlice N (ic * (2 * h) * (2 * w)) x n)) W.flatten idx j * StableHLO.batchSlice N (oc * h * w) cot n j

Batched strided conv weight GRADIENT denotes the certified Σ_n weight gradient. Generic in the kernel size, so the one lemma certifies the 7x7 stem AND every 3x3 downsample W1 AND every 1x1 projection Wp.

theorem Proofs.ResNet34PoCB.convStridedBGradB_den {N ic oc h w kH kW : } (cotN : String) (W : Kernel4 oc ic kH kW) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (b : Vec oc) (cot : Vec (N * (oc * h * w))) (o : Fin oc) :
StableHLO.den (StableHLO.SHlo.convStridedBiasGradB W x b (StableHLO.SHlo.operand cotN cot)) o = n : Fin N, j : Fin (oc * h * w), pdiv (fun (b' : Vec oc) => flatConvStride2 W b' (StableHLO.batchSlice N (ic * (2 * h) * (2 * w)) x n)) b o j * StableHLO.batchSlice N (oc * h * w) cot n j

Batched strided conv bias GRADIENT denotes the certified Σ_n bias gradient.

theorem Proofs.ResNet34PoCB.bnGammaGradB_den {N oc h w : } (vN epsStr cotN : String) (ε : ) (γ β : Vec oc) (v cot : Vec (N * (oc * (h * w)))) (c : Fin oc) :
StableHLO.den (StableHLO.SHlo.bnGammaGradB vN epsStr ε v (StableHLO.SHlo.operand cotN cot)) c = j : Fin (oc * (N * (h * w))), pdiv (fun (γ' : Vec oc) => bnPerChannelFlat oc (N * (h * w)) ε γ' β (bnchwFwd N oc h w v)) γ c j * bnchwFwd N oc h w cot j

Batched BN γ GRADIENT denotes the certified per-channel γ gradient over the merged batch+spatial axis m = N·(h·w). γ enters affinely, so there is no batch coupling in the PARAM gradient and this is bnPerChannel_grad_gamma_correct at that width, through the network→oc-major reindex bnchwFwd. Generic in the free β.

theorem Proofs.ResNet34PoCB.bnBetaGradB_den {N oc h w : } (cotN : String) (ε : ) (γ β : Vec oc) (v : Vec (oc * (N * (h * w)))) (cot : Vec (N * (oc * (h * w)))) (c : Fin oc) :
StableHLO.den (StableHLO.SHlo.operand cotN cot).bnBetaGradB c = j : Fin (oc * (N * (h * w))), pdiv (fun (β' : Vec oc) => bnPerChannelFlat oc (N * (h * w)) ε γ β' v) β c j * bnchwFwd N oc h w cot j

Batched BN β GRADIENT denotes the certified per-channel β gradient Σ_{batch,spatial} cot at m = N·(h·w). Carries a free v/γ — β's gradient is the channel sum and depends on neither.

def Proofs.ResNet34PoCB.BnPairTiedB (N oc h w : ) (vN epsStr cotN : String) (ε : ) (γ β : Vec oc) (v cot : Vec (N * (oc * (h * w)))) :

One batched BN layer's γ and β gradient nodes, tied — the pair every step tie states per BatchNorm: the emitted bnGammaGradB / bnBetaGradB denote the certified per-channel γ and β gradients over the merged batch+spatial axis, at the layer's pre-BN activation v and its output cotangent cot (both in the network layout).

Equations
  • One or more equations did not get rendered due to their size.
Instances For
    theorem Proofs.ResNet34PoCB.bnPairTiedB_holds {N oc h w : } (vN epsStr cotN : String) (ε : ) (γ β : Vec oc) (v cot : Vec (N * (oc * (h * w)))) :
    BnPairTiedB N oc h w vN epsStr cotN ε γ β v cot
    theorem Proofs.ResNet34PoCB.denseWGradB_den {N a c : } (xN cotN : String) (x : Vec (N * a)) (W : Mat a c) (b : Vec c) (cot : Vec (N * c)) (i : Fin a) (j : Fin c) :

    Batched dense weight GRADIENT denotes the certified Σ_n outer product.

    theorem Proofs.ResNet34PoCB.denseBGradB_den {N c : } (cotN : String) (W : Mat c c) (x b : Vec c) (cot : Vec (N * c)) (j : Fin c) :
    StableHLO.den (StableHLO.SHlo.operand cotN cot).denseBiasGradB j = n : Fin N, k : Fin c, pdiv (fun (b' : Vec c) => dense W b' x) b j k * StableHLO.batchSlice N c cot n k

    Batched dense bias GRADIENT denotes the certified Σ_n cotangent sum.

    def Proofs.ResNet34PoCB.ConvWTiedB (N h w : ) {ic oc kH kW : } (xN cotN : String) (b : Vec oc) (x : Vec (N * (ic * h * w))) (W : Kernel4 oc ic kH kW) (cot : Vec (N * (oc * h * w))) :

    A stride-1 conv weight gradient node, tied (convWGradB_den).

    Equations
    • One or more equations did not get rendered due to their size.
    Instances For
      def Proofs.ResNet34PoCB.ConvBTiedB (N h w : ) {ic oc kH kW : } (cotN : String) (W : Kernel4 oc ic kH kW) (x : Vec (N * (ic * h * w))) (b : Vec oc) (cot : Vec (N * (oc * h * w))) :

      A stride-1 conv bias gradient node, tied (convBGradB_den).

      Equations
      • One or more equations did not get rendered due to their size.
      Instances For
        def Proofs.ResNet34PoCB.ConvStridedWTiedB (N h w : ) {ic oc kH kW : } (xN cotN : String) (b : Vec oc) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (W : Kernel4 oc ic kH kW) (cot : Vec (N * (oc * h * w))) :

        A stride-2 (symmetric-pad) conv weight gradient node, tied (convStridedWGradB_den).

        Equations
        • One or more equations did not get rendered due to their size.
        Instances For
          def Proofs.ResNet34PoCB.ConvStridedBTiedB (N h w : ) {ic oc kH kW : } (cotN : String) (W : Kernel4 oc ic kH kW) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (b : Vec oc) (cot : Vec (N * (oc * h * w))) :

          A stride-2 (symmetric-pad) conv bias gradient node, tied (convStridedBGradB_den).

          Equations
          • One or more equations did not get rendered due to their size.
          Instances For
            def Proofs.ResNet34PoCB.ConvStridedXlaWTiedB (N h w : ) {ic oc kH kW : } (xN cotN : String) (b : Vec oc) (x : Vec (N * (ic * (2 * h) * (2 * w)))) (W : Kernel4 oc ic kH kW) (cot : Vec (N * (oc * h * w))) :

            A stride-2 XLA-SAME conv weight gradient node, tied (EnetPoCG.convStridedXlaWGradB_den).

            Equations
            • One or more equations did not get rendered due to their size.
            Instances For
              def Proofs.ResNet34PoCB.DepthwiseWTiedB (N h w : ) {c kH kW : } (xN cotN : String) (b : Vec c) (x : Vec (N * (c * h * w))) (W : DepthwiseKernel c kH kW) (cot : Vec (N * (c * h * w))) :

              A stride-1 depthwise weight gradient node, tied (EnetPoCG.depthwiseWGradB_den).

              Equations
              • One or more equations did not get rendered due to their size.
              Instances For
                def Proofs.ResNet34PoCB.DepthwiseBTiedB (N h w : ) {c kH kW : } (cotN : String) (W : DepthwiseKernel c kH kW) (x : Vec (N * (c * h * w))) (b : Vec c) (cot : Vec (N * (c * h * w))) :

                A stride-1 depthwise bias gradient node, tied (Mnv2PaperPoCG.depthwiseBGradB_den).

                Equations
                • One or more equations did not get rendered due to their size.
                Instances For
                  def Proofs.ResNet34PoCB.DepthwiseStridedWTiedB (N h w : ) {c kH kW : } (xN cotN : String) (b : Vec c) (x : Vec (N * (c * (2 * h) * (2 * w)))) (W : DepthwiseKernel c kH kW) (cot : Vec (N * (c * h * w))) :

                  A stride-2 depthwise weight gradient node, tied (EnetPoCG.depthwiseStridedWGradB_den).

                  Equations
                  • One or more equations did not get rendered due to their size.
                  Instances For
                    def Proofs.ResNet34PoCB.DenseWTiedB (N : ) {a c : } (xN cotN : String) (x : Vec (N * a)) (W : Mat a c) (b : Vec c) (cot : Vec (N * c)) :

                    A dense weight gradient node, tied (denseWGradB_den).

                    Equations
                    • One or more equations did not get rendered due to their size.
                    Instances For
                      def Proofs.ResNet34PoCB.DenseBTiedB (N : ) {a c : } (cotN : String) (W : Mat a c) (x : Vec a) (b : Vec c) (cot : Vec (N * c)) :

                      A dense bias gradient node, tied — free in W and x, which b's gradient ignores.

                      Equations
                      • One or more equations did not get rendered due to their size.
                      Instances For