Documentation

LeanMlir.Proofs.SpecVJP

Spec → math: nine committed VerifiedNetSpecs denote their proven forwards #

The shape #guard beside resnet34Verified in Verified.NetsCore only checks the parameter interface (typechecking). This file ties nine committed specs' layer lists — the linear classifier, MLP, MNIST CNN, CIFAR CNN, and the 10-class Imagenette MobileNetV2, ResNet-34, EfficientNet-B0, ConvNeXt-T and ViT-Tiny — to the math the proofs are about. The other specs (the ImageNet ones, ResNet-50, MobileNetV4, the cifar8* family) are not tied here. Per net, up to three pieces:

  1. a denotation denote* mapping the spec's layer list to the proven forward function, and a *_denote_eq lemma equating the two by rfl (drift-sensitive: any other layer list denotes 0, so editing the spec's layers breaks the rfl);
  2. for the linear classifier, the MLP and ViT-Tiny, a VJP witness at the spec's denotation (linearVerifiedHasVJP is denseHasVJP, vitVerifiedHasVJP transports vitForwardKVHasVJP, and at a smooth input the MLP has the folded mlpVerifiedHasVJPAt). The other nets' whole-net VJPs are stated on their forwards (mnistCnnNoBnHasVJPAt, cifarCnnHasVJPAt, mobilenetv2ForwardBFullHasVJPAt, resnet34ForwardBFullHasVJPAt, efficientnetForwardBFullHasVJP, convNextForwardTChHasVJP), which *_denote_eq equates with the spec's denotation;
  3. a *_fwd_faithful lemma composing the forward graph's faithfulness theorem with the tie, so the generated forward MLIR denotes the spec's function.

ConvNeXt-T's and ViT-Tiny's denotations take the form of the GELU (GeluForm) as an argument. A spec's layer list does not record it: like weight-decay exclusion, clipping or drop-path it is a property of the render variant, and the ties hold at either form.

noncomputable def denoteLinear (layers : List VLayer) (W : Proofs.Mat 784 10) (b : Proofs.Vec 10) :

Math denotation of the linear spec. The Chapter-1 model is a single dense layer, so [.dense 784 10] denotes to the Mathlib dense W b. Any other layer list is not the linear model (0), which makes the tie below drift-sensitive.

Equations
Instances For

    Spec ≡ the proven model. linearVerified's denotation is exactly mnistLinear (the function the Chapter-1 VJP capstone is about) — by rfl, so it's checked by the kernel and breaks if linearVerified.layers changes.

    A VJP for the linear spec's denotation: denseHasVJP, the hand-written dense backward with its correctness proof, at the spec's denotation.

    Equations
    Instances For

      The MLP — a vjpCompAt fold #

      The linear model was the degenerate case (one layer, no fold). The MLP's denotation is a chain — dense ∘ relu ∘ dense ∘ relu ∘ dense (mlpForward) — and its VJP is built by folding vjpCompAt down that chain (mlpHasVJPAt). So this is where the spec→math tie first exercises the chain rule, not just a single op.

      noncomputable def denoteMLP (layers : List VLayer) (W₀ : Proofs.Mat 784 512) (b₀ : Proofs.Vec 512) (W₁ : Proofs.Mat 512 512) (b₁ : Proofs.Vec 512) (W₂ : Proofs.Mat 512 10) (b₂ : Proofs.Vec 10) :

      Math denotation of the MLP spec: the 5-layer list denotes to mlpForward.

      Equations
      Instances For
        theorem mlpVerified_denote_eq (W₀ : Proofs.Mat 784 512) (b₀ : Proofs.Vec 512) (W₁ : Proofs.Mat 512 512) (b₁ : Proofs.Vec 512) (W₂ : Proofs.Mat 512 10) (b₂ : Proofs.Vec 10) :
        denoteMLP mlpVerified.layers W₀ b₀ W₁ b₁ W₂ b₂ = Proofs.mlpForward W₀ b₀ W₁ b₁ W₂ b₂

        Spec ≡ the proven model. mlpVerified's denotation is exactly mlpForward (dense ∘ relu ∘ dense ∘ relu ∘ dense) — by rfl, drift-sensitive.

        noncomputable def mlpVerifiedHasVJPAt (W₀ : Proofs.Mat 784 512) (b₀ : Proofs.Vec 512) (W₁ : Proofs.Mat 512 512) (b₁ : Proofs.Vec 512) (W₂ : Proofs.Mat 512 10) (b₂ : Proofs.Vec 10) (x : Proofs.Vec 784) (h0 : ∀ (k : Fin 512), Proofs.dense W₀ b₀ x k ≠ 0) (h1 : ∀ (k : Fin 512), Proofs.dense W₁ b₁ (Proofs.relu 512 (Proofs.dense W₀ b₀ x)) k ≠ 0) :
        Proofs.HasVJPAt (denoteMLP mlpVerified.layers W₀ b₀ W₁ b₁ W₂ b₂) x

        The folded VJP at a smooth input. When the two ReLU pre-activations avoid zero (h0, h1), the MLP spec's denotation has a VJP built by folding vjpCompAt through dense → relu → dense → relu → dense (mlpHasVJPAt).

        Equations
        Instances For

          The MNIST CNN #

          The CNN's denotation is mnistCnnNoBnForward — a flat Vec 784 → Vec 10 chain flatConv → relu → flatConv → relu → maxPoolFlat → dense → relu → dense → relu → dense. The honest chain-rule fold (via vjpCompAt through conv/maxpool/dense) is the audited mnistCnnNoBnHasVJPAt, conditional on its ReLU and max-pool smoothness hypotheses at the input. The witness below is the canonical one; the spec is the subject of that conditional fold via cnnVerified_denote_eq.

          noncomputable def denoteCNN (layers : List VLayer) (W₁ : Proofs.Kernel4 32 1 3 3) (b₁ : Proofs.Vec 32) (W₂ : Proofs.Kernel4 32 32 3 3) (b₂ : Proofs.Vec 32) (W₃ : Proofs.Mat 6272 512) (b₃ : Proofs.Vec 512) (W₄ : Proofs.Mat 512 512) (b₄ : Proofs.Vec 512) (W₅ : Proofs.Mat 512 10) (b₅ : Proofs.Vec 10) :

          Math denotation of the CNN spec: the 11-layer list denotes to mnistCnnNoBnForward (c=32, h=w=14, the Chapter-3 MNIST CNN).

          Equations
          • One or more equations did not get rendered due to their size.
          • denoteCNN layers W₁ b₁ W₂ b₂ W₃ b₃ W₄ b₄ W₅ b₅ = fun (x : Proofs.Vec 784) => 0
          Instances For
            theorem cnnVerified_denote_eq (W₁ : Proofs.Kernel4 32 1 3 3) (b₁ : Proofs.Vec 32) (W₂ : Proofs.Kernel4 32 32 3 3) (b₂ : Proofs.Vec 32) (W₃ : Proofs.Mat 6272 512) (b₃ : Proofs.Vec 512) (W₄ : Proofs.Mat 512 512) (b₄ : Proofs.Vec 512) (W₅ : Proofs.Mat 512 10) (b₅ : Proofs.Vec 10) :
            denoteCNN cnnVerified.layers W₁ b₁ W₂ b₂ W₃ b₃ W₄ b₄ W₅ b₅ = Proofs.mnistCnnNoBnForward W₁ b₁ W₂ b₂ W₃ b₃ W₄ b₄ W₅ b₅

            Spec ≡ the proven model. cnnVerified's denotation is exactly mnistCnnNoBnForward — the function the Chapter-3 fold mnistCnnNoBnHasVJPAt is about — by rfl.

            Linear: the spec ↔ the generated MLIR #

            The ties above connect the spec to the math (denote = the proven forward, which has the proven VJP). This connects the spec to the StableHLO the trainer actually compiles and runs: the generated forward graph fwdGraph (→ verified_mlir/linear_fwd.mlir, the eval path) and the train-step loss-cotangent graph lossCotGraph (→ linear_train_step.mlir) denote the spec's forward and its softmax-CE gradient — via the audited faithfulness theorems (fwdGraph_faithful, lossCotGraph_isCEgrad) composed with denoteLinear = mnistLinear (rfl). So the generated code provably computes the spec's function.

            What stays trusted (the codegen boundary, per Proofs/README.md): the text render linearFwdModuleV = pretty (emit fwdGraph) and that the committed .mlir equals that text — the pretty-printer + regeneration, NOT the semantics, which are proven here.

            Generated forward MLIR ↔ spec. The forward graph (rendered to linear_fwd.mlir, the eval path) denotes the spec's forward function.

            theorem linearVerified_lossCot_isCEgrad (W : Proofs.Mat 784 10) (b : Proofs.Vec 10) (x : Proofs.Vec 784) (label j : Fin 10) :

            Generated train-step cotangent ↔ spec. The loss-cotangent graph (in linear_train_step.mlir) denotes ∂(softmax-CE)/∂logits at the spec's logits.

            MLP: the spec ↔ the generated MLIR — both forward and backward #

            The MLP has faithfulness for the whole forward graph (mlpFwdGraph_faithful, the graph mlp_fwd.mlir prints) AND for a whole backward input-VJP graph (mlpBackGraph_faithful). Composed with denoteMLP = mlpForward and mlpVerifiedHasVJPAt = mlpHasVJPAt, both denote the spec: the rendered forward computes the spec's forward, and mlpBackGraph computes the spec's VJP backward (at a smooth input). mlpBackGraph is a spec-level graph no committed artifact prints: mlp_train_step.mlir is MlpRender.lean's mlpTrainStepText, whose parameter ops MlpFold ties to the certified step.

            theorem mlpVerified_fwd_faithful (W₀ : Proofs.Mat 784 512) (b₀ : Proofs.Vec 512) (W₁ : Proofs.Mat 512 512) (b₁ : Proofs.Vec 512) (W₂ : Proofs.Mat 512 10) (b₂ : Proofs.Vec 10) (x : Proofs.Vec 784) :
            Proofs.StableHLO.den (Proofs.StableHLO.mlpFwdGraph W₀ b₀ W₁ b₁ W₂ b₂ x) = denoteMLP mlpVerified.layers W₀ b₀ W₁ b₁ W₂ b₂ x

            Generated MLP forward MLIR ↔ spec. The forward graph (→ mlp_fwd.mlir) denotes the spec's forward function.

            theorem mlpVerified_back_faithful (W₀ : Proofs.Mat 784 512) (b₀ : Proofs.Vec 512) (W₁ : Proofs.Mat 512 512) (b₁ : Proofs.Vec 512) (W₂ : Proofs.Mat 512 10) (b₂ : Proofs.Vec 10) (x : Proofs.Vec 784) (h0 : ∀ (k : Fin 512), Proofs.dense W₀ b₀ x k ≠ 0) (h1 : ∀ (k : Fin 512), Proofs.dense W₁ b₁ (Proofs.relu 512 (Proofs.dense W₀ b₀ x)) k ≠ 0) (dy : Proofs.Vec 10) :
            Proofs.StableHLO.den (Proofs.StableHLO.mlpBackGraph W₀ W₁ W₂ (Proofs.dense W₀ b₀ x) (Proofs.dense W₁ b₁ (Proofs.relu 512 (Proofs.dense W₀ b₀ x))) dy) = (mlpVerifiedHasVJPAt W₀ b₀ W₁ b₁ W₂ b₂ x h0 h1).backward dy

            MLP backward graph ↔ spec. mlpBackGraph, the input-VJP graph (spec-level; no committed artifact prints it, see the section header), denotes the spec's VJP backward (mlpVerifiedHasVJPAt), at a smooth input (the two ReLU pre-activations avoid zero).

            MNIST CNN: the spec ↔ the generated MLIR (forward) #

            The generated CNN forward graph (flatConv→relu→flatConv→relu→maxPoolFlat→dense→relu→ dense→relu→dense) denotes the spec's forward. The backward graph faithfulness exists too (cnnBackGraph_faithful denotes mnistCnnNoBnHasVJPAt.backward — the VJP of exactly this spec's forward), but it carries the same five ReLU/maxpool smoothness hypotheses as the conditional fold, so we headline the unconditional forward tie.

            theorem cnnVerified_fwd_faithful (W₁ : Proofs.Kernel4 32 1 3 3) (b₁ : Proofs.Vec 32) (W₂ : Proofs.Kernel4 32 32 3 3) (b₂ : Proofs.Vec 32) (W₃ : Proofs.Mat 6272 512) (b₃ : Proofs.Vec 512) (W₄ : Proofs.Mat 512 512) (b₄ : Proofs.Vec 512) (W₅ : Proofs.Mat 512 10) (b₅ : Proofs.Vec 10) (x : Proofs.Vec 784) :
            Proofs.StableHLO.den (Proofs.StableHLO.cnnFwdGraph W₁ b₁ W₂ b₂ W₃ b₃ W₄ b₄ W₅ b₅ x) = denoteCNN cnnVerified.layers W₁ b₁ W₂ b₂ W₃ b₃ W₄ b₄ W₅ b₅ x

            Generated CNN forward MLIR ↔ spec. The forward graph (→ cnn_fwd.mlir) denotes the spec's forward (mnistCnnNoBnForward, c=32 / h=w=14).

            The CIFAR CNN (no BN) #

            The CIFAR-10 net (ic=3, c1=32, c2=64, h=w=8 — spatial 32→16→8) gets the spec→math denotation (= cifarCnnForward by rfl), the canonical witness, and the forward spec→generated-MLIR tie (cifarFwdGraph_faithful). The conditional fold is cifarCnnHasVJPAt (ReLU and max-pool smoothness hypotheses at the input).

            noncomputable def denoteCifar (layers : List VLayer) (W₁ : Proofs.Kernel4 32 3 3 3) (b₁ : Proofs.Vec 32) (W₂ : Proofs.Kernel4 32 32 3 3) (b₂ : Proofs.Vec 32) (W₃ : Proofs.Kernel4 64 32 3 3) (b₃ : Proofs.Vec 64) (W₄ : Proofs.Kernel4 64 64 3 3) (b₄ : Proofs.Vec 64) (W₅ : Proofs.Mat 4096 512) (b₅ : Proofs.Vec 512) (W₆ : Proofs.Mat 512 512) (b₆ : Proofs.Vec 512) (W₇ : Proofs.Mat 512 10) (b₇ : Proofs.Vec 10) :
            Equations
            • One or more equations did not get rendered due to their size.
            • denoteCifar layers W₁ b₁ W₂ b₂ W₃ b₃ W₄ b₄ W₅ b₅ W₆ b₆ W₇ b₇ = fun (x : Proofs.Vec 3072) => 0
            Instances For
              theorem cifarVerified_denote_eq (W₁ : Proofs.Kernel4 32 3 3 3) (b₁ : Proofs.Vec 32) (W₂ : Proofs.Kernel4 32 32 3 3) (b₂ : Proofs.Vec 32) (W₃ : Proofs.Kernel4 64 32 3 3) (b₃ : Proofs.Vec 64) (W₄ : Proofs.Kernel4 64 64 3 3) (b₄ : Proofs.Vec 64) (W₅ : Proofs.Mat 4096 512) (b₅ : Proofs.Vec 512) (W₆ : Proofs.Mat 512 512) (b₆ : Proofs.Vec 512) (W₇ : Proofs.Mat 512 10) (b₇ : Proofs.Vec 10) :
              denoteCifar cifarVerified.layers W₁ b₁ W₂ b₂ W₃ b₃ W₄ b₄ W₅ b₅ W₆ b₆ W₇ b₇ = Proofs.cifarCnnForward W₁ b₁ W₂ b₂ W₃ b₃ W₄ b₄ W₅ b₅ W₆ b₆ W₇ b₇
              theorem cifarVerified_fwd_faithful (W₁ : Proofs.Kernel4 32 3 3 3) (b₁ : Proofs.Vec 32) (W₂ : Proofs.Kernel4 32 32 3 3) (b₂ : Proofs.Vec 32) (W₃ : Proofs.Kernel4 64 32 3 3) (b₃ : Proofs.Vec 64) (W₄ : Proofs.Kernel4 64 64 3 3) (b₄ : Proofs.Vec 64) (W₅ : Proofs.Mat 4096 512) (b₅ : Proofs.Vec 512) (W₆ : Proofs.Mat 512 512) (b₆ : Proofs.Vec 512) (W₇ : Proofs.Mat 512 10) (b₇ : Proofs.Vec 10) (x : Proofs.Vec 3072) :
              Proofs.StableHLO.den (Proofs.StableHLO.cifarFwdGraph W₁ b₁ W₂ b₂ W₃ b₃ W₄ b₄ W₅ b₅ W₆ b₆ W₇ b₇ x) = denoteCifar cifarVerified.layers W₁ b₁ W₂ b₂ W₃ b₃ W₄ b₄ W₅ b₅ W₆ b₆ W₇ b₇ x

              Generated (no-BN) CIFAR forward MLIR ↔ spec.

              MobileNetV2 (full, batched): the committed spec ↔ the batch-BN net #

              denoteMobilenetB maps mobilenetv2Verified.layers — the committed 21-entry full-paper [t,c,n,s] list the trainer runs (stem-s2 3→32 → 17 bottlenecks → 1×1 head 320→1280 → GAP → dense 1280→10) — to mobilenetv2ForwardBFull (MobileNetV2FullB.lean: batch BN throughout, the t=1 no-expand first block, 4 stride-2 depthwise downsamples 224→7), at every batch size. Weights ride in the MNV2BWeights bundle, so the tie stays readable. The rfl is drift-sensitive: any [t,c,n,s] edit to the spec stops the match reducing; certs.yml re-elaborates it on every spec push.

              noncomputable def denoteMobilenetB (N : ℕ) (layers : List VLayer) (w : Proofs.MNV2BWeights 10) :
              Proofs.Vec (N * (3 * 224 * 224)) → Proofs.Vec (N * 10)

              Math denotation of the committed MobileNetV2 spec at batch BN: the 21-entry full-paper layer list denotes to mobilenetv2ForwardBFull — the batch-statistics net the 10-class Imagenette MobileNetV2 artifact runs (MobileNetV2FullB.lean), at every batch size N. Any other list is not the net (0), so the tie below is drift-sensitive.

              Equations
              Instances For

                Spec ≡ the full batch-BN net. mobilenetv2Verified's denotation at batch N is exactly mobilenetv2ForwardBFull N — by rfl, drift-sensitive.

                Forward graph ↔ the committed spec, batched. The typed graph the Imagenette MobileNetV2 artifact is printed from denotes the committed spec's function at batch BN: mobilenetv2FwdGraphBFull_faithful composed with the tie.

                ResNet-34, EfficientNet-B0, ConvNeXt-T, ViT-Tiny (full, weight bundles) #

                The mnv2 full-paper pattern applied to the remaining imagenette nets: each committed spec's ENTIRE layer list (literal dims, drift-sensitive) denotes the full proven forward, with weights riding a structure bundle so the ties stay readable. Existing bundles are reused where the Full module already has one (B0Weights, CnxTWeightsCh, R34BWeights); vit gets its bundle here (ViTTinyWeights, SpecVJP-local so no proof module's signature changes). Each *_fwd_faithful composes the net's full graph-faithfulness theorem with the tie (vit's is vitFwdGraphKMHV_faithful, the depth-k multi-head vector-LN graph of ViTDepthK). ViT alone gets a VJP witness at the spec: all-smooth, so it is the whole-net VJP vitForwardKVHasVJP (only 0 < ε).

                noncomputable def denoteR34FullB (N : ℕ) (layers : List VLayer) (w : Proofs.R34BWeights 10) :
                Proofs.Vec (N * (3 * 224 * 224)) → Proofs.Vec (N * 10)

                Math denotation of the committed ResNet-34 spec at batch BN: the 8-entry stage-level list denotes to resnet34ForwardBFull — the batch-statistics net the 10-class Imagenette ResNet-34 artifact runs (ResNet34FullB.lean), at every batch size N. Any other list is not the net (0), so the tie below is drift-sensitive.

                Equations
                Instances For

                  Spec ≡ the full batch-BN net. resnet34Verified's denotation at batch N is exactly resnet34ForwardBFull N — by rfl, drift-sensitive.

                  Forward graph ↔ the committed spec, batched. The typed graph the Imagenette ResNet-34 artifact is printed from denotes the committed spec's function at batch BN: resnet34FwdGraphBFull_faithful composed with the tie.

                  noncomputable def denoteEfficientnetB0 (N : ℕ) (layers : List VLayer) (w : Proofs.B0Weights 10) :
                  Proofs.Vec (N * (3 * 224 * 224)) → Proofs.Vec (N * 10)

                  Math denotation of the committed EfficientNet-B0 spec at batch N: the 21-entry [t,c,n,s,k] layer list denotes to efficientnetForwardBFull (all 16 MBConv blocks, true batch-norm + SE). The spec ties the batched net at EVERY batch size.

                  Equations
                  Instances For

                    Spec ≡ the full proven net. efficientnetVerified's denotation is exactly efficientnetForwardBFull (16 MBConv, batched, per-channel BN + SE) — by rfl.

                    Forward graph ↔ the committed spec (batched). The full 16-MBConv batched graph denotes the committed spec's function: efficientnetFwdGraphBFull_faithful ∘ the tie.

                    noncomputable def denoteConvnextT (gf : Proofs.GeluForm) (layers : List VLayer) (w : Proofs.CnxTWeightsCh 10) :
                    Proofs.Vec (3 * 224 * 224) → Proofs.Vec 10

                    Math denotation of the committed ConvNeXt-T spec: the 29-entry [3,3,9,3] layer list denotes to convNextForwardTCh — the channel-LayerNorm net, whose 23 LN sites are 1 stem + 18 block + 3 downsample + 1 head, the first 22 reducing over the c channels at one spatial position with a per-channel [c] affine and the head one over the [768] GAP output (which is the same function at one spatial position — rowLNVecFlat 1 768).

                    Equations
                    Instances For

                      Spec ≡ the full proven net. convnextVerified's denotation is exactly convNextForwardTCh ([3,3,9,3] @ [96,192,384,768], channel LN + head LN, 28,589,128 params at K = 1000 — the JAX reference's own count) — by rfl.

                      Forward graph ↔ the committed spec. The committed-config [3,3,9,3] channel-LN graph denotes the committed spec's function: convNextFwdGraphTCh_faithful ∘ the tie.

                      structure ViTTinyWeights :

                      All ViT-Tiny parameters at the committed config (D=192, 3 heads, d_head=64, mlpDim=768, 12 untied blocks, vector-LN): patch embed + CLS/pos + 12 per-block BlockParamsV bundles + final LN + CLS head. Shared LN ε rides along.

                      Instances For
                        noncomputable def vitForwardTiny (gf : Proofs.GeluForm) (w : ViTTinyWeights) :
                        Proofs.Vec (3 * 224 * 224) → Proofs.Vec 10

                        vitForwardKV at the committed ViT-Tiny config (depth 12, 3 heads × 64).

                        Equations
                        Instances For
                          noncomputable def denoteVitTiny (gf : Proofs.GeluForm) (layers : List VLayer) (w : ViTTinyWeights) :
                          Proofs.Vec (3 * 224 * 224) → Proofs.Vec 10

                          Math denotation of the committed ViT-Tiny spec: the 17-entry layer list (12 untied .transformerBlocks, per-channel [192] LN, 1D CLS) denotes to vitForwardTiny.

                          Equations
                          • One or more equations did not get rendered due to their size.
                          Instances For

                            Spec ≡ the full proven net. vitVerified's denotation is exactly vitForwardKV at the committed config (depth-12 DISTINCT-param multi-head, per-token vector-LN — ViTDepthK.lean) — by rfl.

                            The whole-net VJP at the committed ViT-Tiny spec. ViT is all-smooth (GELU/softmax/LN), so the chain-rule fold applies globally: vitForwardKVHasVJP at the committed config, hypothesis 0 < ε only. Not the canonical witness.

                            Equations
                            Instances For
                              theorem vitVerified_fwd_faithful {gf : Proofs.GeluForm} (epsStr sStr oneStr zeroStr : String) (w : ViTTinyWeights) (x : Proofs.Vec (3 * 224 * 224)) :

                              Forward graph ↔ the committed spec. The depth-12 3-head vector-LN forward graph (vitFwdGraphKMHV — patch embed → 12 spelled multi-head blocks → final vector-LN → CLS → head) denotes the committed spec's function: vitFwdGraphKMHV_faithful composed with the tie.