Documentation

LeanMlir.Proofs.Nets.Small.MlpCanonical

The CANONICAL MNIST MLP — 784→512→512→10 (ReLU, biased) #

mlpVerified (LeanMlir/VerifiedNets.lean, Chapter 2) is the repo's canonical MNIST reference architecture: [.dense 784 512, .relu, .dense 512 512, .relu, .dense 512 10]. Every runnable MNIST MLP path uses it (verified/e4m3/pgd/spectral/smooth trainers, the committed verified_mlir/mlp_train_step.mlir render, the baselines, margin_probe.py).

This file makes the canonical claim a CHECKABLE LEAN SURFACE: the generic MLP proof chain (whole-net VJP, float-gradient closeness, float-SGD descent, the emitted-train-step tie) instantiated at the literal canonical dims. Each declaration below IS the corresponding generic theorem at (784, 512, 512, 10)#check shows the specialized statement; the 3-axiom audit covers them all. Nothing else in Lean consumes them: tests/AuditAxioms.lean is this file's only importer, by design. The spec-level partner is SpecVJP.lean's mlpVerified_denote_eq / mlpVerified_has_vjp* (stated over mlpVerified.layers itself; that file lives outside the Mathlib-only seam).

The OTHER MNIST proof population — the trained-weight certificate instances (LipschitzCert{Instance,Scorecard*,Float}, TrainedMlpWitness, TrainedLinearDescent) — deliberately lives on a REDUCED model (4×4-pooled 49-dim inputs, width-8 hidden, /128–/256 rational weights): every margin, Schatten/Gram sum, and LDLᵀ SOS witness is exact rational arithmetic checked in-kernel, which is infeasible today at 512-wide fan-ins (and the unconstrained canonical net's spectral-product cert is MEASURED vacuous — L ≈ 39 ⇒ 0% certified — which is why randomized smoothing, which DOES run on the canonical net, exists). Those files carry a reduced-model banner pointing here.

noncomputable def Proofs.MlpCanonical.has_vjp_at (W₀ : Mat 784 512) (b₀ : Vec 512) (W₁ : Mat 512 512) (b₁ : Vec 512) (W₂ : Mat 512 10) (b₂ : Vec 10) (x : Vec 784) (h_smooth_0 : ∀ (k : Fin 512), dense W₀ b₀ x k 0) (h_smooth_1 : ∀ (k : Fin 512), dense W₁ b₁ (relu 512 (dense W₀ b₀ x)) k 0) :
HasVJPAt (mlpForward W₀ b₀ W₁ b₁ W₂ b₂) x

Canonical whole-net pointwise VJP: mlp_has_vjp_at at (784, 512, 512, 10) — the honest conditional witness (both hidden layers off-kink at x).

Equations
Instances For
    noncomputable def Proofs.MlpCanonical.has_vjp_correct (W₀ : Mat 784 512) (b₀ : Vec 512) (W₁ : Mat 512 512) (b₁ : Vec 512) (W₂ : Mat 512 10) (b₂ : Vec 10) (x : Vec 784) (dy : Vec 10) (i : Fin 784) :
    (mlp_has_vjp W₀ b₀ W₁ b₁ W₂ b₂).backward x dy i = j : Fin 10, pdiv (mlpForward W₀ b₀ W₁ b₁ W₂ b₂) x i j * dy j

    Canonical backward-correctness: the canonical witness's backward IS the Jacobian-transpose contraction.

    Equations
    Instances For
      noncomputable def Proofs.MlpCanonical.output_float_sgd_descends (M : FloatModel) (W₀ : Mat 784 512) (b₀ : Vec 512) (W₁ : Mat 512 512) (b₁ : Vec 512) (W₂ : Mat 512 10) (b₂ : Vec 10) (x : Vec 784) (label : Fin 10) (fexp : ) {lr a eexp δ : } (ha : 0 a) (hx : ∀ (i : Fin 512), |relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))) i| a) (hlr : 0 lr) (heexp0 : 0 eexp) (heexp1 : eexp 1) (hδ0 : 0 δ) (hfexp : ∀ (t : ), |fexp t - Real.exp t| eexp * Real.exp t) (hρ1 : FloatModel.smRho M.u eexp 10 < 1) ( : ∀ (k' : Fin 10), |M.dense W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x)))) k' - dense W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x)))) k'| δ) (hsmall : 2 * (a * (lr * (idx : Fin (512 * 10), |gradAt (fun (w : Vec (512 * 10)) => crossEntropy 10 (dense (Mat.unflatten w) b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) label) W₂.flatten idx| + ↑(512 * 10) * FloatModel.mulErr M.u a 1 0 (FloatModel.cotErr M.u eexp δ 10)))) < 1) (h1 : lr * FloatModel.mulErr M.u a 1 0 (FloatModel.cotErr M.u eexp δ 10) * idx : Fin (512 * 10), |gradAt (fun (w : Vec (512 * 10)) => crossEntropy 10 (dense (Mat.unflatten w) b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) label) W₂.flatten idx| (lr * idx : Fin (512 * 10), gradAt (fun (w : Vec (512 * 10)) => crossEntropy 10 (dense (Mat.unflatten w) b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) label) W₂.flatten idx ^ 2) / 4) (h2 : 2 * a ^ 2 / (1 - 2 * (a * (lr * (idx : Fin (512 * 10), |gradAt (fun (w : Vec (512 * 10)) => crossEntropy 10 (dense (Mat.unflatten w) b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) label) W₂.flatten idx| + ↑(512 * 10) * FloatModel.mulErr M.u a 1 0 (FloatModel.cotErr M.u eexp δ 10))))) * (lr * (idx : Fin (512 * 10), |gradAt (fun (w : Vec (512 * 10)) => crossEntropy 10 (dense (Mat.unflatten w) b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) label) W₂.flatten idx| + ↑(512 * 10) * FloatModel.mulErr M.u a 1 0 (FloatModel.cotErr M.u eexp δ 10))) ^ 2 (lr * idx : Fin (512 * 10), gradAt (fun (w : Vec (512 * 10)) => crossEntropy 10 (dense (Mat.unflatten w) b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) label) W₂.flatten idx ^ 2) / 4) :
      crossEntropy 10 (dense (Mat.unflatten (W₂.flatten - lr M.linearFloatGrad W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x)))) fexp label)) b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) label crossEntropy 10 (dense (Mat.unflatten W₂.flatten) b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) label - (lr * idx : Fin (512 * 10), gradAt (fun (w : Vec (512 * 10)) => crossEntropy 10 (dense (Mat.unflatten w) b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) label) W₂.flatten idx ^ 2) / 2

      Canonical output-layer float-SGD descent (mlp_output_float_sgd_descends at the canonical dims): one binary32-model SGD step on W₂ decreases the real CE loss, margins carried.

      Equations
      Instances For
        noncomputable def Proofs.MlpCanonical.hidden_float_sgd_descends (M : FloatModel) (W₁ : Mat 512 512) (b₁ : Vec 512) (W₂ : Mat 512 10) (b₂ : Vec 10) (a₀ : Vec 512) (label : Fin 10) (fexp : ) {lr a w₁ β₁ w₂ β₂ eexp : } (ha : 0 a) (hw₁ : 0 w₁) (hβ₁ : 0 β₁) (hw₂ : 0 w₂) (hβ₂ : 0 β₂) (hlr : 0 lr) (heexp0 : 0 eexp) (heexp1 : eexp 1) (hfexp : ∀ (t : ), |fexp t - Real.exp t| eexp * Real.exp t) (hρ1 : FloatModel.smRho M.u eexp 10 < 1) (hx : ∀ (i : Fin 512), |a₀ i| a) (hW₁ : ∀ (i j : Fin 512), |W₁ i j| w₁) (hb₁ : ∀ (j : Fin 512), |b₁ j| β₁) (hW₂ : ∀ (i : Fin 512) (j : Fin 10), |W₂ i j| w₂) (hb₂ : ∀ (j : Fin 10), |b₂ j| β₂) (hmargin_round : ∀ (j' : Fin 512), FloatModel.layerBudget M.u 512 w₁ β₁ a 0 < |dense W₁ b₁ a₀ j'|) (hmargin_step : ∀ (j : Fin 512), a * (lr * (idx : Fin (512 * 512), |gradAt (fun (w : Vec (512 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense (Mat.unflatten w) b₁ a₀))) label) W₁.flatten idx| + ↑(512 * 512) * FloatModel.mulErr M.u a (FloatModel.layerAct 10 w₂ 0 1) 0 (FloatModel.layerBudget M.u 10 w₂ 0 1 (FloatModel.cotErr M.u eexp (FloatModel.layerBudget M.u 512 w₂ β₂ (FloatModel.layerAct 512 w₁ β₁ a) (FloatModel.layerBudget M.u 512 w₁ β₁ a 0)) 10)))) < |dense W₁ b₁ a₀ j|) (hsmall : 2 * (w₂ * (a * (lr * (idx : Fin (512 * 512), |gradAt (fun (w : Vec (512 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense (Mat.unflatten w) b₁ a₀))) label) W₁.flatten idx| + ↑(512 * 512) * FloatModel.mulErr M.u a (FloatModel.layerAct 10 w₂ 0 1) 0 (FloatModel.layerBudget M.u 10 w₂ 0 1 (FloatModel.cotErr M.u eexp (FloatModel.layerBudget M.u 512 w₂ β₂ (FloatModel.layerAct 512 w₁ β₁ a) (FloatModel.layerBudget M.u 512 w₁ β₁ a 0)) 10)))))) < 1) (h1 : lr * FloatModel.mulErr M.u a (FloatModel.layerAct 10 w₂ 0 1) 0 (FloatModel.layerBudget M.u 10 w₂ 0 1 (FloatModel.cotErr M.u eexp (FloatModel.layerBudget M.u 512 w₂ β₂ (FloatModel.layerAct 512 w₁ β₁ a) (FloatModel.layerBudget M.u 512 w₁ β₁ a 0)) 10)) * idx : Fin (512 * 512), |gradAt (fun (w : Vec (512 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense (Mat.unflatten w) b₁ a₀))) label) W₁.flatten idx| (lr * idx : Fin (512 * 512), gradAt (fun (w : Vec (512 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense (Mat.unflatten w) b₁ a₀))) label) W₁.flatten idx ^ 2) / 4) (h2 : 2 * 10 * w₂ ^ 2 * a ^ 2 / (1 - 2 * (w₂ * (a * (lr * (idx : Fin (512 * 512), |gradAt (fun (w : Vec (512 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense (Mat.unflatten w) b₁ a₀))) label) W₁.flatten idx| + ↑(512 * 512) * FloatModel.mulErr M.u a (FloatModel.layerAct 10 w₂ 0 1) 0 (FloatModel.layerBudget M.u 10 w₂ 0 1 (FloatModel.cotErr M.u eexp (FloatModel.layerBudget M.u 512 w₂ β₂ (FloatModel.layerAct 512 w₁ β₁ a) (FloatModel.layerBudget M.u 512 w₁ β₁ a 0)) 10))))))) * (lr * (idx : Fin (512 * 512), |gradAt (fun (w : Vec (512 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense (Mat.unflatten w) b₁ a₀))) label) W₁.flatten idx| + ↑(512 * 512) * FloatModel.mulErr M.u a (FloatModel.layerAct 10 w₂ 0 1) 0 (FloatModel.layerBudget M.u 10 w₂ 0 1 (FloatModel.cotErr M.u eexp (FloatModel.layerBudget M.u 512 w₂ β₂ (FloatModel.layerAct 512 w₁ β₁ a) (FloatModel.layerBudget M.u 512 w₁ β₁ a 0)) 10)))) ^ 2 (lr * idx : Fin (512 * 512), gradAt (fun (w : Vec (512 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense (Mat.unflatten w) b₁ a₀))) label) W₁.flatten idx ^ 2) / 4) :
        crossEntropy 10 (dense W₂ b₂ (relu 512 (dense (Mat.unflatten (W₁.flatten - lr M.mlpHiddenFloatGrad W₁ b₁ W₂ b₂ a₀ fexp label)) b₁ a₀))) label crossEntropy 10 (dense W₂ b₂ (relu 512 (dense (Mat.unflatten W₁.flatten) b₁ a₀))) label - (lr * idx : Fin (512 * 512), gradAt (fun (w : Vec (512 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense (Mat.unflatten w) b₁ a₀))) label) W₁.flatten idx ^ 2) / 2

        Canonical hidden-layer float-SGD descent.

        Equations
        Instances For
          noncomputable def Proofs.MlpCanonical.input_float_sgd_descends (M : FloatModel) (W₀ : Mat 784 512) (b₀ : Vec 512) (W₁ : Mat 512 512) (b₁ : Vec 512) (W₂ : Mat 512 10) (b₂ : Vec 10) (x : Vec 784) (label : Fin 10) (fexp : ) {lr a w₀ β₀ w₁ β₁ w₂ β₂ eexp : } (ha : 0 a) (hw₀ : 0 w₀) (hβ₀ : 0 β₀) (hw₁ : 0 w₁) (hβ₁ : 0 β₁) (hw₂ : 0 w₂) (hβ₂ : 0 β₂) (hlr : 0 lr) (heexp0 : 0 eexp) (heexp1 : eexp 1) (hfexp : ∀ (t : ), |fexp t - Real.exp t| eexp * Real.exp t) (hρ1 : FloatModel.smRho M.u eexp 10 < 1) (hx : ∀ (i : Fin 784), |x i| a) (hW₀ : ∀ (i : Fin 784) (j : Fin 512), |W₀ i j| w₀) (hb₀ : ∀ (j : Fin 512), |b₀ j| β₀) (hW₁ : ∀ (i j : Fin 512), |W₁ i j| w₁) (hb₁ : ∀ (j : Fin 512), |b₁ j| β₁) (hW₂ : ∀ (i : Fin 512) (j : Fin 10), |W₂ i j| w₂) (hb₂ : ∀ (j : Fin 10), |b₂ j| β₂) (hmargin0_round : ∀ (j' : Fin 512), FloatModel.layerBudget M.u 784 w₀ β₀ a 0 < |dense W₀ b₀ x j'|) (hmargin1_round : ∀ (l' : Fin 512), FloatModel.layerBudget M.u 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a) (FloatModel.layerBudget M.u 784 w₀ β₀ a 0) < |dense W₁ b₁ (relu 512 (dense W₀ b₀ x)) l'|) (hmargin0_step : ∀ (j : Fin 512), a * (lr * (idx : Fin (784 * 512), |gradAt (fun (w : Vec (784 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense (Mat.unflatten w) b₀ x))))) label) W₀.flatten idx| + ↑(784 * 512) * FloatModel.mulErr M.u a (FloatModel.layerAct 512 w₁ 0 (FloatModel.layerAct 10 w₂ 0 1)) 0 (FloatModel.layerBudget M.u 512 w₁ 0 (FloatModel.layerAct 10 w₂ 0 1) (FloatModel.layerBudget M.u 10 w₂ 0 1 (FloatModel.cotErr M.u eexp (FloatModel.layerBudget M.u 512 w₂ β₂ (FloatModel.layerAct 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a)) (FloatModel.layerBudget M.u 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a) (FloatModel.layerBudget M.u 784 w₀ β₀ a 0))) 10))))) < |dense W₀ b₀ x j|) (hmargin1_step : ∀ (l : Fin 512), w₁ * (a * (lr * (idx : Fin (784 * 512), |gradAt (fun (w : Vec (784 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense (Mat.unflatten w) b₀ x))))) label) W₀.flatten idx| + ↑(784 * 512) * FloatModel.mulErr M.u a (FloatModel.layerAct 512 w₁ 0 (FloatModel.layerAct 10 w₂ 0 1)) 0 (FloatModel.layerBudget M.u 512 w₁ 0 (FloatModel.layerAct 10 w₂ 0 1) (FloatModel.layerBudget M.u 10 w₂ 0 1 (FloatModel.cotErr M.u eexp (FloatModel.layerBudget M.u 512 w₂ β₂ (FloatModel.layerAct 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a)) (FloatModel.layerBudget M.u 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a) (FloatModel.layerBudget M.u 784 w₀ β₀ a 0))) 10)))))) < |dense W₁ b₁ (relu 512 (dense W₀ b₀ x)) l|) (hsmall : 2 * (w₂ * (512 * (w₁ * (a * (lr * (idx : Fin (784 * 512), |gradAt (fun (w : Vec (784 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense (Mat.unflatten w) b₀ x))))) label) W₀.flatten idx| + ↑(784 * 512) * FloatModel.mulErr M.u a (FloatModel.layerAct 512 w₁ 0 (FloatModel.layerAct 10 w₂ 0 1)) 0 (FloatModel.layerBudget M.u 512 w₁ 0 (FloatModel.layerAct 10 w₂ 0 1) (FloatModel.layerBudget M.u 10 w₂ 0 1 (FloatModel.cotErr M.u eexp (FloatModel.layerBudget M.u 512 w₂ β₂ (FloatModel.layerAct 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a)) (FloatModel.layerBudget M.u 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a) (FloatModel.layerBudget M.u 784 w₀ β₀ a 0))) 10))))))))) < 1) (h1 : lr * FloatModel.mulErr M.u a (FloatModel.layerAct 512 w₁ 0 (FloatModel.layerAct 10 w₂ 0 1)) 0 (FloatModel.layerBudget M.u 512 w₁ 0 (FloatModel.layerAct 10 w₂ 0 1) (FloatModel.layerBudget M.u 10 w₂ 0 1 (FloatModel.cotErr M.u eexp (FloatModel.layerBudget M.u 512 w₂ β₂ (FloatModel.layerAct 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a)) (FloatModel.layerBudget M.u 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a) (FloatModel.layerBudget M.u 784 w₀ β₀ a 0))) 10))) * idx : Fin (784 * 512), |gradAt (fun (w : Vec (784 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense (Mat.unflatten w) b₀ x))))) label) W₀.flatten idx| (lr * idx : Fin (784 * 512), gradAt (fun (w : Vec (784 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense (Mat.unflatten w) b₀ x))))) label) W₀.flatten idx ^ 2) / 4) (h2 : 2 * 10 * 512 ^ 2 * w₁ ^ 2 * w₂ ^ 2 * a ^ 2 / (1 - 2 * (w₂ * (512 * (w₁ * (a * (lr * (idx : Fin (784 * 512), |gradAt (fun (w : Vec (784 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense (Mat.unflatten w) b₀ x))))) label) W₀.flatten idx| + ↑(784 * 512) * FloatModel.mulErr M.u a (FloatModel.layerAct 512 w₁ 0 (FloatModel.layerAct 10 w₂ 0 1)) 0 (FloatModel.layerBudget M.u 512 w₁ 0 (FloatModel.layerAct 10 w₂ 0 1) (FloatModel.layerBudget M.u 10 w₂ 0 1 (FloatModel.cotErr M.u eexp (FloatModel.layerBudget M.u 512 w₂ β₂ (FloatModel.layerAct 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a)) (FloatModel.layerBudget M.u 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a) (FloatModel.layerBudget M.u 784 w₀ β₀ a 0))) 10)))))))))) * (lr * (idx : Fin (784 * 512), |gradAt (fun (w : Vec (784 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense (Mat.unflatten w) b₀ x))))) label) W₀.flatten idx| + ↑(784 * 512) * FloatModel.mulErr M.u a (FloatModel.layerAct 512 w₁ 0 (FloatModel.layerAct 10 w₂ 0 1)) 0 (FloatModel.layerBudget M.u 512 w₁ 0 (FloatModel.layerAct 10 w₂ 0 1) (FloatModel.layerBudget M.u 10 w₂ 0 1 (FloatModel.cotErr M.u eexp (FloatModel.layerBudget M.u 512 w₂ β₂ (FloatModel.layerAct 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a)) (FloatModel.layerBudget M.u 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a) (FloatModel.layerBudget M.u 784 w₀ β₀ a 0))) 10))))) ^ 2 (lr * idx : Fin (784 * 512), gradAt (fun (w : Vec (784 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense (Mat.unflatten w) b₀ x))))) label) W₀.flatten idx ^ 2) / 4) :
          crossEntropy 10 (dense W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense (Mat.unflatten (W₀.flatten - lr M.mlpInputFloatGrad W₀ b₀ W₁ b₁ W₂ b₂ x fexp label)) b₀ x))))) label crossEntropy 10 (dense W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense (Mat.unflatten W₀.flatten) b₀ x))))) label - (lr * idx : Fin (784 * 512), gradAt (fun (w : Vec (784 * 512)) => crossEntropy 10 (dense W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense (Mat.unflatten w) b₀ x))))) label) W₀.flatten idx ^ 2) / 2

          Canonical input-layer float-SGD descent — the whole canonical MLP is float-fused descent, layer by layer.

          Equations
          Instances For
            noncomputable def Proofs.MlpCanonical.w1_grad_close (M : FloatModel) (W₁ : Mat 512 512) (b₁ : Vec 512) (W₂ : Mat 512 10) (b₂ : Vec 10) (a₀ : Vec 512) (label : Fin 10) (fexp : ) {a w₁ β₁ w₂ β₂ eexp : } (ha : 0 a) (hw₁ : 0 w₁) (hβ₁ : 0 β₁) (hw₂ : 0 w₂) (hβ₂ : 0 β₂) (heexp0 : 0 eexp) (heexp1 : eexp 1) (hfexp : ∀ (t : ), |fexp t - Real.exp t| eexp * Real.exp t) (hρ1 : FloatModel.smRho M.u eexp 10 < 1) (hx : ∀ (i : Fin 512), |a₀ i| a) (hW₁ : ∀ (i j : Fin 512), |W₁ i j| w₁) (hb₁ : ∀ (j : Fin 512), |b₁ j| β₁) (hW₂ : ∀ (i : Fin 512) (j : Fin 10), |W₂ i j| w₂) (hb₂ : ∀ (j : Fin 10), |b₂ j| β₂) (hmargin : ∀ (j' : Fin 512), FloatModel.layerBudget M.u 512 w₁ β₁ a 0 < |dense W₁ b₁ a₀ j'|) (i j : Fin 512) :
            |M.mul (a₀ i) (FloatModel.reluMask (M.dense W₁ b₁ a₀) (M.dense (fun (j' : Fin 10) (i' : Fin 512) => W₂ i' j') (fun (x : Fin 512) => 0) (M.softmaxCECotF fexp (M.dense W₂ b₂ (relu 512 (M.dense W₁ b₁ a₀))) label)) j) - a₀ i * FloatModel.reluMask (dense W₁ b₁ a₀) (dense (fun (j' : Fin 10) (i' : Fin 512) => W₂ i' j') (fun (x : Fin 512) => 0) fun (k : Fin 10) => softmax 10 (dense W₂ b₂ (relu 512 (dense W₁ b₁ a₀))) k - oneHot 10 label k) j| FloatModel.mulErr M.u a (FloatModel.layerAct 10 w₂ 0 1) 0 (FloatModel.layerBudget M.u 10 w₂ 0 1 (FloatModel.cotErr M.u eexp (FloatModel.layerBudget M.u 512 w₂ β₂ (FloatModel.layerAct 512 w₁ β₁ a) (FloatModel.layerBudget M.u 512 w₁ β₁ a 0)) 10))

            Canonical W₁ float-gradient closeness.

            Equations
            Instances For
              noncomputable def Proofs.MlpCanonical.w0_grad_close (M : FloatModel) (W₀ : Mat 784 512) (b₀ : Vec 512) (W₁ : Mat 512 512) (b₁ : Vec 512) (W₂ : Mat 512 10) (b₂ : Vec 10) (x : Vec 784) (label : Fin 10) (fexp : ) {a w₀ β₀ w₁ β₁ w₂ β₂ eexp : } (ha : 0 a) (hw₀ : 0 w₀) (hβ₀ : 0 β₀) (hw₁ : 0 w₁) (hβ₁ : 0 β₁) (hw₂ : 0 w₂) (hβ₂ : 0 β₂) (heexp0 : 0 eexp) (heexp1 : eexp 1) (hfexp : ∀ (t : ), |fexp t - Real.exp t| eexp * Real.exp t) (hρ1 : FloatModel.smRho M.u eexp 10 < 1) (hx : ∀ (i : Fin 784), |x i| a) (hW₀ : ∀ (i : Fin 784) (j : Fin 512), |W₀ i j| w₀) (hb₀ : ∀ (j : Fin 512), |b₀ j| β₀) (hW₁ : ∀ (i j : Fin 512), |W₁ i j| w₁) (hb₁ : ∀ (j : Fin 512), |b₁ j| β₁) (hW₂ : ∀ (i : Fin 512) (j : Fin 10), |W₂ i j| w₂) (hb₂ : ∀ (j : Fin 10), |b₂ j| β₂) (hmargin0 : ∀ (j' : Fin 512), FloatModel.layerBudget M.u 784 w₀ β₀ a 0 < |dense W₀ b₀ x j'|) (hmargin1 : ∀ (l' : Fin 512), FloatModel.layerBudget M.u 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a) (FloatModel.layerBudget M.u 784 w₀ β₀ a 0) < |dense W₁ b₁ (relu 512 (dense W₀ b₀ x)) l'|) (i : Fin 784) (j : Fin 512) :
              |M.mul (x i) (FloatModel.reluMask (M.dense W₀ b₀ x) (M.dense (fun (j' i' : Fin 512) => W₁ i' j') (fun (x : Fin 512) => 0) (FloatModel.reluMask (M.dense W₁ b₁ (relu 512 (M.dense W₀ b₀ x))) (M.dense (fun (j' : Fin 10) (i' : Fin 512) => W₂ i' j') (fun (x : Fin 512) => 0) (M.softmaxCECotF fexp (M.dense W₂ b₂ (relu 512 (M.dense W₁ b₁ (relu 512 (M.dense W₀ b₀ x))))) label)))) j) - x i * FloatModel.reluMask (dense W₀ b₀ x) (dense (fun (j' i' : Fin 512) => W₁ i' j') (fun (x : Fin 512) => 0) (FloatModel.reluMask (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))) (dense (fun (j' : Fin 10) (i' : Fin 512) => W₂ i' j') (fun (x : Fin 512) => 0) fun (k : Fin 10) => softmax 10 (dense W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) k - oneHot 10 label k))) j| FloatModel.mulErr M.u a (FloatModel.layerAct 512 w₁ 0 (FloatModel.layerAct 10 w₂ 0 1)) 0 (FloatModel.layerBudget M.u 512 w₁ 0 (FloatModel.layerAct 10 w₂ 0 1) (FloatModel.layerBudget M.u 10 w₂ 0 1 (FloatModel.cotErr M.u eexp (FloatModel.layerBudget M.u 512 w₂ β₂ (FloatModel.layerAct 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a)) (FloatModel.layerBudget M.u 512 w₁ β₁ (FloatModel.layerAct 784 w₀ β₀ a) (FloatModel.layerBudget M.u 784 w₀ β₀ a 0))) 10)))

              Canonical W₀ float-gradient closeness.

              Equations
              Instances For
                noncomputable def Proofs.MlpCanonical.train_step_tied_certified (W₀ : Mat 784 512) (b₀ : Vec 512) (W₁ : Mat 512 512) (b₁ : Vec 512) (W₂ : Mat 512 10) (b₂ : Vec 10) (x : Vec 784) (lr : ) (lrStr aN dyN cN : String) (label : Fin 10) :
                (∀ (i : Fin 512) (j : Fin 10), StableHLO.den (StableHLO.SHlo.weightSgd aN "%W2" lrStr (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x)))) W₂ lr (StableHLO.SHlo.operand dyN fun (k : Fin 10) => softmax 10 (mnistLinear W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) k - oneHot 10 label k)) (finProdFinEquiv (i, j)) = W₂ i j - lr * pdiv (fun (v : Vec (512 * 10)) (x_1 : Fin 1) => crossEntropy 10 (dense (Mat.unflatten v) b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) label) W₂.flatten (finProdFinEquiv (i, j)) 0) (∀ (i : Fin 10), StableHLO.den (StableHLO.SHlo.biasSgd "%b2" lrStr b₂ lr (StableHLO.SHlo.operand dyN fun (k : Fin 10) => softmax 10 (mnistLinear W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) k - oneHot 10 label k)) i = b₂ i - lr * j : Fin 10, pdiv (fun (b' : Vec 10) => dense W₂ b' (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) b₂ i j * (fun (k : Fin 10) => softmax 10 (mnistLinear W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) k - oneHot 10 label k) j) (∀ (i j : Fin 512), StableHLO.den (StableHLO.SHlo.weightSgd aN "%W1" lrStr (relu 512 (dense W₀ b₀ x)) W₁ lr (StableHLO.SHlo.operand cN ((IR.mlpCotOut1 W₂ (dense W₁ b₁ (relu 512 (dense W₀ b₀ x)))).denote fun (k : Fin 10) => softmax 10 (mnistLinear W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) k - oneHot 10 label k))) (finProdFinEquiv (i, j)) = W₁ i j - lr * k : Fin 512, pdiv (fun (v : Vec (512 * 512)) => dense (Mat.unflatten v) b₁ (relu 512 (dense W₀ b₀ x))) W₁.flatten (finProdFinEquiv (i, j)) k * (IR.mlpCotOut1 W₂ (dense W₁ b₁ (relu 512 (dense W₀ b₀ x)))).denote (fun (k : Fin 10) => softmax 10 (mnistLinear W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) k - oneHot 10 label k) k) (∀ (i : Fin 512), StableHLO.den (StableHLO.SHlo.biasSgd "%b1" lrStr b₁ lr (StableHLO.SHlo.operand cN ((IR.mlpCotOut1 W₂ (dense W₁ b₁ (relu 512 (dense W₀ b₀ x)))).denote fun (k : Fin 10) => softmax 10 (mnistLinear W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) k - oneHot 10 label k))) i = b₁ i - lr * j : Fin 512, pdiv (fun (b' : Vec 512) => dense W₁ b' (relu 512 (dense W₀ b₀ x))) b₁ i j * (IR.mlpCotOut1 W₂ (dense W₁ b₁ (relu 512 (dense W₀ b₀ x)))).denote (fun (k : Fin 10) => softmax 10 (mnistLinear W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) k - oneHot 10 label k) j) (∀ (i : Fin 784) (j : Fin 512), StableHLO.den (StableHLO.SHlo.weightSgd "%x" "%W0" lrStr x W₀ lr (StableHLO.SHlo.operand cN ((IR.mlpCotOut0 W₁ W₂ (dense W₀ b₀ x) (dense W₁ b₁ (relu 512 (dense W₀ b₀ x)))).denote fun (k : Fin 10) => softmax 10 (mnistLinear W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) k - oneHot 10 label k))) (finProdFinEquiv (i, j)) = W₀ i j - lr * k : Fin 512, pdiv (fun (v : Vec (784 * 512)) => dense (Mat.unflatten v) b₀ x) W₀.flatten (finProdFinEquiv (i, j)) k * (IR.mlpCotOut0 W₁ W₂ (dense W₀ b₀ x) (dense W₁ b₁ (relu 512 (dense W₀ b₀ x)))).denote (fun (k : Fin 10) => softmax 10 (mnistLinear W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) k - oneHot 10 label k) k) ∀ (i : Fin 512), StableHLO.den (StableHLO.SHlo.biasSgd "%b0" lrStr b₀ lr (StableHLO.SHlo.operand cN ((IR.mlpCotOut0 W₁ W₂ (dense W₀ b₀ x) (dense W₁ b₁ (relu 512 (dense W₀ b₀ x)))).denote fun (k : Fin 10) => softmax 10 (mnistLinear W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) k - oneHot 10 label k))) i = b₀ i - lr * j : Fin 512, pdiv (fun (b' : Vec 512) => dense W₀ b' x) b₀ i j * (IR.mlpCotOut0 W₁ W₂ (dense W₀ b₀ x) (dense W₁ b₁ (relu 512 (dense W₀ b₀ x)))).denote (fun (k : Fin 10) => softmax 10 (mnistLinear W₂ b₂ (relu 512 (dense W₁ b₁ (relu 512 (dense W₀ b₀ x))))) k - oneHot 10 label k) j

                Canonical emitted-train-step tie (MlpPoC.mlp_train_step_tied_certified at the canonical dims): every SGD op of the emitted graph denotes the certified loss-descent step of the REAL canonical forward.

                Equations
                Instances For