Documentation

LeanMlir.Proofs.Nets.Small.LinearFoldB

The MNIST-linear train step at batch N — the fold LinearFold leaves open #

LinearFold certifies linear_train_step.mlir's two outputs for ONE example: its nodes (weightSgd/biasSgd over lossCotGraph) denote per-example values, and the file says so ("the committed module batch-contracts over B examples, which this file does not state"). This file states it. The same text is the print of a graph at the batched index — batchOp lifts of dense/expe/softmaxDiv, subB, and the denseWeightSgdB/denseBiasSgdB nodes whose den sums the per-example gradient over batchSlice n — and tests/TestSmallNetsBatchedEmit.lean pins that print byte-for-byte to linTrainStepText at the committed shape. So:

Scope #

Two node lemmas every batched dense fold uses #

denseWeightSgdB/denseBiasSgdB denote θ − lr·Σₖ (per-example gradient at slice k) by definition; these read that off at an index. The two collapse lemmas turn the certified dense Jacobian contracted with a cotangent into the closed form the node computes.

theorem Proofs.LinFoldB.denseWeightSgdB_den_slices {N a c : ℕ} (xN wN lrS : String) (x : Vec (N * a)) (W : Mat a c) (lr : ℝ) (e : StableHLO.SHlo (N * c)) (i : Fin a) (j : Fin c) :
theorem Proofs.LinFoldB.denseBiasSgdB_den_slices {N c : ℕ} (bN lrS : String) (b : Vec c) (lr : ℝ) (e : StableHLO.SHlo (N * c)) (j : Fin c) :
StableHLO.den (StableHLO.SHlo.denseBiasSgdB bN lrS b lr e) j = b j - lr * ∑ k : Fin N, StableHLO.batchSlice N c (StableHLO.den e) k j

Slice lemmas for the batched backward-chain nodes #

theorem Proofs.LinFoldB.batchSlice_selectPosB {N n : ℕ} (xN : String) (x : Vec (N * n)) (e : StableHLO.SHlo (N * n)) (k : Fin N) :

A selectPosB mask reads its own example's saved pre-activation.

An operand leaf's slice is the slice of the value it carries.

A dotOut lift's slice is W · (slice of the input).

theorem Proofs.LinFoldB.dense_jacW_collapse {a c : ℕ} (W : Mat a c) (b : Vec c) (x : Vec a) (cot : Vec c) (i : Fin a) (j : Fin c) :
∑ k : Fin c, pdiv (fun (v : Vec (a * c)) => dense (Mat.unflatten v) b x) W.flatten (finProdFinEquiv (i, j)) k * cot k = x i * cot j

The certified dense weight Jacobian contracted with a cotangent is x i · c j.

theorem Proofs.LinFoldB.dense_jacB_collapse {a c : ℕ} (W : Mat a c) (b : Vec c) (x : Vec a) (cot : Vec c) (j : Fin c) :
∑ i : Fin c, pdiv (fun (b' : Vec c) => dense W b' x) b j i * cot i = cot j

The certified dense bias Jacobian contracted with a cotangent is c j.

noncomputable def Proofs.LinFoldB.oneHotB {n : ℕ} (N : ℕ) (label : Fin N → Fin n) :
Vec (N * n)

The one-hot batch %onehot carries: row k is oneHot n (label k).

Equations
Instances For
    theorem Proofs.LinFoldB.batchSlice_oneHotB {n : ℕ} (N : ℕ) (label : Fin N → Fin n) (k : Fin N) :
    StableHLO.batchSlice N n (oneHotB N label) k = oneHot n (label k)
    def Proofs.LinFoldB.fwdGraphB {m n : ℕ} (N : ℕ) (W : Mat m n) (b : Vec n) (xB : Vec (N * m)) :

    The batched forward: dense W b lifted across the batch (%x : [N, m]).

    Equations
    Instances For
      noncomputable def Proofs.LinFoldB.lossCotGraphB {m n : ℕ} (N : ℕ) (W : Mat m n) (b : Vec n) (xB : Vec (N * m)) (label : Fin N → Fin n) :

      The batched loss cotangent softmax(logits) − onehot, every op a batched node.

      Equations
      • One or more equations did not get rendered due to their size.
      Instances For
        theorem Proofs.LinFoldB.lossCotB_slice {m n : ℕ} (N : ℕ) (W : Mat m n) (b : Vec n) (xB : Vec (N * m)) (label : Fin N → Fin n) (k : Fin N) (j : Fin n) :
        StableHLO.den (lossCotGraphB N W b xB label) (finProdFinEquiv (k, j)) = softmax n (mnistLinear W b (StableHLO.batchSlice N m xB k)) j - oneHot n (label k) j

        Example k's slice of the batched cotangent is the per-example softmax − oneHot.

        theorem Proofs.LinFoldB.batchSlice_lossCotB {m n : ℕ} (N : ℕ) (W : Mat m n) (b : Vec n) (xB : Vec (N * m)) (label : Fin N → Fin n) (k : Fin N) :
        StableHLO.batchSlice N n (StableHLO.den (lossCotGraphB N W b xB label)) k = fun (j : Fin n) => softmax n (mnistLinear W b (StableHLO.batchSlice N m xB k)) j - oneHot n (label k) j

        The slice form of lossCotB_slice.

        theorem Proofs.LinFoldB.weightSgdB_tied {m n : ℕ} (N : ℕ) (W : Mat m n) (b : Vec n) (xB : Vec (N * m)) (label : Fin N → Fin n) (lrStr : String) (lr : ℝ) (i : Fin m) (j : Fin n) :
        StableHLO.den (StableHLO.SHlo.denseWeightSgdB "%x" "%W0" lrStr xB W lr (lossCotGraphB N W b xB label)) (finProdFinEquiv (i, j)) = W i j - lr * ∑ k : Fin N, pdiv (fun (v : Vec (m * n)) (x : Fin 1) => crossEntropy n (dense (Mat.unflatten v) b (StableHLO.batchSlice N m xB k)) (label k)) W.flatten (finProdFinEquiv (i, j)) 0

        Weight output at batch N. The emitted denseWeightSgdB node over the batched cotangent denotes W − lr·Σₖ ∂(crossEntropy ∘ dense)/∂W at example k — the sum over the batch of the per-example loss gradient LinearFold proves at one example.

        theorem Proofs.LinFoldB.biasSgdB_tied {m n : ℕ} (N : ℕ) (W : Mat m n) (b : Vec n) (xB : Vec (N * m)) (label : Fin N → Fin n) (lrStr : String) (lr : ℝ) (j : Fin n) :
        StableHLO.den (StableHLO.SHlo.denseBiasSgdB "%b0" lrStr b lr (lossCotGraphB N W b xB label)) j = b j - lr * ∑ k : Fin N, ∑ i : Fin n, pdiv (fun (b' : Vec n) => dense W b' (StableHLO.batchSlice N m xB k)) b j i * (softmax n (mnistLinear W b (StableHLO.batchSlice N m xB k)) i - oneHot n (label k) i)

        Bias output at batch N. The emitted denseBiasSgdB node denotes b − lr·Σₖ (certified ∂dense/∂b · (softmax(zₖ) − onehotₖ)).

        theorem Proofs.LinFoldB.lin_train_step_tiedB {m n : ℕ} (N : ℕ) (W : Mat m n) (b : Vec n) (xB : Vec (N * m)) (label : Fin N → Fin n) (lrStr : String) (lr : ℝ) :
        (∀ (i : Fin m) (j : Fin n), StableHLO.den (StableHLO.SHlo.denseWeightSgdB "%x" "%W0" lrStr xB W lr (lossCotGraphB N W b xB label)) (finProdFinEquiv (i, j)) = W i j - lr * ∑ k : Fin N, pdiv (fun (v : Vec (m * n)) (x : Fin 1) => crossEntropy n (dense (Mat.unflatten v) b (StableHLO.batchSlice N m xB k)) (label k)) W.flatten (finProdFinEquiv (i, j)) 0) ∧ ∀ (j : Fin n), StableHLO.den (StableHLO.SHlo.denseBiasSgdB "%b0" lrStr b lr (lossCotGraphB N W b xB label)) j = b j - lr * ∑ k : Fin N, ∑ i : Fin n, pdiv (fun (b' : Vec n) => dense W b' (StableHLO.batchSlice N m xB k)) b j i * (softmax n (mnistLinear W b (StableHLO.batchSlice N m xB k)) i - oneHot n (label k) i)

        The linear train step at batch N, tied. Both outputs of linear_train_step.mlir, read as the batched graph it is the print of, denote the certified SGD step on the sum of the N examples' losses: the weight against the loss gradient, the bias against the certified bias Jacobian contracted with softmax − onehot. N is a binder; the artifact is N = 128.

        theorem Proofs.LinFoldB.lin_weight_meanlossB {m n : ℕ} (N : ℕ) (W : Mat m n) (b : Vec n) (xB : Vec (N * m)) (label : Fin N → Fin n) (lrStr : String) (α : ℝ) (hN : 0 < N) (i : Fin m) (j : Fin n) :
        StableHLO.den (StableHLO.SHlo.denseWeightSgdB "%x" "%W0" lrStr xB W (α / ↑N) (lossCotGraphB N W b xB label)) (finProdFinEquiv (i, j)) = W i j - α * pdiv (fun (v : Vec (m * n)) (x : Fin 1) => 1 / ↑N * ∑ k : Fin N, crossEntropy n (dense (Mat.unflatten v) b (StableHLO.batchSlice N m xB k)) (label k)) W.flatten (finProdFinEquiv (i, j)) 0

        The mean-loss reading. At lr = α / N (the artifact's 0.00078125 = 0.1 / 128) the weight output is one gradient-descent step at rate α on the batch-MEAN cross-entropy.