The MNIST-linear train step at batch N — the fold LinearFold leaves open #
LinearFold certifies linear_train_step.mlir's two outputs for ONE example: its nodes
(weightSgd/biasSgd over lossCotGraph) denote per-example values, and the file says so
("the committed module batch-contracts over B examples, which this file does not state").
This file states it. The same text is the print of a graph at the batched index — batchOp
lifts of dense/expe/softmaxDiv, subB, and the denseWeightSgdB/denseBiasSgdB nodes
whose den sums the per-example gradient over batchSlice n — and
tests/TestSmallNetsBatchedEmit.lean pins that print byte-for-byte to linTrainStepText at the
committed shape. So:
lin_train_step_tiedB— at a batch ofNexamples with labelslabel : Fin N → Fin n, the weight output denotesW − lr·Σₖ ∂CE(dense(W,b)·xₖ, labelₖ)/∂Wand the bias outputb − lr·Σₖ (certified ∂dense/∂b · (softmax(zₖ) − onehotₖ)), for everyN.lin_weight_meanlossB— atlr = α/Nthe weight output is one step of gradient descent at rateαon the batch-MEAN loss, which is the step the trainer takes (0.00078125 = 0.1/128).
Scope #
- per-op
den⇄ MLIR text, as inLinearFold; the committed-bytes tie is CI's regenerate-and-diff plus the batched-emit pin; ℝ → Float32 isFloatBridge.
Two node lemmas every batched dense fold uses #
denseWeightSgdB/denseBiasSgdB denote θ − lr·Σₖ (per-example gradient at slice k) by
definition; these read that off at an index. The two collapse lemmas turn the certified dense
Jacobian contracted with a cotangent into the closed form the node computes.
Slice lemmas for the batched backward-chain nodes #
A selectPosB mask reads its own example's saved pre-activation.
An operand leaf's slice is the slice of the value it carries.
A dotOut lift's slice is W · (slice of the input).
The one-hot batch %onehot carries: row k is oneHot n (label k).
Equations
- Proofs.LinFoldB.oneHotB N label idx = Proofs.oneHot n (label (finProdFinEquiv.symm idx).1) (finProdFinEquiv.symm idx).2
Instances For
The batched forward: dense W b lifted across the batch (%x : [N, m]).
Equations
- Proofs.LinFoldB.fwdGraphB N W b xB = Proofs.StableHLO.SHlo.batchOp (Proofs.StableHLO.BatchableOp.dense "%W0" "%b0" W b) (Proofs.StableHLO.SHlo.operand "%x" xB)
Instances For
The batched loss cotangent softmax(logits) − onehot, every op a batched node.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Example k's slice of the batched cotangent is the per-example softmax − oneHot.
The slice form of lossCotB_slice.
Weight output at batch N. The emitted denseWeightSgdB node over the batched cotangent
denotes W − lr·Σₖ ∂(crossEntropy ∘ dense)/∂W at example k — the sum over the batch of the
per-example loss gradient LinearFold proves at one example.
Bias output at batch N. The emitted denseBiasSgdB node denotes
b − lr·Σₖ (certified ∂dense/∂b · (softmax(zₖ) − onehotₖ)).
The linear train step at batch N, tied. Both outputs of linear_train_step.mlir, read as
the batched graph it is the print of, denote the certified SGD step on the sum of the N
examples' losses: the weight against the loss gradient, the bias against the certified bias
Jacobian contracted with softmax − onehot. N is a binder; the artifact is N = 128.
The mean-loss reading. At lr = α / N (the artifact's 0.00078125 = 0.1 / 128) the weight
output is one gradient-descent step at rate α on the batch-MEAN cross-entropy.