ConvNeXt-T train step rendered ENTIRELY from the verified AST (the §1 render) #
⭐ Since 4c leg 3 (2026-09-07) this file writes ONE artifact: the SGD-inline
verified_mlir/convnext_train_step.mlir. The thirteen AdamW/EMA train steps and the four
drop-free forwards render from the batched chain in ConvNeXtRenderB.lean
(planning/archive/renderer_convergence.md, leg 3), whose Proofs tier is ConvNeXtFoldGB.lean.
This chain stays for two reasons: the batched traversal has no fused-SGD arm, and
ConvNeXtStepTie.lean's 182-parameter tie is stated at exactly these bytes.
tests/TestConvNeXtFwdBTie.lean pins the two chains against each other — identical forwards,
backwards differing on the conv-VJP transpose/reverse pair (78 lines) and nothing else.
The ConvNeXt peer of EfficientNetRender: the FULL [3,3,9,3] ConvNeXt-T train
step (BS=32, 3×224²→10) rendered as pretty of verified SHlo nodes — forward, backward-cotangent
chain, AND the param-SGD tail (the new ConvNeXtFold ops + the existing conv/depthwise/dense
ops). Adapted from the committed emitter tests/TestConvNeXtTTrainPC.lean: its forward + backward
cotangent chain were already pretty(SHlo); here the hand-written param-GRAD strings are replaced by
the SHlo param-SGD ops, which BUNDLE the gradient + SGD wrap into one op (producing the updated param).
The two weight-gradient residuals are CLOSED (2026-07-28); all 180 params are now SHlo ops. They were never the same kind of gap:
- the stem 4×4/s4 weight (
psW) needed a genuinely missing cert.flatConvStride4(forward) andflatConvStride4_has_vjp(input) already existed;flatConvStride4_weight_grad_has_vjpis new — twovjp_compsteps over the stride-1 weight-VJP and the two decimations, mirroring the stride-2 sibling. It backs the new.convStride4WeightGradop. - the 2×2/s2 downsample (
d{i}W) needed NO new cert.flatConvStride2_weight_grad_has_vjpis kernel-generic and.convStridedWeightGradalready existed; the blocker was purely emit-side —(kH−1)/2symmetric SAME padding floors to 0 atkH = 2and emitted a 1×1 convolution against a declared 2×2 result, i.e. type-invalid MLIR.StableHLO.sWGradGeomsplits odd/even (odd byte-for-byte unchanged) and the site is now certified.
(A note here used to claim the scalar-LN γ/β params render as tensor<1xf32> against a
committed tensor<f32> signature. Checked 2026-07-29 and retired: neither is true of any committed
artifact. ty [] = "tensor<f32>", and grep -c 'tensor<1xf32>' is 0 in both
convnext_train_step.mlir and convnext_adam_train_step.mlir — the scalar params are tensor<f32>
on both sides. Handoff §0b repeats the stale claim.)
Every other param (depthwise-7×7 W/b, 1×1 expand/project W/b, per-channel layer-scale γ,
channel-LN γ/β, downsample 2×2 W/b, dense W/b) denotes the certified loss-descent step
(ConvNeXtFold + M2/M3). Render is value-independent (skel erases values), so placeholders + lr:=0
are passed; the emitted lrStr/epsStr literals carry the real values.
The ConvNeXt size — depths and channel dims TOGETHER, in one record.
⚠⚠ They are bundled deliberately, and this is the lesson ConvNeXt-B taught that ConvNeXt-S
did not. S is pure depth, so it was served by a bare Array Nat of depths — and that shape
admits (depths := S, dims := T), which is not a ConvNeXt of any size but type-checks, renders
and trains. B moves BOTH, so a second bare array would have made the mismatch reachable in the
direction that matters. One record, three instances, and a caller cannot spell a net that does
not exist. Same reasoning as VitDims, arrived at from the opposite direction: ViT bundled to
keep d = heads * hd definitional, this bundles to keep two tables from drifting.
⚠ deriving DecidableEq is load-bearing: ConvNeXtRenderB restates these and #guards the
restatement against them, and cnxModelName matches on the whole record rather than on the
block count — which is what stops B (36 blocks, like S) from introducing itself as an S.
Instances For
Equations
Instances For
Equations
- One or more equations did not get rendered due to their size.
Instances For
Equations
Equations
ConvNeXt-T: [3,3,9,3] @ [96,192,384,768], 28.6 M at K=1000. The DEFAULT everywhere, so
every pre-existing call site is unchanged and every committed artifact re-renders byte-identical.
Instances For
ConvNeXt-B: S's depth AND a wider net — [128,256,512,1024]. 88.6 M at K=1000.
⚠ B is the size that made the dims a parameter: it shares S's depth table exactly, so anything
keying on block count alone cannot tell them apart.
Instances For
Total ConvNeXt blocks = the reference's totalDrop, i.e. the ramp DENOMINATOR is this
minus 1. Derived from the stage table the render traverses, not restated beside it — which is
what makes it follow D to 36 at ConvNeXt-S without a second edit.
Equations
- Proofs.StableHLO.cnxDropTotal V = Array.foldl (fun (x1 x2 : ℕ) => x1 + x2) 0 V.depths
Instances For
The ramp index of stage si's block j — the reference's dbi, which counts blocks over
the WHOLE net. The single source for the drop-site numbering: the forward calls it walking the
stages forwards, the backward walking them backwards, and they cannot disagree.
Equations
- Proofs.StableHLO.cnxBlockIdx si j V = List.foldl (fun (x1 x2 : ℕ) => x1 + x2) 0 (List.map (fun (k : ℕ) => V.depths[k]!) (List.range si)) + j
Instances For
The number of per-example drop-path scale inputs an SD ConvNeXt render takes: one per block.
Equations
Instances For
The model name a render's banner claims, DERIVED from the stage table it was rendered at.
⚠ A banner is a render's own description of what it did, and the one thing worse than not
having one is having one that lies — cnxDropFwdBanner's own note says exactly that about
reusing a drop-free banner on an SD artifact. Passing the name as a second parameter beside D
would be two writers for one fact, the shape (§2k's α/K, the wx variant marker) that has
shipped a real defect in this net three times. So it is read off D.
⚠⚠ It matches on the WHOLE RECORD, not on the block count, and that is not fussiness.
This function keyed on cnxDropTotal while ConvNeXt-S was the only new size, and B broke it:
B is [3,3,27,3] too, so 36 blocks names S and B alike and every B artifact would have opened
by calling itself a ConvNeXt-S. Caught by the guards below, which is why they enumerate all
three rather than spot-checking one. An unrecognised table gets a name that SAYS it is
unrecognised, rather than falling back to "ConvNeXt-T" the way a getD would.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The %dp<i>: tensor<Bxf32> inputs an SD render appends to its signature — one per block, in
ramp-index order, which is the order the driver's dropScales writes them into the blob.
Empty when off, which is what keeps every committed artifact byte-identical.
Equations
- One or more equations did not get rendered due to their size.
Instances For
What was wrong. Every LN site here normalises with .bnF ⇒ bnForward n ε γ β, which takes
ONE mean and ONE variance over the whole c·h·w map per example and applies a scalar γ/β.
ConvNeXt's channel_layer_norm takes h·w statistics per example, each over the c channels at
one spatial position, with a per-channel [c] affine — a different function, on 21 of the 22
sites. (The 22nd, the head, runs after GAP where there is no spatial extent left, so its axis is
already right; only its affine is wrong.) §2m first recorded the axis as correct by matching the
literal across dimensions = [1] against the reference's axis=1, but the artifact's tensor is
rank-2 [B, c·h·w] and the reference's is rank-4 NCHW — §4's one-tensor-layout rule.
Route A: no new SHlo op. ConvNeXt's channel-LN IS ViT's row-LN under a transpose — view an
example as [c, s] with s = h·w, transpose to [s, c], and each row is one spatial position
holding its c channels, which is exactly what rowLNFlat normalises. Settled on device before
any of this was written (lake build channel-ln): forward and all three backward pieces tie the
closed form at rel 0, the incumbent .bnF control fires at rel 0.82, and the transposes measure
free (Δ 0.00 ms on 16.1 ms of whole-net LN — XLA folds a transpose into the consumer's layout).
⚠ Nat multiplication is not definitionally associative, and the ambient index here is
c*h*h = (c*h)*h while the transpose needs c*(h*h). reassoc/unassoc transport along
Nat.mul_assoc; they are casts on the index, not on the value, so pretty walks the same tree.
The %one/%zero constants the channel-LN chain binds lnRowF/lnRowBack's SCALAR γ/β to —
the real per-channel affine is rowScaleF/rowBiasF downstream, exactly as ViT does it.
Emitted once per module body.
⚠ This is the enet zeroBiasPrelude defect one net over (§2m): wire the operand and forget the
prelude, and the artifact uses an SSA name nothing defines — iree-compile/XLA say "use of
undeclared SSA value name" and nothing before them says anything. It happened HERE too, on the
first flag-on render, and regen_verified_mlir.sh check's prelude audit is what generalises.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Equations
Every leaf below is one of the *Sgd/*Grad pairs whose dens differ by exactly θ − lr · ·
(*Sgd_eq_grad, all rfl) and whose emits differ by exactly the const-lr/multiply/subtract tail
(tests/TestBatchedEmitTie.lean, byte-PREFIX). So one traversal serves both renders.
ConvNeXt is the cheapest of the five nets to thread, because its param tails were already
factored out of the cotangent traversal — bwdBlock computes cotangents and nothing else, and is
untouched by this. lrStr is threaded but unused in adam mode: AdamW's learning rate is the
runtime %lr argument, not a baked literal.
Shapes are List Nat rather than rendered tensor<…> strings because the AdamW render needs
both forms: %{nm}/%{nm}m/%{nm}v for the moment slots and the raw dimensions for the emitted
Adam ops. The scalar-LN γ/β are rank 0, i.e. [] — and ty [] = "tensor<f32>", exactly the
string that used to be hardcoded, so the SGD render's emitted text is unchanged.
Does this parameter get weight decay? ConvNeXt's half of the timm rule.
⚠ It is the PLAIN RANK TEST, with no name carve-out, and that is the difference from ViT.
The reference's _wd_mask also excludes anything matching _WD_POS_SHAPE, but ConvNeXt has no
patch-embedding positional parameter, so its generated reference sets _WD_POS_SHAPE = None
and that branch can never fire. Checked in the generated file rather than assumed — carrying
ViT's nm != "pos" over would have been a transcription of a rule this net does not have.
What that leaves excluded here: every LN γ/β ([c]), every conv bias, and LayerScale γ
(lg, [c]) — which is 1-D and therefore excluded for the same structural reason, not as a
special case.
Equations
- Proofs.StableHLO.cnxWdDecays _nm ds = decide (ds.length ≥ 2)
Instances For
allParams is private (it is this file's internal signature source); this is the one thing
outside it that legitimately needs the list — tests/TestWdExcludeTie.lean, which must read
the SAME names and shapes the renderer chose %wd/%wdz from. Exposing an alias rather than
dropping private keeps the surface one definition wide.
Equations
- Proofs.StableHLO.cnxAllParams nClasses V = Proofs.StableHLO.allParams✝ nClasses V
Instances For
Every SSA name the ConvNeXt-T forward produces. convNextFwdFaithfulV returns just logits;
the train steps additionally consume the block records, the two downsample names and the
head's gap/hn on the way back.
Instances For
Equations
@convnext_fwd rendered ENTIRELY from the verified AST — the peer of the train-step
render, sharing its forward chain and its 180-parameter signature. Takes %x plus the 180
params in allParams (= func-arg) order (181 inputs) and returns logits [32, 10].
This replaces the independent hand-written string emitter in tests/TestConvNeXtFwd.lean: the
forward the driver evals is now the same graph the train step differentiates, by construction
rather than by inspection. Because it shares the chain, the emitted body is a byte-identical
PREFIX of convnext_train_step.mlir's, ending exactly where the loss begins — which is what
scripts/regen_verified_mlir.sh check audits.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The whole-net forward + cotangent + backward traversal, SHARED by the SGD and AdamW renders.
Returns (code, params, softmax): every emitted line; one SSA per parameter — the updated
param at adam := false, the un-fused gradient at adam := true; and the softmax SSA,
which the AdamW render's report-only %loss reads.
smooth is the cotangent recipe. none → the SGD render's (softmax − onehot)/B, unchanged
down to the hand-written %dy divide. some (α, −α/K, B) → the label-smoothed
((softmax − onehot) + α·onehot − α/K)/B composed from kit ops, which is what the AdamW recipe
trains on. ConvNeXt already spells the ÷B explicitly (unlike ViT/R34, which fold the mean into
lr), so here the smoothing is the only difference between the two cotangents.
Gate on the refactor: convnext_train_step.mlir must come back byte-identical. It does — the
softmax is now prettyd on its own line instead of nested inside the .sub so that %loss can
read it, but .operand is a leaf that emits nothing, so the fresh-name sequence is unchanged.
Equations
- One or more equations did not get rendered due to their size.
Instances For
ConvNeXt-T (full [3,3,9,3]) SGD train step rendered from the verified AST (except the two
documented weight-grad gaps — the stem 4×4/s4 patchify and the even-kernel 2×2/s2 downsample,
neither of which has a VJP-cert SHlo op). Every other line is pretty of a verified node:
forward + backward cotangent chain + the param-SGD ops, whose output IS the updated param.
The cotangent is plain CE with an explicit ÷B — unlike ViT/R34, which fold the batch mean
into lr — so the committed cLR = 0.1 is an effective 0.1, the house convention spelled
differently (§2a-quinquies).
Equations
- One or more equations did not get rendered due to their size.
Instances For
The driver's variant slug for a given replica count: the artifact is
verified_mlir/convnext_<variant>_train_step.mlir, the entry point is
@convnext_<variant>_train_step, and LEAN_MLIR_VARIANT selects it. All three must agree —
the shim checks the entry name and refuses a mismatch outright ("entry mismatch") rather than
running the wrong graph. The #guards at the bottom pin the literal #eval paths against this.
ConvNeXt has only one batch (32), so unlike mnv2AdamVariant/r34AdamVariant there is no
batch suffix — rendering another batch would need cBS to stop being a private constant.
⚠ The EMA renders get their OWN slugs (ema/emadp), for the reason the RMSProp ones do: a
render carrying a fourth [θ|m|v|ema] region must never be able to overwrite the artifact the
AdamW trainer runs, whose blob has three. That is §2a's last-writer-wins race, and here it would
also be an arity mismatch the driver could not survive.
Equations
- One or more equations did not get rendered due to their size.
Instances For
ConvNeXt-T AdamW train step rendered from the verified AST. The certified peer of the
hand-written render in tests/TestConvNeXtTrain.lean that convnext-verified-adam trains on.
Same backward as convnext_train_step (convNextBackAll, one traversal) but taking the
un-fused gradients, each fed to the proven AdamW triple. The cotangent adds label smoothing
(α = 0.1, K = 10); ConvNeXt already spelled the ÷B explicitly, so that is the only difference.
Interface: 545 in (%x, 180 θ, 180 m, 180 v, %lr/%bc1/%bc2, %onehot) / 543 out
(180 θ', 180 m', 180 v', %loss/%bc1/%bc2) — positionally identical to the hand-written
render, so trainAdamSched's packed [θ|m|v] protocol is unchanged.
What this certifies. As of 2026-07-28 all 180 params are pretty(AST) end to end —
the two weight-grad gaps this render used to carry (the stem 4×4/s4 patchify and the even-kernel
2×2/s2 downsample) are closed, by a new cert (flatConvStride4_weight_grad_has_vjp) and an
emit-side odd/even padding split (StableHLO.sWGradGeom) respectively. Licensed by
convnext-adam-tie against the previously committed hand-written render: bit-exact on all
83,434,629 returned floats, spread 0/180, against a bit-exact A-vs-A floor.
Still outside the AST here, and unchanged: %loss (report-only, no gradient path).
replicas > 1 renders the DATA-PARALLEL variant (handoff §2h-quater) to its own entry name
and artifact path via cnxAdamVariant, so producing it can never clobber the one the trainer
runs. The only difference is one all_reduce(add)/N per parameter gradient, between the
certified gradient and the certified AdamW triple: certified gradient → trusted collective →
certified AdamW. See convnextAdamOne for the carve-out.
Equations
- One or more equations did not get rendered due to their size.