T3 §1 fold for ConvNeXt-T at the BATCHED index — the op set every ImageNet artifact renders #
ConvNeXtFoldG.lean folds the fourteen gradient nodes of the PER-EXAMPLE traversal
(ConvNeXtRender.convNextBackAll at adam := true). This is its batched peer, at
ConvNeXtRenderB.convNextBackAllB's constructors — and unlike ViT's (ViTFoldGB.lean), it
was owed BEFORE any renderer swap: every convnextin_* train step, every *drop* variant and the
ConvNeXt-S/B artifacts have rendered from the batched chain since they existed, so the artifact
behind this net's quoted ImageNet accuracy (convnextin_adamdpwxclipdrop) had a fold at the
per-example constructors that no committed byte of it is pretty of. 4b's "one lemma per op kind
certifies every optimizer tail" was, for ConvNeXt, a statement about the Imagenette pair only.
⭐ The bytes are the same on the forward and differ on 78 backward lines. The batched
convBackBatched emits the conv input-VJP's transpose/reverse in the other order from the
per-example convBack — commuting ops on disjoint axes, one kernel — and tests/TestConvNeXtFwdBTie.lean
allows exactly that pair and nothing else. 4c leg 3 moves the drop-free writers onto this chain;
this file lands first, per leg 1's ordering rule, so that no committed artifact is ever pretty of
an AST without a fold.
The op table of convnext_adam_train_step.mlir and every convnextin_* train step #
| emitted node | lemma | per-example peer it batches |
|---|---|---|
layerScaleChGammaGradB (18 block γ) | layerScaleChGammaGradB_den | CnxPoCG.layerScaleChGammaGrad_den |
convWeightGradB / convBiasGradB (18 expand + 18 project 1×1, + the stem bias) | convWGradB_den / convBGradB_den | ResNet34PoCB's, verbatim |
depthwiseWeightGradB / depthwiseBiasGradB (18 × 7×7) | depthwiseWGradB_den / depthwiseBGradB_den | EnetPoCG / Mnv2PaperPoCG |
convStridedWeightGradB / convStridedBiasGradB (3 × 2×2/s2 downsample) | convStridedWGradB_den / convStridedBGradB_den | ResNet34PoCB's, verbatim |
convStride4WeightGradB (patchify stem) | psWGradB_den | flatConvStride4_weight_grad_has_vjp, per example |
veclnGammaGradB / rowDenseBiasGradB at R = h·w (22 spatial LN sites) | chanLnGammaGradB_den / chanLnBetaGradB_den | CnxPoCG.chanLnGammaGrad_den / chanLnBetaGrad_den |
veclnGammaGradB / rowDenseBiasGradB at R = 1 (the head LN, after GAP) | headLnGammaGradB_den / headLnBetaGradB_den | ViTPoCGB's two-level LN lemmas |
weightGradB / biasGradB (the classifier) | headWGradB_den / headBGradB_den | ViTPoCGB.headWGradB_den / headBGradB_den |
convWeightGradBBf16 / depthwiseWeightGradBBf16 / convStridedWeightGradBBf16 / convStride4WeightGradBBf16 (the bf16 artifacts) | Bf16PoC.convWGradBBf16_den and its siblings, Foundation/Bf16GradNodes.lean | none — a bf16 node is its own op kind |
⭐ No new mathematics. Every proof is Finset.sum_congr rfl over the batch and then the
per-example bridge at batchSlice n — ResNet34FoldB.denseWGradB_den's shape — because
each batched den arm is literally the per-example one under a batch sum. The channel-LN sites
add one step: the batched render hands the LN ops batchMap N (chanLNRows c h w) of the saved
input and of the cotangent (the [h·w, c] transposed views, lifted per example), and
batchSlice_batchMap peels the lift so ConvNeXtChannelLN's permutation argument applies at each
slice.
⭐ The bf16 artifacts (convnextin_adamwxclipdropbf16, the S/B twins) emit *GradBBf16
constructors, not these nodes: their den rounds the operands and the result once, outside the
batch sum. Those are their own op kinds, folded once for every net in
Foundation/Bf16GradNodes.lean (first stated in this file, 2026-09-07).
⭐ One lemma per op kind certifies every optimizer tail at once — AdamW, the wx/clip
variants, the EMA shadow, drop-path and the data-parallel twins all consume the same *GradB
node, and convnextin_adamdpwxclipdrop, whose accuracy the book quotes, is one of them.
Honest residual #
- ⚠
biasGradBis the IDENTITY on its operand and the classifier bias's batch reduce is in the emitted text, outside the AST — soheadBGradB_denis stated PER EXAMPLE atbatchSlice n, the per-examplebiasGradcarve-out carried over unchanged (as inViTFoldGB). - Every lemma is
∀ cot. Pinning each to the emitted backward subgraph is the §1a tie; ConvNeXt's capstone (ConvNeXtStepTie.lean, 182 params) is at the per-example SGD-inlineconvnext_train_step.mlir, which stays on the per-example chain (the batched traversal has no fused-SGD arm). Re-pointing it at these nodes withSmoothedLossCotis 4b's ConvNeXt capstone, which this file is the prerequisite for. convnextin_adamdp*is four replicas: the all-reduce is emitted text outside the AST, so these lemmas are about the per-replica gradient node (4d).- ⛔ SYMMETRIC padding at the three 2×2/s2 downsamples and the 4×4/s4 stem (
flatConvStride2,flatConvStride4); ConvNeXt is PyTorch-origin and has no XLA-SAMEsite.
Batched per-channel layer-scale γ GRADIENT denotes the certified Σ_n gradient. The
emitted reduce contracts batch and spatial in one op; den reads it as the batch sum of the
per-example dγ_c = Σ_{k : chanIdx k = c} x_k·dy_k. All 18 blocks.
Batched conv weight GRADIENT denotes the certified Σ_n weight gradient. Kernel-generic:
every 1×1 expand and project. ResNet34PoCB.convWGradB_den verbatim.
Batched conv bias GRADIENT denotes the certified Σ_n bias gradient. The expand/project
biases and the patchify stem's psb (at 4×4).
Batched depthwise weight GRADIENT denotes the certified Σ_n weight gradient. All 18
blocks; the kernel size is a variable, so 7×7 is an instance. EnetPoCG.depthwiseWGradB_den.
Batched depthwise bias GRADIENT denotes the certified Σ_n bias gradient. ConvNeXt's
depthwises carry a bias (no BatchNorm follows them), which is Mnv2PaperPoCG's lemma.
Batched strided conv weight GRADIENT denotes the certified Σ_n weight gradient. The
three 2×2/s2 downsamples; kernel-generic, ResNet34PoCB.convStridedWGradB_den verbatim.
Batched strided conv bias GRADIENT denotes the certified Σ_n bias gradient.
Batched patchify-stem weight GRADIENT denotes the certified Σ_n weight gradient.
⚠ The emitted convolution contracts the batch axis itself (the transpose trick), so the outer
sum is inside one op rather than across N of them — same as the strided ops.
Batched channel-LN γ GRADIENT denotes the certified Σ_n γ gradient. All 22 spatial sites
(1 stem + 18 block + 3 downsample). Two levels: the outer sum is the batch, the inner the
h·w rows within one example.
Batched channel-LN β GRADIENT denotes the certified Σ_n β gradient. The β gradient is
the plain two-level row reduce, so the render uses the same rowDenseBiasGradB op ViT's LN β
does.
Batched head-LN γ GRADIENT denotes the certified Σ_n γ gradient. ViT's two-level vector
LayerNorm lemma; the head LN runs after GAP, so the render instantiates it at R = 1.
Batched head-LN β GRADIENT denotes the certified Σ_n β gradient.
Batched classifier weight GRADIENT denotes the certified Σ_n outer product.
Batched classifier bias GRADIENT denotes the certified cotangent, PER EXAMPLE. biasGradB
is the identity on its operand — the batch reduce is emitted text outside the AST — so the
statement is at every batchSlice n, the per-example carve-out carried over.
A batched channel-LN γ gradient node, tied (chanLnGammaGradB_den).
Equations
- One or more equations did not get rendered due to their size.
Instances For
A batched channel-LN β gradient node, tied (chanLnBetaGradB_den).
Equations
- One or more equations did not get rendered due to their size.