ConvNeXt at the BATCHED index N := B — forward, backward and AdamW renders #
The ConvNeXt peer of ResNet34RenderB / MobileNetV2RenderB, and the reason it exists is
stochastic depth: the drop mask is per-EXAMPLE, and in the per-example-indexed render
(ConvNeXtRender) a node denotes ONE example — pretty B lifts it across the batch, so the node
cannot see j. dropPathB needs its operand at index B·n, which is what this file's chain
produces.
The trap this closes is that the wrong thing TYPECHECKS. pretty B already emits
tensor<B×n>, so a broadcast_in_dim %mask, dims = [0] + multiply against a per-example node
compiles, trains and descends — with no faithful den behind it. Every node below is instead a
batchOp/*B form whose den is batchMap N (…) or batchMapAux N (…), i.e. honest about which
index is the batch.
What this file writes — every ConvNeXt artifact but one. The forwards (drop-free and
stochastic-depth), and the AdamW/EMA train steps through ConvNeXtRender.convNextAdamTrainStepText
with this file's traversal, at ConvNeXt-T/S/B, Imagenette and ImageNet, f32 and bf16. Only the
SGD-inline convnext_train_step.mlir is written by ConvNeXtRender: this traversal has no
fused-SGD arm, and ConvNeXtStepTie's 182-parameter tie is stated at those bytes. The Proofs tier
for this chain is Nets/ConvNeXt/ConvNeXtFoldGB.lean.
The gate (lake build convnext-fwd-b-tie): the per-example chain and this one must emit
byte-identical forwards, and train steps that differ on the conv-VJP transpose/reverse pair
and nothing else (commuting ops on disjoint axes). That is available because every batched form
was built to emit its per-example peer's text byte-for-byte
(tests/TestBatchedEmitTie.lean checks each form). So the whole-net statement is the
per-form statement composed — and if it ever fails, the tie file localises which form did it in
one run. The committed bytes are this chain's, so the gate renders the per-example chain and
compares it against them.
The batched shape helpers #
reassoc/unassoc in the per-example renderer cast SHlo (c*h*h) ↔ SHlo (c*(h*h)). At the
batched index the same cast has to happen UNDER N * ·, which is bnBatchLA's existing move
(congrArg (N * ·) (Nat.mul_assoc …)). It is a reindex, not a function change — the emitted text
is unaffected, since skel never sees the index.
ConvNeXt-T / ImageNet's per-replica batch, spelled ONCE for the whole convnextin_*
family.
64, not the 32 every other ConvNeXt render uses, and the reason is a PAIRING fact rather
than a throughput one. This net's JAX reference trains at global 256 = 4 × 64 (its own
banner: batch 256 (4x64) · SPE 5004). At 4 × 32 the verified job would run half the batch,
twice the updates, and an LR off the linear-scaling rule: a RECIPE difference, not a lowering
difference, and it would forfeit the only thing this pair is in the book for: ConvNeXt has no
BatchNorm, so a tie here says the LOWERER is clean and an offset here implicates the lowerer
or the feed fleet-wide. Neither reading survives if the batch does not match.
Spelled once because it has to reach every convnextin_* #eval AND both halves of every
train step (body at N := bB, wrapper via cBS).
ConvNeXt-S's and -B's EMA pair renders and eval forwards are at this batch too; their adam*
train steps stay at the 32 default. A job's LEAN_MLIR_BATCH must match its render —
the PRECHECK in scripts/jobs/cnx-default-4gpu.conf greps the artifact for its own baked
batch and is the template for that.
Equations
Instances For
The sites, one per per-example site in ConvNeXtRender.lean #
The full ConvNeXt-T [3,3,9,3] forward at the batched index. Node for node the same chain
convNextFwdChain emits — 4×4/s4 patchify stem (3→96, 224→56) → stem channel-LN → 4 stages at
56/28/14/7 with 2×2/s2 downsamples between → GAP(7×7) → dense(768→nClasses).
Every node is a batchOp/*B form, so den is a batchMap/batchMapAux at N := bB and
the batch is an index of the AST rather than a number only pretty knows. That is the entire
content of the move; the emitted text is unchanged, which the tie checks.
sd := true adds the 18 stochastic-depth sites, one per block, at ramp index cnxBlockIdx si j
— and the sites are emitted in the FORWARD as well as the train step deliberately: at eval the driver supplies an all-ones mask, so they are the exact
identity, and the forward ⊂ train-step prefix audit keeps a partner for the SD render instead
of quietly not covering it.
Equations
- One or more equations did not get rendered due to their size.
Instances For
@convnext_fwd_b — the batched-index peer of convNextFwdText, same signature
(182 parameters at ConvNeXt-T) and same %x. This WRITES convnext_fwd, convnextin_fwd,
convnextsin_fwd and convnextbin_fwd (the #evals at the bottom of this file);
convnext-fwd-b-tie renders the per-example chain against these bytes.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The backward #
Every site below is the per-example site with its node swapped for the batched form. Two of them are where increments 2 and 3's constructors earn their keep, and both are cases the emit tie alone cannot judge:
lnRowBackBtakes the whole-batch saved LN input and hands examplekbatchSlice k x(batchMapAux). A descriptor here would hand example 0's activation to allN— same types, same bytes, different function (den_lnRowBackB_per_example).veclnGammaGradB/rowDenseBiasGradBcontract the batch and the spatial rows. The per-example peers contract only the rows, and the two spellings agree atN = 1(den_rowDenseBiasGradB_at_one) — so a render that dropped the batch sum would pass a one-example check.
The whole-net batched traversal — forward + cotangent + every parameter gradient, the
batched peer of convNextBackAll true (some …). Returns (code, gradMap, softmaxSSA) with the
same shape, so the AdamW tail in ConvNeXtRender.lean can consume either.
AdamW only, deliberately. The per-example traversal serves both renders off one adam
flag; this one does not, because the SGD path bakes lr as a literal where AdamW takes it as a
runtime %lr operand, and an SGD render at the batched index is not something any config asks
for yet. Adding the flag later is cheap; an artifact nobody loads is a silent-hyperparameter
hazard.
The %dgi/%dgb/%dgn/%dgd/%dgapf GAP-backward block is hand-written text on both
sides, carried over verbatim. It is one of the declared non-AST carve-outs, so the batched
move neither improves nor degrades it — but note it is parameterised by bB and therefore
already batch-correct, which is why it needs no peer.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The ConvNeXt-T AdamW train step at the batched index. It is the SAME renderer the
per-example path uses — convNextAdamTrainStepText with traversal pointed at
convNextBackAllB — not a copy.
That is possible because the AdamW tail is entirely parameter-space: adamMNextF,
adamVNextF, adamWParamF, gradSumSqAccF and clipScaleF are indexed by the parameter's
own size and never see the batch. So "the AdamW tail at the batched index" is no
work at all — the batch is factored out of it by the ops' own shapes. The only
thing that moves is which traversal produced the gradients.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The drop-free forward's banner — the line ConvNeXtRender.convNextFwdText emits, restated
here so the byte tie can demand byte-identity rather than "identical apart from a comment".
Worth the parameter: a tie that compares modulo one line is a tie with a hole in it, and the
hole is exactly where a renderer's own description of what it did would live. This is the banner
the committed convnext_fwd / convnextin_fwd / convnextsin_fwd / convnextbin_fwd carry,
which is why it takes the size: the model name is derived from the stage table, exactly as the
per-example line derives it.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The SD forward's banner. Its own, and not cnxFwdBanner, because these bytes ARE a
different render and a banner claiming otherwise would misdescribe the artifact it heads.
Equations
- One or more equations did not get rendered due to their size.