ConvNeXt-T — every parameter gradient node IS the loss's derivative in that parameter #
cnx_net_tiedGB says each of the 182 parameter gradient nodes denotes its layer's parameter
Jacobian contracted with the cotangent the emitted backward chain threads to it, the chain's top
being the smoothed-loss cotangent. cnx_net_lossGrad composes that with the chain: for any loss
L of the logits whose gradient at the net's output is g, the loss of the WHOLE net cnxNetB
with that one parameter varied is differentiable in it and every node is its gradient
(HasGradAt). cnxNetB is the tie's forward, batched, at its stochastic-depth sites sd; at
none it is batchMap N of the canonical convNextForwardTCh (cnxNetB_eq_convNextForwardTCh). cnx_net_lossGrad_smoothedCE discharges hL for
the label-smoothed loss the artifacts ship (smoothedBatchLossDiv, whose gradient is the
softmaxDiv cotangent the render emits).
How. No ConvNeXt op couples examples, so the work is per example and lifted once:
- Per example (at variable widths): the loss read at each activation inside a block, a
downsample, the stem and the head has the tie's own per-example cotangent as its gradient
(
cnxBlk_hasGradAt,cnxDown_hasGradAt, …). Every stage VJP is global — GELU has no kink and LayerNorm needs only0 < ε— so each step is oneHasGradAt.comp_global, and the channel-LN step ischanLNTensor3Back_eq_chanLN_vjp. - Lifted (
HasGradAt.param_batchMapIdx_through,Foundation.Batched.Indexed, for a block;param_batchMap_throughfor the stem, downsamples and head): read against the linear loss⟨·, dyₙ⟩per example, those gradients turn each tied node'sΣ_n Σ_j ∂per/∂θ · cotₙinto∂G/∂θof the batched block,Gthe loss at the block's output. Examplen's block suffix ends in its own drop site (cnxPostP'ssiteScale), so every block cotangent is the drop-free one ats ⊙ dy. - Per net: the loss read after each stage (
cnxSuf*), pulled back through the certified batched block, downsample and head VJPs (cnxBlockCotInB_eq_vjp,cnxDownCotInB_eq_vjp,cnxHeadDyB_eq_vjp), and eachΦidentified with the whole net at updated weights by a standalonecnx_factor_*theorem.
The stem's bias node is emitted as a stride-1 convBiasGradB at the output resolution, and
reads no input; its Jacobian in the bias is the channel indicator whatever the conv, so it is the
patchify conv's (GradNodeB.pdiv_flatConvStride4_bias_eq_conv2d), as in the tie.
Hypotheses. 0 < ε (the LayerNorms' VJPs); no smoothness hypothesis. For the smoothed loss,
every example's target sums to one and 0 < nC. Stochastic depth is the tie's sd binder,
none the drop-free chain.
Scope. One replica, at either precision: every conv, depthwise, strided-conv and patchify
weight node is stated on the renderers' switch (convWeightGradBAt bf16 id …,
depthwiseWeightGradBAt bf16 id …, convStridedWeightGradBAt bf16 id …,
convStride4WeightGradBAt bf16 id …; StableHLO.PrecisionSwitch), so bf16 := true is the bf16
kind the convnextin_*bf16 artifacts emit, read over ℝ at the identity rounding (Bf16Erasure),
and bf16 := false the f32 artifacts'. The bias, LayerNorm, layer-scale and classifier nodes carry
no flag because no render switches them.
The block after the project conv: layer scale, the drop site, then the identity skip.
Equations
- Proofs.CnxTieGB.cnxPostP h w p s y u i = Proofs.siteScale s (Proofs.layerScale (fun (k : Fin (c * h * w)) => p.sL (Proofs.StableHLO.chanIdx c h w k)) u i) + y i
Instances For
The block after the expand conv (pre-GELU).
Equations
- Proofs.CnxTieGB.cnxPostE gf h w p s y u = Proofs.CnxTieGB.cnxPostP h w p s y (Proofs.flatConv p.pW p.pB (gf.map (cExp * h * w) u))
Instances For
The block after the channel LN.
Equations
- Proofs.CnxTieGB.cnxPostN gf h w p s y u = Proofs.CnxTieGB.cnxPostE gf h w p s y (Proofs.flatConv p.eW p.eB u)
Instances For
The block after the depthwise conv.
Equations
- Proofs.CnxTieGB.cnxPostD gf h w ε p s y u = Proofs.CnxTieGB.cnxPostN gf h w p s y (Proofs.chanLNTensor3 c h w ε p.nG p.nB u)
Instances For
The depthwise conv's output.
Equations
- Proofs.CnxTieGB.cnxActD h w p y = Proofs.depthwiseFlat p.aW p.aB y
Instances For
The channel LN's output.
Equations
- Proofs.CnxTieGB.cnxActNl h w ε p y = Proofs.chanLNTensor3 c h w ε p.nG p.nB (Proofs.CnxTieGB.cnxActD h w p y)
Instances For
The GELU's output (the project conv's input).
Equations
- Proofs.CnxTieGB.cnxActG gf h w ε p y = gf.map (cExp * h * w) (Proofs.flatConv p.eW p.eB (Proofs.CnxTieGB.cnxActNl h w ε p y))
Instances For
The project conv's output (the layer scale's input).
Equations
- Proofs.CnxTieGB.cnxActP gf h w ε p y = Proofs.flatConv p.pW p.pB (Proofs.CnxTieGB.cnxActG gf h w ε p y)
Instances For
A block's cotangents are loss gradients, per example at its drop site s: from the gradient
dy at the block output, the loss read after each activation has the tie's cotangent there —
s ⊙ dy at the layer scale's output, cnxCotP of it at the project conv's, then blkCotE,
blkCotN, blkCotD at s ⊙ dy.
ConvNeXt block, every parameter node a loss derivative — the nine nodes
cnxBlockChTiedGB ties, at the tie's batched activations and cotangents, Φ the loss at the
block's output as a function of the block's weight record.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Downsample, every parameter node a loss derivative — the four nodes cnxDownChTiedGB
ties.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stem, every parameter node a loss derivative — the four nodes cnxStemChTiedGB ties. The
bias node is the emitted stride-1 convBiasGradB, which reads no input (stated at zero).
Equations
- One or more equations did not get rendered due to their size.
Instances For
The head per example: GAP, then LayerNorm at one row, then the dense classifier.
Equations
- Proofs.CnxTieGB.cnxHeadO h w ε hng hnbt Wfc bfc = Proofs.dense Wfc bfc ∘ Proofs.rowLNVecFlat 1 768 ε hng hnbt ∘ Proofs.globalAvgPoolFlat 768 h w
Instances For
Head, every parameter node a loss derivative — the four nodes cnxHeadChTiedGB ties.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Pull the loss gradient back through a batched block's certified VJP at its drop site: the
cotangent is the tie's batchMapAuxIdx N (p.cotInD ε …) (cnxBlockCotInB_eq_vjp).
…through a batched downsample (cnxDownCotInB_eq_vjp).
…and through the batched head (cnxHeadDyB_eq_vjp).
ConvNeXt-T, batched: the tie's forward, stage by stage — batchMap N of the stem,
batchMapIdx N of each block at its site, batchMap N of each downsample and the head. At
sd = none it is batchMap N (convNextForwardTCh (w.toCh ε))
(cnxNetB_eq_convNextForwardTCh).
Equations
- One or more equations did not get rendered due to their size.
Instances For
The stem's output — block b1's input (the tie's ib1).
Equations
- Proofs.CnxTieGB.cnxPreS N ε w = Proofs.StableHLO.batchMap N (Proofs.CnxTie.cnxStemFwdO ε w.sW w.sb w.sγ w.sβ)
Instances For
Stage b1's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage b2's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage b3's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage d0's output.
Equations
- Proofs.CnxTieGB.cnxPreD0 gf N ε w sd = Proofs.StableHLO.batchMap N (w.d0.fwdO ε) ∘ Proofs.CnxTieGB.cnxPreB3 gf N ε w sd
Instances For
Stage b4's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage b5's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage b6's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage d1's output.
Equations
- Proofs.CnxTieGB.cnxPreD1 gf N ε w sd = Proofs.StableHLO.batchMap N (w.d1.fwdO ε) ∘ Proofs.CnxTieGB.cnxPreB6 gf N ε w sd
Instances For
Stage b7's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage b8's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage b9's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage b10's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage b11's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage b12's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage b13's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage b14's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage b15's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage d2's output.
Equations
- Proofs.CnxTieGB.cnxPreD2 gf N ε w sd = Proofs.StableHLO.batchMap N (w.d2.fwdO ε) ∘ Proofs.CnxTieGB.cnxPreB15 gf N ε w sd
Instances For
Stage b16's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage b17's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Stage b18's output.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after block b18 — the head.
Equations
- Proofs.CnxTieGB.cnxSufB18 N ε w = Proofs.StableHLO.batchMap N (Proofs.CnxTieGB.cnxHeadO 7 7 ε w.hG w.hT w.Wfc w.bfc)
Instances For
The net after stage b17: stage b18, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage b16: stage b17, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage d2: stage b16, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage b15: stage d2, then the rest.
Equations
- Proofs.CnxTieGB.cnxSufB15 gf N ε w sd y = Proofs.CnxTieGB.cnxSufD2 gf N ε w sd (Proofs.StableHLO.batchMap N (w.d2.fwdO ε) y)
Instances For
The net after stage b14: stage b15, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage b13: stage b14, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage b12: stage b13, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage b11: stage b12, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage b10: stage b11, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage b9: stage b10, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage b8: stage b9, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage b7: stage b8, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage d1: stage b7, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage b6: stage d1, then the rest.
Equations
- Proofs.CnxTieGB.cnxSufB6 gf N ε w sd y = Proofs.CnxTieGB.cnxSufD1 gf N ε w sd (Proofs.StableHLO.batchMap N (w.d1.fwdO ε) y)
Instances For
The net after stage b5: stage b6, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage b4: stage b5, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage d0: stage b4, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage b3: stage d0, then the rest.
Equations
- Proofs.CnxTieGB.cnxSufB3 gf N ε w sd y = Proofs.CnxTieGB.cnxSufD0 gf N ε w sd (Proofs.StableHLO.batchMap N (w.d0.fwdO ε) y)
Instances For
The net after stage b2: stage b3, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after stage b1: stage b2, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net after the stem: stage b1, then the rest.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net with the stem's parameters varied is the suffix after the stem at the varied stem.
The net with stage b1's weights varied is the suffix after it at the varied stage.
The net with stage b2's weights varied is the suffix after it at the varied stage.
The net with stage b3's weights varied is the suffix after it at the varied stage.
The net with stage d0's weights varied is the suffix after it at the varied stage.
The net with stage b4's weights varied is the suffix after it at the varied stage.
The net with stage b5's weights varied is the suffix after it at the varied stage.
The net with stage b6's weights varied is the suffix after it at the varied stage.
The net with stage d1's weights varied is the suffix after it at the varied stage.
The net with stage b7's weights varied is the suffix after it at the varied stage.
The net with stage b8's weights varied is the suffix after it at the varied stage.
The net with stage b9's weights varied is the suffix after it at the varied stage.
The net with stage b10's weights varied is the suffix after it at the varied stage.
The net with stage b11's weights varied is the suffix after it at the varied stage.
The net with stage b12's weights varied is the suffix after it at the varied stage.
The net with stage b13's weights varied is the suffix after it at the varied stage.
The net with stage b14's weights varied is the suffix after it at the varied stage.
The net with stage b15's weights varied is the suffix after it at the varied stage.
The net with stage d2's weights varied is the suffix after it at the varied stage.
The net with stage b16's weights varied is the suffix after it at the varied stage.
The net with stage b17's weights varied is the suffix after it at the varied stage.
The net with stage b18's weights varied is the suffix after it at the varied stage.
The net with the head varied is the head at the varied parameters.
The logits the tie's loss cotangent reads are cnxNetB's. The tie spells the head as
three batched ops (batchMap_comp).
Without stochastic depth cnxNetB is the canonical ConvNeXt-T forward, batched: convNextForwardTCh at
w.toCh ε, the FullT forward whose VJP is convNextForwardTChHasVJP, applied per example. The
per-example identity is CnxTieWeights.forward_eq_convNextForwardTCh; batchMap_comp splits
the batched composite into the capstone's stage-by-stage chain.
Every ConvNeXt-T parameter gradient node is the derivative of L in that parameter, for a
loss L of the logits and g the cotangent the chain starts from: the 182 nodes
cnx_net_tiedGB ties, each at the cotangent the tie threads to it from g, stated against
L of cnxNetB with that one parameter varied.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Every ConvNeXt-T parameter gradient node is the derivative of the loss in that parameter.
For any loss L of the logits with gradient g at the net's output, each of the 182 nodes
cnx_net_tiedGB ties — at the same cotangent — is ∂L/∂θ of the WHOLE net, cnxNetB with that
one parameter varied (a stem field, a block's or downsample's record w.bk := p with one slot
changed, or a head field).
Hypothesis: 0 < ε, the LayerNorms' (the tie itself needs none). The loss enters only through
hL; cnx_net_lossGrad_smoothedCE discharges it for the loss the artifacts ship.
The loss the artifacts ship: every node is the derivative of the batched label-smoothed
cross-entropy smoothedBatchLossDiv, g the softmaxDiv cotangent the render emits — the
tie's own g, whose logits are cnxNetB N ε w x (cnx_logitsB_eq).
The emitted ConvNeXt-T step's gradient nodes ARE the loss's gradient, at one chain. For
each of the 182 parameter slots, at ONE cotangent chain (the tie's own, from the emitted
smoothed-loss cotangent g): the node denotes its layer's Jacobian against the chain cotangent
(cnx_net_tiedGB), and the batched smoothed loss of cnxNetB with that one slot varied is
differentiable there with the node as its gradient (cnx_net_lossGrad_smoothedCE). The tie
spells each block input as its own let; the proof rewrites the loss side's cnxPre* into those
lets (cnxPreS_apply, …) and the loss side's logits into the tie's (cnx_logitsB_eq).