ResNet-50's T3 §1a TIE — the un-fused, batched whole-net thread #
ResNet34FoldB.lean's op-kind folds (ResNet34PoCB.*GradB_den) make every parameter GRADIENT
node of ResNet-50's batched train step den-faithful for an ARBITRARY cotangent. This removes the "arbitrary": each is pinned to the one
the emitted backward chain delivers, so the whole train step is den-composed forward → loss →
backward with no free activation and no symbolic cotangent. With T1 and T2 that is ResNet-50's T3,
and it makes this the third net whose train-step tie is about the artifact its quoted accuracy
comes from.
⭐⭐ The block cotangents are NOT derived here. ResNet50FullBVJP.lean's
r50{Id,Proj,Down}B_has_vjp_at ARE the certified block backwards, and ResNet50BackB0.lean's
r50{Bottleneck,ProjBlock,DownBlock}BackBatchedGraph_faithful family already proves the emitted
backward subgraph denotes exactly them. The three *CotIn_eq_vjp lemmas below are that statement
in this file's vocabulary, so the cross-block chain is a composition of certified VJPs rather than
a re-derivation — the economy ResNet-34's 4.2a and MobileNetV2's 4.2c both took.
⭐⭐ THE LOSS COTANGENT IS A BINDER, and for this net it had to be. ResNet-34's and
MobileNetV2's capstones compute g internally from smoothedLossCotGraph. ResNet-50 ships BOTH
losses: bce := false artifacts carry the six-op label-smoothed softmax chain and bce := true
ones — including resnet50in160_lambaccdp8x64bce, where the 76.66% comes from — carry
BCE-with-logits' three-op chain. So r50_net_tiedB takes g as a hypothesis and the two loss
corollaries instantiate it: r50_lossCot_is_smoothedCE_grad and r50_lossCot_is_bce_grad. That is
4b's "the head takes g as a BINDER" made necessary rather than merely tidier.
⭐ The stem tie is three nodes, not four, and the head tie is ResNet-34's. ResNet50RenderB has
no convBias flag at all — its zb bakes false — so no conv-bias gradient op is ever emitted and
there is nothing to keep "to cover the flag", unlike r34's and MobileNetV2's ties.
ResNet34TieB.r34HeadTiedB is generic in {c nCls} and r34HeadCotBlk in the same, so the head
is reused verbatim at 2048 channels — as r34HeadB itself was in T1.
The emitted chain, node for node #
ResNet50RenderB.bnkIdBackGradB, from the block-output cotangent dyOut:
%da = selectPosB(a) %dn3 = bnBatchBack(g3, c3) %dc3 = convBackBatched(W3)
%dr2 = selectPosB(n2) %dn2 = bnBatchBack(g2, c2) %dc2 = convBackBatched(W2)
%dr1 = selectPosB(n1) %dn1 = bnBatchBack(g1, c1) %dc1 = convBackBatched(W1)
%dx = addVB(%dc1, %da)
and the nine parameter nodes read W1 ← %dn1, g1/bt1 ← %dr1, W2 ← %dn2, g2/bt2 ← %dr2,
W3 ← %dn3, g3/bt3 ← %da. ⚠ Off by one on any of those and the gradient is silently wrong; the
render's own comment records the same trap on the stochastic-depth cotangent.
The projection blocks add %dnp = bnBatchBack(gp, cp, %da) and %dcp = convBack(Wp), and their
fan-in is addVB(%dc1, %dcp) — both branches nontrivial.
Honest residual #
⚠ One add_comm per projection form. The render emits addVB(body, projection) where
residualProj proj body adds proj + body, so r50{Proj,Down}CotIn_eq_vjp carry a commutation.
The identity block needs none. Same seam T2's graph faithfulness has, for the same reason.
⛔ ONE REPLICA. In resnet50in160_lambaccdp8x64bce every gradient node feeds
allReduceMeanF — the collective as an AST node since 4d piece 2 (2026-09-07), until then emitted
text outside the AST — so every statement here is at the per-replica gradient node and
Foundation/DataParallelNode.lean composes it with the replica mean (§4d). The 8× accumulation sits between the gradient
and the optimizer as momVNextF at (μ := akeep), and the LAMB tail is lamb_triple_faithful —
both certified, neither part of this file.
⚠ No smoothness hypothesis in the capstone, exactly as r34's and mnv2's: the folds are ∀ cot
statements instantiated at explicitly constructed cotangents. The relu-kink and positivity
conditions enter ONLY in the three *CotIn_eq_vjp lemmas, which say those cotangents ARE the
certified whole-net backward. N and q are both binders.
Cotangent at the block's pre-relu sum a — the outer relu's mask applied to dyOut. Feeds
bn₃'s γ/β directly AND, through the identity skip, the block-input fan-in.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at bn₂'s output — conv₃'s input-VJP masked by the second relu. Feeds γ₂/β₂.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The block-INPUT cotangent: the residual fan-in addVB(convBack(dn1), da) the render emits.
Equations
- Proofs.ResNet50TieB.r50IdCotIn N h w p xin dyOut i = Proofs.EnetTiePoC.cInB N p.W₁ p.b₁ (Proofs.ResNet50TieB.r50IdCotC1 N h w p xin dyOut) i + Proofs.ResNet50TieB.r50IdCotA N h w p xin dyOut i
Instances For
⭐⭐ The emitted fan-in IS the certified bottleneck VJP's backward. rfl after the graph
lemma: the render's ten-node backward subgraph denotes (r50IdB_has_vjp_at …).backward dyOut.
Cotangent at the pre-relu sum. Feeds bn₃'s AND the projection's γ/β, and the projection's whole backward branch.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the PROJECTION conv's output — r50ProjCotA through the skip BN's backward.
Feeds Wp. ⚠ It reads the UNMASKED-by-drop %da, which is the render's rule: the projection
branch is never dropped.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The block-INPUT cotangent: addVB(convBack(dn1), convBack(dnp)) — both branches
nontrivial, unlike the identity block's.
Equations
- One or more equations did not get rendered due to their size.
Instances For
⭐ The projected fan-in IS the certified stride-1 projection block's backward. ⚠ One
add_comm: the render emits addVB(body, projection) and the graph builds
addV(projection, body).
Cotangent at conv₃'s output. Feeds W₃ (a stride-1 1×1 at h × w).
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at bn₂'s output, at h × w. Feeds γ₂/β₂.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the STRIDED conv₂'s output. Feeds W₂.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at bn₁'s output — at the INPUT grid 2h × 2w, because conv₂ upsamples. Feeds
γ₁/β₁. ⚠⚠ Writing this at h × w typechecks nowhere, and it is the one place a reader can
get v1.5's shape wrong.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at conv₁'s output, at 2h × 2w. Feeds W₁, an ORDINARY conv weight gradient.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Cotangent at the STRIDED projection conv's output. Feeds Wp.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The block-INPUT cotangent: addVB(convBack(dn1), convStridedBack(dnp)).
Equations
- One or more equations did not get rendered due to their size.
Instances For
⭐ The projected fan-in IS the certified strided block's backward. One add_comm, as the
stride-1 projection's is.
Each conjunct is a ResNet34PoCB op-kind fold instantiated at the cotangent the
render's chain delivers, so nothing here is a new proof: the bundles are the §1 fold with the
freedom removed. reassocB bridges the conv/relu index N·(c·h·w) to the BatchNorm parameter
ops' N·(c·(h·w)). ⛔ There are NO conv-bias conjuncts: ResNet50RenderB has no convBias flag,
so those ops are never emitted and every slot here is exercised by the artifact.
Identity bottleneck, tied. All NINE parameter nodes — three conv weights and three
BatchNorm γ/β pairs — denote the certified batched Σ_n gradient at the real forward
activations and the real backward-chain cotangent driven by dyOut. ⚠ Each BatchNorm's γ/β
reads the cotangent at THAT BatchNorm's output (cotN1, cotN2, cotA) while its conv reads
the one at the conv's output (cotC1, cotC2, cotC3); off by one and the gradient is
silently wrong.
Equations
- One or more equations did not get rendered due to their size.
Instances For
⭐ Stride-1 projection bottleneck, tied. Twelve nodes: the identity block's nine plus the
1×1 skip's weight and its BatchNorm γ/β. ⚠ The skip's conv is an ORDINARY convWeightGradB —
stage 1 block 0 changes channels but not resolution, which is the whole reason this block form
exists. ⚠ The skip's three nodes read cotA, the UNMASKED post-relu cotangent: the projection
branch is never stochastic-depth dropped, which is the render's own rule.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Strided projection bottleneck, tied. Twelve nodes, and TWO of the four conv weights are
the strided op. ⚠⚠ v1.5: W₁ is an ordinary convWeightGradB at the INPUT grid 2h × 2w and
its BatchNorm reduces there too; only W₂ (the 3×3) and Wp (the skip) are strided. Both
strided nodes are SYMMETRIC padding — flatConvStride2, not the XLA-SAME twin B0 and
MobileNetV2 use.
Equations
- One or more equations did not get rendered due to their size.
Instances For
⭐⭐ The whole batch-BN ResNet-50 train step, tied. Threading resnet50ForwardB_full's own
prefixes as the block inputs and an arbitrary loss cotangent g down through the certified head
backward and the sixteen certified bottleneck backwards, every parameter GRADIENT node of the
net — stem 3, twelve identity bottlenecks × 9, four projection bottlenecks × 12, dense 2 —
denotes the certified batched Σ_n gradient. That is 161, the render's own census and the
signature of resnet50_fwd.mlir minus %x. No free activation and no symbolic cotangent
below the loss.
⭐⭐ g IS A BINDER, and for this net it had to be. ResNet-50 ships both losses — the
label-smoothed softmax chain on the bce := false artifacts and BCE-with-logits' three-op
chain on the bce := true ones, including resnet50in160_lambaccdp8x64bce where the quoted
76.66% comes from. r50_lossCot_is_smoothedCE_grad and r50_lossCot_is_bce_grad instantiate
it; neither is privileged.
⭐ No smoothness hypothesis, no 0 < ε, and N and q are both binders. The folds are
∀ cot statements at explicitly constructed cotangents. The kink and positivity conditions
enter only in r50{Id,Proj,Down}CotIn_eq_vjp, which say those cotangents ARE the certified
whole-net backward — the two halves of the tie, kept apart because they have different
hypotheses.
⛔ One replica. In resnet50in160_lambaccdp8x64bce every gradient node feeds
allReduceMeanF, an AST node since 4d piece 2 (DataParallelNode.lean, §4d), and the 8×
accumulation and the LAMB tail sit downstream of every node named here.
⭐ The label-smoothed cotangent, for every bce := false artifact. Row by row, the six-op
chain ResNet50RenderB emits is (1/B)·∂/∂logits of soft-target cross-entropy against the
SMOOTHED target, at that example's real logits. The only hypothesis is that the example's
target sums to 1 — a one-hot, or mixup's convex combination.
⭐⭐ The BCE-with-logits cotangent, for every bce := true artifact — including the one the
76.66% comes from. Row by row, the three-op chain sigmoidB → subB → divConstB is
∂/∂logits of Σ_k (softplus(z_k) − t_k·z_k) at that example's real logits, over the baked
N·K. ⚠⚠ The divisor is N·K, not N: timm's BinaryCrossEntropy is reduction='mean' over
B×C, and at K = 1000 the two differ by 1000× on the effective step. ⭐ NO hypothesis on the
target at all, where the smoothed-CE row needs its mass to be 1 — BCE is per-class and
separable, which is the point under mixup.