ResNet-50's whole-net input-VJP at TRUE BATCH-NORM (T1, the VJP half) #
ResNet50FullB.lean states the batch-BN forward at the [3,4,6,3] bottleneck ladder. This file
gives that forward a certified HasVJPAt — the last piece of T1 for the largest hole in the
Proofs tier (planning/archive/proofs_tier_to_paper_nets.md §3.5(a)).
No new mathematics, and nothing new one tier down #
Every block VJP is already proven at bnBatchLA: r50BottleneckB_has_vjp_at,
r50ProjBlockB_has_vjp_at and r50DownBlockB_has_vjp_at (ResNet50BackB0.lean) are exactly the
three shapes r50IdB / r50ProjB / r50DownB unfold to. The bundle lemmas below are delegations
in ResNet34FullBVJP.lean's style.
⭐ And unlike ResNet-34's, this file needed no new Foundation lemma. r34's T1 was blocked on
batchMap_has_vjp_at (4.1c) for its stem pool. ResNet-50 has the same stem — and reuses
r34StemB_has_vjp_at verbatim, so that lemma is spent rather than re-derived. The head is
ResNet-34's too (r34HeadB_has_vjp, GLOBAL: GAP and dense are smooth and each is batchMap of a
per-example op, so no hypothesis appears).
The hypothesis budget #
⚠ Pointwise (HasVJPAt), not global, and necessarily. relu is kinked. ⛔ And a BOTTLENECK
carries THREE kink clauses, where ResNet-34's basic block carries two: the two interior relus
and the post-residual OUTER relu. Sixteen blocks give 48 clauses, plus the stem's relu and the stem
pool's no-tie condition — bundled per block into R50IdSmoothAt / R50ProjSmoothAt /
R50DownSmoothAt so the apex binds 18 bundles rather than 50 loose hypotheses.
⚠ The pool's condition is per example (R34PoolSmoothAt, reused): a tie is a property of one
image's 3×3 window, not of the batch.
⛔ 0 < q is a real hypothesis here, where ResNet-34 needed none. r34's ladder is at literals,
so 0 < 56 closes by norm_num; R50's is at the binder q, and the stem pool's VJP needs its
output grid nonempty. At q = 0 the net is degenerate and the statement says so.
The running activations are named r50Pre1 … r50Pre16 so each bundle can be STATED at the
activation entering its block without a sixteen-deep nested application inline; r50Pre16 doubles
as the trunk, and resnet50ForwardB_full_eq_chain bridges it back to the committed
nested-application forward.
⭐ N and q are both variables: this tier carries no numerals, so ONE statement covers
resnet50in_fwd (q = 7, 224 px) and resnet50in160_fwd (q = 5, 160 px) — the net the quoted
76.66% trains — at every batch size.
All four BatchNorm epsilons of a projection bottleneck are positive (body three, skip one).
⭐ One bundle for BOTH projection forms, as R50ProjW is one record for both.
Instances For
The THREE relu sites of an identity bottleneck are away from the kink at v: the 1×1 reduce's
BN output, the 3×3's BN output, and the post-residual sum. The 1×1 expand has no activation.
Instances For
The three relu sites of the STRIDE-1 projection bottleneck at v. Identical to the identity
block's except that the outer relu sits at the PROJECTED residual — both paths nontrivial.
Instances For
The three relu sites of the STRIDED projection bottleneck at v.
⚠⚠ v1.5: the FIRST relu is at the input resolution 2h × 2w and only the second is at
h × w, because the stride is on the 3×3. Writing hm1 at h w typechecks nowhere, which is
the one place a reader can get this shape wrong.
- hm2 (k : Fin (N * (mid * h * w))) : StableHLO.bnBatchLA N mid h w p.ε₂ p.γ₂ p.β₂ (StableHLO.batchMap N (flatConvStride2 p.W₂ p.b₂) (StableHLO.cbReluB N p.W₁ p.b₁ p.ε₁ p.γ₁ p.β₁ v)) k ≠ 0
Instances For
Identity bottleneck VJP — r50BottleneckB_has_vjp_at at the bundle's fields.
Equations
Instances For
Stride-1 projection bottleneck VJP — r50ProjBlockB_has_vjp_at at the bundle's fields.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Strided projection bottleneck VJP — r50DownBlockB_has_vjp_at at the bundle's fields.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Equations
Instances For
Equations
- Proofs.r50Pre1 N q w = Proofs.r50ProjB N (2 * (2 * (2 * q))) (2 * (2 * (2 * q))) w.s1b0 ∘ Proofs.r50Pre0 N q w
Instances For
Equations
- Proofs.r50Pre2 N q w = Proofs.r50IdB N (2 * (2 * (2 * q))) (2 * (2 * (2 * q))) w.s1b1 ∘ Proofs.r50Pre1 N q w
Instances For
Equations
- Proofs.r50Pre3 N q w = Proofs.r50IdB N (2 * (2 * (2 * q))) (2 * (2 * (2 * q))) w.s1b2 ∘ Proofs.r50Pre2 N q w
Instances For
Equations
- Proofs.r50Pre4 N q w = Proofs.r50DownB N (2 * (2 * q)) (2 * (2 * q)) w.s2b0 ∘ Proofs.r50Pre3 N q w
Instances For
Equations
- Proofs.r50Pre5 N q w = Proofs.r50IdB N (2 * (2 * q)) (2 * (2 * q)) w.s2b1 ∘ Proofs.r50Pre4 N q w
Instances For
Equations
- Proofs.r50Pre6 N q w = Proofs.r50IdB N (2 * (2 * q)) (2 * (2 * q)) w.s2b2 ∘ Proofs.r50Pre5 N q w
Instances For
Equations
- Proofs.r50Pre7 N q w = Proofs.r50IdB N (2 * (2 * q)) (2 * (2 * q)) w.s2b3 ∘ Proofs.r50Pre6 N q w
Instances For
Equations
- Proofs.r50Pre8 N q w = Proofs.r50DownB N (2 * q) (2 * q) w.s3b0 ∘ Proofs.r50Pre7 N q w
Instances For
Equations
- Proofs.r50Pre9 N q w = Proofs.r50IdB N (2 * q) (2 * q) w.s3b1 ∘ Proofs.r50Pre8 N q w
Instances For
Equations
- Proofs.r50Pre10 N q w = Proofs.r50IdB N (2 * q) (2 * q) w.s3b2 ∘ Proofs.r50Pre9 N q w
Instances For
Equations
- Proofs.r50Pre11 N q w = Proofs.r50IdB N (2 * q) (2 * q) w.s3b3 ∘ Proofs.r50Pre10 N q w
Instances For
Equations
- Proofs.r50Pre12 N q w = Proofs.r50IdB N (2 * q) (2 * q) w.s3b4 ∘ Proofs.r50Pre11 N q w
Instances For
Equations
- Proofs.r50Pre13 N q w = Proofs.r50IdB N (2 * q) (2 * q) w.s3b5 ∘ Proofs.r50Pre12 N q w
Instances For
⭐⭐ ResNet-50 at TRUE BATCH-NORM has a certified input-VJP at a smooth point — all sixteen
bottlenecks. Chains stem → the [3,4,6,3] ladder → head with vjp_comp_at, one positivity
bundle and one smoothness bundle per block. T1's VJP half, and the first tier ResNet-50 has
ever had at the net level.
⚠ Pointwise, and necessarily: relu is kinked. ⛔ Each block contributes THREE clauses — the two interior relus and the post-residual OUTER relu — where ResNet-34's basic block contributes two and EfficientNet's MBConv none.
⭐ The head takes no hypothesis at all, and N and q are both variables, so this covers the
224-px and 160-px artifacts at every batch size. ⛔ 0 < q is needed for the stem pool.
Equations
- One or more equations did not get rendered due to their size.
Instances For
⭐ The committed nested-application forward IS the layered chain the VJP is stated on — the
ResNet-50 peer of resnet34ForwardB_full_eq_chain, and what lets the VJP be about
resnet50ForwardB_full rather than about a re-spelling of it.
⭐⭐ Public correctness theorem: the sixteen-bottleneck batch-BN backward equals the
pdiv-contracted Jacobian of resnet50ForwardB_full ITSELF — the committed
nested-application forward ResNet50FullB.lean defines — not of the layered chain the VJP is
assembled on. Tied back through resnet50ForwardB_full_eq_chain.