ResNet-34's non-degeneracy seal, on the full-width batched net (levels 2 and 3) #
planning/full_width_seals.md §4.1. ResNet34FullBVJP.lean proves
resnet34ForwardB_full_has_vjp_at: the whole-net VJP at any (w, x) that satisfies 32 relu
clauses, a stem clause and the stem pool's no-tie. A conditional theorem of that shape says
nothing unless its hypotheses are jointly satisfiable at a point with a nonzero Jacobian — and
until now that was exhibited only on a 2-channel per-example proxy (ResNet34Live*, retired with
this file). This file exhibits it on resnet34ForwardB_full itself: 64→512 channels, [3,4,6,3]
blocks, the 7×7/s2 stem, the 3×3/s2 pool, batch BatchNorm, at 224×224.
The witness #
Weights are structural, not trained (a trained-weight witness is a numeric fact about millions of
floats, which is what the training runs evidence — planning/full_width_seals.md §7):
- every residual body is zeroed, so it is the constant
β₂ = 1whateverγ₂is (BatchSeal.bnBatchLA_const: a constant channel has variance 0, so batch BN returnsβ). On a nonnegative activation the identity block is therefore the affine shifta ↦ a + 1(sealIdB_eq) — and the shift is batch-uniform, hence invisible to the carrier; - every channel-changing conv (the stem, the three 1×1/s2 projections) is a centre-tap
broadcast (
BatchSeal.ctK): every output channel is a copy of input channel 0 read through the kernel's centre tap. ⚠ Centre tap, not a general kernel, becauseconv2dzero-pads: a conv of a constant is not constant at the border, but the centre tap is in range at every cell. ⚠⚠ And a broadcast, not the plan's diagonal-on-channel-0: a kernel feeding only output channel 0 leaves the stem's other 63 channels constant, and a constant channel ties every 3×3 window of the pool —R34PoolSmoothAtquantifies over channels, so the tap has to reach all of them; γ = 1,β = 160at the stem and the projections,ε = 1everywhere.√(N·h·w) ≤ √25088 < 160at every one of those four BN widths, soBatchSeal.bnBatchLA_posputs every relu strictly off its kink at every input — the net needs no eventually-argument for its relus, and the only genuine kink left is the stem pool;- the head reads channel 0 into class 0 (
Wd 0 0 = 1,bd = 0), so0 < nClsis the only constraint on the class count.
N = 2 and the input is sealX t = sealBase + t • sealV: both examples carry the same strictly
decreasing ramp in channel 0, and sealV adds t to all of example 0's channel 0.
Why the carrier is a batch difference (§3.2 of the plan) #
⭐⭐ bnBatchLA normalizes each channel over all N·h·w cells, so the proxies' channel-difference
carrier is exactly what a channel's own mean subtracts, and at N = 1 this net is constant in its
input. The carrier here is EDiff: example 0's slab is example 1's plus a per-channel constant.
Batch BN keeps it and scales it by γ_c · istd_c (BatchSeal.bnBatchLA_exdiff) because the two
examples share one mean and one istd; the centre-tap convs copy channel 0's offset to every
channel; the pool shifts with it (BatchSeal.maxPool3s2_shift, for every t, no argmax argument);
the zeroed bodies add a batch-uniform constant and so are transparent. The class-0 output
difference between the two examples is therefore t · R t with R a product of the four
carrier-path istds, continuous and positive, so g'(0) = R 0 ≠ 0 and no BN-variance derivative
is ever taken.
The structural identity-block weights: both convs zeroed, every BN (ε, γ, β) = (1, 1, 1).
The body is then the constant 1 and the block is a ↦ a + 1 on a nonnegative activation.
Equations
- One or more equations did not get rendered due to their size.
Instances For
The structural downsample weights: zeroed body, centre-tap 1×1/s2 projection, β_p = 160
(the margin that keeps the post-residual relu off its kink at every input).
Equations
- One or more equations did not get rendered due to their size.
Instances For
The witness weights, generic in the class count.
Equations
- One or more equations did not get rendered due to their size.
Instances For
A zeroed final conv makes a body the constant β₂ — projB at a zero kernel is
bnBatchLA of the constant 0, which is β₂ (variance 0). Used for both block kinds.
⭐ The structural identity block is the shift a ↦ a + 1 on a nonnegative activation: the
body is the constant 1 and the post-residual relu is off (1 + a ≥ 1 > 0).
A strided conv-bn-relu stage whose BN is everywhere positive has no relu left.
⭐ The stem with its relu removed: pool ∘ bn ∘ strided conv. The pool stays — it is the
net's only remaining kink, and the carrier crosses it by BatchSeal.maxPool3s2_shift.
Both εs of a structural identity block are positive.
All three εs of a structural downsample are positive.
⭐ The identity block's two relu clauses: the mid-relu sees the constant β₁ = 1
(weight-only), the outer one sees 1 + activation > 0.
⭐ The downsample's two relu clauses — both weight-only: the mid-relu sees β₁ = 1, the
outer one proj + 1 > 0 with proj > 0 by the margin. No hypothesis on the activation.
The stem's relu clause — weight-only, from the β = 160 margin.
The stem's centre-tap conv output — the pre-BN activation on the carrier's path.
Equations
- Proofs.R34FullBSeal.Zs t = Proofs.BatchSeal.ctConv 64 7 7 56 56 t
Instances For
⭐ The stem pool has no tie at the witness — the ramp is positionally injective and BN is injective within a channel.
The stem, collapsed: its relu is off, so it is pool ∘ bn ∘ centre-tap conv.
The stem's relu is off at the witness, so the pool's no-tie condition can be stated on the
BN output (sealPoolSmooth).
⭐⭐ The whole-net VJP at the witness — every one of the 32 relu clauses, the stem clause
and the pool's no-tie discharged at (sealW nCls, sealX t), on resnet34ForwardB_full
itself (transported through resnet34ForwardB_full_eq_chain).
Equations
- One or more equations did not get rendered due to their size.
Instances For
The net is differentiable at the witness — fderiv_ne_zero_of_ray's first hypothesis. Built
block by block, as the apex's own proof builds its chain.
sealProj, unfolded — bn of the centre-tap strided conv.
The carrier after the stem BN: t scaled by that BN's istd.
Equations
- Proofs.R34FullBSeal.dS t ci = t * Proofs.bnIstd (2 * (2 * 56 * (2 * 56))) (Proofs.BatchSeal.bnRowLA 2 64 (2 * 56) (2 * 56) (Proofs.R34FullBSeal.Zs t) ci) 1
Instances For
The carrier after d2's projection BN.
Equations
- Proofs.R34FullBSeal.dP2 nCls t ci = Proofs.R34FullBSeal.dS t 0 * Proofs.bnIstd (2 * (28 * 28)) (Proofs.BatchSeal.bnRowLA 2 128 28 28 (Proofs.R34FullBSeal.Zp2 nCls t) ci) 1
Instances For
The carrier after d3's projection BN.
Equations
- Proofs.R34FullBSeal.dP3 nCls t ci = Proofs.R34FullBSeal.dP2 nCls t 0 * Proofs.bnIstd (2 * (14 * 14)) (Proofs.BatchSeal.bnRowLA 2 256 14 14 (Proofs.R34FullBSeal.Zp3 nCls t) ci) 1
Instances For
The carrier after d4's projection BN — the one the head reads.
Equations
- Proofs.R34FullBSeal.dP4 nCls t ci = Proofs.R34FullBSeal.dP3 nCls t 0 * Proofs.bnIstd (2 * (7 * 7)) (Proofs.BatchSeal.bnRowLA 2 512 7 7 (Proofs.R34FullBSeal.Zp4 nCls t) ci) 1
Instances For
⭐ The carrier at the stem's output: the centre-tap conv copies channel 0's t to every
channel, the BN scales it by istd, and the pool carries it through unchanged.
The head reads channel 0 into class 0 — BatchSeal.head_diff_ct at this net's widths.
⭐⭐ The positive, continuous nonlinear factor: one istd per BN on the carrier's path —
the stem's and the three projections'. ⚠ No γ appears because every carrier-path γ is 1,
and no BN-variance derivative is ever taken: R enters only through the factor t · R t,
whose derivative at 0 is R 0 for any R continuous there.
Equations
- One or more equations did not get rendered due to their size.
Instances For
⭐⭐ The class-0 difference between the two examples, along the ray, is t · R t. The
carrier vanishes at the base (both examples carry the same ramp), so the product rule's cross
terms all carry a factor t.
⭐⭐ Level 2 — the witness is non-degenerate: the full-width batch-BN ResNet-34 at the
structural weights is NOT constant in its input. Straight from the ray: the class-0 difference
between the two examples is R 1 > 0 at t = 1 and 0 at the base.
⭐⭐ Level 3 — the whole-net Jacobian is nonzero at the witness. fderiv_ne_zero_of_ray at
the readout "example 0's class 0 minus example 1's class 0": along the ray it is t · R t with
R continuous and R 0 > 0, so its derivative at 0 is R 0 ≠ 0.
⭐⭐ The seal: the proven whole-network backward of the full-width, batch-BatchNorm,
[3,4,6,3], 224×224 ResNet-34 — resnet34ForwardB_full, the forward the ImageNet artifacts
run — is not the zero map at the witness. The conditional apex
resnet34ForwardB_full_has_vjp_at is therefore not vacuous, and it is not vacuous on the net
itself rather than on a 2-channel proxy of it.