The ResNet-34 / ResNet-50 backward chains — the ℝ maps the ResNet ties are about #
The hand-composed reverse of the committed ResNet forwards, as plain defs on the cotangent:
the batched chains r34InputGradB / r50InputGradB (the reverses of resnet34ForwardBFull /
resnet50ForwardBFull, at a variable batch N, R50 also at a variable resolution q). Each
chain keeps its block backwards and its BatchNorm backwards as supplied maps and spells only
the endpoints — the stem's strided conv-back, the 3×3/s2 pool-back, the GAP-back and the
dense-back — so that the certified tie (ResNet34BackCertifiedTieB,
ResNet50WholeBackCertifiedTieB, on the leaf ties of ConvBackCertifiedTie) is a statement
about a NAMED chain of the forward's shape.
Note: maxPool3s2FlatBackB, the batched pool backward, is StableHLO.batchMapAux and not
batchMap: the pool's backward is indexed by the saved forward activation and every example
has its own, so a batchMap would hand example 0's argmax pattern to all of them. It is the one
endpoint the batch axis changed, and the reason this leaf imports StableHLO.Basic.
Padding is SYMMETRIC at every stride-2 site of both nets (flatConvStride2Back), the
PyTorch-origin convention — not the XLA-SAME flatConvStride2XlaBack the TF-origin stems take.
At a variable q every dimension is written as an explicit 2 * (…) nest, never 8 * q: those
are equal Nats and NOT definitionally equal terms, and each stage demands its operand at exactly
the spelling it names.
No number is stated about any of these chains.
The batched whole-net input-gradient backward of ResNet-34 — the exact reverse of
resnet34ForwardBFull = head ∘ [3,4,6,3] ∘ stem: dense-back → GAP-back → the sixteen basic
blocks' backwards → the 3×3/s2 pool back → the stem's relu mask, BatchNorm back and 7×7/s2
conv back. The block backwards and the stem's BatchNorm back are supplied; the conv, pool, GAP
and dense leaves are concrete and lifted over the N examples. N is a variable: the batch
size is never pinned (the 224-px resolution and the 64…512 widths are literals).
Equations
- One or more equations did not get rendered due to their size.
Instances For
The batched whole-net input-gradient backward of ResNet-50 — the exact reverse of
resnet50ForwardBFull = head ∘ [3,4,6,3] bottlenecks ∘ stem: dense-back → GAP-back → the
sixteen bottleneck backwards → the 3×3/s2 pool back → the stem's relu mask, BatchNorm back
and 7×7/s2 conv back. The bottleneck backwards and the stem's BatchNorm back are supplied;
the conv, pool, GAP and dense leaves are concrete and lifted over the N examples. q is a
binder, so one chain covers resnet50in_fwd (q = 7, 224 px) and resnet50in160_fwd
(q = 5, 160 px). Stem and head ARE ResNet-34's
(r34StemB / r34HeadB) at R50's widths.
Equations
- One or more equations did not get rendered due to their size.