IBP at conv depth — a compositional interval-bound engine #
IntervalBound.lean proves the L∞ certificate for exactly one shape:
dense ∘ relu ∘ dense, dense-only, hard-wired at two layers
(ibp2_certified_at_eps). That is the shape the 784→16→10 scorecards use, and
it is the reason the IBP tier stops at one hidden layer.
This file removes both restrictions. The soundness argument for IBP is
compositional — "the box propagates" is a property of one layer, and depth is
just ∘ — so the engine here is:
InBox3/InBox— a point lies in an axis-aligned box (tensor / the head's logits);BoxSound3 f Flo Fhi—(Flo, Fhi)is a sound interval transformer forf;BoxSound3.comp— depth: sound transformers compose exactly as the layers do, so ann-layer net's transformer isncompositions and no new proof;- per-layer instances —
relu(endpointmax),conv2d(sign-split over the padded taps),maxPool2(maxis monotone, so pool the endpoints), and the dense headdenseT(sign-split); ibp3_certified_of_boxSound— the capstone: a sound tensor body and dense head whose output boxes separate onx ∓ εcertifyxatL∞radiusε, at any depth.
The conv transformer is stated against the repo's own conv2d
(CNN.lean, SAME padding, the definition the VJP suite and the codegen use) —
not a re-modelled convolution. Padding contributes 0 to both endpoints, which
is why the sign-split can sit outside the pad test (convTap is monotone,
and constant 0 where the window falls off the image).
convLo_uniform / convHi_uniform are the conv peers of
denseLo_uniform/denseHi_uniform: on the first layer's uniform box x ∓ ε
the interval image collapses to conv2d W b x ∓ ε · conv2d |W| 0 𝟙, i.e. one
extra convolution by the absolute kernel against the all-ones image. That keeps
the generated per-image data at "one forward pass + one |W| pass" instead of
two full sign-split sums — the conv analogue of the ⟨w,x⟩ ∓ ε‖w‖₁ collapse the
784-dim files already exploit.
Nonlinearity note: as in IntervalBound.lean, unstable (sign-crossing) ReLUs
and tied max-pool windows are handled soundly, not assumed away — the box
simply contains both branches. The certificate is sound and incomplete; counts
derived from it are lower bounds only.
Everything is elementary and closes under propext / Classical.choice / Quot.sound.
References #
- Gowal et al. 2018, On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models. https://arxiv.org/abs/1810.12715
- Mirman, Gehr, Vechev 2018, Differentiable Abstract Interpretation for Provably Robust Neural Networks. https://proceedings.mlr.press/v80/mirman18b.html
A sound interval transformer. Flo lo hi / Fhi lo hi bracket f on
every point of the box [lo, hi]. This is the only property a layer needs
to contribute to a certificate — it composes (BoxSound3.comp) and it is
what the capstone consumes.
Equations
- Proofs.IBP.BoxSound3 f Flo Fhi = ∀ (lo hi u : Proofs.Tensor3 c h w), Proofs.IBP.InBox3 lo hi u → Proofs.IBP.InBox3 (Flo lo hi) (Fhi lo hi) (f u)
Instances For
Depth, for free. Sound transformers compose exactly as their layers do:
feed the first layer's output box into the second layer's transformer, so a conv
body (maxpool ∘ relu ∘ conv ∘ …) of n layers needs n per-layer soundness
facts and n-1 uses of this lemma — no new argument per depth.
Coordinatewise ReLU on tensors — the form a conv body uses between conv and pool, so the body never leaves tensor space.
Equations
- Proofs.IBP.reluT x o i j = max (x o i j) 0
Instances For
The padded input lookup conv2d performs: the tap at kernel offset (kh, kw) for output
position (hi, wi), or 0 where the window falls off the image. convPad (ConvIndex)
with the kernel extent implicit.
Equations
- Proofs.IBP.convTap x c kh kw hi wi = Proofs.convPad kH kW x c kh kw hi wi
Instances For
Taps are monotone in the input tensor (the pad branch is the constant 0,
which is monotone trivially) — the only fact the conv sign-split needs.
Sign-split LOWER image of conv2d W b. The split sits outside the pad test
because convTap is 0 on both endpoints there.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Max-pool is monotone, so its box endpoints are the pooled endpoints. Tied
windows need no special handling: max is monotone regardless of which
entry attains it, so the certificate never touches the argmax.
A conv classifier is Tensor3 → Vec k: a tensor body then a dense head that
reads the activation directly. Stating the certificate there keeps flatten out
of every per-image proof (it is an Equiv computation, and paying for it at each
of the head's inputs is the difference between a cheap instance and an
unaffordable one).
A sound interval transformer for a Tensor3 → Vec map (the head rung).
Equations
- Proofs.IBP.BoxSound3V f Flo Fhi = ∀ (lo hi u : Proofs.Tensor3 c h w), Proofs.IBP.InBox3 lo hi u → Proofs.IBP.InBox (Flo lo hi) (Fhi lo hi) (f u)
Instances For
Compose a tensor body into a head — the last link of a conv net's chain.
f is certified at pixel L∞ radius ε on the image x with class y:
every perturbation bounded by ε in EVERY pixel keeps y the strict argmax.
The image-shaped peer of Robustness.CertifiedAtLinf.
Equations
Instances For
Certification is monotone in the radius: a certificate at ε is a
certificate at every ε' ≤ ε, directly from the statement. Instances
therefore only need to prove (and carry the box data for) the LARGEST radius
at which an image certifies — every smaller radius on the ε-grid is a
one-line corollary, not another propagated box.
IBP L∞ certificate for a conv net, in the shape a convolutional
classifier has: an arbitrary-depth tensor body (conv/relu/pool, composed by
BoxSound3.comp) followed by a dense head. Separation of the propagated
output boxes on the pixel box x ∓ ε certifies x.
A deeper stack than anything the dense ibp2_certified_at_eps could state:
conv → relu → conv → relu → max-pool → dense head. Weights are arbitrary —
this is a statement about the ENGINE, not about one trained net.
Equations
- Proofs.IBP.deepNet W₁ b₁ W₂ b₂ Wh bh = Proofs.IBP.denseT Wh bh ∘ Proofs.maxPool2 ∘ Proofs.IBP.reluT ∘ Proofs.conv2d W₂ b₂ ∘ Proofs.IBP.reluT ∘ Proofs.conv2d W₁ b₁
Instances For
Depth costs nothing. The six-layer stack's sound interval transformer is
five .comps of the per-layer facts — no induction, no new lemma, and the
proof term is one line. Feeding this to ibp3_certified_of_boxSound gives a
pixel-L∞ certificate for the deep net exactly as for the shallow one; the
only thing that grows with depth is how loose the box gets.
The all-ones input, used to read off each output position's active tap
count (SAME padding drops taps at the border, so the ℓ1 weight is
position-dependent — conv2d (absK W) 0 ones computes exactly that).
Equations
- Proofs.IBP.onesT ic h w x✝² x✝¹ x✝ = 1
Instances For
Uniform-box collapse, LOWER. On the first layer's box x ∓ ε the conv
sign split evaluates to conv2d W b x − ε · conv2d |W| 0 𝟙 — one ordinary
forward pass plus one absolute-kernel pass, instead of two full sign-split
sums. The conv peer of denseLo_uniform.
Uniform-box collapse, UPPER (sibling of convLo_uniform).