Documentation

LeanMlir.Proofs.Foundation.CrownBound

CROWN: a linear-relaxation bound on the seam IBP already certifies through #

IntervalBound.lean concretizes to an interval after every layer, so the box grows multiplicatively with depth: each layer throws away the correlations between neurons. CROWN (Zhang et al. 2018) never concretizes in the middle. It carries a linear function of the input backward through the net — relaxing each unstable ReLU by a linear lower/upper envelope — and concretizes once, at the end.

Why that is tighter, in one line: IBP bounds each pre-activation with ⟨W1ᵢ, x₀⟩ ∓ ε‖W1ᵢ‖₁ and then combines; CROWN forms the composite row A = Σₜ aₜ · W1ₜ first and takes ε‖A‖₁ once. Cancellation between the rows of W1 survives in A and is destroyed by the per-row ‖·‖₁.

CROWN-IBP, which is what this file supports: the per-neuron pre-activation bounds [l, u] that decide each ReLU's relaxation are taken from IBP — here literally denseLo W1 … / denseHi W1 …, the box IntervalBound.lean already proves — and only the output pass is CROWN. (Full CROWN would re-derive [l,u] by running itself on every prefix: strictly tighter, much more work.)

What is certified, and why it is a MARGIN and not a box #

certified_of_boxSound (IntervalBound.lean) consumes a bracket on the logit vector and asks the boxes to separate. CROWN does not produce one usefully: bounding f · y and f · j independently and then separating discards the correlation between them, which is a large fraction of the available tightening. So the capstone here is stated on the margin f · y − f · j directly, via certified_of_marginPos — the same seam idea (the certificate never inspects how the bound was obtained), one rung lower. certified_of_marginPos is generic: a margin-direct interval bound plugs into it too, with no CROWN.

Relaxation, and the rational-size question #

Both envelopes are stated so a generator can discharge them by norm_num on rationals, and — critically — so the slope may be rounded:

That rounding is what keeps the coefficients small. Left unrounded, u/(u−l) carries the layer-1 denominators into every entry of A and reproduces the LipSDP tier's ~230-digit blow-up. Measured (scripts/crown_ibp_probe.py, planning/archive/crown_ibp.md §5.5): at k = 8 the rounding costs ZERO images on both trained nets at every radius, so the coefficients stay at the same /256 scale as the weights themselves.

Depth: this file does the single backward step a dense ∘ relu ∘ dense net needs, where crownRow IS the matrix-level back-substitution. A general multi-layer LinSound would be real machinery with no consumer yet — the conv tier stays on IBP (max-pool is not an elementwise nonlinearity), and the trained dense nets are two-layer.

Everything is elementary and closes under propext / Classical.choice / Quot.sound. Engine only; the generated instance is a separate file.

theorem Proofs.LipschitzCertDemo.certified_of_marginPos {n k : } {f : EuclideanSpace (Fin n)EuclideanSpace (Fin k)} {x : EuclideanSpace (Fin n)} {ε : } {y : Fin k} (h : ∀ (j : Fin k), j y∀ (x' : EuclideanSpace (Fin n)), InBoxE (fun (i : Fin n) => x.ofLp i - ε) (fun (i : Fin n) => x.ofLp i + ε) x'0 < (f x').ofLp y - (f x').ofLp j) :

Margin positivity ⇒ L∞ certificate. If for every wrong class the margin f · y − f · j is strictly positive at every point of the box x ∓ ε, then y is the strict argmax under every perturbation with |δ i| ≤ ε coordinatewise.

The peer of certified_of_boxSound one rung lower: it says nothing about HOW the margin was bounded, so a CROWN bound and a margin-direct interval bound discharge it the same way. Bounding the margin rather than the two logits separately is itself a tightening — it keeps the correlation between f · y and f · j that a box separation throws away.

theorem Proofs.LipschitzCertDemo.relu_lower_envelope {α z : } (h0 : 0 α) (h1 : α 1) :
α * z max z 0

Lower envelope. α · z ≤ relu z for ANY α ∈ [0,1], with no reference to [l,u] at all. Soundness is therefore insensitive to the choice of α: optimizing it (α-CROWN) is a tightness question, never a correctness one, and a rounded α needs no re-verification.

theorem Proofs.LipschitzCertDemo.relu_upper_envelope {l u s z : } (hl : l 0) (hlz : l z) (hzu : z u) (hs0 : 0 s) (hs : u s * (u - l)) :
max z 0 s * (z - l)

Upper envelope. On [l, u] with l ≤ 0, any nonneg s at or above the chord slope dominates relu. The chord condition is stated as u ≤ s * (u - l) rather than s = u / (u - l), which is what lets a generator ROUND s up to a /2^k grid and still discharge it by norm_num — the rational-size lever (see the header).

Proof: g z = s·(z−l) − relu z is affine on [0,u] with g 0 = −s·l ≥ 0 and g u = s·(u−l) − u ≥ 0, hence nonneg between; below 0 it is trivial.

a · z + c is a sound linear lower bound for v · relu z on [l, u].

One neuron's contribution to a margin, relaxed. v is that neuron's coefficient in the margin row W2 y · − W2 j ·; its SIGN decides which envelope may be used, which is why the instances below split on it and not on the neuron alone.

Equations
Instances For
    theorem Proofs.LipschitzCertDemo.reluLB_dead {v l u : } (hu : u 0) :
    ReluLB v l u 0 0

    A neuron that is dead on the whole box (u ≤ 0) contributes exactly 0 — both envelopes are exact and the coefficient vanishes.

    theorem Proofs.LipschitzCertDemo.reluLB_active {v l u : } (hl : 0 l) :
    ReluLB v l u v 0

    A neuron that is active on the whole box (0 ≤ l) is EXACT: relu z = z, so it passes its margin coefficient through unrelaxed. This is where CROWN keeps everything IBP would have widened.

    theorem Proofs.LipschitzCertDemo.reluLB_unstable_pos {v l u α : } (hv : 0 v) (h0 : 0 α) (h1 : α 1) :
    ReluLB v l u (v * α) 0

    Unstable neuron, POSITIVE margin coefficient: take the lower envelope, whose slope α is unconstrained in [0,1].

    theorem Proofs.LipschitzCertDemo.reluLB_unstable_neg {v l u s : } (hv : v 0) (hl : l 0) (hs0 : 0 s) (hs : u s * (u - l)) :
    ReluLB v l u (v * s) (-(v * s * l))

    Unstable neuron, NEGATIVE margin coefficient: multiplying by v ≤ 0 flips the inequality, so the upper envelope is what lower-bounds the contribution. This is the only branch that carries a constant, and the only one whose slope must be verified against the chord.

    theorem Proofs.LipschitzCertDemo.ReluLB.congr {v l u a a' c c' : } (h : ReluLB v l u a c) (ha : a' = a) (hc : c' = c) :
    ReluLB v l u a' c'

    Transport a ReluLB along equalities of its coefficient and constant — the join between the engine's branch-selected form and the generator's emitted literals.

    noncomputable def Proofs.LipschitzCertDemo.relaxA (l u α s v : ) :

    The relaxation coefficient, with the branch chosen by the box and the sign of the margin coefficient v.

    Equations
    Instances For
      noncomputable def Proofs.LipschitzCertDemo.relaxC (l u s v : ) :

      The relaxation constant. Only the unstable/negative branch carries one.

      Equations
      Instances For
        theorem Proofs.LipschitzCertDemo.reluLB_relax {l u α s v : } (h0 : 0 α) (h1 : α 1) (hs0 : 0 s) (hs : 0 < ul < 0u s * (u - l)) :
        ReluLB v l u (relaxA l u α s v) (relaxC l u α s v)

        All four branches in one lemma, uniform in v. The chord condition is required only where it is used — on a genuinely unstable neuron — so a stable neuron discharges it vacuously and may carry s = 0.

        This is what keeps a generated instance affordable: the relaxation is proved once per NEURON (16 facts), not once per (class, neuron) pair (144), because nothing here depends on which wrong class is being separated.

        noncomputable def Proofs.LipschitzCertDemo.crownRow {n h : } (a : Fin h) (W1 : Fin hFin n) :
        Fin n

        The CROWN row. Back-substitute the per-neuron relaxation coefficients through the first layer: A = Σₜ aₜ · W1ₜ. Forming this composite BEFORE taking any norm is the entire mechanism — cancellation between the rows of W1 survives here and is destroyed by IBP's per-row ‖·‖₁.

        Equations
        Instances For
          theorem Proofs.LipschitzCertDemo.crownRow_dot {n h : } (a : Fin h) (W1 : Fin hFin n) (x' : EuclideanSpace (Fin n)) :
          t : Fin h, a t * (denseE W1 x').ofLp t = i : Fin n, crownRow a W1 i * x'.ofLp i

          Back-substitution is exact: relaxing in pre-activation space and then substituting z = W1 x' is the same linear function of x' as the CROWN row applied directly.

          theorem Proofs.LipschitzCertDemo.linf_lower_bound {n : } (A : Fin n) (x : EuclideanSpace (Fin n)) (ε : ) {x' : EuclideanSpace (Fin n)} (hbox : InBoxE (fun (i : Fin n) => x.ofLp i - ε) (fun (i : Fin n) => x.ofLp i + ε) x') :
          i : Fin n, A i * x.ofLp i - ε * i : Fin n, |A i| i : Fin n, A i * x'.ofLp i

          Concretization, once. On the uniform box x ∓ ε a linear functional bottoms out at ⟨A, x₀⟩ − ε‖A‖₁. This is denseLo_uniform read at the one-row matrix A — reused, not re-derived. ONE ℓ1 norm per (image, class), where IBP pays one per (image, neuron).

          theorem Proofs.LipschitzCertDemo.mlp2_apply {n h k : } (W1 : Fin hFin n) (W2 : Fin kFin h) (x' : EuclideanSpace (Fin n)) (c : Fin k) :
          ((denseE W2 reluE denseE W1) x').ofLp c = t : Fin h, W2 c t * max ((denseE W1 x').ofLp t) 0

          The two-layer net, evaluated one logit at a time — denseE/reluE peeled exactly once, so the inner denseE W1 x' stays folded (unfolding it would put a 784-term sum under every max).

          theorem Proofs.LipschitzCertDemo.crown_margin_ge {h k : } (W2 : Fin kFin h) (y j : Fin k) (a cc l u z : Fin h) (hz : ∀ (t : Fin h), l t z t z t u t) (hr : ∀ (t : Fin h), ReluLB (W2 y t - W2 j t) (l t) (u t) (a t) (cc t)) :
          t : Fin h, a t * z t + t : Fin h, cc t t : Fin h, W2 y t * max (z t) 0 - t : Fin h, W2 j t * max (z t) 0

          The margin bound. Summing the per-neuron relaxations gives a linear lower bound on f · y − f · j in pre-activation space. z is abstract here: the bound is a statement about the relaxation, not about the net.

          theorem Proofs.LipschitzCertDemo.crown2_certified_at_eps {n h k : } (W1 : Fin hFin n) (W2 : Fin kFin h) {x : EuclideanSpace (Fin n)} {ε : } {y : Fin k} (a cc : Fin kFin h) (hrelax : ∀ (j : Fin k), j y∀ (t : Fin h), ReluLB (W2 y t - W2 j t) (denseLo W1 (fun (i : Fin n) => x.ofLp i - ε) (fun (i : Fin n) => x.ofLp i + ε) t) (denseHi W1 (fun (i : Fin n) => x.ofLp i - ε) (fun (i : Fin n) => x.ofLp i + ε) t) (a j t) (cc j t)) (hcert : ∀ (j : Fin k), j y0 < i : Fin n, crownRow (a j) W1 i * x.ofLp i - ε * i : Fin n, |crownRow (a j) W1 i| + t : Fin h, cc j t) :

          CROWN-IBP L∞ certificate for dense ∘ relu ∘ dense.

          For each wrong class j, the generator supplies relaxation coefficients a j and constants cc j for the margin row W2 y · − W2 j ·, discharged against the IBP pre-activation box by the reluLB_* instances; the certificate fires when the ONE concretized linear bound is positive:

          ⟨A, x₀⟩ − ε‖A‖₁ + Σ cc  >  0,    A = crownRow (a j) W1.
          

          Note the box in hrelax is literally denseLo W1 … / denseHi W1 … — the interval box IntervalBound.lean proves — not a float recomputation. That is forced by the statement, which is the point: CROWN-IBP is sound only if the [l,u] you relax against are the ones that actually hold on the box the certificate quantifies over.

          ‖A‖₁ = Σᵢ |Σₜ aₜ·W1ₜᵢ| does NOT decompose over t — the absolute value is taken after the combination, which is exactly the point of CROWN. So it needs a fact of its own, and the naive route (emit A's 784 numerators per (image, class)) makes the exhibit enormous (gotcha 2).

          Instead the kernel forms A from the weight rows the corpus already commits: a generator emits the 16 coefficient numerators, and absSumZ (combZ …) folds the 16×784 combination and the absolute sum in one decide +kernel.

          ⟨A, x₀⟩ needs no new fact at all — crownRow_dot turns it back into Σₜ aₜ·⟨W1ₜ, x₀⟩, i.e. a 16-term rational sum over the committed hpre dots.

          Entrywise c · row + acc.

          Equations
          Instances For

            Σₜ csₜ · rowsₜ, entrywise: the CROWN row A, at integer scale.

            Equations
            Instances For
              theorem Proofs.LipschitzCertDemo.length_combZ (n : ) (cs : List ) (rows : List (List )) :
              (∀ rrows, r.length = n)(combZ n cs rows).length = n
              theorem Proofs.LipschitzCertDemo.getD_scaleAddZ (c : ) (row acc : List ) :
              row.length = acc.length∀ (i : ), (scaleAddZ c row acc).getD i 0 = c * row.getD i 0 + acc.getD i 0
              theorem Proofs.LipschitzCertDemo.getD_map_getD (rows : List (List )) (i t : ) :
              (List.map (fun (r : List ) => r.getD i 0) rows).getD t 0 = (rows.getD t []).getD i 0
              theorem Proofs.LipschitzCertDemo.getD_combZ (n : ) (cs : List ) (rows : List (List )) :
              cs.length = rows.length(∀ rrows, r.length = n)∀ (i : ), (combZ n cs rows).getD i 0 = dotZ cs (List.map (fun (r : List ) => r.getD i 0) rows)

              The combined row, entry by entry, is a plain dotZ of the coefficients against the rows' i-th column.

              theorem Proofs.LipschitzCertDemo.crownRow_comb {n h : } (cs : List ) (rows : List (List )) (W1 : Fin hFin n) (dc dw : ) (hW : ∀ (t : Fin h) (i : Fin n), W1 t i = ((rows.getD t []).getD (↑i) 0) / dw) (hcl : cs.length = h) (hrl : rows.length = h) (hrn : rrows, r.length = n) (i : Fin n) :
              crownRow (fun (t : Fin h) => (cs.getD (↑t) 0) / dc) W1 i = ((combZ n cs rows).getD (↑i) 0) / (dc * dw)

              The CROWN row, in one kernel-checkable object. With coefficients aₜ = csₜ / dc and weights W1ₜᵢ = rowsₜᵢ / dw, the ℝ-level crownRow is the integer combZ over dc·dw.

              theorem Proofs.LipschitzCertDemo.crownRow_l1 {n h : } (cs : List ) (rows : List (List )) (W1 : Fin hFin n) (dc dw : ) (hW : ∀ (t : Fin h) (i : Fin n), W1 t i = ((rows.getD t []).getD (↑i) 0) / dw) (hcl : cs.length = h) (hrl : rows.length = h) (hrn : rrows, r.length = n) (hd : 0 dc * dw) {v : } (hv : absSumZ (combZ n cs rows) = v) :
              i : Fin n, |crownRow (fun (t : Fin h) => (cs.getD (↑t) 0) / dc) W1 i| = v / (dc * dw)

              ‖A‖₁ from one absSumZ … := by decide +kernel fact. One ℓ1 norm per (image, class), where IBP pays one per (image, neuron) — and the 784 entries of A are never emitted, only folded.