Documentation

LeanMlir.Proofs.Nets.ViT.ViTWholeBackCertifiedTieB

⭐⭐ vitInputGradKB IS the certified whole-net ViT-Tiny gradient AT A BATCH #

ViTWholeBackCertifiedTie.lean closed T6 for ONE image: vitInputGradK, the reverse of vitForwardKV over that image's N + 1 tokens, IS the certified gradient. Every shipped ViT artifact runs a batch — vit_adam_train_step and the vitin_* family at 128 or 512 per device — and its batched T3 tie (ViTStepTieGB.lean) states every activation as StableHLO.batchMap B of the per-example prefix and every cotangent as batchMapAux B of the per-example chain, because no ViT op couples examples. This file closes T6 at that index: the five-stage batched chain vitInputGradKB (ViTBackChains.lean) IS the certified gradient of batchMap B vitForwardKV at every batch x, for every B. ViT was the one net without a batched whole-net tie; with this the *InputGradB_eq_*_vjp family covers all seven nets.

Nothing here is new mathematics, and it is lighter than ResNet-34's batched tie because ViT is smooth everywhere: every stage has a GLOBAL HasVJP, so its batched witness is batchMap_has_vjp_at (4.1c's field-by-field lift, BatchMapVJPAt.lean) over HasVJP.toHasVJPAt at each row — no smooth-point hypothesis anywhere, only 0 < ε.

  1. batchMap_comp (BatchMapVJPAt.lean, shared with ConvNeXt's batched tie) — batchMap B (g ∘ f) = batchMap B g ∘ batchMap B f, the lemma the shape check needs and the reason the chain saves its activations stage by stage: the two spellings agree only up to finProdFinEquiv.symm_apply_apply, which is not rfl.
  2. The batched leaf ties. The patch-embed stage needs none (its backward is rfl); the other three are funext to one example, one rewrite of the per-example tie at that example's row (vitTowerBackK_eq_vjp, vitFinalLNBack_eq_vjp, vitHeadBack_eq_classifier_vjp), then rflbatchMapAux's slice and the lift's .backward row are the same term, as r34's pool endpoint found.
  3. vitKVB_has_vjp_at — the four-stage apex, three vjp_comp_diff_ats over the batched stage witnesses — and vitInputGradKB_eq_vitKVB_vjp, the tie: three leaf rewrites, then rfl.
  4. vitForwardKVB_eq_chain — the shape check: the four batched stages compose to batchMap B vitForwardKV, by vitForwardKV_eq_chain and three batchMap_comps — and vitInputGradKB_eq_batchMap_vitForwardKV_vjp, the tie carried to the committed GLOBAL witness batchMap_has_vjp (vitForwardKV …) through HasVJPAt.backward_unique_of_eq (BatchMapVJPAt.lean; batchMap_has_vjp is -transported, so its .backward does not reduce; uniqueness is the escape every whole-net tie in this repo takes), plus the ∑ pdiv reading.
  5. vitTinyInputGradB_eq_vitTiny_vjp — the capstone at ViT-Tiny's literal dims, B a binder.
noncomputable def Proofs.vitEmbedB_at (B ic H W patchSize N D : ) (W_conv : Kernel4 D ic patchSize patchSize) (b_conv cls_token : Vec D) (pos_embed : Mat (N + 1) D) (x : Vec (B * (ic * H * W))) :
HasVJPAt (StableHLO.batchMap B (patchEmbed_flat ic H W patchSize N D W_conv b_conv cls_token pos_embed)) x

The batched patch-embed witness at x: batchMap_has_vjp_at over the global per-example witness at each row.

Equations
  • One or more equations did not get rendered due to their size.
Instances For
    noncomputable def Proofs.vitTowerB_at (B Np1 heads d_head mlpDim : ) (ε : ) ( : 0 < ε) (k : ) (ps : Fin kBlockParamsV (heads * d_head) mlpDim) (v : Vec (B * (Np1 * (heads * d_head)))) :
    HasVJPAt (StableHLO.batchMap B (vitBodyKVFlat Np1 heads d_head mlpDim ε k ps)) v

    The batched tower witness at v.

    Equations
    • One or more equations did not get rendered due to their size.
    Instances For
      noncomputable def Proofs.vitLNB_at (B n D : ) (ε : ) ( : 0 < ε) (γF βF : Vec D) (v : Vec (B * (n * D))) :
      HasVJPAt (StableHLO.batchMap B fun (v : Vec (n * D)) => Mat.flatten fun (r : Fin n) => layerNormVec D ε γF βF (Mat.unflatten v r)) v

      The batched final-LayerNorm witness at v.

      Equations
      • One or more equations did not get rendered due to their size.
      Instances For
        noncomputable def Proofs.vitHeadB_at (B N D nClasses : ) (Wcls : Mat D nClasses) (bcls : Vec nClasses) (v : Vec (B * ((N + 1) * D))) :
        HasVJPAt (StableHLO.batchMap B (classifier_flat N D nClasses Wcls bcls)) v

        The batched classifier-head witness at v.

        Equations
        • One or more equations did not get rendered due to their size.
        Instances For
          theorem Proofs.vitTowerBackB_eq_vjp (B Np1 heads d_head mlpDim : ) (ε : ) ( : 0 < ε) (k : ) (ps : Fin kBlockParamsV (heads * d_head) mlpDim) (v : Vec (B * (Np1 * (heads * d_head)))) :
          StableHLO.batchMapAux B (vitTowerBackK Np1 heads d_head mlpDim ε k ps) v = (vitTowerB_at B Np1 heads d_head mlpDim ε k ps v).backward

          The batched tower tie. batchMapAux B of the depth-k tower backward at the batched saved input IS the lift's backward: one example, one rewrite of vitTowerBackK_eq_vjp at that example's row, rfl.

          theorem Proofs.vitLNBackB_eq_vjp (B n D : ) (ε : ) ( : 0 < ε) (γF βF : Vec D) (v : Vec (B * (n * D))) :
          StableHLO.batchMapAux B (rowLNVecFlatBack n D ε γF) v = (vitLNB_at B n D ε γF βF v).backward

          The batched final-LayerNorm tie. batchMapAux B of rowLNVecFlatBack at the batched tower output IS the lift's backward — vitFinalLNBack_eq_vjp at one example's row.

          theorem Proofs.vitHeadBackB_eq_vjp (B N D nClasses : ) (Wcls : Mat D nClasses) (bcls : Vec nClasses) (v : Vec (B * ((N + 1) * D))) :
          StableHLO.batchMap B (clsScatter N D) StableHLO.batchMap B (dense Wcls.transpose 0) = (vitHeadB_at B N D nClasses Wcls bcls v).backward

          The batched head tie. batchMap B of the CLS scatter after batchMap B of the free dense backward IS the lift's backward at any saved v (the head is linear): fuse the two lifts by batchMap_comp, then vitHeadBack_eq_classifier_vjp at one example's row.

          noncomputable def Proofs.vitKVB_has_vjp_at (B ic H W patchSize N mlpDim heads d_head nClasses k : ) (W_conv : Kernel4 (heads * d_head) ic patchSize patchSize) (b_conv cls_token : Vec (heads * d_head)) (pos_embed : Mat (N + 1) (heads * d_head)) (ε : ) ( : 0 < ε) (ps : Fin kBlockParamsV (heads * d_head) mlpDim) (γF βF : Vec (heads * d_head)) (Wcls : Mat (heads * d_head) nClasses) (bcls : Vec nClasses) (x : Vec (B * (ic * H * W))) :
          HasVJPAt (StableHLO.batchMap B (classifier_flat N (heads * d_head) nClasses Wcls bcls) (StableHLO.batchMap B fun (v : Vec ((N + 1) * (heads * d_head))) => Mat.flatten fun (r : Fin (N + 1)) => layerNormVec (heads * d_head) ε γF βF (Mat.unflatten v r)) StableHLO.batchMap B (vitBodyKVFlat (N + 1) heads d_head mlpDim ε k ps) StableHLO.batchMap B (patchEmbed_flat ic H W patchSize N (heads * d_head) W_conv b_conv cls_token pos_embed)) x

          The batched whole-net witness, four batched stages composed by vjp_comp_diff_at, each at the batched saved activation the chain uses (vitSavedPEB, vitSavedBodyB).

          Equations
          • One or more equations did not get rendered due to their size.
          Instances For
            theorem Proofs.vitInputGradKB_eq_vitKVB_vjp (B ic H W patchSize N mlpDim heads d_head nClasses k : ) (W_conv : Kernel4 (heads * d_head) ic patchSize patchSize) (b_conv cls_token : Vec (heads * d_head)) (pos_embed : Mat (N + 1) (heads * d_head)) (ε : ) ( : 0 < ε) (ps : Fin kBlockParamsV (heads * d_head) mlpDim) (γF βF : Vec (heads * d_head)) (Wcls : Mat (heads * d_head) nClasses) (bcls : Vec nClasses) (x : Vec (B * (ic * H * W))) :
            vitInputGradKB B ic H W patchSize N mlpDim heads d_head nClasses k W_conv b_conv cls_token pos_embed ε ps γF Wcls x = (vitKVB_has_vjp_at B ic H W patchSize N mlpDim heads d_head nClasses k W_conv b_conv cls_token pos_embed ε ps γF βF Wcls bcls x).backward

            ⭐⭐ THE BATCHED TIE. vitInputGradKB — the five-stage batched chain, every slot a lift of the per-example backward at the batched saved activation — IS the batched apex's backward. Three leaf rewrites (tower, final LN, head), then rfl: the patch-embed leaf is definitional.

            theorem Proofs.vitForwardKVB_eq_chain (B ic H W patchSize N mlpDim heads d_head nClasses k : ) (W_conv : Kernel4 (heads * d_head) ic patchSize patchSize) (b_conv cls_token : Vec (heads * d_head)) (pos_embed : Mat (N + 1) (heads * d_head)) (ε : ) (ps : Fin kBlockParamsV (heads * d_head) mlpDim) (γF βF : Vec (heads * d_head)) (Wcls : Mat (heads * d_head) nClasses) (bcls : Vec nClasses) :
            StableHLO.batchMap B (vitForwardKV ic H W patchSize N mlpDim heads d_head nClasses k W_conv b_conv cls_token pos_embed ε ps γF βF Wcls bcls) = StableHLO.batchMap B (classifier_flat N (heads * d_head) nClasses Wcls bcls) (StableHLO.batchMap B fun (v : Vec ((N + 1) * (heads * d_head))) => Mat.flatten fun (r : Fin (N + 1)) => layerNormVec (heads * d_head) ε γF βF (Mat.unflatten v r)) StableHLO.batchMap B (vitBodyKVFlat (N + 1) heads d_head mlpDim ε k ps) StableHLO.batchMap B (patchEmbed_flat ic H W patchSize N (heads * d_head) W_conv b_conv cls_token pos_embed)

            The shape check. The four batched stages the apex is stated at compose to batchMap B vitForwardKV, the committed per-example forward lifted whole: the per-example shape check vitForwardKV_eq_chain and three batchMap_comps.

            theorem Proofs.vitForwardKV_differentiable (ic H W patchSize N mlpDim heads d_head nClasses k : ) (W_conv : Kernel4 (heads * d_head) ic patchSize patchSize) (b_conv cls_token : Vec (heads * d_head)) (pos_embed : Mat (N + 1) (heads * d_head)) (ε : ) ( : 0 < ε) (ps : Fin kBlockParamsV (heads * d_head) mlpDim) (γF βF : Vec (heads * d_head)) (Wcls : Mat (heads * d_head) nClasses) (bcls : Vec nClasses) :
            Differentiable (vitForwardKV ic H W patchSize N mlpDim heads d_head nClasses k W_conv b_conv cls_token pos_embed ε ps γF βF Wcls bcls)

            vitForwardKV is differentiable everywhere (only 0 < ε): the four stage lemmas composed.

            theorem Proofs.vitInputGradKB_eq_batchMap_vitForwardKV_vjp (B ic H W patchSize N mlpDim heads d_head nClasses k : ) (W_conv : Kernel4 (heads * d_head) ic patchSize patchSize) (b_conv cls_token : Vec (heads * d_head)) (pos_embed : Mat (N + 1) (heads * d_head)) (ε : ) ( : 0 < ε) (ps : Fin kBlockParamsV (heads * d_head) mlpDim) (γF βF : Vec (heads * d_head)) (Wcls : Mat (heads * d_head) nClasses) (bcls : Vec nClasses) (x : Vec (B * (ic * H * W))) :
            vitInputGradKB B ic H W patchSize N mlpDim heads d_head nClasses k W_conv b_conv cls_token pos_embed ε ps γF Wcls x = (batchMap_has_vjp (vitForwardKV ic H W patchSize N mlpDim heads d_head nClasses k W_conv b_conv cls_token pos_embed ε ps γF βF Wcls bcls) (vitForwardKV_has_vjp ic H W patchSize N mlpDim heads d_head nClasses k W_conv b_conv cls_token pos_embed ε ps γF βF Wcls bcls) ).backward x

            ⭐⭐ THE APEX, at the committed batched witness. vitInputGradKB IS (batchMap_has_vjp (vitForwardKV …) …).backward x — the certified gradient of the per-example net lifted whole over B examples. Carried from the chain-shaped apex by HasVJPAt.backward_unique_of_eq along the shape check.

            theorem Proofs.vitInputGradKB_correct (B ic H W patchSize N mlpDim heads d_head nClasses k : ) (W_conv : Kernel4 (heads * d_head) ic patchSize patchSize) (b_conv cls_token : Vec (heads * d_head)) (pos_embed : Mat (N + 1) (heads * d_head)) (ε : ) ( : 0 < ε) (ps : Fin kBlockParamsV (heads * d_head) mlpDim) (γF βF : Vec (heads * d_head)) (Wcls : Mat (heads * d_head) nClasses) (bcls : Vec nClasses) (x : Vec (B * (ic * H * W))) (dy : Vec (B * nClasses)) (i : Fin (B * (ic * H * W))) :
            vitInputGradKB B ic H W patchSize N mlpDim heads d_head nClasses k W_conv b_conv cls_token pos_embed ε ps γF Wcls x dy i = j : Fin (B * nClasses), pdiv (StableHLO.batchMap B (vitForwardKV ic H W patchSize N mlpDim heads d_head nClasses k W_conv b_conv cls_token pos_embed ε ps γF βF Wcls bcls)) x i j * dy j

            The batched apex, read as the Jacobian. vitInputGradKB is the pdiv-contracted Jacobian transpose of batchMap B vitForwardKV, at EVERY batch and EVERY cotangent. Only 0 < ε.

            theorem Proofs.vitTinyInputGradB_eq_vitTiny_vjp (B : ) (W_conv : Kernel4 (3 * 64) 3 16 16) (b_conv cls_token : Vec (3 * 64)) (pos_embed : Mat (196 + 1) (3 * 64)) (ε : ) ( : 0 < ε) (ps : Fin 12BlockParamsV (3 * 64) 768) (γF βF : Vec (3 * 64)) (Wcls : Mat (3 * 64) 10) (bcls : Vec 10) (x : Vec (B * (3 * 224 * 224))) :
            vitInputGradKB B 3 224 224 16 196 768 3 64 10 12 W_conv b_conv cls_token pos_embed ε ps γF Wcls x = (batchMap_has_vjp (vitForwardKV 3 224 224 16 196 768 3 64 10 12 W_conv b_conv cls_token pos_embed ε ps γF βF Wcls bcls) (vitForwardKV_has_vjp 3 224 224 16 196 768 3 64 10 12 W_conv b_conv cls_token pos_embed ε ps γF βF Wcls bcls) ).backward x

            ⭐⭐ ViT-Tiny's BATCHED whole-net backward tie — tier T6 at the paper net and the shipped index. vitInputGradKB_eq_batchMap_vitForwardKV_vjp at the exact vitTiny spec (3×224×224, 16×16 patches, 196 + CLS tokens, D = 192 = 3 × 64, MLP 768, 12 distinct blocks, vector-[D] LayerNorm, 10 classes), at a variable batch B — 128 or 512 per device in the shipped vitin_* artifacts, and neither number appears here. ViT's entry in the batched T6 column beside r34InputGradB_eq_r34B_full_vjp and mnv2InputGradB_eq_mobilenetv2B_full_vjp.