Documentation

LeanMlir.Proofs.Foundation.DropSites

Optional drop sites — stochastic depth and classifier dropout as Option binders #

A verified render carries stochastic depth and classifier dropout as HOST-DRAWN mask inputs (Proofs.Training.DropPath: the keep probability is folded into the mask, the op is a diagonal scale, its VJP is itself). Whether a site is RENDERED is a renderer flag (sd, cd), so a graph or chain statement that covers both the drop-free artifact and the *drop* / *do* one takes the site as an Option: none emits no node and denotes the identity, some s emits the dropPathB / dropoutB node at the mask s. The forward graphs state their sites this way (LeanMlir.Proofs.Nets.EfficientNet.EfficientNetFullB0Drop's efficientnetFwdGraphBFullDrop); the step, sync and loss-gradient ties thread the same binder through their cotangent chains (Proofs.MobileNetV2TieB's classifier dropout first), so the drop-free tie is the none instance verbatim and the some instance is the artifact that trained.

noncomputable def Proofs.dropPathOpt (N n : ℕ) :
Option (Vec N) → Vec (N * n) → Vec (N * n)

Drop-path at a site that may be absent: none is the identity.

Equations
Instances For
    noncomputable def Proofs.dropoutOpt {m : ℕ} :
    Option (Vec m) → Vec m → Vec m

    Dropout at a site that may be absent: none is the identity.

    Equations
    Instances For
      @[simp]
      @[simp]
      theorem Proofs.dropPathOpt_some (N n : ℕ) (s : Vec N) :
      dropPathOpt N n (some s) = dropPath N n s
      @[simp]
      theorem Proofs.dropoutOpt_some {m : ℕ} (mk : Vec m) :
      theorem Proofs.dropPathOpt_ones (N n : ℕ) :
      dropPathOpt N n (some fun (x : Fin N) => 1) = id

      At the all-ones scale drop-path is the identity — the masks the driver passes at eval.

      theorem Proofs.dropoutOpt_ones {m : ℕ} :
      dropoutOpt (some fun (x : Fin m) => 1) = id

      At the all-ones mask dropout is the identity.

      noncomputable def Proofs.dropoutOptHasVJP {m : ℕ} (cd : Option (Vec m)) :

      The backward is the forward at the same mask, at either value — stated as the witness's backward field, not derived by cases, so (dropoutOptHasVJP cd).backward x dy unfolds to dropoutOpt cd dy for a symbolic cd.

      Equations
      Instances For
        theorem Proofs.dropoutOpt_vjp_is_self {m : ℕ} (cd : Option (Vec m)) (x dy : Vec m) :
        noncomputable def Proofs.dropPathOptHasVJP (N n : ℕ) (sd : Option (Vec N)) :

        dropoutOptHasVJP one rank down — the per-example scale.

        Equations
        Instances For
          theorem Proofs.dropPathOpt_vjp_is_self (N n : ℕ) (sd : Option (Vec N)) (x dy : Vec (N * n)) :
          (dropPathOptHasVJP N n sd).backward x dy = dropPathOpt N n sd dy
          theorem Proofs.dropoutOpt_smul {m : ℕ} (cd : Option (Vec m)) :

          The site is linear in what flows through it.

          theorem Proofs.dropoutOpt_shard {R N a : ℕ} (cd : Option (Vec (R * N * a))) (X : Vec (R * N * a)) (r : Fin R) :
          dropoutOpt (Option.map (fun (M : Vec (R * N * a)) => batchShard R N a M r) cd) (batchShard R N a X r) = batchShard R N a (dropoutOpt cd X) r

          Replica r applying ITS shard of the mask to ITS shard of a value is shard r of the global site — batchShard_zipWith at the site's multiply.

          noncomputable def Proofs.exampleShard (R N : ℕ) (S : Vec (R * N)) (r : Fin R) :
          Vec N

          Replica r's block of a per-example scalar family laid out [R·N]: example (r, n) of the global batch is example n of shard r — batchShard's cut at width one, for the stochastic-depth masks (Vec N per site, not Vec (N * 1)). The DP renders' per-replica %dp<i> inputs are these.

          Equations
          Instances For
            theorem Proofs.dropPathOpt_smul (N n : ℕ) (sd : Option (Vec N)) :

            The per-example site is linear in what flows through it.

            theorem Proofs.dropPathOpt_shard {R N n : ℕ} (sd : Option (Vec (R * N))) (X : Vec (R * N * n)) (r : Fin R) :
            dropPathOpt N n (Option.map (fun (S : Vec (R * N)) => exampleShard R N S r) sd) (batchShard R N n X r) = batchShard R N n (dropPathOpt (R * N) n sd X) r

            Replica r scaling ITS shard of a value by ITS shard of the per-example mask is shard r of the global site: the example a cell belongs to is the same on both sides of the cut (Equiv.symm_apply_apply at batchShard's index).

            theorem Proofs.dropPath_shard {R N n : ℕ} (S : Vec (R * N)) (X : Vec (R * N * n)) (r : Fin R) :
            dropPath N n (exampleShard R N S r) (batchShard R N n X r) = batchShard R N n (dropPath (R * N) n S X) r

            dropPathOpt_shard at a rendered site.

            noncomputable def Proofs.dropPathOptFam (R N n : ℕ) (sd : Option (Vec (R * N))) (dys : Fin R → Vec (N * n)) :
            Fin R → Vec (N * n)

            The replica family a data-parallel chain feeds a drop site: replica r's cotangent through replica r's shard of the mask.

            Equations
            Instances For
              theorem Proofs.dropPathOptFam_shard {R N n : ℕ} (sd : Option (Vec (R * N))) (dys : Fin R → Vec (N * n)) (DY : Vec (R * N * n)) (hdys : ∀ (r : Fin R), dys r = batchShard R N n DY r) (r : Fin R) :
              dropPathOptFam R N n sd dys r = batchShard R N n (dropPathOpt (R * N) n sd DY) r
              theorem Proofs.dropPathOptFam_scaled {R N n : ℕ} (sd : Option (Vec (R * N))) (dys : Fin R → Vec (N * n)) (DY : Vec (R * N * n)) (hdys : ∀ (r : Fin R), dys r = batchShard R N n (fun (i : Fin (R * N * n)) => ↑R * DY i) r) (r : Fin R) :
              dropPathOptFam R N n sd dys r = batchShard R N n (fun (i : Fin (R * N * n)) => ↑R * dropPathOpt (R * N) n sd DY i) r

              The scaled-shard invariant passes a drop site: replicas at R × the shards of a global cotangent hand the branch R × the shards of the global branch cotangent.

              def Proofs.StableHLO.dropPathOptG (mN : String) {N n : ℕ} :
              Option (Vec N) → SHlo (N * n) → SHlo (N * n)

              A dropPathB node when the site is rendered, nothing otherwise.

              Equations
              Instances For
                theorem Proofs.StableHLO.den_dropPathOptG (mN : String) {N n : ℕ} (s : Option (Vec N)) (e : SHlo (N * n)) :
                den (dropPathOptG mN s e) = dropPathOpt N n s (den e)
                def Proofs.StableHLO.dropoutOptG (mN : String) {N n : ℕ} :
                Option (Vec (N * n)) → SHlo (N * n) → SHlo (N * n)

                A dropoutB node when the site is rendered, nothing otherwise.

                Equations
                Instances For
                  theorem Proofs.StableHLO.den_dropoutOptG (mN : String) {N n : ℕ} (m : Option (Vec (N * n))) (e : SHlo (N * n)) :
                  den (dropoutOptG mN m e) = dropoutOpt m (den e)