Spec helpers: parameter layouts and counts (Layer.paramSlots, NetSpec.totalParams), feature
queries the codegen gates helper emission on, architecture display, and validation.
Param shapes owned by a .fpnDetect layer, in the ONE canonical order that
emitTrainStepSig, the optimizer, the grad-clip list, the backward's
grad-binding, NetSpec.paramShapes and heInitLayer must all agree on:
Wn3 [oc,c3], Wn4 [oc,c4], Wn5 [oc,c5] -- neck laterals
(W [oc,oc,3,3], b [oc]) × tower, per level P3 → P4 → P5
Wh3, Wh4, Wh5 [A·15, oc] -- head convs
bh3, bh4, bh5 [A·15] -- head biases, LAST
The biases MUST stay last: NetSpec.applyDetPriorBias installs the RetinaNet
prior by splicing the final 3·A·15 floats of the buffer.
Defined here (rather than in the codegen) because SpecHelpers and MlirCodegen both need it and neither should own it — a mismatch between any two of these consumers is a silent parameter-aliasing bug, not a compile error.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Squeeze width of an .mbConv block's SE: a quarter of the block's input channels
(timm's se_ratio taken over the block input, not the expanded width). Every
parameter-layout writer and emitter reads it from here.
Equations
- mbConvSeMid blockIc = Nat.max 1 (blockIc / 4)
Instances For
How one parameter tensor is initialized. A random draw consumes one seed; a constant does not.
zeroSeeded is zeros that still consume a seed — .timeCondAdd's weight, which starts as a
no-op but keeps the seed stream every later layer draws from.
- he (fanIn : Nat) : ParamInit
- normal (σ : Float) : ParamInit
- const (v : Float) : ParamInit
- zeroSeeded : ParamInit
Instances For
Equations
- instInhabitedParamInit = { default := instInhabitedParamInit.default }
One trainable tensor of a layer: its shape, its initialization, and its name in the emitted
MLIR signatures (W weight — also a LayerNorm's γ, b bias — also β, g / bt a conv's BN
γ / β). A W opens a group: the group shares one index, %W3, %g3, %bt3. decay: whether
the optimizer applies weight decay — matrices and kernels do; vectors (biases, norm scales) and
positional embeddings do not.
Instances For
Equations
- instInhabitedParamSlot = { default := instInhabitedParamSlot.default }
The one writer of a trainable layer's parameter layout — the tensors in signature order,
each with its initialization. NetSpec.paramShapes (the packed [θ|m|v] blob), heInitLayer
(the initial weights) and Layer.nParams (hence totalParams) all read it, so they cannot
disagree. none for layers the codegen does not train (their count is nParamsUntrained).
The MLIR signatures (MlirCodegen.fwdSigParts, emitTrainStepSig) render it too.
Equations
- One or more equations did not get rendered due to their size.
- (Layer.conv2d ic oc k pad act).paramSlots = some (slotsConvB✝ oc ic k)
- (Layer.convBn ic oc k stride pad).paramSlots = some (slotsConvBn✝ oc ic k)
- (Layer.dense fi fo act).paramSlots = some (slotsDense✝ fi fo)
- (Layer.layerNorm d).paramSlots = some (slotsLN✝ d)
- (Layer.convNextDownsample ic oc _norm).paramSlots = some (slotsLN✝ ic ++ slotsConvB✝ oc ic 2)
- (Layer.convNextStem ic oc p).paramSlots = some (slotsConvB✝ oc ic p ++ slotsLN✝ oc)
- (Layer.unetDown ic oc).paramSlots = some (slotsConvBn✝ oc ic 3 ++ slotsConvBn✝ oc oc 3)
- (Layer.unetUp ic oc).paramSlots = some (slotsConvBn✝ oc (ic + oc) 3 ++ slotsConvBn✝ oc oc 3)
- (Layer.lmHead d v seqLen).paramSlots = some (slotsDense✝ d v)
- x✝.paramSlots = none
Instances For
The displayed parameter count of a layer the codegen does NOT train (Layer.paramSlots is
none): the Bestiary's architectures, some counted approximately (.mambaBlock).
Equations
- One or more equations did not get rendered due to their size.
- (Layer.separableConv ic oc stride).nParamsUntrained = ic * 9 + 2 * ic + (oc * ic + 2 * oc)
- (Layer.fireModule ic sq e1 e3).nParamsUntrained = sq * ic + 2 * sq + (e1 * sq + 2 * e1) + (e3 * sq * 9 + 2 * e3)
- (Layer.mambaBlock dim stateSize expand nBlocks).nParamsUntrained = nBlocks * (3 * expand * dim * dim + 3 * expand * dim * stateSize + 5 * expand * dim + dim)
- (Layer.patchMerging inDim outDim).nParamsUntrained = 2 * 4 * inDim + (4 * inDim * outDim + outDim)
- (Layer.detrHeads dim nClasses).nParamsUntrained = dim * (nClasses + 1) + (nClasses + 1) + (dim * dim + dim) + (dim * dim + dim) + (dim * 4 + 4)
- (Layer.shuffleV2Block ic oc nUnits).nParamsUntrained = 12 * ic + ic * oc + 11 * oc + oc * oc / 4 + (nUnits - 1) * (oc * oc / 2 + 19 * oc / 2)
- (Layer.asppModule ic oc).nParamsUntrained = ic * oc + oc + 2 * oc + 3 * (9 * ic * oc + oc + 2 * oc) + (ic * oc + oc + 2 * oc) + (5 * oc * oc + oc + 2 * oc)
- (Layer.fpnModule c2 c3 c4 c5 target).nParamsUntrained = (c2 + c3 + c4 + c5) * target + 4 * target + 4 * (9 * target * target + target)
- (Layer.structureModule singleChannels pairChannels nBlocks).nParamsUntrained = 6 * singleChannels * singleChannels + singleChannels * pairChannels / 4
- (Layer.waveNetBlock residualCh skipCh nLayers).nParamsUntrained = nLayers * (4 * residualCh * residualCh + 2 * residualCh + residualCh * residualCh + residualCh + residualCh * skipCh + skipCh)
- (Layer.positionalEncoding _inputDim _numFrequencies).nParamsUntrained = 0
- (Layer.darknetBlock channels nBlocks).nParamsUntrained = nBlocks * (5 * channels * channels + 3 * channels)
- (Layer.transitionLayer ic oc).nParamsUntrained = 2 * ic + ic * oc + oc
- x✝.nParamsUntrained = 0
Instances For
Equations
- One or more equations did not get rendered due to their size.
Instances For
Equations
- s.totalParams = List.foldl (fun (acc : Nat) (l : Layer) => acc + l.nParams) 0 s.layers
Instances For
Equations
- One or more equations did not get rendered due to their size.
Instances For
Equations
- One or more equations did not get rendered due to their size.
Instances For
Equations
- One or more equations did not get rendered due to their size.
Instances For
Equations
- s.hasBottleneck = s.layers.any fun (x : Layer) => match x with | Layer.bottleneckBlock ic oc nBlocks firstStride => true | x => false
Instances For
Equations
- One or more equations did not get rendered due to their size.
Instances For
Equations
- s.hasInvertedResidual = s.layers.any fun (x : Layer) => match x with | Layer.invertedResidual ic oc expand stride nBlocks => true | x => false
Instances For
Equations
- One or more equations did not get rendered due to their size.
Instances For
Equations
- s.hasGlobalAvgPool = s.layers.any fun (x : Layer) => match x with | Layer.globalAvgPool => true | x => false
Instances For
Equations
- s.hasTransformer = s.layers.any fun (x : Layer) => match x with | Layer.transformerEncoder dim heads mlpDim nBlocks causalMask keepSequence flashAttn rope => true | x => false
Instances For
Equations
- s.numClasses = match s.layers.getLast? with | some (Layer.dense fanIn fo act) => fo | some (Layer.lmHead dModel v seqLen) => v | x => 0
Instances For
Equations
- One or more equations did not get rendered due to their size.
Instances For
Output channels of a layer. Returns 0 for structural layers (pool, flatten, GAP).
Equations
- (Layer.conv2d ic oc kSize pad act).outChannels = oc
- (Layer.convBn ic oc kSize stride pad).outChannels = oc
- (Layer.layerNorm d).outChannels = d
- (Layer.dense fanIn fo act).outChannels = fo
- (Layer.residualBlock ic oc nBlocks firstStride).outChannels = oc
- (Layer.bottleneckBlock ic oc nBlocks firstStride).outChannels = oc
- (Layer.separableConv ic oc stride).outChannels = oc
- (Layer.invertedResidual ic oc expand stride nBlocks).outChannels = oc
- (Layer.mbConv ic oc expand kSize stride nBlocks useSE act).outChannels = oc
- (Layer.mbConvV3 ic oc expandCh kSize stride useSE act).outChannels = oc
- (Layer.fusedMbConv ic oc expand kSize stride nBlocks useSE act).outChannels = oc
- (Layer.uib ic oc expand stride preDWk postDWk).outChannels = oc
- (Layer.fireModule ic squeeze e1 e3).outChannels = e1 + e3
- (Layer.patchEmbed ic dim patchSize nPatches).outChannels = dim
- (Layer.transformerEncoder dim heads mlpDim nBlocks causalMask keepSequence flashAttn rope).outChannels = dim
- (Layer.mambaBlock dim stateSize expand nBlocks).outChannels = dim
- (Layer.swinStage dim heads mlpDim windowSize nBlocks).outChannels = dim
- (Layer.patchMerging inDim outDim).outChannels = outDim
- (Layer.unetDown ic oc).outChannels = oc
- (Layer.unetUp ic oc).outChannels = oc
- (Layer.transformerDecoder dim heads mlpDim nBlocks nQueries).outChannels = dim
- (Layer.detrHeads dim nClasses).outChannels = nClasses + 1
- (Layer.shuffleBlock ic oc groups nUnits).outChannels = oc
- (Layer.shuffleV2Block ic oc nUnits).outChannels = oc
- (Layer.asppModule ic oc).outChannels = oc
- (Layer.fpnModule c2 c3 c4 c5 target).outChannels = target
- (Layer.fpnDetect oc c3 c4 c5 g5 A tower).outChannels = 315 * A * g5 * g5
- (Layer.evoformerBlock msaCh pairChannels nBlocks).outChannels = msaCh
- (Layer.structureModule sCh pairChannels nBlocks).outChannels = sCh
- (Layer.mobileVitBlock ic dim heads mlpDim nTxBlocks).outChannels = ic
- (Layer.convNextStage c nBlocks norm act).outChannels = c
- (Layer.convNextDownsample ic oc norm).outChannels = oc
- (Layer.convNextStem ic oc patch).outChannels = oc
- (Layer.waveNetBlock residualCh skipCh nLayers).outChannels = skipCh
- (Layer.positionalEncoding inputDim numFreq).outChannels = inputDim * 2 * numFreq
- (Layer.nerfMLP encodedPosDim encodedDirDim hiddenDim).outChannels = 4
- (Layer.darknetBlock c nBlocks).outChannels = c
- (Layer.cspBlock ic oc nBlocks).outChannels = oc
- (Layer.inceptionModule ic b1 b2reduce b2 b3reduce b3 b4).outChannels = b1 + b2 + b3 + b4
- (Layer.denseBlock ic gr nLayers).outChannels = ic + nLayers * gr
- (Layer.transitionLayer ic oc).outChannels = oc
- (Layer.tokenPositionEmbed vocabSize seqLen d idsInput gather).outChannels = d
- (Layer.lmHead dModel v t).outChannels = t * v
- (Layer.timeCondAdd c nFreq).outChannels = c
- (Layer.pairTile seqLen inDim c p).outChannels = c + p
- Layer.spatialFlatten.outChannels = 0
- (Layer.spatialUnflatten c height width).outChannels = c
- x✝.outChannels = 0
Instances For
Input channels expected by a layer. Returns 0 for layers that accept any input.
Equations
- (Layer.conv2d ic oc kSize pad act).inChannels = ic
- (Layer.convBn ic oc kSize stride pad).inChannels = ic
- (Layer.layerNorm d).inChannels = d
- (Layer.dense fi fanOut act).inChannels = fi
- (Layer.residualBlock ic oc nBlocks firstStride).inChannels = ic
- (Layer.bottleneckBlock ic oc nBlocks firstStride).inChannels = ic
- (Layer.separableConv ic oc stride).inChannels = ic
- (Layer.invertedResidual ic oc expand stride nBlocks).inChannels = ic
- (Layer.mbConv ic oc expand kSize stride nBlocks useSE act).inChannels = ic
- (Layer.mbConvV3 ic oc expandCh kSize stride useSE act).inChannels = ic
- (Layer.fusedMbConv ic oc expand kSize stride nBlocks useSE act).inChannels = ic
- (Layer.uib ic oc expand stride preDWk postDWk).inChannels = ic
- (Layer.fireModule ic squeeze expand1x1 expand3x3).inChannels = ic
- (Layer.patchEmbed ic dim patchSize nPatches).inChannels = ic
- (Layer.transformerEncoder dim heads mlpDim nBlocks causalMask keepSequence flashAttn rope).inChannels = dim
- (Layer.mambaBlock dim stateSize expand nBlocks).inChannels = dim
- (Layer.swinStage dim heads mlpDim windowSize nBlocks).inChannels = dim
- (Layer.patchMerging inDim outDim).inChannels = inDim
- (Layer.unetDown ic oc).inChannels = ic
- (Layer.unetUp ic oc).inChannels = ic
- (Layer.transformerDecoder dim heads mlpDim nBlocks nQueries).inChannels = dim
- (Layer.detrHeads dim nClasses).inChannels = dim
- (Layer.shuffleBlock ic oc groups nUnits).inChannels = ic
- (Layer.shuffleV2Block ic oc nUnits).inChannels = ic
- (Layer.asppModule ic oc).inChannels = ic
- (Layer.fpnModule c2 c3 c4 c5 target).inChannels = c5
- (Layer.fpnDetect oc c3 c4 c5 g5 A tower).inChannels = c5
- (Layer.evoformerBlock msaCh pairChannels nBlocks).inChannels = msaCh
- (Layer.structureModule sCh pairChannels nBlocks).inChannels = sCh
- (Layer.mobileVitBlock ic dim heads mlpDim nTxBlocks).inChannels = ic
- (Layer.convNextStage c nBlocks norm act).inChannels = c
- (Layer.convNextDownsample ic oc norm).inChannels = ic
- (Layer.convNextStem ic oc patch).inChannels = ic
- (Layer.waveNetBlock residualCh skipCh nLayers).inChannels = residualCh
- (Layer.positionalEncoding inputDim numFrequencies).inChannels = inputDim
- (Layer.nerfMLP encodedPosDim encodedDirDim hiddenDim).inChannels = encodedPosDim
- (Layer.darknetBlock c nBlocks).inChannels = c
- (Layer.cspBlock ic oc nBlocks).inChannels = ic
- (Layer.inceptionModule ic b1out b2reduce b2out b3reduce b3out b4out).inChannels = ic
- (Layer.denseBlock ic growthRate nLayers).inChannels = ic
- (Layer.transitionLayer ic oc).inChannels = ic
- (Layer.tokenPositionEmbed v t dModel ids gather posEmb).inChannels = if (ids || gather) = true then t else v * t
- (Layer.lmHead d vocabSize seqLen).inChannels = d
- Layer.spatialFlatten.inChannels = 0
- (Layer.spatialUnflatten c height width).inChannels = c
- x✝.inChannels = 0
Instances For
Validate that channel dimensions chain correctly through the spec.
Returns none if valid, or some errorMessage describing the first mismatch.
Equations
- One or more equations did not get rendered due to their size.
Instances For
Which quantity the summary's first line reports. Bestiary entries cover images, token streams, residue chains and raw audio, and a few (NeRF) have no meaningful input extent.
- image : SummarySize
- tokens : SummarySize
- residues : SummarySize
- samples : SummarySize
- omitted : SummarySize
Instances For
Equations
- instReprSummarySize = { reprPrec := instReprSummarySize.repr }
Equations
- instReprSummarySize.repr SummarySize.image prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "SummarySize.image")).group prec✝
- instReprSummarySize.repr SummarySize.tokens prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "SummarySize.tokens")).group prec✝
- instReprSummarySize.repr SummarySize.residues prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "SummarySize.residues")).group prec✝
- instReprSummarySize.repr SummarySize.samples prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "SummarySize.samples")).group prec✝
- instReprSummarySize.repr SummarySize.omitted prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "SummarySize.omitted")).group prec✝
Instances For
Equations
- instReprParamUnit = { reprPrec := instReprParamUnit.repr }
Equations
- instReprParamUnit.repr ParamUnit.millions prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "ParamUnit.millions")).group prec✝
- instReprParamUnit.repr ParamUnit.thousands prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "ParamUnit.thousands")).group prec✝
- instReprParamUnit.repr ParamUnit.bare prec✝ = Repr.addAppParen (Std.Format.nest (if prec✝ ≥ 1024 then 1 else 2) (Std.Format.text "ParamUnit.bare")).group prec✝
Instances For
The shared Bestiary summary block, printed by every Bestiary/*.lean entry.
The first line's label, the params abbreviation (unit) and an optional parenthetical on
the validate : OK line (okNote) are the parts that vary.
The ── {name} ── and params<pad>: {n} lines are parsed, not decoration:
tests/test_bestiary_params.py parses exactly those two shapes out of each binary's stdout
to pin every variant against tests/bestiary_params.yml (the CI drift guard in
.github/workflows/certs.yml). Keep both shapes if you touch this.
Equations
- One or more equations did not get rendered due to their size.