Mobilint IR Operation Formats¶
This page describes the behavior, inputs, attributes, and outputs of Mobilint IR operations. It is a format reference: an operation appearing here does not by itself establish NPU or CPU Offload support. See NPU Operator Support or CPU Offloading for execution scope.
Reading the operation definitions¶
- Four-dimensional activations use NHWC order: batch (
N), height (H), width (W), and channels (C). The batch size is 1 in the shapes below. - Inputs and outputs use positional names such as
in[0]andout[0]. - A
Weightattribute stores a tensor index (TensorIndex), not tensor values. It refers to the subgraph tensor table, which records the constant tensor's name, dtype, shape, and storage location.-1means no tensor; omitted optional tensor attributes use this value. - In attribute tables, required means the attribute must be set and optional means it may be omitted.
Detailed operation definitions¶
Adding¶
Element-wise sum of two activations.
Adds two activations element by element.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
in[1] |
broadcastable to in[0] | Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
broadcast(in[0], in[1]) |
Output activation. |
AddingConstant¶
Adds a constant tensor to an activation.
Adds a constant stored in the model to one activation. The constant may be a scalar, a per-channel vector or a full tensor.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
constant |
Weight |
required | Constant operand. Broadcasts against in[0]. |
is_const_first |
bool |
false |
true if the constant is the left operand. |
shape |
int32[] |
[] |
Shape of the constant operand before the stored constant was reduced (e.g. to a per-channel vector). The output shape is broadcast(shape(in[0]), shape). The default [] means the tensor's shape was not recorded; the field is optional. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
broadcast(in[0], shape) |
Output activation. |
Batchnorm¶
Per-channel normalization with stored mean and variance, then scale and shift.
Inference-form batch normalization. Each channel is normalized with the constant moving_mean and moving_var stored in the model, then scaled by gamma and shifted by beta.
out[0][n, h, w, c] = (in[0][n, h, w, c] - moving_mean[c]) / sqrt(moving_var[c] + epsilon)
* gamma[c] + beta[c]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
gamma |
Weight |
required | Per-channel scale. |
beta |
Weight |
required | Per-channel shift. |
moving_mean |
Weight |
required | Per-channel mean subtracted from the input. |
moving_var |
Weight |
required | Per-channel variance the input is divided by (after adding epsilon and taking the square root). |
epsilon |
Weight |
required | Scalar added to the variance for numerical stability. |
channel_last |
bool |
true |
Must be true; the NPU uses the NHWC layout. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Celu¶
Continuously differentiable ELU with parameter alpha.
Passes positive inputs through and maps negative inputs to an exponential that saturates at -alpha.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
alpha |
float32 |
required | Saturation level of the negative branch and divisor of the exponent. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Clip¶
Clamps each element to [clip_min_value, clip_max_value].
Limits every element to the given bounds. Either bound may be absent, and that side is then left unlimited.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
clip_max_value |
float32 |
optional | Upper bound. Absent means no upper bound. |
clip_min_value |
float32 |
optional | Lower bound. Absent means no lower bound. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Concatenate¶
Joins activations along one axis.
Concatenates in[0..K-1], in operand order, along axis. All operands have the same rank and agree on every other axis.
out[0] = concat(in[0], in[1], ..., in[K-1]) along axis
out[0].shape[axis] = in[0].shape[axis] + ... + in[K-1].shape[axis]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0..K-1] |
equal except along axis | Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
original_key |
str |
optional | Deprecated and no longer used. The key is always present, but its value is always None. |
axis |
int32 |
optional | Axis to join along; negative values count from the last axis. It indexes the tensor as laid out, so 3 is the channel axis of an NHWC activation. |
original_axis |
int32 |
optional | Deprecated and no longer used. The key is always present, but its value is always None. |
original_inputs |
int32[] |
optional | Deprecated and no longer used. The key is always present, but its value is always None. |
input_index |
int32[] |
optional | Always [0, 1, ..., K-1]. |
input_names |
str[] |
optional | Always the names of this operation's input activations. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
in[0] with the axis extent summed over operands | Output activation. |
Convolution¶
2-D convolution of an NHWC activation with a constant filter.
Slides weight over the spatial axes of in[0] with the given stride, dilation and zero padding, and adds bias. The weight is stored output-channel first, as [out_channels, in_channels / group_number, h_kernel, w_kernel].
G = group_number, Cg = in_channels / G, Og = out_channels / G (// : integer division)
xp = zero_pad(in[0], top=north_padding, bottom=south_padding,
left=west_padding, right=east_padding)
out[0][n, y, x, o] = bias[o] + sum_{ky, kx, c < Cg}
weight[o, c, ky, kx]
* xp[n, y*h_stride + ky*h_dilation, x*w_stride + kx*w_dilation, (o // Og)*Cg + c]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, in_channels] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
in_channels |
int32 |
required | Number of input channels. |
out_channels |
int32 |
required | Number of output channels. |
h_kernel |
int32 |
required | Kernel height. |
w_kernel |
int32 |
required | Kernel width. |
h_stride |
int32 |
required | Stride along H. |
w_stride |
int32 |
required | Stride along W. |
h_dilation |
int32 |
required | Dilation along H. |
w_dilation |
int32 |
required | Dilation along W. |
west_padding |
int32 |
required | Zero padding at the start of W (left). |
east_padding |
int32 |
required | Zero padding at the end of W (right). |
north_padding |
int32 |
required | Zero padding at the start of H (top). |
south_padding |
int32 |
required | Zero padding at the end of H (bottom). |
padding_list |
int32[] |
[] |
The original 2-D padding as [top, left, bottom, right], kept for reference. The computation uses the four *_padding attributes. |
group_number |
int32 |
required | Number of channel groups; must be 1. A convolution with more than one group must be a GroupConvolution or DepthwiseConvolution. |
weight |
Weight |
required | Filter, [out_channels, in_channels / group_number, h_kernel, w_kernel]. |
bias |
Weight |
optional | Bias, [out_channels]. Absent means zero. |
channel_last |
bool |
true |
Must be true; the NPU uses the NHWC layout. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[1, H_out, W_out, out_channels] |
Output activation. |
Cos¶
Element-wise cosine.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
DepthwiseConvolution¶
Depthwise 2-D convolution; each input channel has its own filters.
A Convolution with one group per input channel (group_number = in_channels). Each input channel is convolved with its own M = out_channels / in_channels filters and is never mixed with other channels; output channel o reads input channel o / M. The weight is stored as [out_channels, 1, h_kernel, w_kernel].
M = out_channels / in_channels
xp = zero_pad(in[0], top=north_padding, bottom=south_padding,
left=west_padding, right=east_padding)
out[0][n, y, x, o] = bias[o] + sum_{ky, kx}
weight[o, 0, ky, kx]
* xp[n, y*h_stride + ky*h_dilation, x*w_stride + kx*w_dilation, o / M]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, in_channels] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
in_channels |
int32 |
required | Number of input channels. |
out_channels |
int32 |
required | Number of output channels, in_channels * M. |
h_kernel |
int32 |
required | Kernel height. |
w_kernel |
int32 |
required | Kernel width. |
h_stride |
int32 |
required | Stride along H. |
w_stride |
int32 |
required | Stride along W. |
h_dilation |
int32 |
required | Dilation along H. |
w_dilation |
int32 |
required | Dilation along W. |
west_padding |
int32 |
required | Zero padding at the start of W (left). |
east_padding |
int32 |
required | Zero padding at the end of W (right). |
north_padding |
int32 |
required | Zero padding at the start of H (top). |
south_padding |
int32 |
required | Zero padding at the end of H (bottom). |
padding_list |
int32[] |
[] |
The original 2-D padding as [top, left, bottom, right], kept for reference. The computation uses the four *_padding attributes. |
group_number |
int32 |
required | Number of channel groups; equals in_channels, one group per input channel. |
weight |
Weight |
required | Filter, [out_channels, 1, h_kernel, w_kernel]. |
bias |
Weight |
optional | Bias, [out_channels]. Absent means zero. |
channel_last |
bool |
true |
Must be true; the NPU uses the NHWC layout. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[1, H_out, W_out, out_channels] |
Output activation. |
Div¶
Element-wise quotient of two activations.
Divides in[0] by in[1] element by element (true division).
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
in[1] |
broadcastable to in[0] | Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
broadcast(in[0], in[1]) |
Output activation. |
DivConstant¶
Divides an activation by a constant, or a constant by an activation.
Divides with one operand taken from a constant stored in the model. The constant may be a scalar, a per-channel vector or a full tensor. With is_const_first true the result is constant / in[0], otherwise in[0] / constant.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
constant |
Weight |
required | Constant operand. Broadcasts against in[0]. |
is_const_first |
bool |
required | true if the constant is the dividend (left operand), false if it is the divisor. |
shape |
int32[] |
[] |
Shape of the constant operand before the stored constant was reduced (e.g. to a per-channel vector). The output shape is broadcast(shape(in[0]), shape). The default [] means the tensor's shape was not recorded; the field is optional. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
broadcast(in[0], shape) |
Output activation. |
Elu¶
Exponential linear unit with negative-side scale leaky_alpha.
Passes positive inputs through and maps the rest to leaky_alpha * (exp(x) - 1), which saturates at -leaky_alpha.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
leaky_alpha |
float32 |
required | ELU alpha, the scale of the negative branch. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Erf¶
Element-wise Gauss error function.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Exp¶
Element-wise natural exponential.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Gelu¶
Gaussian error linear unit.
Applies GELU element by element. approximate selects between the erf form and the tanh approximation.
approximate = false: out[0] = 0.5 * x * (1 + erf(x / sqrt(2)))
approximate = true: out[0] = 0.5 * x * (1 + tanh(sqrt(2/pi) * (x + 0.044715 * x^3)))
where x = in[0]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
approximate |
bool |
false |
true for the tanh approximation, false for the exact erf form. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
GroupConvolution¶
Grouped 2-D convolution of an NHWC activation with a constant filter.
Computes the same grouped convolution as Convolution. The channels are split into group_number groups, and each group's out_channels / group_number outputs see only that group's in_channels / group_number inputs.
G = group_number, Cg = in_channels / G, Og = out_channels / G (// : integer division)
xp = zero_pad(in[0], top=north_padding, bottom=south_padding,
left=west_padding, right=east_padding)
out[0][n, y, x, o] = bias[o] + sum_{ky, kx, c < Cg}
weight[o, c, ky, kx]
* xp[n, y*h_stride + ky*h_dilation, x*w_stride + kx*w_dilation, (o // Og)*Cg + c]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, in_channels] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
in_channels |
int32 |
required | Number of input channels. |
out_channels |
int32 |
required | Number of output channels. |
h_kernel |
int32 |
required | Kernel height. |
w_kernel |
int32 |
required | Kernel width. |
h_stride |
int32 |
required | Stride along H. |
w_stride |
int32 |
required | Stride along W. |
h_dilation |
int32 |
required | Dilation along H. |
w_dilation |
int32 |
required | Dilation along W. |
west_padding |
int32 |
required | Zero padding at the start of W (left). |
east_padding |
int32 |
required | Zero padding at the end of W (right). |
north_padding |
int32 |
required | Zero padding at the start of H (top). |
south_padding |
int32 |
required | Zero padding at the end of H (bottom). |
padding_list |
int32[] |
[] |
The original 2-D padding as [top, left, bottom, right], kept for reference. The computation uses the four *_padding attributes. |
group_number |
int32 |
required | Number of channel groups. Divides both in_channels and out_channels. |
weight |
Weight |
required | Filter, [out_channels, in_channels / group_number, h_kernel, w_kernel]. |
bias |
Weight |
optional | Bias, [out_channels]. Absent means zero. |
channel_last |
bool |
true |
Must be true; the NPU uses the NHWC layout. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[1, H_out, W_out, out_channels] |
Output activation. |
GroupNormalization¶
Normalizes each group of channels over H, W and the group, then scales and shifts.
Splits the C channels into num_groups contiguous groups and normalizes each group of each batch item to zero mean and unit variance over H, W and the group's channels. A per-channel scale and bias follow. The variance is the population variance.
G = num_groups, Cg = C / G, g(c) = floor(c / Cg)
S(n, g) = in[0][n, :, :, g*Cg : (g+1)*Cg]
mu[n, g] = mean(S(n, g))
var[n, g] = mean((S(n, g) - mu[n, g])^2)
out[0][n, h, w, c] = (in[0][n, h, w, c] - mu[n, g(c)]) / sqrt(var[n, g(c)] + epsilon)
* scale[c] + bias[c]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
num_groups |
int32 |
required | Number of channel groups. Divides C; each group is C / num_groups consecutive channels. |
scale |
Weight |
required | Per-channel scale (gamma). |
bias |
Weight |
optional | Per-channel shift (beta). Absent means zero. |
epsilon |
Weight |
required | Scalar added to the variance for numerical stability. |
channel_last |
bool |
true |
Must be true; the NPU uses the NHWC layout. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
HardSigmoid¶
Piecewise-linear sigmoid, clamp(alpha * x + beta, 0, 1).
A linear ramp clamped to [0, 1]. An absent alpha or beta takes 1/6 or 0.5.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
alpha |
float32 |
optional | Slope of the ramp. Absent means 1/6. |
beta |
float32 |
optional | Offset of the ramp. Absent means 0.5. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
HardSwish¶
Piecewise swish, x * clamp(x + 3, 0, 6) / 6.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
HeaderView¶
Splits the channel axis into attention heads and stacks them along H.
One operation for the reshape-transpose-reshape that turns a [1, H, W, heads*D] projection into per-head rows of D channels. The rows of all heads are stacked along H.
D = C / num_heads
batch_order = 0 (head-minor rows, row = h*num_heads + head):
[1, H, W, C] -reshape-> [1, H, W, num_heads, D]
-transpose(0,1,3,2,4)-> [1, H, num_heads, W, D]
-reshape-> [1, H*num_heads, W, D]
batch_order = 1 (head-major rows, row = head*H + h):
[1, H, W, C] -reshape-> [1, H, W, num_heads, D]
-transpose(0,3,1,2,4)-> [1, num_heads, H, W, D]
-reshape-> [1, num_heads*H, W, D]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
view_shape |
int32[] |
[] |
Deprecated; ignored. Use num_heads. |
num_heads |
int32 |
-1 |
Number of heads. Divides C. |
batch_order |
int32 |
0 |
Row order of the stacked heads: 0 interleaves heads within each row of H, 1 places each head's H rows contiguously. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[1, H*num_heads, W, C/num_heads] |
Output activation. |
HeaderViewRevert¶
Unstacks attention heads from H and merges them back into channels.
Takes per-head rows of D channels stacked along H, [1, num_heads*H, W, D], and concatenates the heads along the channel axis into [1, H, W, num_heads*D]. batch_order must be the value the rows were stacked with.
in[0] is [1, num_heads*H, W, D]
batch_order = 0 (input row = h*num_heads + head):
-reshape-> [1, H, num_heads, W, D]
-transpose(0,1,3,2,4)-> [1, H, W, num_heads, D]
-reshape-> [1, H, W, num_heads*D]
batch_order = 1 (input row = head*H + h):
-reshape-> [1, num_heads, H, W, D]
-transpose(0,2,3,1,4)-> [1, H, W, num_heads, D]
-reshape-> [1, H, W, num_heads*D]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, num_heads*H, W, D] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
revert_shape |
int32[] |
[] |
Deprecated; ignored. Use num_heads. |
num_heads |
int32 |
-1 |
Number of heads. Divides the H extent of in[0]. |
batch_order |
int32 |
0 |
Row order of the stacked heads in in[0]: 0 if heads alternate within each row of H, 1 if each head's H rows are contiguous. Same meaning as in HeaderView. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[1, H, W, num_heads*D] |
Output activation. |
Input¶
Marks an activation as a graph input.
Declares that in[0] is supplied from outside the graph. The operand and the result are the same activation, so the operation computes nothing; it only fixes the activation as a graph input.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
orig_name |
str |
"" |
User-facing name of this input in the source model, kept so the compiled model exposes the same input names. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
InputConstant¶
Provides a constant stored in the model as an activation.
Has no operands; its result is the stored constant, with element type dtype and shape shape.
Inputs¶
None.
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
constant |
Weight |
required | The constant value. |
dtype |
str |
"" |
Element type name of the constant, e.g. float32 or int64. |
shape |
int32[] |
[] |
Shape of the constant, which is the shape of out[0]. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
shape |
Output activation. |
InstanceNormalization¶
Normalizes each channel over H and W.
Normalizes every channel of every batch item to zero mean and unit variance over H and W, then applies per-channel scale and bias. The variance is the population variance. Normalization that shares statistics across several channels (num_groups other than -1) is expressed with GroupNormalization.
mu[n, c] = mean(in[0][n, :, :, c])
var[n, c] = mean((in[0][n, :, :, c] - mu[n, c])^2)
out[0][n, h, w, c] = (in[0][n, h, w, c] - mu[n, c]) / sqrt(var[n, c] + epsilon)
* scale[c] + bias[c]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
scale |
Weight |
required | Scale (gamma). |
bias |
Weight |
optional | Shift (beta), shaped like scale. Absent means zero. |
epsilon |
Weight |
required | Scalar added to the variance for numerical stability. |
channel_last |
bool |
true |
Must be true; the NPU uses the NHWC layout. |
num_groups |
int32 |
-1 |
Always -1. Grouped normalization is GroupNormalization. |
group_order |
int32 |
-1 |
Always -1; unused. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
InverseSigmoid¶
Inverse of the sigmoid, log(x / (1 - x)), kept finite by eps.
Clamps in[0] to [0, 1], then keeps the numerator and the denominator of x / (1 - x) at or above eps, so inputs at or beyond 0 and 1 still give finite results.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
eps |
float32 |
1e-05 |
Lower bound applied to the numerator and the denominator. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
L1Normalization¶
Divides by the sum of absolute values over the given axes.
Scales in[0] so that its absolute values along norm_axis sum to 1, per position of the other axes. The divisor is bounded below by 1e-12.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
norm_axis |
int32[] |
required | Axes to normalize over, as indices into in[0] as laid out. Negative values count from the last axis. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
L2Normalization¶
Divides by the Euclidean norm over the given axes.
Scales in[0] to unit L2 norm along norm_axis, per position of the other axes. The norm is bounded below by epsilon to avoid division by zero.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
norm_axis |
int32[] |
required | Axes to normalize over, as indices into in[0] as laid out. Negative values count from the last axis. |
epsilon |
Weight |
optional | Lower bound on the norm. Absent means 1e-12. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
LayerNormalization¶
Normalizes over the trailing axes, then scales and shifts.
Normalizes in[0] to zero mean and unit variance over its last len(normalized_shape) axes, then applies the element-wise affine scale and bias. Mean and variance are taken per position of the remaining leading axes; the variance is the biased (population) variance.
axes = the last len(normalized_shape) axes of in[0]
mu = mean(in[0], axes)
var = mean((in[0] - mu)^2, axes)
out[0] = (in[0] - mu) / sqrt(var + epsilon) * scale + bias
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[..., *normalized_shape] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
normalized_shape |
int64[] |
[] |
Shape of the normalized trailing axes, e.g. [C] to normalize over the channel axis of an NHWC activation. |
scale |
Weight |
required | Per-element scale (gamma), shape normalized_shape. |
bias |
Weight |
optional | Per-element shift (beta), shape normalized_shape. Absent means zero. |
epsilon |
Weight |
required | Scalar added to the variance for numerical stability. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
LeakyRelu¶
ReLU with the scalar slope leaky_alpha for negative inputs.
Multiplies negative elements by one scalar slope.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
leaky_alpha |
float32 |
required | Slope for negative inputs. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Log¶
Element-wise natural logarithm.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
MatMul¶
Batched matrix product of two activations over their last two axes.
Multiplies in[0] by in[1] as matrices over their last two axes; all leading axes are batch axes and broadcast. For 4-D NHWC activations the matrices are [W, C] and N, H are batch axes.
a = in[0], b = in[1]
repeat = [0, R]: a = repeat_interleave(a, R, axis=1)
repeat = [1, R]: b = repeat_interleave(b, R, axis=1)
where repeat_interleave(x, R, axis=1)[n, h, ...] = x[n, floor(h / R), ...]
out[0][..., m, p] = sum_k a[..., m, k] * b[..., k, p]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[..., M, K] |
Input activation. |
in[1] |
[..., K, P] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
constant |
Weight |
optional | Unused; always -1. A constant matrix is passed as an activation from InputConstant. |
is_const_first |
bool |
false |
Unused; always false. |
repeat |
int32[2] |
[-1, -1] |
[operand, R]: before the product, repeat operand operand (0 = in[0], 1 = in[1]) R times along axis 1 (H), each H slice R times in place (h0 h0 h1 h1 ..., not h0 h1 h0 h1 ...). [-1, -1]: no repeat. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[broadcast of the leading axes, M, P] | Output activation. |
Mish¶
Self-gated activation x * tanh(softplus(x)).
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Multiply¶
Element-wise product of two activations.
Multiplies two activations element by element.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
in[1] |
broadcastable to in[0] | Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
broadcast(in[0], in[1]) |
Output activation. |
MultiplyConstant¶
Multiplies an activation by a constant tensor.
Multiplies one activation by a constant stored in the model. The constant may be a scalar, a per-channel vector or a full tensor.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
constant |
Weight |
required | Constant operand. Broadcasts against in[0]. |
is_const_first |
bool |
false |
true if the constant is the left operand. |
shape |
int32[] |
[] |
Shape of the constant operand before the stored constant was reduced (e.g. to a per-channel vector). The output shape is broadcast(shape(in[0]), shape). The default [] means the tensor's shape was not recorded; the field is optional. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
broadcast(in[0], shape) |
Output activation. |
OrderedPatchify¶
Partitions H x W into windows and lays the pixels out window by window.
One operation for the reshape-transpose-reshape that cuts a [1, H, W, C] map into a gh x gw grid of h_kernel x w_kernel windows and flattens it into a 2-D layout on H and W. order=0 gives one row per window, holding its pixels in raster order; order=1 is its H/W transpose, one row per pixel offset inside the window; order=2 undoes order=1. Channels are moved as a whole and never mixed.
gh = H / h_kernel, gw = W / w_kernel
x = reshape(in[0], [1, gh, h_kernel, gw, w_kernel, C])
order 0: out[0] = reshape(transpose(x, 0,1,3,2,4,5), [1, gh*gw, h_kernel*w_kernel, C])
order 1: out[0] = reshape(transpose(x, 0,2,4,1,3,5), [1, h_kernel*w_kernel, gh*gw, C])
order 2: out[0] = reshape(transpose(x, 0,3,1,4,2,5), [1, gw*gh, w_kernel*h_kernel, C])
Element map, p < gh, q < h_kernel, u < gw, v < w_kernel:
in[0][0, p*h_kernel + q, u*w_kernel + v, c] =
order 0: out[0][0, p*gw + u, q*w_kernel + v, c]
order 1: out[0][0, q*w_kernel + v, p*gw + u, c]
order 2: out[0][0, u*gh + p, v*h_kernel + q, c]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
h_kernel |
int32 |
required | Window height. Divides H. |
w_kernel |
int32 |
required | Window width. Divides W. |
order |
int32 |
0 |
Layout: 0 one row per window, 1 one row per in-window offset (the transpose of 0), 2 the inverse of 1. See the formula. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
order 0, 2: [1, ghgw, h_kernelw_kernel, C]; order 1: [1, h_kernelw_kernel, ghgw, C] | Output activation. |
Output¶
Marks an activation as a graph output.
Declares that in[0] is returned from the graph. The operand and the result are the same activation, so the operation computes nothing; it only fixes the activation as a graph output.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
orig_name |
str |
"" |
User-facing name of this output in the source model, kept so the compiled model exposes the same output names. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Pad¶
Pads the H and W axes of an NHWC activation with a constant or reflection.
Pads in[0] by north_padding above, south_padding below, west_padding to its left and east_padding to its right. The batch and channel axes are not padded.
H_out = north_padding + H + south_padding
W_out = west_padding + W + east_padding
out[0][n, north_padding + i, west_padding + j, c] = in[0][n, i, j, c]
other cells:
constant: constant
reflect: in[0] mirrored about its first/last row and column, edge not
repeated (the row above row 0 is row 1)
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
west_padding |
int32 |
-1 |
Columns added at the start of W (left). |
east_padding |
int32 |
-1 |
Columns added at the end of W (right). |
south_padding |
int32 |
-1 |
Rows added at the end of H (bottom). |
north_padding |
int32 |
-1 |
Rows added at the start of H (top). |
pad_mode |
str |
required | constant fills the new cells with constant; reflect mirrors in[0] about its border without repeating the edge row or column. |
constant |
float32 |
0.0 |
Fill value for pad_mode = constant. |
padding_list |
int32[] |
optional | The same padding written for every axis as [x1_begin, x2_begin, ..., x1_end, x2_end, ...], for NHWC [0, top, left, 0, 0, bottom, right, 0], kept for reference. The computation uses the four *_padding attributes. |
channel_last |
bool |
true |
Must be true; the NPU uses the NHWC layout. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[1, H_out, W_out, C] |
Output activation. |
Pooling¶
2-D max or average pooling over sliding windows of an NHWC activation.
Pads in[0] by the *_padding attributes with the fill chosen by pad_mode, then takes the maximum (MaxPool) or the average (AvgPool) of each h_kernel x w_kernel window, moved by the stride and spread by the dilation. Channels are pooled independently.
xp = pad(in[0], top=north_padding, bottom=south_padding,
left=west_padding, right=east_padding) (fill per pad_mode)
win(n, y, x, c) = { xp[n, y*h_stride + ky*h_dilation, x*w_stride + kx*w_dilation, c]
: ky < h_kernel, kx < w_kernel }
MaxPool: out[0][n, y, x, c] = max(win(n, y, x, c))
AvgPool: out[0][n, y, x, c] = sum(win(n, y, x, c)) / cnt
cnt = h_kernel * w_kernel if is_include_pad
number of window cells inside in[0] otherwise
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
h_kernel |
int32 |
required | Window height. |
w_kernel |
int32 |
required | Window width. |
h_stride |
int32 |
required | Stride along H. |
w_stride |
int32 |
required | Stride along W. |
h_dilation |
int32 |
1 |
Dilation along H. |
w_dilation |
int32 |
1 |
Dilation along W. |
west_padding |
int32 |
required | Padding at the start of W (left). |
east_padding |
int32 |
required | Padding at the end of W (right). |
north_padding |
int32 |
required | Padding at the start of H (top). |
south_padding |
int32 |
required | Padding at the end of H (bottom). |
padding_list |
int32[] |
[] |
The original 2-D padding as [top, left, bottom, right], kept for reference; may be empty. The computation uses the four *_padding attributes. |
pool_type |
str |
required | MaxPool for the window maximum, AvgPool for the window average. |
pad_mode |
str |
required | Fill of the padded border: constant (zeros), -inf (negative infinity, so padding never wins a maximum) or reflect (mirror image of the border, edge not repeated). -inf is not combined with AvgPool and is_include_pad = true. |
is_ceil_mode |
bool |
false |
Records that the source pooling rounded the output size up. Informational; the output size always follows the floor formula below, so a rounded-up size appears as extra padding in south_padding (bottom) and east_padding (right). |
is_include_pad |
bool |
false |
AvgPool only. true divides by the full window size h_kernel * w_kernel, counting padded cells; false divides by the number of window cells that lie inside in[0]. |
channel_last |
bool |
true |
Must be true; the NPU uses the NHWC layout. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[1, H_out, W_out, C] |
Output activation. |
Pow¶
Raises each element to a constant scalar power.
Raises every element of in[0] to the same exponent.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
exponent |
float32 |
required | Scalar exponent. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
PRelu¶
Leaky ReLU whose negative slope is a constant tensor.
Multiplies negative elements by the slope leaky_alpha, a tensor stored in the model. The slope broadcasts against in[0], aligned from the last axis.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
leaky_alpha |
Weight |
required | Slope for negative inputs; broadcastable to in[0]. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
QuickGelu¶
Sigmoid approximation of GELU, x * sigmoid(1.702 * x).
Approximates Gelu by gating with sigmoid(1.702 * x).
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
ReduceMax¶
Maximum over the given axes.
Takes the largest element of in[0] over reduce_axes, separately for each position of the remaining axes.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
keep_dims |
bool |
required | true keeps each reduced axis with size 1; false removes it. |
reduce_axes |
int32[] |
required | Axes to reduce, as indices into in[0] as laid out (for an NHWC activation 3 is the channel axis). Negative values count from the last axis. |
noop_with_empty_axes |
int32 |
required | Only matters when reduce_axes is empty - 1 returns in[0] unchanged, 0 reduces over every axis. The IR normally lists the reduced axes explicitly. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
in[0] with the reduced axes set to 1 or removed | Output activation. |
ReduceMean¶
Arithmetic mean over the given axes.
Averages in[0] over reduce_axes, separately for each position of the remaining axes.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
keep_dims |
bool |
required | true keeps each reduced axis with size 1; false removes it. |
reduce_axes |
int32[] |
required | Axes to reduce, as indices into in[0] as laid out (for an NHWC activation 3 is the channel axis). Negative values count from the last axis. |
noop_with_empty_axes |
int32 |
required | Only matters when reduce_axes is empty - 1 returns in[0] unchanged, 0 reduces over every axis. The IR normally lists the reduced axes explicitly. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
in[0] with the reduced axes set to 1 or removed | Output activation. |
ReduceMin¶
Minimum over the given axes.
Takes the smallest element of in[0] over reduce_axes, separately for each position of the remaining axes.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
keep_dims |
bool |
required | true keeps each reduced axis with size 1; false removes it. |
reduce_axes |
int32[] |
required | Axes to reduce, as indices into in[0] as laid out (for an NHWC activation 3 is the channel axis). Negative values count from the last axis. |
noop_with_empty_axes |
int32 |
required | Only matters when reduce_axes is empty - 1 returns in[0] unchanged, 0 reduces over every axis. The IR normally lists the reduced axes explicitly. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
in[0] with the reduced axes set to 1 or removed | Output activation. |
ReduceSum¶
Sum over the given axes.
Adds up the elements of in[0] over reduce_axes, separately for each position of the remaining axes.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
keep_dims |
bool |
required | true keeps each reduced axis with size 1; false removes it. |
reduce_axes |
int32[] |
required | Axes to reduce, as indices into in[0] as laid out (for an NHWC activation 3 is the channel axis). Negative values count from the last axis. |
noop_with_empty_axes |
int32 |
required | Only matters when reduce_axes is empty - 1 returns in[0] unchanged, 0 reduces over every axis. The IR normally lists the reduced axes explicitly. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
in[0] with the reduced axes set to 1 or removed | Output activation. |
RelShift¶
Re-indexes relative-offset attention scores by absolute key position.
Relative-position shift of Transformer-XL style attention. Row i of in[0] holds the scores of query i for 2Q - 1 relative offsets, with column k meaning query minus key = Q - 1 - k. The op shifts row i left by Q - 1 - i and keeps Q columns, so column j of the result is the score for key j.
out[0][n, h, i, j] = in[0][n, h, i, Q - 1 - i + j], i, j < Q
Equivalent sequence on [1, H, Q, P], P = 2Q - 1:
pad one zero at the start of axis 3 -> [1, H, Q, P+1]
reshape -> [1, H, P+1, Q]
drop the first row of axis 2 -> [1, H, P, Q]
reshape -> [1, H, Q, P]
slice axis 3 to [0, Q) -> [1, H, Q, Q]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, Q, 2Q-1] |
Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[1, H, Q, Q] |
Output activation. |
Relu¶
Rectified linear unit, max(x, 0).
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Reshape¶
Changes the shape of an activation without moving its elements.
Returns in[0] with the shape new_shape. Elements keep their row-major order (last axis fastest).
out[0] = reshape(in[0], new_shape)
flatten(out[0]) == flatten(in[0]) (same elements, row-major order)
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
new_shape |
int32[] |
optional | Target shape. One entry may be -1; it is inferred so that the element count stays the same. A 0 entry is read as allowzero says; graph builders write explicit extents, so 0 normally does not appear. |
allowzero |
int32 |
0 |
How a 0 in new_shape is read: 0 copies the extent of in[0] at the same index, 1 makes a zero-length axis. |
is_const_first |
bool |
false |
Not used by the computation; in[0] is always the data. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
new_shape, with -1 resolved | Output activation. |
Resize¶
Resizes the spatial axes by nearest, bilinear or bicubic interpolation.
Resamples in[0] to a new H x W. sizes and scales hold one entry per axis of in[0], in its own axis order, and only the two spatial axes may differ from the input.
shape(out[0]) = sizes if sizes is given
shape(in[0]) * scales otherwise, axis by axis
s_h = H_out / H, s_w = W_out / W (or the matching scales entries)
nearest: out[0][n, y, x, c] = in[0][n, floor(y / s_h), floor(x / s_w), c]
linear: bilinear blend of the 2 x 2 input pixels around (ys, xs)
cubic: bicubic blend of the 4 x 4 input pixels around (ys, xs)
where the source row ys of output row y follows transform_mode
(columns likewise with x, W, W_out, s_w):
half_pixel: ys = (y + 0.5) / s_h - 0.5
align_corners: ys = y * (H - 1) / (H_out - 1)
asymmetric: ys = y / s_h
source positions outside the input are clamped to the border
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
resize_type |
str |
optional | Interpolation: nearest, linear (bilinear over H and W) or cubic (bicubic). |
resize_mode |
str |
optional | The interpolation mode as recorded when the graph was built; same values as resize_type, may be absent. The computation reads resize_type. |
nearest_mode |
str |
optional | Rounding rule of the source model for nearest (round_prefer_floor, round_prefer_ceil, floor or ceil). Recorded only; nearest always rounds down, as in the formula. |
transform_mode |
str |
optional | Mapping from output to input coordinates for linear and cubic: half_pixel (pixel centres aligned; also applies to any other recorded value), align_corners (corner pixels aligned) or asymmetric (output index divided by the scale); see the formula. nearest always uses the asymmetric mapping. |
roi |
int32[] |
optional | Region of interest of a crop-and-resize. Recorded only; the computation does not read it. |
scales |
float32[] |
optional | Scale factor per axis of in[0], 1.0 on axes that keep their size; for example [1, 2, 2, 1] doubles H and W of an NHWC tensor. |
sizes |
int32[] |
optional | Output shape, one entry per axis of in[0]. |
cubic_coeff_a |
float32 |
optional | Coefficient a of the cubic kernel for cubic. Recorded only; the computation does not read it. |
exclude_outside |
int32 |
optional | For cubic, 1 drops kernel taps outside the input and renormalizes the rest. Recorded only; the computation does not read it. |
extrapolation_value |
float32 |
optional | Value for output pixels whose source lies outside the input in a crop-and-resize. Recorded only; the computation does not read it. |
antialias |
int32 |
optional | 1 applies a low-pass (antialiasing) filter when downscaling with linear or cubic; 0 or absent does not. |
axes |
int32[] |
optional | Axes the source resize named for its scales or sizes. Recorded only; in the IR, scales and sizes always cover every axis. |
keep_aspect_ratio_policy |
str |
optional | How the source fitted its target size to keep the aspect ratio (stretch, not_larger, not_smaller). Recorded only; the computation does not read it. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[1, H_out, W_out, C] |
Output activation. |
RmsNormalization¶
Divides by the root mean square over the trailing axes, then scales.
Divides in[0] by its root mean square over the last len(normalized_shape) axes and multiplies by scale. With rms_center, the RMS is taken of in[0] - rms_center while the numerator stays in[0].
axes = the last len(normalized_shape) axes of in[0]
center = rms_center if present, else 0
rms = sqrt(mean((in[0] - center)^2, axes, keep_dims=true) + epsilon)
out[0] = in[0] / rms * scale
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[..., *normalized_shape] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
normalized_shape |
int64[] |
[] |
Shape of the trailing axes to normalize over, normally [C], the channel axis of an NHWC activation. |
epsilon |
Weight |
required | Scalar added to the mean square for numerical stability. |
scale |
Weight |
required | Per-element scale (gamma), shape normalized_shape. |
rms_center |
Weight |
optional | Per-channel offset subtracted from in[0] inside the RMS only. Absent means 0. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Sigmoid¶
Logistic sigmoid with an optional input scale alpha.
Computes 1 / (1 + exp(-alpha * x)).
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
alpha |
float32 |
1.0 |
Scale applied to the input before the sigmoid. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Sin¶
Element-wise sine.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Slice¶
Extracts a strided sub-range along one or more axes.
On each listed axis, keeps the elements from start up to but not including end, every step; the other axes stay whole. The lists are read position by position, so entry k of start, end and step applies to axis axis[k].
for each k, along axis a = axis[k] with length D = in[0].shape[a]:
s = start[k] (+ D if negative), clamped to [0, D]
e = end[k] (+ D if negative), clamped to [0, D]
keep indices s, s + step[k], s + 2*step[k], ... while < e
out[0] = in[0] restricted to the kept indices on every listed axis
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
axis |
int32[] |
required | Axes to slice; negative values count from the last axis. |
start |
int64[] |
required | First index kept on each axis (inclusive). |
end |
int64[] |
required | End index on each axis (exclusive). A value past the axis length is clamped, so a large value such as 2147483647 means "to the end". |
step |
int64[] |
optional | Stride on each axis. Absent means 1 on every axis. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
in[0] with each sliced axis shortened | Output activation. |
Softmax¶
Normalizes exponentials along one axis so they sum to 1.
Computes softmax over the axis axis of in[0], optionally scaling the input by beta first.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
beta |
float32 |
1.0 |
Scale applied to the input before the exponential. Must not be 0. |
axis |
int32 |
required | Axis to normalize over. A negative value counts from the last axis. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Softplus¶
Smooth ReLU log(1 + exp(beta * x)) / beta, linear above threshold.
Where beta * x exceeds threshold the output is x itself.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
beta |
float32 |
1.0 |
Sharpness of the curve; larger values approach ReLU. |
threshold |
float32 |
20.0 |
Value of beta * x above which the output is linear. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Sqrt¶
Element-wise square root.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
StaggeredPadding¶
Shifts row r of each matrix by r columns with zero fill, then crops.
Treats the last two axes of [1, H, W, C] as W rows of length C and shifts each row by its row index into a zero-filled row, giving a staircase layout. reverse=false shifts row r right by r and drops window_size columns at each side; reverse=true reads, for each row r, the 2*window_size + 1 columns centered on column r.
x[n, h, r, k] = in[0][n, h, r, k] for 0 <= k < C, and 0 for any other k
reverse = false:
o = 0 if window_size = -1, else window_size
out[0][n, h, r, j] = x[n, h, r, j + o - r], j < W + C - 1 - 2*o
reverse = true:
out[0][n, h, r, j] = x[n, h, r, r + j - window_size], j < 2*window_size + 1
Equivalent sequence for reverse = false:
[1, H, W, C] -pad axis 3 end by W-> [1, H, W, C+W] -reshape-> [1, H, W*(C+W)]
-drop the last W-> -reshape-> [1, H, W, W+C-1] -slice axis 3 [o, W+C-1-o)
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
axis |
int32 |
3 |
Axis whose entries are shifted. Only 3 (the last axis) is defined; rows run along axis 2. |
reverse |
bool |
false |
false: row r starts at column r (staircase toward the lower right). true: row r starts at column W - 1 - r (toward the lower left), and the result is cropped to a 2*window_size + 1 band. |
window_size |
int32 |
-1 |
reverse=false: columns dropped at each side, -1 to keep the full width W + C - 1. reverse=true: half-width of the band; must be given (not -1), and W and C are each at least window_size + 1. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
reverse=false: [1, H, W, W + C - 1 - 2o]; reverse=true: [1, H, W, 2window_size + 1] | Output activation. |
StatefulAttentionMaskWrapper¶
Adds a causal, sliding-window or supplied mask to attention scores.
Applies the attention mask to raw attention scores. With causal_mask the bias is derived from the score shape alone, taking the Q queries as the last Q of the K key positions. With custom_mask the bias is a second input.
custom_mask:
out[0] = in[0] + in[1]
causal_mask:
qp(i) = K - Q + i key position of query i
masked(i, j) = j > qp(i) or (is_sliding and j <= qp(i) - sliding_window)
out[0][..., i, j] = -inf if masked(i, j) else in[0][..., i, j]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, heads, Q, K] |
Input activation. |
in[1] |
broadcastable to in[0] | Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
past_seqlen |
int32 |
0 |
Number of cached positions in front of the queries when the graph was traced. Informational; the mask follows the runtime Q and K. |
is_sliding |
bool |
false |
true also masks keys sliding_window or more positions behind the query (causal_mask only). |
sliding_window |
int32 |
1024 |
Number of most recent key positions, the query's own included, that a query may attend to when is_sliding. |
mask_type |
str |
"causal_mask" |
causal_mask: mask synthesized from positions, one input. custom_mask: adds in[1], two inputs. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
StatefulCachedStack¶
Prepends the last entries kept from previous calls along an axis.
Keeps the last cache_len entries along axis across calls and prepends them to each new input. The first call sees zeros; later calls see the tail of the earlier inputs. The kept window is state held by the op, not an operand.
state: entries of shape in[0] with axis extent cache_len, initially zeros
out[0] = concat(state, in[0], axis)
state = last cache_len entries of out[0] along axis
cache_len = 0: out[0] = in[0]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
axis |
int32 |
2 |
Axis to stack along, by default 2 (the sequence axis W of NHWC). Negative counts from the last axis. |
cache_len |
int32 |
0 |
Number of entries kept and prepended. 0 makes the op an identity. |
cache_name |
str |
required | Name of the kept window. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
in[0] with axis extent + cache_len | Output activation. |
StatefulGRUWrapper¶
GRU over a sequence, returning per-step outputs and the final hidden state.
Runs a gated recurrent unit recurrence along the time axis of in[0] with the constant weights W, R, B, in one or both directions. input_mask names the operands that are present; the results are always out[0] (hidden state at every step) and out[1] (final hidden state).
batch_first = true; T = steps; @ = matrix product, * = element-wise product
for each direction d (0 = forward, 1 = backward):
act_f, act_g = activations[2d .. 2d+1] (activations[0..1] if only 2)
Wz, Wr, Wh = W[d] split into 3 row blocks; Rz, Rr, Rh = R[d] likewise
Wbz, Wbr, Wbh, Rbz, Rbr, Rbh = B[d] split into 6 (zero if no B)
h = in[h][:, 0, d, :] (zero if absent)
for t = 0 .. T-1 (forward) or t = T-1 .. 0 (backward):
x = in[0][:, 0, t, :]
z = act_f(x @ Wz^T + h @ Rz^T + Wbz + Rbz)
r = act_f(x @ Wr^T + h @ Rr^T + Wbr + Rbr)
n = act_g(x @ Wh^T + Wbh + (r * h) @ Rh^T + Rbh) if not linear_before_reset
act_g(x @ Wh^T + Wbh + r * (h @ Rh^T + Rbh)) if linear_before_reset
h = (1 - z) * n + z * h
out[0][:, d, t, :] = h
out[1][:, 0, d, :] = h
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, 1, T, input_size] |
Input activation. |
in[s] |
any |
Input activation. |
in[h] |
[1, 1, num_directions, hidden_size] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
W |
Weight |
required | Input weights, [num_directions, 3*hidden_size, input_size], gate blocks in the order z (update), r (reset), h (candidate). |
R |
Weight |
required | Recurrent weights, [num_directions, 3*hidden_size, hidden_size], gate order z, r, h. |
B |
Weight |
optional | Biases, [num_directions, 6*hidden_size]: the input biases (z, r, h) followed by the recurrent biases (z, r, h). Absent means zero. |
input_mask |
str |
required | Letters naming the operands present, in operand order. i input sequence (always first), s sequence lengths, h initial hidden state; for example ih means in[0] is the sequence and in[1] the hidden state. |
output_mask |
str |
required | Letters among o, h naming the results that are used (out[0], out[1]). The positions of the two results do not change. |
direction |
str |
"forward" |
forward, backward (reverse is treated the same) or bidirectional. bidirectional runs both and stacks forward (index 0) and backward (index 1) on the direction axis, so num_directions is 2; otherwise it is 1. |
batch_first |
bool |
false |
true for the batch-first layout used in the shapes here. false is time-major: in[0] is [T, 1, input_size], the state is [num_directions, 1, hidden_size] and out[0] is [T, num_directions, 1, hidden_size]. |
hidden_size |
int32 |
required | Number of hidden units per direction. |
hidden_state_name |
str |
required | Name of out[1] and of the state output of the cell subgraph. |
subgraph |
str |
required | Name of the cell subgraph that computes one time step from the step input and the previous hidden state. |
activation_alpha |
float32[] |
optional | Alpha coefficients for the functions in activations, in the same order. sigmoid and tanh take none. |
activation_beta |
float32[] |
optional | Beta coefficients for the functions in activations, in the same order. sigmoid and tanh take none. |
activations |
str[] |
optional | Gate functions [act_f, act_g], usually [sigmoid, tanh]. With bidirectional, four names give the forward then the backward functions; two names apply to both. The reference computation defines sigmoid and tanh. |
clip |
float32 |
optional | Cell clip threshold. When set, the gate inputs are limited to [-clip, clip] before the activation; absent means no clipping. |
linear_before_reset |
bool |
false |
false applies the reset gate to the previous state before the recurrent product; true applies it to the product h @ Rh^T + Rbh. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[1, num_directions, T, hidden_size] |
Output activation. |
out[1] |
[1, 1, num_directions, hidden_size] |
Output activation. |
StatefulKVCacheWrapper¶
Appends new key or value states to a persistent cache and returns it.
Keeps one layer's key or value cache across calls. Each call writes the S new states of in[0] at their absolute token positions and returns every state the current attention step can see, ordered by position. The cache and the token positions are runtime state, not IR operands.
pos = absolute positions of the S new tokens (from the runtime), p0 = pos[0]
the cache holds the states of earlier calls by position; in[0] is stored at pos
is_sliding = false:
out[0] = states of positions 0 .. pos[S-1], concatenated on axis 2
is_sliding = true:
out[0] = states of positions max(0, p0 - (sliding_window - 1)) .. pos[S-1]
only the last sliding_window positions stay stored for later calls
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, heads, S, D] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
cache_name |
str |
required | Name of the cache state this op reads and writes. |
past_seqlen |
int32 |
0 |
Number of tokens already in the cache when the graph was traced. The static output length is S + past_seqlen (a dynamic extent when non-zero); at run time the length follows the token positions. At most sliding_window - 1 when is_sliding. |
is_sliding |
bool |
false |
true keeps only the most recent sliding_window positions (sliding-window attention). |
sliding_window |
int32 |
1024 |
Window length in tokens when is_sliding. Positive and at most max_cache_len. |
max_cache_len |
int32 |
8192 |
Capacity of the cache in tokens. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[1, heads, P + S, D], P = past tokens visible (p0, or min(p0, sliding_window - 1) when sliding) | Output activation. |
StatefulLSTMWrapper¶
LSTM over a sequence, returning per-step outputs and final hidden/cell states.
Runs a long short-term memory recurrence along the time axis of in[0] with the constant weights W, R, B and optional peepholes P, in one or both directions. input_mask names the operands that are present; the results are always out[0] (hidden state at every step), out[1] (final hidden state) and out[2] (final cell state).
batch_first = true; T = steps; @ = matrix product, * = element-wise product
for each direction d (0 = forward, 1 = backward):
act_f, act_g, act_h = activations[3d .. 3d+2] (activations[0..2] if only 3)
Wi, Wo, Wf, Wc = W[d] split into 4 row blocks; Ri, Ro, Rf, Rc = R[d] likewise
Wbi, Wbo, Wbf, Wbc, Rbi, Rbo, Rbf, Rbc = B[d] split into 8 (zero if no B)
Pi, Po, Pf = P[d] split into 3 (zero if no P)
h = in[h][:, 0, d, :], c = in[c][:, 0, d, :] (zero if absent)
for t = 0 .. T-1 (forward) or t = T-1 .. 0 (backward):
x = in[0][:, 0, t, :]
i = act_f(x @ Wi^T + h @ Ri^T + Wbi + Rbi + Pi * c)
f = act_f(x @ Wf^T + h @ Rf^T + Wbf + Rbf + Pf * c)
g = act_g(x @ Wc^T + h @ Rc^T + Wbc + Rbc)
c = f * c + i * g
o = act_f(x @ Wo^T + h @ Ro^T + Wbo + Rbo + Po * c)
h = o * act_h(c)
out[0][:, d, t, :] = h
out[1][:, 0, d, :] = h
out[2][:, 0, d, :] = c
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, 1, T, input_size] |
Input activation. |
in[s] |
any |
Input activation. |
in[h] |
[1, 1, num_directions, hidden_size] |
Input activation. |
in[c] |
[1, 1, num_directions, hidden_size] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
W |
Weight |
required | Input weights, [num_directions, 4*hidden_size, input_size], gate blocks in the order i, o, f, c. |
R |
Weight |
required | Recurrent weights, [num_directions, 4*hidden_size, hidden_size], gate order i, o, f, c. |
B |
Weight |
optional | Biases, [num_directions, 8*hidden_size]: the input biases (i, o, f, c) followed by the recurrent biases (i, o, f, c). Absent means zero. |
input_mask |
str |
required | Letters naming the operands present, in operand order. i input sequence (always first), s sequence lengths, h initial hidden state, c initial cell state; for example ihc means in[0..2] are the sequence, the hidden state and the cell state. |
output_mask |
str |
required | Letters among o, h, c naming the results that are used (out[0], out[1], out[2]). The positions of the three results do not change. |
P |
Weight |
optional | Peephole weights, [num_directions, 3*hidden_size], in the order i, o, f. Absent means no peephole. |
direction |
str |
"forward" |
forward, backward (reverse is treated the same) or bidirectional. bidirectional runs both and stacks forward (index 0) and backward (index 1) on the direction axis, so num_directions is 2; otherwise it is 1. |
batch_first |
bool |
false |
true for the batch-first layout used in the shapes here. false is time-major: in[0] is [T, 1, input_size], the states are [num_directions, 1, hidden_size] and out[0] is [T, num_directions, 1, hidden_size]. |
hidden_size |
int32 |
required | Number of hidden units per direction. |
hidden_state_name |
str |
required | Name of out[1] and of the hidden-state output of the cell subgraph. |
cell_state_name |
str |
required | Name of out[2] and of the cell-state output of the cell subgraph. |
subgraph |
str |
required | Name of the cell subgraph that computes one time step from the step input and the previous hidden and cell states. |
activation_alpha |
float32[] |
optional | Alpha coefficients for the functions in activations, in the same order. sigmoid and tanh take none. |
activation_beta |
float32[] |
optional | Beta coefficients for the functions in activations, in the same order. sigmoid and tanh take none. |
activations |
str[] |
optional | Gate functions [act_f, act_g, act_h], usually [sigmoid, tanh, tanh]. With bidirectional, six names give the forward then the backward functions; three names apply to both. The reference computation defines sigmoid and tanh. |
clip |
float32 |
optional | Cell clip threshold. When set, the gate inputs are limited to [-clip, clip] before the activation; absent means no clipping. |
input_forget |
bool |
false |
true couples the input and forget gates. The formula above is the false case. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[1, num_directions, T, hidden_size] |
Output activation. |
out[1] |
[1, 1, num_directions, hidden_size] |
Output activation. |
out[2] |
[1, 1, num_directions, hidden_size] |
Output activation. |
StatefulRNNWrapper¶
Simple RNN over a sequence, returning per-step outputs and the final state.
Runs the recurrence h = act_f(x W^T + h R^T + b) along the time axis of in[0] with the constant weights W, R, B, in one or both directions. input_mask names the operands that are present; the results are always out[0] (hidden state at every step) and out[1] (final hidden state).
batch_first = true; T = steps; @ = matrix product
for each direction d (0 = forward, 1 = backward):
act_f = activations[d] (activations[0] if only 1)
Wb, Rb = B[d] split into 2 (zero if no B)
h = in[h][:, 0, d, :] (zero if absent)
for t = 0 .. T-1 (forward) or t = T-1 .. 0 (backward):
x = in[0][:, 0, t, :]
h = act_f(x @ W[d]^T + h @ R[d]^T + Wb + Rb)
out[0][:, d, t, :] = h
out[1][:, 0, d, :] = h
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, 1, T, input_size] |
Input activation. |
in[s] |
any |
Input activation. |
in[h] |
[1, 1, num_directions, hidden_size] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
W |
Weight |
required | Input weights, [num_directions, hidden_size, input_size]. |
R |
Weight |
required | Recurrent weights, [num_directions, hidden_size, hidden_size]. |
B |
Weight |
optional | Biases, [num_directions, 2*hidden_size]: the input bias followed by the recurrent bias. Absent means zero. |
input_mask |
str |
required | Letters naming the operands present, in operand order. i input sequence (always first), s sequence lengths, h initial hidden state; for example ih means in[0] is the sequence and in[1] the hidden state. |
output_mask |
str |
required | Letters among o, h naming the results that are used (out[0], out[1]). The positions of the two results do not change. |
direction |
str |
"forward" |
forward, backward (reverse is treated the same) or bidirectional. bidirectional runs both and stacks forward (index 0) and backward (index 1) on the direction axis, so num_directions is 2; otherwise it is 1. |
batch_first |
bool |
false |
true for the batch-first layout used in the shapes here. false is time-major: in[0] is [T, 1, input_size], the state is [num_directions, 1, hidden_size] and out[0] is [T, num_directions, 1, hidden_size]. |
hidden_size |
int32 |
required | Number of hidden units per direction. |
hidden_state_name |
str |
required | Name of out[1] and of the state output of the cell subgraph. |
subgraph |
str |
required | Name of the cell subgraph that computes one time step from the step input and the previous hidden state. |
activation_alpha |
float32[] |
optional | Alpha coefficients for the functions in activations, in the same order. sigmoid and tanh take none. |
activation_beta |
float32[] |
optional | Beta coefficients for the functions in activations, in the same order. sigmoid and tanh take none. |
activations |
str[] |
optional | Recurrence function [act_f], usually [tanh]. With bidirectional, two names give the forward then the backward function; one name applies to both. The reference computation defines sigmoid and tanh. |
clip |
float32 |
optional | Cell clip threshold. When set, the input of the activation is limited to [-clip, clip]; absent means no clipping. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[1, num_directions, T, hidden_size] |
Output activation. |
out[1] |
[1, 1, num_directions, hidden_size] |
Output activation. |
StatefulRoPEWrapper¶
Applies rotary position embedding to queries or keys at their positions.
Rotates channel pairs of a query or key tensor by position-dependent angles, using precomputed cos/sin tables whose row p belongs to position p. The rows are picked by the absolute positions of the tokens in the current call.
x = in[0]; c, s = rows of in[1], in[2] for the S current positions
rot0(y) = concat(-y[..., h:], y[..., :h]), h = width(y) / 2
rot1(y)[..., 2i] = -y[..., 2i+1], rot1(y)[..., 2i+1] = y[..., 2i]
rope_type 0: R = rotary_emb_dim if > 0, else width(c)
y = x[..., :R]
out[0] = concat(y * c[..., :R] + rot0(y) * s[..., :R], x[..., R:])
rope_type 1: out[0] = x * c + rot1(x) * s
rope_type 2: rope_type 0 with R = width(c)
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, heads, S, D] |
Input activation. |
in[1] |
[1, 1, L, R] |
Input activation. |
in[2] |
[1, 1, L, R] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
rope_type |
int32 |
0 |
Channel pairing: 0 pairs channel i with i + R/2 (rotary channels split in halves); 1 pairs adjacent channels 2i, 2i+1; 2 is 0 with the rotary width taken from the cos table. |
seqlen |
int32 |
0 |
Sequence length S of in[0] when the graph was built. Informational; the computation uses the runtime shape. |
rotary_emb_dim |
int32 |
0 |
Number of leading channels of D that are rotated (partial rotary); the rest pass through unchanged. 0 means the width of the cos table. Used with rope_type=0 only. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Sub¶
Element-wise difference of two activations.
Subtracts in[1] from in[0] element by element.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
in[1] |
broadcastable to in[0] | Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
broadcast(in[0], in[1]) |
Output activation. |
SubConstant¶
Subtracts a constant from an activation, or an activation from a constant.
Subtracts with one operand taken from a constant stored in the model. The constant may be a scalar, a per-channel vector or a full tensor. With is_const_first true the result is constant - in[0], otherwise in[0] - constant.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
constant |
Weight |
required | Constant operand. Broadcasts against in[0]. |
is_const_first |
bool |
required | true if the constant is the minuend (left operand), false if it is subtracted. |
shape |
int32[] |
[] |
Shape of the constant operand before the stored constant was reduced (e.g. to a per-channel vector). The output shape is broadcast(shape(in[0]), shape). The default [] means the tensor's shape was not recorded; the field is optional. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
broadcast(in[0], shape) |
Output activation. |
Swish¶
Self-gated activation x * sigmoid(x).
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Tan¶
Element-wise tangent.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
Tanh¶
Element-wise hyperbolic tangent.
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
None.
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
same as in[0] | Output activation. |
TensorCmp¶
Element-wise comparison of two activations, giving a boolean tensor.
Compares in[0] with in[1] element by element, after broadcasting, using the relation named by kind. The output is boolean - true where the relation holds.
kind = "eq": out[0] = (in[0] == in[1])
kind = "neq": out[0] = (in[0] != in[1])
kind = "lt": out[0] = (in[0] < in[1])
kind = "leq": out[0] = (in[0] <= in[1])
kind = "gt": out[0] = (in[0] > in[1])
kind = "geq": out[0] = (in[0] >= in[1])
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
in[1] |
broadcastable to in[0] | Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
kind |
str |
"eq" |
Relation: eq, neq, lt, leq, gt or geq, read as in[0] <kind> in[1]. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
broadcast(in[0], in[1]) |
Output activation. |
Transpose¶
Permutes the axes of an activation.
Reorders the axes of in[0] so that axis i of out[0] is axis transpose_axes[i] of in[0].
P = transpose_axes
out[0].shape[i] = in[0].shape[P[i]]
out[0][i_0, ..., i_{r-1}] = in[0][j] where j[P[k]] = i_k
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, C] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
transpose_axes |
int64[] |
required | Permutation of 0..rank-1; entry i names the input axis that becomes output axis i. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[in[0].shape[transpose_axes[i]] for each i] |
Output activation. |
TransposeConvolution¶
Transposed 2-D convolution, stored as zero insertion plus a convolution.
Upsamples in[0] by inserting h_stride - 1 zero rows and w_stride - 1 zero columns between neighbouring pixels, zero-pads that tensor by the *_padding attributes, and convolves it with weight at stride 1 and the given dilation, adding bias. The weight is therefore stored in convolution layout, [out_channels, in_channels / group_number, h_kernel, w_kernel], holding the transposed-convolution kernel flipped along H and W, and the paddings apply to the zero-inserted tensor rather than being cropped from the output.
G = group_number, Cg = in_channels / G, Og = out_channels / G (// : integer division)
z = zeros([1, (H-1)*h_stride + 1, (W-1)*w_stride + 1, in_channels])
z[n, i*h_stride, j*w_stride, c] = in[0][n, i, j, c]
zp = zero_pad(z, top=north_padding, bottom=south_padding,
left=west_padding, right=east_padding)
out[0][n, y, x, o] = bias[o] + sum_{ky, kx, c < Cg}
weight[o, c, ky, kx]
* zp[n, y + ky*h_dilation, x + kx*w_dilation, (o // Og)*Cg + c]
Inputs¶
| Name | Shape | Description |
|---|---|---|
in[0] |
[1, H, W, in_channels] |
Input activation. |
Attributes¶
| Name | Type | Default | Description |
|---|---|---|---|
in_channels |
int32 |
required | Number of input channels. |
out_channels |
int32 |
required | Number of output channels. |
h_kernel |
int32 |
required | Kernel height. |
w_kernel |
int32 |
required | Kernel width. |
h_stride |
int32 |
required | Upsampling stride along H; h_stride - 1 zero rows go between input rows. |
w_stride |
int32 |
required | Upsampling stride along W; w_stride - 1 zero columns go between input columns. |
h_dilation |
int32 |
required | Dilation along H. |
w_dilation |
int32 |
required | Dilation along W. |
west_padding |
int32 |
required | Zero padding at the start of W (left) of the zero-inserted tensor. |
east_padding |
int32 |
required | Zero padding at the end of W (right) of the zero-inserted tensor. |
north_padding |
int32 |
required | Zero padding at the start of H (top) of the zero-inserted tensor. |
south_padding |
int32 |
required | Zero padding at the end of H (bottom) of the zero-inserted tensor. |
padding_list |
int32[] |
[] |
The original transposed-convolution padding as [top, left, bottom, right] (amounts removed from the output), kept for reference. The computation uses the four *_padding attributes. |
og_padding_list |
int32[] |
[] |
Slot for the original padding of the transposed convolution. Informational; the computation does not read it, and it is normally empty. |
og_out_padding_list |
int32[] |
[] |
Slot for the original output padding of the transposed convolution (extra rows and columns at the end of the output). Informational; the computation does not read it, and it is normally empty. |
group_number |
int32 |
required | Number of channel groups. Divides both in_channels and out_channels. |
weight |
Weight |
required | Filter in convolution layout, [out_channels, in_channels / group_number, h_kernel, w_kernel]: the transposed-convolution kernel flipped along H and W. |
bias |
Weight |
optional | Bias, [out_channels]. Absent means zero. |
channel_last |
bool |
true |
Must be true; the NPU uses the NHWC layout. |
Outputs¶
| Name | Shape | Description |
|---|---|---|
out[0] |
[1, H_out, W_out, out_channels] |
Output activation. |
Other Mobilint IR operations¶
The remaining operations have input and attribute metadata in the table below. Their presence in this table does not imply NPU or CPU Offload support.
| Operation | Notes |
|---|---|
| Abs | num inputs: 1 num outputs: 1 |
| AddPartial | num inputs: 2 num outputs: 1 attributes: - is_input0_larger: bool |
| Cast | num inputs: 1 num outputs: 1 attributes: - dtype: str |
| Ceil | num inputs: 1 num outputs: 1 |
| Convolution3d | num inputs: 1 num outputs: 1 attributes: - in_channels: int32 - out_channels: int32 - h_kernel: int32 - w_kernel: int32 - d_kernel: int32 - h_stride: int32 - w_stride: int32 - d_stride: int32 - h_dilation: int32 - w_dilation: int32 - d_dilation: int32 - west_padding: int32 - east_padding: int32 - north_padding: int32 - south_padding: int32 - front_padding: int32 - back_padding: int32 - padding_list: int32[] (default=[]) - group_number: int32 - weight: Weight - bias: Weight (optional) - channel_last: bool (default=true) |
| Crop | num inputs: 1 num outputs: 1 attributes: - w_offset: int32 (default=0) - h_offset: int32 (default=0) - crop_width: int32 - crop_height: int32 |
| DeviceBridge | num inputs: 1 num outputs: 1 attributes: - squeeze_axes: int32[] - unsqueeze_axes: int32[] - permute_axes: int64[] - device: str (optional) |
| Expand | num inputs: 1 num outputs: 1 attributes: - constant: Weight (optional) - is_const_first: bool |
| GLU | num inputs: 1 num outputs: 1 |
| GridSample | num inputs: 2 num outputs: 1 attributes: - align_corners: int32 (default=0) - mode: str (default="linear") - padding_mode: str (default="zeros") description: inputs: X, grid X is in channel-last format. X: (n, h, w, c) grid: (n, h_out, w_out, num_spatial_dims). in this case, num_spatial_dims=2. |
| Hardtanh | num inputs: 1 num outputs: 1 attributes: - min_val: float32 (default=-1.0) - max_val: float32 (default=1.0) |
| Hardshrink | num inputs: 1 num outputs: 1 attributes: - lambd: float32 (default=0.5) |
| Softshrink | num inputs: 1 num outputs: 1 attributes: - lambd: float32 (default=0.5) |
| Identity | num inputs: 1 num outputs: 1 |
| SumNormalization | num inputs: 1 num outputs: 1 attributes: - norm_axis: int32[] - epsilon: float32 (optional) |
| LSTM | num inputs: variable num outputs: 1 attributes: - W: Weight - R: Weight - B: Weight (optional) - lstm_group: str - input_mask: str - output_mask: str - P: Weight (optional) - direction: str (default="forward") - batch_first: bool (default=false) - hidden_size: int32 description: inputs:[input, sequence_lens, initial_h, initial_c] outputs:[output] or [hidden_out] or [cell_out] input_mask: "ishc" or "i", etc,... output_mask: "o" or "h" or "c" |
| Logit | num inputs: 1 num outputs: 1 attributes: - eps: float32 (optional) description: Compute y = ln(z / (1 - z)). If eps is given, we clamp input z = clamp(z, min=eps, max=1-eps) |
| RNN | num inputs: variable num outputs: 1 attributes: - W: Weight - R: Weight - B: Weight (optional) - rnn_group: str - input_mask: str - output_mask: str - direction: str (default="forward") - batch_first: bool (default=false) - hidden_size: int32 description: inputs:[input, sequence_lens, initial_h] outputs:[output] or [hidden_out] input_mask: "ish" or "i", etc,... output_mask: "o" or "h" |
| GRU | num inputs: variable num outputs: 1 attributes: - W: Weight - R: Weight - B: Weight (optional) - gru_group: str - input_mask: str - output_mask: str - direction: str (default="forward") - batch_first: bool (default=false) - linear_before_reset: bool (default=false) - hidden_size: int32 description: inputs:[input, sequence_lens, initial_h] outputs:[output] or [hidden_out] input_mask: "ish" or "i", etc,... output_mask: "o" or "h" |
| LogSigmoid | num inputs: 1 num outputs: 1 |
| MaskSoftmax | num inputs: 1 num outputs: 1 attributes: - beta: float32 - axis: int32 |
| NonMaxSuppression | num inputs: 2 num outputs: variable attributes: - max_output_size: int64 - iou_threshold: float32 - score_threshold: float32 - soft_nms_sigma: float32 - pad_to_max_output_size: bool - center_point_box: int32 (default=0) |
| GlobalAveragePooling | num inputs: 1 num outputs: 1 attributes: - channel_last: bool (default=true) |
| GlobalResponseNormalization | num inputs: 1 num outputs: 1 attributes: - scale: Weight - bias: Weight - epsilon: float32 - channel_last: bool (default=true) |
| ReduceL2 | num inputs: 1 num outputs: 1 attributes: - keep_dims: bool - reduce_axes: int32[] - noop_with_empty_axes: int32 |
| ReduceProd | num inputs: 1 num outputs: 1 attributes: - keep_dims: bool - reduce_axes: int32[] - noop_with_empty_axes: int32 |
| ReduceRMS | num inputs: 1 num outputs: 1 attributes: - keep_dims: bool - reduce_axes: int32[] - epsilon: float32 (optional) description: reduce rms with epsilon if exists. i.e.,sqrt(mean**2 + eps) we let eps to be optional so that it has more general fusing opportunities. |
| MeanCentering | num inputs: 1 num outputs: 1 attributes: - axes: int32[] description: compute x - mean(x, axes) |
| ReshapeV2 | num inputs: 2 num outputs: 1 attributes: - allowzero: int32 (default=0) - is_const_first: bool (default=false) description: Reshape with variable shape argument. inputs[1] will be used as a shape. |
| RotaryEmbedding | num inputs: 1 num outputs: 2 attributes: - inv_freq: Weight - attention_scaling: float32 description: following the LlamaRotaryEmbedding of the Huggingface Transformer library. inputs => (position_ids,), outputs => (cos, sin) |
| Rsqrt | num inputs: 1 num outputs: 1 |
| Rstd | num inputs: 1 num outputs: 1 attributes: - reduce_axes: int32[] - epsilon: Weight - num_groups: int32 (default=1) description: computes 1/std(x). if gruops > 1, then activation is grouped along channel axis. |
| Gather | num inputs: 2 num outputs: 1 attributes: - axis: int32 (default=0) |
| GatherElements | num inputs: 2 num outputs: 1 attributes: - axis: int32 (default=0) |
| ScatterND | num inputs: variable num outputs: 1 attributes: - constant_data: Weight (optional) - constant_indices: Weight (optional) |
| Selu | num inputs: 1 num outputs: 1 |
| Shape | num inputs: 1 num outputs: 1 attributes: - start: int32 (optional) - end: int32 (optional) |
| LogSoftmax | num inputs: 1 num outputs: 1 attributes: - axis: int32 |
| Softsign | num inputs: 1 num outputs: 1 |
| Unsqueeze | num inputs: 1 num outputs: 1 attributes: - axes: int32[] |
| DynamicSliceInputConst | num inputs: 1 num outputs: 1 attributes: - mod: int32 - start: int64[] - end: int64[] - axis: int64[] - step: int64[] (optional) |
| MeloTTSPiecewiseRationalQuadraticTransform | num inputs: 2 num outputs: 1 attributes: - num_bins: int32 - filter_channels: int32 - tail_bound: float32 - reverse: bool description: helper function for parsing melotts model. |
| Neg | num inputs: 1 num outputs: 1 |
| Einsum | num inputs: variable num outputs: 1 attributes: - equation: str - constant: Weight (optional) - is_const_first: bool (default=false) |
| Where | num inputs: 3 num outputs: 1 description: condition,X,Y, if cond == True: else y |
| Logical | num inputs: variable num outputs: 1 attributes: - kind: str (default="and") description: this function represents the logical operation of two inputs. single input if kind is "not" else two inputs kind: and, or, xor, not |
| MaskedFill | num inputs: 2 num outputs: 1 attributes: - value: float32 description: inputs: (x, mask) |
| SubgraphCall | num inputs: variable num outputs: variable attributes: - subgraph: str - scope: str - kind: str |
| MoeDispatcher | num inputs: 2 num outputs: 2 attributes: - top_k: int32 - norm_top_k_prob: bool (default=true) description: inputs: (hidden_states, router_logits) outputs: (hidden_states, top_k_logits) if norm_top_k_prob is true topk_vals, topk_idx = torch.topk( routing_weights, self.top_k, dim=-1 ) # (18,8), (18,8) weight_new = torch.zeros_like(routing_weights) weight_new.scatter_(1, topk_idx, topk_vals) routing_weights = weight_new routing_weights /= routing_weights.sum(dim=1, keepdim=True) |
| MoeAggregator | num inputs: variable num outputs: 1 description: inputs: (router_weights, experts_output0, expert_output1, ...) |
| TopKValues | num inputs: 1 num outputs: 1 attributes: - axis: int32 - k: int32 (default=1) - largest: bool (default=true) - sorted: bool (default=true) |
| TopKIndices | num inputs: 1 num outputs: 1 attributes: - axis: int32 - k: int32 (default=1) - largest: bool (default=true) - sorted: bool (default=true) |