Skip to content

Mobilint IR Operation Formats

This page describes the behavior, inputs, attributes, and outputs of Mobilint IR operations. It is a format reference: an operation appearing here does not by itself establish NPU or CPU Offload support. See NPU Operator Support or CPU Offloading for execution scope.

Reading the operation definitions

  • Four-dimensional activations use NHWC order: batch (N), height (H), width (W), and channels (C). The batch size is 1 in the shapes below.
  • Inputs and outputs use positional names such as in[0] and out[0].
  • A Weight attribute stores a tensor index (TensorIndex), not tensor values. It refers to the subgraph tensor table, which records the constant tensor's name, dtype, shape, and storage location. -1 means no tensor; omitted optional tensor attributes use this value.
  • In attribute tables, required means the attribute must be set and optional means it may be omitted.

Detailed operation definitions

Adding

Element-wise sum of two activations.

Adds two activations element by element.

out[0] = in[0] + in[1]

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.
in[1] broadcastable to in[0] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] broadcast(in[0], in[1]) Output activation.

AddingConstant

Adds a constant tensor to an activation.

Adds a constant stored in the model to one activation. The constant may be a scalar, a per-channel vector or a full tensor.

out[0] = constant + in[0]   if is_const_first
         in[0] + constant   otherwise

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
constant Weight required Constant operand. Broadcasts against in[0].
is_const_first bool false true if the constant is the left operand.
shape int32[] [] Shape of the constant operand before the stored constant was reduced (e.g. to a per-channel vector). The output shape is broadcast(shape(in[0]), shape). The default [] means the tensor's shape was not recorded; the field is optional.

Outputs

Name Shape Description
out[0] broadcast(in[0], shape) Output activation.

Batchnorm

Per-channel normalization with stored mean and variance, then scale and shift.

Inference-form batch normalization. Each channel is normalized with the constant moving_mean and moving_var stored in the model, then scaled by gamma and shifted by beta.

out[0][n, h, w, c] = (in[0][n, h, w, c] - moving_mean[c]) / sqrt(moving_var[c] + epsilon)
                     * gamma[c] + beta[c]

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
gamma Weight required Per-channel scale.
beta Weight required Per-channel shift.
moving_mean Weight required Per-channel mean subtracted from the input.
moving_var Weight required Per-channel variance the input is divided by (after adding epsilon and taking the square root).
epsilon Weight required Scalar added to the variance for numerical stability.
channel_last bool true Must be true; the NPU uses the NHWC layout.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Celu

Continuously differentiable ELU with parameter alpha.

Passes positive inputs through and maps negative inputs to an exponential that saturates at -alpha.

out[0] = max(0, in[0]) + min(0, alpha * (exp(in[0] / alpha) - 1))

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
alpha float32 required Saturation level of the negative branch and divisor of the exponent.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Clip

Clamps each element to [clip_min_value, clip_max_value].

Limits every element to the given bounds. Either bound may be absent, and that side is then left unlimited.

out[0] = min(max(in[0], clip_min_value), clip_max_value)
an absent bound is skipped

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
clip_max_value float32 optional Upper bound. Absent means no upper bound.
clip_min_value float32 optional Lower bound. Absent means no lower bound.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Concatenate

Joins activations along one axis.

Concatenates in[0..K-1], in operand order, along axis. All operands have the same rank and agree on every other axis.

out[0] = concat(in[0], in[1], ..., in[K-1]) along axis
out[0].shape[axis] = in[0].shape[axis] + ... + in[K-1].shape[axis]

Inputs

Name Shape Description
in[0..K-1] equal except along axis Input activation.

Attributes

Name Type Default Description
original_key str optional Deprecated and no longer used. The key is always present, but its value is always None.
axis int32 optional Axis to join along; negative values count from the last axis. It indexes the tensor as laid out, so 3 is the channel axis of an NHWC activation.
original_axis int32 optional Deprecated and no longer used. The key is always present, but its value is always None.
original_inputs int32[] optional Deprecated and no longer used. The key is always present, but its value is always None.
input_index int32[] optional Always [0, 1, ..., K-1].
input_names str[] optional Always the names of this operation's input activations.

Outputs

Name Shape Description
out[0] in[0] with the axis extent summed over operands Output activation.

Convolution

2-D convolution of an NHWC activation with a constant filter.

Slides weight over the spatial axes of in[0] with the given stride, dilation and zero padding, and adds bias. The weight is stored output-channel first, as [out_channels, in_channels / group_number, h_kernel, w_kernel].

G  = group_number,  Cg = in_channels / G,  Og = out_channels / G   (// : integer division)
xp = zero_pad(in[0], top=north_padding, bottom=south_padding,
                     left=west_padding, right=east_padding)
out[0][n, y, x, o] = bias[o] + sum_{ky, kx, c < Cg}
    weight[o, c, ky, kx]
    * xp[n, y*h_stride + ky*h_dilation, x*w_stride + kx*w_dilation, (o // Og)*Cg + c]

Inputs

Name Shape Description
in[0] [1, H, W, in_channels] Input activation.

Attributes

Name Type Default Description
in_channels int32 required Number of input channels.
out_channels int32 required Number of output channels.
h_kernel int32 required Kernel height.
w_kernel int32 required Kernel width.
h_stride int32 required Stride along H.
w_stride int32 required Stride along W.
h_dilation int32 required Dilation along H.
w_dilation int32 required Dilation along W.
west_padding int32 required Zero padding at the start of W (left).
east_padding int32 required Zero padding at the end of W (right).
north_padding int32 required Zero padding at the start of H (top).
south_padding int32 required Zero padding at the end of H (bottom).
padding_list int32[] [] The original 2-D padding as [top, left, bottom, right], kept for reference. The computation uses the four *_padding attributes.
group_number int32 required Number of channel groups; must be 1. A convolution with more than one group must be a GroupConvolution or DepthwiseConvolution.
weight Weight required Filter, [out_channels, in_channels / group_number, h_kernel, w_kernel].
bias Weight optional Bias, [out_channels]. Absent means zero.
channel_last bool true Must be true; the NPU uses the NHWC layout.

Outputs

Name Shape Description
out[0] [1, H_out, W_out, out_channels] Output activation.

Cos

Element-wise cosine.

out[0] = cos(in[0])

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

DepthwiseConvolution

Depthwise 2-D convolution; each input channel has its own filters.

A Convolution with one group per input channel (group_number = in_channels). Each input channel is convolved with its own M = out_channels / in_channels filters and is never mixed with other channels; output channel o reads input channel o / M. The weight is stored as [out_channels, 1, h_kernel, w_kernel].

M  = out_channels / in_channels
xp = zero_pad(in[0], top=north_padding, bottom=south_padding,
                     left=west_padding, right=east_padding)
out[0][n, y, x, o] = bias[o] + sum_{ky, kx}
    weight[o, 0, ky, kx]
    * xp[n, y*h_stride + ky*h_dilation, x*w_stride + kx*w_dilation, o / M]

Inputs

Name Shape Description
in[0] [1, H, W, in_channels] Input activation.

Attributes

Name Type Default Description
in_channels int32 required Number of input channels.
out_channels int32 required Number of output channels, in_channels * M.
h_kernel int32 required Kernel height.
w_kernel int32 required Kernel width.
h_stride int32 required Stride along H.
w_stride int32 required Stride along W.
h_dilation int32 required Dilation along H.
w_dilation int32 required Dilation along W.
west_padding int32 required Zero padding at the start of W (left).
east_padding int32 required Zero padding at the end of W (right).
north_padding int32 required Zero padding at the start of H (top).
south_padding int32 required Zero padding at the end of H (bottom).
padding_list int32[] [] The original 2-D padding as [top, left, bottom, right], kept for reference. The computation uses the four *_padding attributes.
group_number int32 required Number of channel groups; equals in_channels, one group per input channel.
weight Weight required Filter, [out_channels, 1, h_kernel, w_kernel].
bias Weight optional Bias, [out_channels]. Absent means zero.
channel_last bool true Must be true; the NPU uses the NHWC layout.

Outputs

Name Shape Description
out[0] [1, H_out, W_out, out_channels] Output activation.

Div

Element-wise quotient of two activations.

Divides in[0] by in[1] element by element (true division).

out[0] = in[0] / in[1]

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.
in[1] broadcastable to in[0] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] broadcast(in[0], in[1]) Output activation.

DivConstant

Divides an activation by a constant, or a constant by an activation.

Divides with one operand taken from a constant stored in the model. The constant may be a scalar, a per-channel vector or a full tensor. With is_const_first true the result is constant / in[0], otherwise in[0] / constant.

out[0] = constant / in[0]   if is_const_first
         in[0] / constant   otherwise

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
constant Weight required Constant operand. Broadcasts against in[0].
is_const_first bool required true if the constant is the dividend (left operand), false if it is the divisor.
shape int32[] [] Shape of the constant operand before the stored constant was reduced (e.g. to a per-channel vector). The output shape is broadcast(shape(in[0]), shape). The default [] means the tensor's shape was not recorded; the field is optional.

Outputs

Name Shape Description
out[0] broadcast(in[0], shape) Output activation.

Elu

Exponential linear unit with negative-side scale leaky_alpha.

Passes positive inputs through and maps the rest to leaky_alpha * (exp(x) - 1), which saturates at -leaky_alpha.

out[0] = in[0]                            if in[0] > 0
         leaky_alpha * (exp(in[0]) - 1)   otherwise

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
leaky_alpha float32 required ELU alpha, the scale of the negative branch.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Erf

Element-wise Gauss error function.

out[0] = erf(in[0]) = 2 / sqrt(pi) * integral_0^in[0] exp(-t^2) dt

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Exp

Element-wise natural exponential.

out[0] = exp(in[0])

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Gelu

Gaussian error linear unit.

Applies GELU element by element. approximate selects between the erf form and the tanh approximation.

approximate = false:  out[0] = 0.5 * x * (1 + erf(x / sqrt(2)))
approximate = true:   out[0] = 0.5 * x * (1 + tanh(sqrt(2/pi) * (x + 0.044715 * x^3)))
where x = in[0]

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
approximate bool false true for the tanh approximation, false for the exact erf form.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

GroupConvolution

Grouped 2-D convolution of an NHWC activation with a constant filter.

Computes the same grouped convolution as Convolution. The channels are split into group_number groups, and each group's out_channels / group_number outputs see only that group's in_channels / group_number inputs.

G  = group_number,  Cg = in_channels / G,  Og = out_channels / G   (// : integer division)
xp = zero_pad(in[0], top=north_padding, bottom=south_padding,
                     left=west_padding, right=east_padding)
out[0][n, y, x, o] = bias[o] + sum_{ky, kx, c < Cg}
    weight[o, c, ky, kx]
    * xp[n, y*h_stride + ky*h_dilation, x*w_stride + kx*w_dilation, (o // Og)*Cg + c]

Inputs

Name Shape Description
in[0] [1, H, W, in_channels] Input activation.

Attributes

Name Type Default Description
in_channels int32 required Number of input channels.
out_channels int32 required Number of output channels.
h_kernel int32 required Kernel height.
w_kernel int32 required Kernel width.
h_stride int32 required Stride along H.
w_stride int32 required Stride along W.
h_dilation int32 required Dilation along H.
w_dilation int32 required Dilation along W.
west_padding int32 required Zero padding at the start of W (left).
east_padding int32 required Zero padding at the end of W (right).
north_padding int32 required Zero padding at the start of H (top).
south_padding int32 required Zero padding at the end of H (bottom).
padding_list int32[] [] The original 2-D padding as [top, left, bottom, right], kept for reference. The computation uses the four *_padding attributes.
group_number int32 required Number of channel groups. Divides both in_channels and out_channels.
weight Weight required Filter, [out_channels, in_channels / group_number, h_kernel, w_kernel].
bias Weight optional Bias, [out_channels]. Absent means zero.
channel_last bool true Must be true; the NPU uses the NHWC layout.

Outputs

Name Shape Description
out[0] [1, H_out, W_out, out_channels] Output activation.

GroupNormalization

Normalizes each group of channels over H, W and the group, then scales and shifts.

Splits the C channels into num_groups contiguous groups and normalizes each group of each batch item to zero mean and unit variance over H, W and the group's channels. A per-channel scale and bias follow. The variance is the population variance.

G = num_groups,  Cg = C / G,  g(c) = floor(c / Cg)
S(n, g)   = in[0][n, :, :, g*Cg : (g+1)*Cg]
mu[n, g]  = mean(S(n, g))
var[n, g] = mean((S(n, g) - mu[n, g])^2)
out[0][n, h, w, c] = (in[0][n, h, w, c] - mu[n, g(c)]) / sqrt(var[n, g(c)] + epsilon)
                     * scale[c] + bias[c]

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
num_groups int32 required Number of channel groups. Divides C; each group is C / num_groups consecutive channels.
scale Weight required Per-channel scale (gamma).
bias Weight optional Per-channel shift (beta). Absent means zero.
epsilon Weight required Scalar added to the variance for numerical stability.
channel_last bool true Must be true; the NPU uses the NHWC layout.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

HardSigmoid

Piecewise-linear sigmoid, clamp(alpha * x + beta, 0, 1).

A linear ramp clamped to [0, 1]. An absent alpha or beta takes 1/6 or 0.5.

out[0] = clamp(alpha * in[0] + beta, 0, 1)

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
alpha float32 optional Slope of the ramp. Absent means 1/6.
beta float32 optional Offset of the ramp. Absent means 0.5.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

HardSwish

Piecewise swish, x * clamp(x + 3, 0, 6) / 6.

out[0] = in[0] * clamp(in[0] + 3, 0, 6) / 6

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

HeaderView

Splits the channel axis into attention heads and stacks them along H.

One operation for the reshape-transpose-reshape that turns a [1, H, W, heads*D] projection into per-head rows of D channels. The rows of all heads are stacked along H.

D = C / num_heads
batch_order = 0 (head-minor rows, row = h*num_heads + head):
  [1, H, W, C] -reshape-> [1, H, W, num_heads, D]
               -transpose(0,1,3,2,4)-> [1, H, num_heads, W, D]
               -reshape-> [1, H*num_heads, W, D]
batch_order = 1 (head-major rows, row = head*H + h):
  [1, H, W, C] -reshape-> [1, H, W, num_heads, D]
               -transpose(0,3,1,2,4)-> [1, num_heads, H, W, D]
               -reshape-> [1, num_heads*H, W, D]

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
view_shape int32[] [] Deprecated; ignored. Use num_heads.
num_heads int32 -1 Number of heads. Divides C.
batch_order int32 0 Row order of the stacked heads: 0 interleaves heads within each row of H, 1 places each head's H rows contiguously.

Outputs

Name Shape Description
out[0] [1, H*num_heads, W, C/num_heads] Output activation.

HeaderViewRevert

Unstacks attention heads from H and merges them back into channels.

Takes per-head rows of D channels stacked along H, [1, num_heads*H, W, D], and concatenates the heads along the channel axis into [1, H, W, num_heads*D]. batch_order must be the value the rows were stacked with.

in[0] is [1, num_heads*H, W, D]
batch_order = 0 (input row = h*num_heads + head):
  -reshape-> [1, H, num_heads, W, D]
  -transpose(0,1,3,2,4)-> [1, H, W, num_heads, D]
  -reshape-> [1, H, W, num_heads*D]
batch_order = 1 (input row = head*H + h):
  -reshape-> [1, num_heads, H, W, D]
  -transpose(0,2,3,1,4)-> [1, H, W, num_heads, D]
  -reshape-> [1, H, W, num_heads*D]

Inputs

Name Shape Description
in[0] [1, num_heads*H, W, D] Input activation.

Attributes

Name Type Default Description
revert_shape int32[] [] Deprecated; ignored. Use num_heads.
num_heads int32 -1 Number of heads. Divides the H extent of in[0].
batch_order int32 0 Row order of the stacked heads in in[0]: 0 if heads alternate within each row of H, 1 if each head's H rows are contiguous. Same meaning as in HeaderView.

Outputs

Name Shape Description
out[0] [1, H, W, num_heads*D] Output activation.

Input

Marks an activation as a graph input.

Declares that in[0] is supplied from outside the graph. The operand and the result are the same activation, so the operation computes nothing; it only fixes the activation as a graph input.

out[0] = in[0]      (the same activation)

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
orig_name str "" User-facing name of this input in the source model, kept so the compiled model exposes the same input names.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

InputConstant

Provides a constant stored in the model as an activation.

Has no operands; its result is the stored constant, with element type dtype and shape shape.

out[0] = constant

Inputs

None.

Attributes

Name Type Default Description
constant Weight required The constant value.
dtype str "" Element type name of the constant, e.g. float32 or int64.
shape int32[] [] Shape of the constant, which is the shape of out[0].

Outputs

Name Shape Description
out[0] shape Output activation.

InstanceNormalization

Normalizes each channel over H and W.

Normalizes every channel of every batch item to zero mean and unit variance over H and W, then applies per-channel scale and bias. The variance is the population variance. Normalization that shares statistics across several channels (num_groups other than -1) is expressed with GroupNormalization.

mu[n, c]  = mean(in[0][n, :, :, c])
var[n, c] = mean((in[0][n, :, :, c] - mu[n, c])^2)
out[0][n, h, w, c] = (in[0][n, h, w, c] - mu[n, c]) / sqrt(var[n, c] + epsilon)
                     * scale[c] + bias[c]

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
scale Weight required Scale (gamma).
bias Weight optional Shift (beta), shaped like scale. Absent means zero.
epsilon Weight required Scalar added to the variance for numerical stability.
channel_last bool true Must be true; the NPU uses the NHWC layout.
num_groups int32 -1 Always -1. Grouped normalization is GroupNormalization.
group_order int32 -1 Always -1; unused.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

InverseSigmoid

Inverse of the sigmoid, log(x / (1 - x)), kept finite by eps.

Clamps in[0] to [0, 1], then keeps the numerator and the denominator of x / (1 - x) at or above eps, so inputs at or beyond 0 and 1 still give finite results.

x1 = max(clamp(in[0], 0, 1), eps)
x2 = max(1 - x1, eps)
out[0] = log(x1 / x2)

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
eps float32 1e-05 Lower bound applied to the numerator and the denominator.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

L1Normalization

Divides by the sum of absolute values over the given axes.

Scales in[0] so that its absolute values along norm_axis sum to 1, per position of the other axes. The divisor is bounded below by 1e-12.

s      = sum(abs(in[0]), norm_axis, keep_dims=true)
out[0] = in[0] / max(s, 1e-12)

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
norm_axis int32[] required Axes to normalize over, as indices into in[0] as laid out. Negative values count from the last axis.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

L2Normalization

Divides by the Euclidean norm over the given axes.

Scales in[0] to unit L2 norm along norm_axis, per position of the other axes. The norm is bounded below by epsilon to avoid division by zero.

r      = sqrt(sum(in[0]^2, norm_axis, keep_dims=true))
out[0] = in[0] / max(r, epsilon)

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
norm_axis int32[] required Axes to normalize over, as indices into in[0] as laid out. Negative values count from the last axis.
epsilon Weight optional Lower bound on the norm. Absent means 1e-12.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

LayerNormalization

Normalizes over the trailing axes, then scales and shifts.

Normalizes in[0] to zero mean and unit variance over its last len(normalized_shape) axes, then applies the element-wise affine scale and bias. Mean and variance are taken per position of the remaining leading axes; the variance is the biased (population) variance.

axes   = the last len(normalized_shape) axes of in[0]
mu     = mean(in[0], axes)
var    = mean((in[0] - mu)^2, axes)
out[0] = (in[0] - mu) / sqrt(var + epsilon) * scale + bias

Inputs

Name Shape Description
in[0] [..., *normalized_shape] Input activation.

Attributes

Name Type Default Description
normalized_shape int64[] [] Shape of the normalized trailing axes, e.g. [C] to normalize over the channel axis of an NHWC activation.
scale Weight required Per-element scale (gamma), shape normalized_shape.
bias Weight optional Per-element shift (beta), shape normalized_shape. Absent means zero.
epsilon Weight required Scalar added to the variance for numerical stability.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

LeakyRelu

ReLU with the scalar slope leaky_alpha for negative inputs.

Multiplies negative elements by one scalar slope.

out[0] = in[0]                 if in[0] >= 0
         leaky_alpha * in[0]   otherwise

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
leaky_alpha float32 required Slope for negative inputs.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Log

Element-wise natural logarithm.

out[0] = ln(in[0])

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

MatMul

Batched matrix product of two activations over their last two axes.

Multiplies in[0] by in[1] as matrices over their last two axes; all leading axes are batch axes and broadcast. For 4-D NHWC activations the matrices are [W, C] and N, H are batch axes.

a = in[0],  b = in[1]
repeat = [0, R]:  a = repeat_interleave(a, R, axis=1)
repeat = [1, R]:  b = repeat_interleave(b, R, axis=1)
  where repeat_interleave(x, R, axis=1)[n, h, ...] = x[n, floor(h / R), ...]
out[0][..., m, p] = sum_k a[..., m, k] * b[..., k, p]

Inputs

Name Shape Description
in[0] [..., M, K] Input activation.
in[1] [..., K, P] Input activation.

Attributes

Name Type Default Description
constant Weight optional Unused; always -1. A constant matrix is passed as an activation from InputConstant.
is_const_first bool false Unused; always false.
repeat int32[2] [-1, -1] [operand, R]: before the product, repeat operand operand (0 = in[0], 1 = in[1]) R times along axis 1 (H), each H slice R times in place (h0 h0 h1 h1 ..., not h0 h1 h0 h1 ...). [-1, -1]: no repeat.

Outputs

Name Shape Description
out[0] [broadcast of the leading axes, M, P] Output activation.

Mish

Self-gated activation x * tanh(softplus(x)).

out[0] = in[0] * tanh(log(1 + exp(in[0])))

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Multiply

Element-wise product of two activations.

Multiplies two activations element by element.

out[0] = in[0] * in[1]

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.
in[1] broadcastable to in[0] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] broadcast(in[0], in[1]) Output activation.

MultiplyConstant

Multiplies an activation by a constant tensor.

Multiplies one activation by a constant stored in the model. The constant may be a scalar, a per-channel vector or a full tensor.

out[0] = constant * in[0]   if is_const_first
         in[0] * constant   otherwise

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
constant Weight required Constant operand. Broadcasts against in[0].
is_const_first bool false true if the constant is the left operand.
shape int32[] [] Shape of the constant operand before the stored constant was reduced (e.g. to a per-channel vector). The output shape is broadcast(shape(in[0]), shape). The default [] means the tensor's shape was not recorded; the field is optional.

Outputs

Name Shape Description
out[0] broadcast(in[0], shape) Output activation.

OrderedPatchify

Partitions H x W into windows and lays the pixels out window by window.

One operation for the reshape-transpose-reshape that cuts a [1, H, W, C] map into a gh x gw grid of h_kernel x w_kernel windows and flattens it into a 2-D layout on H and W. order=0 gives one row per window, holding its pixels in raster order; order=1 is its H/W transpose, one row per pixel offset inside the window; order=2 undoes order=1. Channels are moved as a whole and never mixed.

gh = H / h_kernel,  gw = W / w_kernel
x = reshape(in[0], [1, gh, h_kernel, gw, w_kernel, C])
order 0: out[0] = reshape(transpose(x, 0,1,3,2,4,5), [1, gh*gw, h_kernel*w_kernel, C])
order 1: out[0] = reshape(transpose(x, 0,2,4,1,3,5), [1, h_kernel*w_kernel, gh*gw, C])
order 2: out[0] = reshape(transpose(x, 0,3,1,4,2,5), [1, gw*gh, w_kernel*h_kernel, C])
Element map, p < gh, q < h_kernel, u < gw, v < w_kernel:
  in[0][0, p*h_kernel + q, u*w_kernel + v, c] =
    order 0: out[0][0, p*gw + u, q*w_kernel + v, c]
    order 1: out[0][0, q*w_kernel + v, p*gw + u, c]
    order 2: out[0][0, u*gh + p, v*h_kernel + q, c]

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
h_kernel int32 required Window height. Divides H.
w_kernel int32 required Window width. Divides W.
order int32 0 Layout: 0 one row per window, 1 one row per in-window offset (the transpose of 0), 2 the inverse of 1. See the formula.

Outputs

Name Shape Description
out[0] order 0, 2: [1, ghgw, h_kernelw_kernel, C]; order 1: [1, h_kernelw_kernel, ghgw, C] Output activation.

Output

Marks an activation as a graph output.

Declares that in[0] is returned from the graph. The operand and the result are the same activation, so the operation computes nothing; it only fixes the activation as a graph output.

out[0] = in[0]      (the same activation)

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
orig_name str "" User-facing name of this output in the source model, kept so the compiled model exposes the same output names.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Pad

Pads the H and W axes of an NHWC activation with a constant or reflection.

Pads in[0] by north_padding above, south_padding below, west_padding to its left and east_padding to its right. The batch and channel axes are not padded.

H_out = north_padding + H + south_padding
W_out = west_padding  + W + east_padding
out[0][n, north_padding + i, west_padding + j, c] = in[0][n, i, j, c]
other cells:
  constant:  constant
  reflect:   in[0] mirrored about its first/last row and column, edge not
             repeated (the row above row 0 is row 1)

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
west_padding int32 -1 Columns added at the start of W (left).
east_padding int32 -1 Columns added at the end of W (right).
south_padding int32 -1 Rows added at the end of H (bottom).
north_padding int32 -1 Rows added at the start of H (top).
pad_mode str required constant fills the new cells with constant; reflect mirrors in[0] about its border without repeating the edge row or column.
constant float32 0.0 Fill value for pad_mode = constant.
padding_list int32[] optional The same padding written for every axis as [x1_begin, x2_begin, ..., x1_end, x2_end, ...], for NHWC [0, top, left, 0, 0, bottom, right, 0], kept for reference. The computation uses the four *_padding attributes.
channel_last bool true Must be true; the NPU uses the NHWC layout.

Outputs

Name Shape Description
out[0] [1, H_out, W_out, C] Output activation.

Pooling

2-D max or average pooling over sliding windows of an NHWC activation.

Pads in[0] by the *_padding attributes with the fill chosen by pad_mode, then takes the maximum (MaxPool) or the average (AvgPool) of each h_kernel x w_kernel window, moved by the stride and spread by the dilation. Channels are pooled independently.

xp = pad(in[0], top=north_padding, bottom=south_padding,
                left=west_padding, right=east_padding)    (fill per pad_mode)
win(n, y, x, c) = { xp[n, y*h_stride + ky*h_dilation, x*w_stride + kx*w_dilation, c]
                    : ky < h_kernel, kx < w_kernel }
MaxPool:  out[0][n, y, x, c] = max(win(n, y, x, c))
AvgPool:  out[0][n, y, x, c] = sum(win(n, y, x, c)) / cnt
          cnt = h_kernel * w_kernel                    if is_include_pad
                number of window cells inside in[0]    otherwise

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
h_kernel int32 required Window height.
w_kernel int32 required Window width.
h_stride int32 required Stride along H.
w_stride int32 required Stride along W.
h_dilation int32 1 Dilation along H.
w_dilation int32 1 Dilation along W.
west_padding int32 required Padding at the start of W (left).
east_padding int32 required Padding at the end of W (right).
north_padding int32 required Padding at the start of H (top).
south_padding int32 required Padding at the end of H (bottom).
padding_list int32[] [] The original 2-D padding as [top, left, bottom, right], kept for reference; may be empty. The computation uses the four *_padding attributes.
pool_type str required MaxPool for the window maximum, AvgPool for the window average.
pad_mode str required Fill of the padded border: constant (zeros), -inf (negative infinity, so padding never wins a maximum) or reflect (mirror image of the border, edge not repeated). -inf is not combined with AvgPool and is_include_pad = true.
is_ceil_mode bool false Records that the source pooling rounded the output size up. Informational; the output size always follows the floor formula below, so a rounded-up size appears as extra padding in south_padding (bottom) and east_padding (right).
is_include_pad bool false AvgPool only. true divides by the full window size h_kernel * w_kernel, counting padded cells; false divides by the number of window cells that lie inside in[0].
channel_last bool true Must be true; the NPU uses the NHWC layout.

Outputs

Name Shape Description
out[0] [1, H_out, W_out, C] Output activation.

Pow

Raises each element to a constant scalar power.

Raises every element of in[0] to the same exponent.

out[0] = in[0] ^ exponent

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
exponent float32 required Scalar exponent.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

PRelu

Leaky ReLU whose negative slope is a constant tensor.

Multiplies negative elements by the slope leaky_alpha, a tensor stored in the model. The slope broadcasts against in[0], aligned from the last axis.

out[0] = in[0]                 if in[0] >= 0
         leaky_alpha * in[0]   otherwise   (leaky_alpha broadcast to in[0])

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
leaky_alpha Weight required Slope for negative inputs; broadcastable to in[0].

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

QuickGelu

Sigmoid approximation of GELU, x * sigmoid(1.702 * x).

Approximates Gelu by gating with sigmoid(1.702 * x).

out[0] = in[0] * sigmoid(1.702 * in[0])

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

ReduceMax

Maximum over the given axes.

Takes the largest element of in[0] over reduce_axes, separately for each position of the remaining axes.

out[0] = max(in[0], reduce_axes, keep_dims)

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
keep_dims bool required true keeps each reduced axis with size 1; false removes it.
reduce_axes int32[] required Axes to reduce, as indices into in[0] as laid out (for an NHWC activation 3 is the channel axis). Negative values count from the last axis.
noop_with_empty_axes int32 required Only matters when reduce_axes is empty - 1 returns in[0] unchanged, 0 reduces over every axis. The IR normally lists the reduced axes explicitly.

Outputs

Name Shape Description
out[0] in[0] with the reduced axes set to 1 or removed Output activation.

ReduceMean

Arithmetic mean over the given axes.

Averages in[0] over reduce_axes, separately for each position of the remaining axes.

out[0] = sum(in[0], reduce_axes, keep_dims) / (product of in[0]'s sizes on reduce_axes)

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
keep_dims bool required true keeps each reduced axis with size 1; false removes it.
reduce_axes int32[] required Axes to reduce, as indices into in[0] as laid out (for an NHWC activation 3 is the channel axis). Negative values count from the last axis.
noop_with_empty_axes int32 required Only matters when reduce_axes is empty - 1 returns in[0] unchanged, 0 reduces over every axis. The IR normally lists the reduced axes explicitly.

Outputs

Name Shape Description
out[0] in[0] with the reduced axes set to 1 or removed Output activation.

ReduceMin

Minimum over the given axes.

Takes the smallest element of in[0] over reduce_axes, separately for each position of the remaining axes.

out[0] = min(in[0], reduce_axes, keep_dims)

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
keep_dims bool required true keeps each reduced axis with size 1; false removes it.
reduce_axes int32[] required Axes to reduce, as indices into in[0] as laid out (for an NHWC activation 3 is the channel axis). Negative values count from the last axis.
noop_with_empty_axes int32 required Only matters when reduce_axes is empty - 1 returns in[0] unchanged, 0 reduces over every axis. The IR normally lists the reduced axes explicitly.

Outputs

Name Shape Description
out[0] in[0] with the reduced axes set to 1 or removed Output activation.

ReduceSum

Sum over the given axes.

Adds up the elements of in[0] over reduce_axes, separately for each position of the remaining axes.

out[0] = sum(in[0], reduce_axes, keep_dims)

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
keep_dims bool required true keeps each reduced axis with size 1; false removes it.
reduce_axes int32[] required Axes to reduce, as indices into in[0] as laid out (for an NHWC activation 3 is the channel axis). Negative values count from the last axis.
noop_with_empty_axes int32 required Only matters when reduce_axes is empty - 1 returns in[0] unchanged, 0 reduces over every axis. The IR normally lists the reduced axes explicitly.

Outputs

Name Shape Description
out[0] in[0] with the reduced axes set to 1 or removed Output activation.

RelShift

Re-indexes relative-offset attention scores by absolute key position.

Relative-position shift of Transformer-XL style attention. Row i of in[0] holds the scores of query i for 2Q - 1 relative offsets, with column k meaning query minus key = Q - 1 - k. The op shifts row i left by Q - 1 - i and keeps Q columns, so column j of the result is the score for key j.

out[0][n, h, i, j] = in[0][n, h, i, Q - 1 - i + j],   i, j < Q
Equivalent sequence on [1, H, Q, P], P = 2Q - 1:
  pad one zero at the start of axis 3   -> [1, H, Q, P+1]
  reshape                               -> [1, H, P+1, Q]
  drop the first row of axis 2          -> [1, H, P, Q]
  reshape                               -> [1, H, Q, P]
  slice axis 3 to [0, Q)                -> [1, H, Q, Q]

Inputs

Name Shape Description
in[0] [1, H, Q, 2Q-1] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] [1, H, Q, Q] Output activation.

Relu

Rectified linear unit, max(x, 0).

out[0] = max(in[0], 0)

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Reshape

Changes the shape of an activation without moving its elements.

Returns in[0] with the shape new_shape. Elements keep their row-major order (last axis fastest).

out[0] = reshape(in[0], new_shape)
flatten(out[0]) == flatten(in[0])   (same elements, row-major order)

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
new_shape int32[] optional Target shape. One entry may be -1; it is inferred so that the element count stays the same. A 0 entry is read as allowzero says; graph builders write explicit extents, so 0 normally does not appear.
allowzero int32 0 How a 0 in new_shape is read: 0 copies the extent of in[0] at the same index, 1 makes a zero-length axis.
is_const_first bool false Not used by the computation; in[0] is always the data.

Outputs

Name Shape Description
out[0] new_shape, with -1 resolved Output activation.

Resize

Resizes the spatial axes by nearest, bilinear or bicubic interpolation.

Resamples in[0] to a new H x W. sizes and scales hold one entry per axis of in[0], in its own axis order, and only the two spatial axes may differ from the input.

shape(out[0]) = sizes                     if sizes is given
                shape(in[0]) * scales     otherwise, axis by axis
s_h = H_out / H,  s_w = W_out / W         (or the matching scales entries)
nearest:  out[0][n, y, x, c] = in[0][n, floor(y / s_h), floor(x / s_w), c]
linear:   bilinear blend of the 2 x 2 input pixels around (ys, xs)
cubic:    bicubic blend of the 4 x 4 input pixels around (ys, xs)
where the source row ys of output row y follows transform_mode
(columns likewise with x, W, W_out, s_w):
  half_pixel:     ys = (y + 0.5) / s_h - 0.5
  align_corners:  ys = y * (H - 1) / (H_out - 1)
  asymmetric:     ys = y / s_h
source positions outside the input are clamped to the border

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
resize_type str optional Interpolation: nearest, linear (bilinear over H and W) or cubic (bicubic).
resize_mode str optional The interpolation mode as recorded when the graph was built; same values as resize_type, may be absent. The computation reads resize_type.
nearest_mode str optional Rounding rule of the source model for nearest (round_prefer_floor, round_prefer_ceil, floor or ceil). Recorded only; nearest always rounds down, as in the formula.
transform_mode str optional Mapping from output to input coordinates for linear and cubic: half_pixel (pixel centres aligned; also applies to any other recorded value), align_corners (corner pixels aligned) or asymmetric (output index divided by the scale); see the formula. nearest always uses the asymmetric mapping.
roi int32[] optional Region of interest of a crop-and-resize. Recorded only; the computation does not read it.
scales float32[] optional Scale factor per axis of in[0], 1.0 on axes that keep their size; for example [1, 2, 2, 1] doubles H and W of an NHWC tensor.
sizes int32[] optional Output shape, one entry per axis of in[0].
cubic_coeff_a float32 optional Coefficient a of the cubic kernel for cubic. Recorded only; the computation does not read it.
exclude_outside int32 optional For cubic, 1 drops kernel taps outside the input and renormalizes the rest. Recorded only; the computation does not read it.
extrapolation_value float32 optional Value for output pixels whose source lies outside the input in a crop-and-resize. Recorded only; the computation does not read it.
antialias int32 optional 1 applies a low-pass (antialiasing) filter when downscaling with linear or cubic; 0 or absent does not.
axes int32[] optional Axes the source resize named for its scales or sizes. Recorded only; in the IR, scales and sizes always cover every axis.
keep_aspect_ratio_policy str optional How the source fitted its target size to keep the aspect ratio (stretch, not_larger, not_smaller). Recorded only; the computation does not read it.

Outputs

Name Shape Description
out[0] [1, H_out, W_out, C] Output activation.

RmsNormalization

Divides by the root mean square over the trailing axes, then scales.

Divides in[0] by its root mean square over the last len(normalized_shape) axes and multiplies by scale. With rms_center, the RMS is taken of in[0] - rms_center while the numerator stays in[0].

axes   = the last len(normalized_shape) axes of in[0]
center = rms_center if present, else 0
rms    = sqrt(mean((in[0] - center)^2, axes, keep_dims=true) + epsilon)
out[0] = in[0] / rms * scale

Inputs

Name Shape Description
in[0] [..., *normalized_shape] Input activation.

Attributes

Name Type Default Description
normalized_shape int64[] [] Shape of the trailing axes to normalize over, normally [C], the channel axis of an NHWC activation.
epsilon Weight required Scalar added to the mean square for numerical stability.
scale Weight required Per-element scale (gamma), shape normalized_shape.
rms_center Weight optional Per-channel offset subtracted from in[0] inside the RMS only. Absent means 0.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Sigmoid

Logistic sigmoid with an optional input scale alpha.

Computes 1 / (1 + exp(-alpha * x)).

out[0] = 1 / (1 + exp(-alpha * in[0]))

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
alpha float32 1.0 Scale applied to the input before the sigmoid.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Sin

Element-wise sine.

out[0] = sin(in[0])

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Slice

Extracts a strided sub-range along one or more axes.

On each listed axis, keeps the elements from start up to but not including end, every step; the other axes stay whole. The lists are read position by position, so entry k of start, end and step applies to axis axis[k].

for each k, along axis a = axis[k] with length D = in[0].shape[a]:
    s = start[k] (+ D if negative), clamped to [0, D]
    e = end[k]   (+ D if negative), clamped to [0, D]
    keep indices s, s + step[k], s + 2*step[k], ... while < e
out[0] = in[0] restricted to the kept indices on every listed axis

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
axis int32[] required Axes to slice; negative values count from the last axis.
start int64[] required First index kept on each axis (inclusive).
end int64[] required End index on each axis (exclusive). A value past the axis length is clamped, so a large value such as 2147483647 means "to the end".
step int64[] optional Stride on each axis. Absent means 1 on every axis.

Outputs

Name Shape Description
out[0] in[0] with each sliced axis shortened Output activation.

Softmax

Normalizes exponentials along one axis so they sum to 1.

Computes softmax over the axis axis of in[0], optionally scaling the input by beta first.

z = beta * in[0]
out[0] = exp(z) / sum(exp(z), axis)

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
beta float32 1.0 Scale applied to the input before the exponential. Must not be 0.
axis int32 required Axis to normalize over. A negative value counts from the last axis.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Softplus

Smooth ReLU log(1 + exp(beta * x)) / beta, linear above threshold.

Where beta * x exceeds threshold the output is x itself.

out[0] = in[0]                               if beta * in[0] > threshold
         log(1 + exp(beta * in[0])) / beta   otherwise

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
beta float32 1.0 Sharpness of the curve; larger values approach ReLU.
threshold float32 20.0 Value of beta * x above which the output is linear.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Sqrt

Element-wise square root.

out[0] = sqrt(in[0])

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

StaggeredPadding

Shifts row r of each matrix by r columns with zero fill, then crops.

Treats the last two axes of [1, H, W, C] as W rows of length C and shifts each row by its row index into a zero-filled row, giving a staircase layout. reverse=false shifts row r right by r and drops window_size columns at each side; reverse=true reads, for each row r, the 2*window_size + 1 columns centered on column r.

x[n, h, r, k] = in[0][n, h, r, k] for 0 <= k < C, and 0 for any other k
reverse = false:
  o = 0 if window_size = -1, else window_size
  out[0][n, h, r, j] = x[n, h, r, j + o - r],        j < W + C - 1 - 2*o
reverse = true:
  out[0][n, h, r, j] = x[n, h, r, r + j - window_size],   j < 2*window_size + 1
Equivalent sequence for reverse = false:
  [1, H, W, C] -pad axis 3 end by W-> [1, H, W, C+W] -reshape-> [1, H, W*(C+W)]
  -drop the last W-> -reshape-> [1, H, W, W+C-1] -slice axis 3 [o, W+C-1-o)

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
axis int32 3 Axis whose entries are shifted. Only 3 (the last axis) is defined; rows run along axis 2.
reverse bool false false: row r starts at column r (staircase toward the lower right). true: row r starts at column W - 1 - r (toward the lower left), and the result is cropped to a 2*window_size + 1 band.
window_size int32 -1 reverse=false: columns dropped at each side, -1 to keep the full width W + C - 1. reverse=true: half-width of the band; must be given (not -1), and W and C are each at least window_size + 1.

Outputs

Name Shape Description
out[0] reverse=false: [1, H, W, W + C - 1 - 2o]; reverse=true: [1, H, W, 2window_size + 1] Output activation.

StatefulAttentionMaskWrapper

Adds a causal, sliding-window or supplied mask to attention scores.

Applies the attention mask to raw attention scores. With causal_mask the bias is derived from the score shape alone, taking the Q queries as the last Q of the K key positions. With custom_mask the bias is a second input.

custom_mask:
  out[0] = in[0] + in[1]
causal_mask:
  qp(i) = K - Q + i                        key position of query i
  masked(i, j) = j > qp(i)  or  (is_sliding and j <= qp(i) - sliding_window)
  out[0][..., i, j] = -inf if masked(i, j) else in[0][..., i, j]

Inputs

Name Shape Description
in[0] [1, heads, Q, K] Input activation.
in[1] broadcastable to in[0] Input activation.

Attributes

Name Type Default Description
past_seqlen int32 0 Number of cached positions in front of the queries when the graph was traced. Informational; the mask follows the runtime Q and K.
is_sliding bool false true also masks keys sliding_window or more positions behind the query (causal_mask only).
sliding_window int32 1024 Number of most recent key positions, the query's own included, that a query may attend to when is_sliding.
mask_type str "causal_mask" causal_mask: mask synthesized from positions, one input. custom_mask: adds in[1], two inputs.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

StatefulCachedStack

Prepends the last entries kept from previous calls along an axis.

Keeps the last cache_len entries along axis across calls and prepends them to each new input. The first call sees zeros; later calls see the tail of the earlier inputs. The kept window is state held by the op, not an operand.

state: entries of shape in[0] with axis extent cache_len, initially zeros
out[0] = concat(state, in[0], axis)
state  = last cache_len entries of out[0] along axis
cache_len = 0:  out[0] = in[0]

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
axis int32 2 Axis to stack along, by default 2 (the sequence axis W of NHWC). Negative counts from the last axis.
cache_len int32 0 Number of entries kept and prepended. 0 makes the op an identity.
cache_name str required Name of the kept window.

Outputs

Name Shape Description
out[0] in[0] with axis extent + cache_len Output activation.

StatefulGRUWrapper

GRU over a sequence, returning per-step outputs and the final hidden state.

Runs a gated recurrent unit recurrence along the time axis of in[0] with the constant weights W, R, B, in one or both directions. input_mask names the operands that are present; the results are always out[0] (hidden state at every step) and out[1] (final hidden state).

batch_first = true; T = steps; @ = matrix product, * = element-wise product
for each direction d (0 = forward, 1 = backward):
  act_f, act_g = activations[2d .. 2d+1]          (activations[0..1] if only 2)
  Wz, Wr, Wh = W[d] split into 3 row blocks;  Rz, Rr, Rh = R[d] likewise
  Wbz, Wbr, Wbh, Rbz, Rbr, Rbh = B[d] split into 6   (zero if no B)
  h = in[h][:, 0, d, :]                              (zero if absent)
  for t = 0 .. T-1 (forward) or t = T-1 .. 0 (backward):
    x = in[0][:, 0, t, :]
    z = act_f(x @ Wz^T + h @ Rz^T + Wbz + Rbz)
    r = act_f(x @ Wr^T + h @ Rr^T + Wbr + Rbr)
    n = act_g(x @ Wh^T + Wbh + (r * h) @ Rh^T + Rbh)   if not linear_before_reset
        act_g(x @ Wh^T + Wbh + r * (h @ Rh^T + Rbh))   if linear_before_reset
    h = (1 - z) * n + z * h
    out[0][:, d, t, :] = h
  out[1][:, 0, d, :] = h

Inputs

Name Shape Description
in[0] [1, 1, T, input_size] Input activation.
in[s] any Input activation.
in[h] [1, 1, num_directions, hidden_size] Input activation.

Attributes

Name Type Default Description
W Weight required Input weights, [num_directions, 3*hidden_size, input_size], gate blocks in the order z (update), r (reset), h (candidate).
R Weight required Recurrent weights, [num_directions, 3*hidden_size, hidden_size], gate order z, r, h.
B Weight optional Biases, [num_directions, 6*hidden_size]: the input biases (z, r, h) followed by the recurrent biases (z, r, h). Absent means zero.
input_mask str required Letters naming the operands present, in operand order. i input sequence (always first), s sequence lengths, h initial hidden state; for example ih means in[0] is the sequence and in[1] the hidden state.
output_mask str required Letters among o, h naming the results that are used (out[0], out[1]). The positions of the two results do not change.
direction str "forward" forward, backward (reverse is treated the same) or bidirectional. bidirectional runs both and stacks forward (index 0) and backward (index 1) on the direction axis, so num_directions is 2; otherwise it is 1.
batch_first bool false true for the batch-first layout used in the shapes here. false is time-major: in[0] is [T, 1, input_size], the state is [num_directions, 1, hidden_size] and out[0] is [T, num_directions, 1, hidden_size].
hidden_size int32 required Number of hidden units per direction.
hidden_state_name str required Name of out[1] and of the state output of the cell subgraph.
subgraph str required Name of the cell subgraph that computes one time step from the step input and the previous hidden state.
activation_alpha float32[] optional Alpha coefficients for the functions in activations, in the same order. sigmoid and tanh take none.
activation_beta float32[] optional Beta coefficients for the functions in activations, in the same order. sigmoid and tanh take none.
activations str[] optional Gate functions [act_f, act_g], usually [sigmoid, tanh]. With bidirectional, four names give the forward then the backward functions; two names apply to both. The reference computation defines sigmoid and tanh.
clip float32 optional Cell clip threshold. When set, the gate inputs are limited to [-clip, clip] before the activation; absent means no clipping.
linear_before_reset bool false false applies the reset gate to the previous state before the recurrent product; true applies it to the product h @ Rh^T + Rbh.

Outputs

Name Shape Description
out[0] [1, num_directions, T, hidden_size] Output activation.
out[1] [1, 1, num_directions, hidden_size] Output activation.

StatefulKVCacheWrapper

Appends new key or value states to a persistent cache and returns it.

Keeps one layer's key or value cache across calls. Each call writes the S new states of in[0] at their absolute token positions and returns every state the current attention step can see, ordered by position. The cache and the token positions are runtime state, not IR operands.

pos = absolute positions of the S new tokens (from the runtime), p0 = pos[0]
the cache holds the states of earlier calls by position; in[0] is stored at pos
is_sliding = false:
  out[0] = states of positions 0 .. pos[S-1], concatenated on axis 2
is_sliding = true:
  out[0] = states of positions max(0, p0 - (sliding_window - 1)) .. pos[S-1]
  only the last sliding_window positions stay stored for later calls

Inputs

Name Shape Description
in[0] [1, heads, S, D] Input activation.

Attributes

Name Type Default Description
cache_name str required Name of the cache state this op reads and writes.
past_seqlen int32 0 Number of tokens already in the cache when the graph was traced. The static output length is S + past_seqlen (a dynamic extent when non-zero); at run time the length follows the token positions. At most sliding_window - 1 when is_sliding.
is_sliding bool false true keeps only the most recent sliding_window positions (sliding-window attention).
sliding_window int32 1024 Window length in tokens when is_sliding. Positive and at most max_cache_len.
max_cache_len int32 8192 Capacity of the cache in tokens.

Outputs

Name Shape Description
out[0] [1, heads, P + S, D], P = past tokens visible (p0, or min(p0, sliding_window - 1) when sliding) Output activation.

StatefulLSTMWrapper

LSTM over a sequence, returning per-step outputs and final hidden/cell states.

Runs a long short-term memory recurrence along the time axis of in[0] with the constant weights W, R, B and optional peepholes P, in one or both directions. input_mask names the operands that are present; the results are always out[0] (hidden state at every step), out[1] (final hidden state) and out[2] (final cell state).

batch_first = true; T = steps; @ = matrix product, * = element-wise product
for each direction d (0 = forward, 1 = backward):
  act_f, act_g, act_h = activations[3d .. 3d+2]   (activations[0..2] if only 3)
  Wi, Wo, Wf, Wc = W[d] split into 4 row blocks;  Ri, Ro, Rf, Rc = R[d] likewise
  Wbi, Wbo, Wbf, Wbc, Rbi, Rbo, Rbf, Rbc = B[d] split into 8   (zero if no B)
  Pi, Po, Pf = P[d] split into 3                                (zero if no P)
  h = in[h][:, 0, d, :],  c = in[c][:, 0, d, :]                 (zero if absent)
  for t = 0 .. T-1 (forward) or t = T-1 .. 0 (backward):
    x = in[0][:, 0, t, :]
    i = act_f(x @ Wi^T + h @ Ri^T + Wbi + Rbi + Pi * c)
    f = act_f(x @ Wf^T + h @ Rf^T + Wbf + Rbf + Pf * c)
    g = act_g(x @ Wc^T + h @ Rc^T + Wbc + Rbc)
    c = f * c + i * g
    o = act_f(x @ Wo^T + h @ Ro^T + Wbo + Rbo + Po * c)
    h = o * act_h(c)
    out[0][:, d, t, :] = h
  out[1][:, 0, d, :] = h
  out[2][:, 0, d, :] = c

Inputs

Name Shape Description
in[0] [1, 1, T, input_size] Input activation.
in[s] any Input activation.
in[h] [1, 1, num_directions, hidden_size] Input activation.
in[c] [1, 1, num_directions, hidden_size] Input activation.

Attributes

Name Type Default Description
W Weight required Input weights, [num_directions, 4*hidden_size, input_size], gate blocks in the order i, o, f, c.
R Weight required Recurrent weights, [num_directions, 4*hidden_size, hidden_size], gate order i, o, f, c.
B Weight optional Biases, [num_directions, 8*hidden_size]: the input biases (i, o, f, c) followed by the recurrent biases (i, o, f, c). Absent means zero.
input_mask str required Letters naming the operands present, in operand order. i input sequence (always first), s sequence lengths, h initial hidden state, c initial cell state; for example ihc means in[0..2] are the sequence, the hidden state and the cell state.
output_mask str required Letters among o, h, c naming the results that are used (out[0], out[1], out[2]). The positions of the three results do not change.
P Weight optional Peephole weights, [num_directions, 3*hidden_size], in the order i, o, f. Absent means no peephole.
direction str "forward" forward, backward (reverse is treated the same) or bidirectional. bidirectional runs both and stacks forward (index 0) and backward (index 1) on the direction axis, so num_directions is 2; otherwise it is 1.
batch_first bool false true for the batch-first layout used in the shapes here. false is time-major: in[0] is [T, 1, input_size], the states are [num_directions, 1, hidden_size] and out[0] is [T, num_directions, 1, hidden_size].
hidden_size int32 required Number of hidden units per direction.
hidden_state_name str required Name of out[1] and of the hidden-state output of the cell subgraph.
cell_state_name str required Name of out[2] and of the cell-state output of the cell subgraph.
subgraph str required Name of the cell subgraph that computes one time step from the step input and the previous hidden and cell states.
activation_alpha float32[] optional Alpha coefficients for the functions in activations, in the same order. sigmoid and tanh take none.
activation_beta float32[] optional Beta coefficients for the functions in activations, in the same order. sigmoid and tanh take none.
activations str[] optional Gate functions [act_f, act_g, act_h], usually [sigmoid, tanh, tanh]. With bidirectional, six names give the forward then the backward functions; three names apply to both. The reference computation defines sigmoid and tanh.
clip float32 optional Cell clip threshold. When set, the gate inputs are limited to [-clip, clip] before the activation; absent means no clipping.
input_forget bool false true couples the input and forget gates. The formula above is the false case.

Outputs

Name Shape Description
out[0] [1, num_directions, T, hidden_size] Output activation.
out[1] [1, 1, num_directions, hidden_size] Output activation.
out[2] [1, 1, num_directions, hidden_size] Output activation.

StatefulRNNWrapper

Simple RNN over a sequence, returning per-step outputs and the final state.

Runs the recurrence h = act_f(x W^T + h R^T + b) along the time axis of in[0] with the constant weights W, R, B, in one or both directions. input_mask names the operands that are present; the results are always out[0] (hidden state at every step) and out[1] (final hidden state).

batch_first = true; T = steps; @ = matrix product
for each direction d (0 = forward, 1 = backward):
  act_f = activations[d]            (activations[0] if only 1)
  Wb, Rb = B[d] split into 2        (zero if no B)
  h = in[h][:, 0, d, :]             (zero if absent)
  for t = 0 .. T-1 (forward) or t = T-1 .. 0 (backward):
    x = in[0][:, 0, t, :]
    h = act_f(x @ W[d]^T + h @ R[d]^T + Wb + Rb)
    out[0][:, d, t, :] = h
  out[1][:, 0, d, :] = h

Inputs

Name Shape Description
in[0] [1, 1, T, input_size] Input activation.
in[s] any Input activation.
in[h] [1, 1, num_directions, hidden_size] Input activation.

Attributes

Name Type Default Description
W Weight required Input weights, [num_directions, hidden_size, input_size].
R Weight required Recurrent weights, [num_directions, hidden_size, hidden_size].
B Weight optional Biases, [num_directions, 2*hidden_size]: the input bias followed by the recurrent bias. Absent means zero.
input_mask str required Letters naming the operands present, in operand order. i input sequence (always first), s sequence lengths, h initial hidden state; for example ih means in[0] is the sequence and in[1] the hidden state.
output_mask str required Letters among o, h naming the results that are used (out[0], out[1]). The positions of the two results do not change.
direction str "forward" forward, backward (reverse is treated the same) or bidirectional. bidirectional runs both and stacks forward (index 0) and backward (index 1) on the direction axis, so num_directions is 2; otherwise it is 1.
batch_first bool false true for the batch-first layout used in the shapes here. false is time-major: in[0] is [T, 1, input_size], the state is [num_directions, 1, hidden_size] and out[0] is [T, num_directions, 1, hidden_size].
hidden_size int32 required Number of hidden units per direction.
hidden_state_name str required Name of out[1] and of the state output of the cell subgraph.
subgraph str required Name of the cell subgraph that computes one time step from the step input and the previous hidden state.
activation_alpha float32[] optional Alpha coefficients for the functions in activations, in the same order. sigmoid and tanh take none.
activation_beta float32[] optional Beta coefficients for the functions in activations, in the same order. sigmoid and tanh take none.
activations str[] optional Recurrence function [act_f], usually [tanh]. With bidirectional, two names give the forward then the backward function; one name applies to both. The reference computation defines sigmoid and tanh.
clip float32 optional Cell clip threshold. When set, the input of the activation is limited to [-clip, clip]; absent means no clipping.

Outputs

Name Shape Description
out[0] [1, num_directions, T, hidden_size] Output activation.
out[1] [1, 1, num_directions, hidden_size] Output activation.

StatefulRoPEWrapper

Applies rotary position embedding to queries or keys at their positions.

Rotates channel pairs of a query or key tensor by position-dependent angles, using precomputed cos/sin tables whose row p belongs to position p. The rows are picked by the absolute positions of the tokens in the current call.

x = in[0];  c, s = rows of in[1], in[2] for the S current positions
rot0(y) = concat(-y[..., h:], y[..., :h]),   h = width(y) / 2
rot1(y)[..., 2i] = -y[..., 2i+1],  rot1(y)[..., 2i+1] = y[..., 2i]
rope_type 0:  R = rotary_emb_dim if > 0, else width(c)
              y = x[..., :R]
              out[0] = concat(y * c[..., :R] + rot0(y) * s[..., :R], x[..., R:])
rope_type 1:  out[0] = x * c + rot1(x) * s
rope_type 2:  rope_type 0 with R = width(c)

Inputs

Name Shape Description
in[0] [1, heads, S, D] Input activation.
in[1] [1, 1, L, R] Input activation.
in[2] [1, 1, L, R] Input activation.

Attributes

Name Type Default Description
rope_type int32 0 Channel pairing: 0 pairs channel i with i + R/2 (rotary channels split in halves); 1 pairs adjacent channels 2i, 2i+1; 2 is 0 with the rotary width taken from the cos table.
seqlen int32 0 Sequence length S of in[0] when the graph was built. Informational; the computation uses the runtime shape.
rotary_emb_dim int32 0 Number of leading channels of D that are rotated (partial rotary); the rest pass through unchanged. 0 means the width of the cos table. Used with rope_type=0 only.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Sub

Element-wise difference of two activations.

Subtracts in[1] from in[0] element by element.

out[0] = in[0] - in[1]

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.
in[1] broadcastable to in[0] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] broadcast(in[0], in[1]) Output activation.

SubConstant

Subtracts a constant from an activation, or an activation from a constant.

Subtracts with one operand taken from a constant stored in the model. The constant may be a scalar, a per-channel vector or a full tensor. With is_const_first true the result is constant - in[0], otherwise in[0] - constant.

out[0] = constant - in[0]   if is_const_first
         in[0] - constant   otherwise

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
constant Weight required Constant operand. Broadcasts against in[0].
is_const_first bool required true if the constant is the minuend (left operand), false if it is subtracted.
shape int32[] [] Shape of the constant operand before the stored constant was reduced (e.g. to a per-channel vector). The output shape is broadcast(shape(in[0]), shape). The default [] means the tensor's shape was not recorded; the field is optional.

Outputs

Name Shape Description
out[0] broadcast(in[0], shape) Output activation.

Swish

Self-gated activation x * sigmoid(x).

out[0] = in[0] * sigmoid(in[0]) = in[0] / (1 + exp(-in[0]))

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Tan

Element-wise tangent.

out[0] = tan(in[0])

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

Tanh

Element-wise hyperbolic tangent.

out[0] = tanh(in[0])

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

None.

Outputs

Name Shape Description
out[0] same as in[0] Output activation.

TensorCmp

Element-wise comparison of two activations, giving a boolean tensor.

Compares in[0] with in[1] element by element, after broadcasting, using the relation named by kind. The output is boolean - true where the relation holds.

kind = "eq":   out[0] = (in[0] == in[1])
kind = "neq":  out[0] = (in[0] != in[1])
kind = "lt":   out[0] = (in[0] <  in[1])
kind = "leq":  out[0] = (in[0] <= in[1])
kind = "gt":   out[0] = (in[0] >  in[1])
kind = "geq":  out[0] = (in[0] >= in[1])

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.
in[1] broadcastable to in[0] Input activation.

Attributes

Name Type Default Description
kind str "eq" Relation: eq, neq, lt, leq, gt or geq, read as in[0] <kind> in[1].

Outputs

Name Shape Description
out[0] broadcast(in[0], in[1]) Output activation.

Transpose

Permutes the axes of an activation.

Reorders the axes of in[0] so that axis i of out[0] is axis transpose_axes[i] of in[0].

P = transpose_axes
out[0].shape[i] = in[0].shape[P[i]]
out[0][i_0, ..., i_{r-1}] = in[0][j]   where j[P[k]] = i_k

Inputs

Name Shape Description
in[0] [1, H, W, C] Input activation.

Attributes

Name Type Default Description
transpose_axes int64[] required Permutation of 0..rank-1; entry i names the input axis that becomes output axis i.

Outputs

Name Shape Description
out[0] [in[0].shape[transpose_axes[i]] for each i] Output activation.

TransposeConvolution

Transposed 2-D convolution, stored as zero insertion plus a convolution.

Upsamples in[0] by inserting h_stride - 1 zero rows and w_stride - 1 zero columns between neighbouring pixels, zero-pads that tensor by the *_padding attributes, and convolves it with weight at stride 1 and the given dilation, adding bias. The weight is therefore stored in convolution layout, [out_channels, in_channels / group_number, h_kernel, w_kernel], holding the transposed-convolution kernel flipped along H and W, and the paddings apply to the zero-inserted tensor rather than being cropped from the output.

G  = group_number,  Cg = in_channels / G,  Og = out_channels / G   (// : integer division)
z  = zeros([1, (H-1)*h_stride + 1, (W-1)*w_stride + 1, in_channels])
z[n, i*h_stride, j*w_stride, c] = in[0][n, i, j, c]
zp = zero_pad(z, top=north_padding, bottom=south_padding,
                 left=west_padding, right=east_padding)
out[0][n, y, x, o] = bias[o] + sum_{ky, kx, c < Cg}
    weight[o, c, ky, kx]
    * zp[n, y + ky*h_dilation, x + kx*w_dilation, (o // Og)*Cg + c]

Inputs

Name Shape Description
in[0] [1, H, W, in_channels] Input activation.

Attributes

Name Type Default Description
in_channels int32 required Number of input channels.
out_channels int32 required Number of output channels.
h_kernel int32 required Kernel height.
w_kernel int32 required Kernel width.
h_stride int32 required Upsampling stride along H; h_stride - 1 zero rows go between input rows.
w_stride int32 required Upsampling stride along W; w_stride - 1 zero columns go between input columns.
h_dilation int32 required Dilation along H.
w_dilation int32 required Dilation along W.
west_padding int32 required Zero padding at the start of W (left) of the zero-inserted tensor.
east_padding int32 required Zero padding at the end of W (right) of the zero-inserted tensor.
north_padding int32 required Zero padding at the start of H (top) of the zero-inserted tensor.
south_padding int32 required Zero padding at the end of H (bottom) of the zero-inserted tensor.
padding_list int32[] [] The original transposed-convolution padding as [top, left, bottom, right] (amounts removed from the output), kept for reference. The computation uses the four *_padding attributes.
og_padding_list int32[] [] Slot for the original padding of the transposed convolution. Informational; the computation does not read it, and it is normally empty.
og_out_padding_list int32[] [] Slot for the original output padding of the transposed convolution (extra rows and columns at the end of the output). Informational; the computation does not read it, and it is normally empty.
group_number int32 required Number of channel groups. Divides both in_channels and out_channels.
weight Weight required Filter in convolution layout, [out_channels, in_channels / group_number, h_kernel, w_kernel]: the transposed-convolution kernel flipped along H and W.
bias Weight optional Bias, [out_channels]. Absent means zero.
channel_last bool true Must be true; the NPU uses the NHWC layout.

Outputs

Name Shape Description
out[0] [1, H_out, W_out, out_channels] Output activation.

Other Mobilint IR operations

The remaining operations have input and attribute metadata in the table below. Their presence in this table does not imply NPU or CPU Offload support.

Operation Notes
Abs num inputs: 1
num outputs: 1
AddPartial num inputs: 2
num outputs: 1
attributes:
- is_input0_larger: bool
Cast num inputs: 1
num outputs: 1
attributes:
- dtype: str
Ceil num inputs: 1
num outputs: 1
Convolution3d num inputs: 1
num outputs: 1
attributes:
- in_channels: int32
- out_channels: int32
- h_kernel: int32
- w_kernel: int32
- d_kernel: int32
- h_stride: int32
- w_stride: int32
- d_stride: int32
- h_dilation: int32
- w_dilation: int32
- d_dilation: int32
- west_padding: int32
- east_padding: int32
- north_padding: int32
- south_padding: int32
- front_padding: int32
- back_padding: int32
- padding_list: int32[] (default=[])
- group_number: int32
- weight: Weight
- bias: Weight (optional)
- channel_last: bool (default=true)
Crop num inputs: 1
num outputs: 1
attributes:
- w_offset: int32 (default=0)
- h_offset: int32 (default=0)
- crop_width: int32
- crop_height: int32
DeviceBridge num inputs: 1
num outputs: 1
attributes:
- squeeze_axes: int32[]
- unsqueeze_axes: int32[]
- permute_axes: int64[]
- device: str (optional)
Expand num inputs: 1
num outputs: 1
attributes:
- constant: Weight (optional)
- is_const_first: bool
GLU num inputs: 1
num outputs: 1
GridSample num inputs: 2
num outputs: 1
attributes:
- align_corners: int32 (default=0)
- mode: str (default="linear")
- padding_mode: str (default="zeros")

description:
inputs: X, grid
X is in channel-last format.
X: (n, h, w, c)
grid: (n, h_out, w_out, num_spatial_dims). in this case, num_spatial_dims=2.
Hardtanh num inputs: 1
num outputs: 1
attributes:
- min_val: float32 (default=-1.0)
- max_val: float32 (default=1.0)
Hardshrink num inputs: 1
num outputs: 1
attributes:
- lambd: float32 (default=0.5)
Softshrink num inputs: 1
num outputs: 1
attributes:
- lambd: float32 (default=0.5)
Identity num inputs: 1
num outputs: 1
SumNormalization num inputs: 1
num outputs: 1
attributes:
- norm_axis: int32[]
- epsilon: float32 (optional)
LSTM num inputs: variable
num outputs: 1
attributes:
- W: Weight
- R: Weight
- B: Weight (optional)
- lstm_group: str
- input_mask: str
- output_mask: str
- P: Weight (optional)
- direction: str (default="forward")
- batch_first: bool (default=false)
- hidden_size: int32

description:
inputs:[input, sequence_lens, initial_h, initial_c]
outputs:[output] or [hidden_out] or [cell_out]
input_mask: "ishc" or "i", etc,...
output_mask: "o" or "h" or "c"
Logit num inputs: 1
num outputs: 1
attributes:
- eps: float32 (optional)

description:
Compute y = ln(z / (1 - z)). If eps is given, we clamp input z = clamp(z, min=eps, max=1-eps)
RNN num inputs: variable
num outputs: 1
attributes:
- W: Weight
- R: Weight
- B: Weight (optional)
- rnn_group: str
- input_mask: str
- output_mask: str
- direction: str (default="forward")
- batch_first: bool (default=false)
- hidden_size: int32

description:
inputs:[input, sequence_lens, initial_h]
outputs:[output] or [hidden_out]
input_mask: "ish" or "i", etc,...
output_mask: "o" or "h"
GRU num inputs: variable
num outputs: 1
attributes:
- W: Weight
- R: Weight
- B: Weight (optional)
- gru_group: str
- input_mask: str
- output_mask: str
- direction: str (default="forward")
- batch_first: bool (default=false)
- linear_before_reset: bool (default=false)
- hidden_size: int32

description:
inputs:[input, sequence_lens, initial_h]
outputs:[output] or [hidden_out]
input_mask: "ish" or "i", etc,...
output_mask: "o" or "h"
LogSigmoid num inputs: 1
num outputs: 1
MaskSoftmax num inputs: 1
num outputs: 1
attributes:
- beta: float32
- axis: int32
NonMaxSuppression num inputs: 2
num outputs: variable
attributes:
- max_output_size: int64
- iou_threshold: float32
- score_threshold: float32
- soft_nms_sigma: float32
- pad_to_max_output_size: bool
- center_point_box: int32 (default=0)
GlobalAveragePooling num inputs: 1
num outputs: 1
attributes:
- channel_last: bool (default=true)
GlobalResponseNormalization num inputs: 1
num outputs: 1
attributes:
- scale: Weight
- bias: Weight
- epsilon: float32
- channel_last: bool (default=true)
ReduceL2 num inputs: 1
num outputs: 1
attributes:
- keep_dims: bool
- reduce_axes: int32[]
- noop_with_empty_axes: int32
ReduceProd num inputs: 1
num outputs: 1
attributes:
- keep_dims: bool
- reduce_axes: int32[]
- noop_with_empty_axes: int32
ReduceRMS num inputs: 1
num outputs: 1
attributes:
- keep_dims: bool
- reduce_axes: int32[]
- epsilon: float32 (optional)

description:
reduce rms with epsilon if exists. i.e.,sqrt(mean**2 + eps)
we let eps to be optional so that it has more general fusing opportunities.
MeanCentering num inputs: 1
num outputs: 1
attributes:
- axes: int32[]

description:
compute x - mean(x, axes)
ReshapeV2 num inputs: 2
num outputs: 1
attributes:
- allowzero: int32 (default=0)
- is_const_first: bool (default=false)

description:
Reshape with variable shape argument. inputs[1] will be used as a shape.
RotaryEmbedding num inputs: 1
num outputs: 2
attributes:
- inv_freq: Weight
- attention_scaling: float32

description:
following the LlamaRotaryEmbedding of the Huggingface Transformer library.
inputs => (position_ids,), outputs => (cos, sin)
Rsqrt num inputs: 1
num outputs: 1
Rstd num inputs: 1
num outputs: 1
attributes:
- reduce_axes: int32[]
- epsilon: Weight
- num_groups: int32 (default=1)

description:
computes 1/std(x). if gruops > 1, then activation is grouped along channel axis.
Gather num inputs: 2
num outputs: 1
attributes:
- axis: int32 (default=0)
GatherElements num inputs: 2
num outputs: 1
attributes:
- axis: int32 (default=0)
ScatterND num inputs: variable
num outputs: 1
attributes:
- constant_data: Weight (optional)
- constant_indices: Weight (optional)
Selu num inputs: 1
num outputs: 1
Shape num inputs: 1
num outputs: 1
attributes:
- start: int32 (optional)
- end: int32 (optional)
LogSoftmax num inputs: 1
num outputs: 1
attributes:
- axis: int32
Softsign num inputs: 1
num outputs: 1
Unsqueeze num inputs: 1
num outputs: 1
attributes:
- axes: int32[]
DynamicSliceInputConst num inputs: 1
num outputs: 1
attributes:
- mod: int32
- start: int64[]
- end: int64[]
- axis: int64[]
- step: int64[] (optional)
MeloTTSPiecewiseRationalQuadraticTransform num inputs: 2
num outputs: 1
attributes:
- num_bins: int32
- filter_channels: int32
- tail_bound: float32
- reverse: bool

description:
helper function for parsing melotts model.
Neg num inputs: 1
num outputs: 1
Einsum num inputs: variable
num outputs: 1
attributes:
- equation: str
- constant: Weight (optional)
- is_const_first: bool (default=false)
Where num inputs: 3
num outputs: 1


description:
condition,X,Y, if cond == True: else y
Logical num inputs: variable
num outputs: 1
attributes:
- kind: str (default="and")

description:
this function represents the logical operation of two inputs.
single input if kind is "not" else two inputs
kind: and, or, xor, not
MaskedFill num inputs: 2
num outputs: 1
attributes:
- value: float32

description:
inputs: (x, mask)
SubgraphCall num inputs: variable
num outputs: variable
attributes:
- subgraph: str
- scope: str
- kind: str
MoeDispatcher num inputs: 2
num outputs: 2
attributes:
- top_k: int32
- norm_top_k_prob: bool (default=true)

description:
inputs: (hidden_states, router_logits)
outputs: (hidden_states, top_k_logits)
if norm_top_k_prob is true
topk_vals, topk_idx = torch.topk(
routing_weights, self.top_k, dim=-1
) # (18,8), (18,8)
weight_new = torch.zeros_like(routing_weights)
weight_new.scatter_(1, topk_idx, topk_vals)
routing_weights = weight_new
routing_weights /= routing_weights.sum(dim=1, keepdim=True)
MoeAggregator num inputs: variable
num outputs: 1


description:
inputs: (router_weights, experts_output0, expert_output1, ...)
TopKValues num inputs: 1
num outputs: 1
attributes:
- axis: int32
- k: int32 (default=1)
- largest: bool (default=true)
- sorted: bool (default=true)
TopKIndices num inputs: 1
num outputs: 1
attributes:
- axis: int32
- k: int32 (default=1)
- largest: bool (default=true)
- sorted: bool (default=true)