# Mobilint IR Operation Formats

This page describes the behavior, inputs, attributes, and outputs of Mobilint IR operations. It is a format reference: an operation appearing here does not by itself establish NPU or CPU Offload support. See [NPU Operator Support](<https://docs.mobilint.com/compiler/en/v1.4/operation_support/index.md>) or [CPU Offloading](<https://docs.mobilint.com/compiler/en/v1.4/cpu_offloading/#cpu-offload-operator-scope>) for execution scope.

## Reading the operation definitions

- Four-dimensional activations use NHWC order: batch (`N`), height (`H`), width (`W`), and channels (`C`). The batch size is 1 in the shapes below.
- Inputs and outputs use positional names such as `in[0]` and `out[0]`.
- A `Weight` attribute stores a tensor index (`TensorIndex`), not tensor values. It refers to the subgraph tensor table, which records the constant tensor's name, dtype, shape, and storage location. `-1` means no tensor; omitted optional tensor attributes use this value.
- In attribute tables, **required** means the attribute must be set and **optional** means it may be omitted.

## Detailed operation definitions

### Adding

Element-wise sum of two activations.

Adds two activations element by element.

```text
out[0] = in[0] + in[1]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |
| `in[1]` | broadcastable to in\[0\] | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `broadcast(in[0], in[1])` | Output activation. |

### AddingConstant

Adds a constant tensor to an activation.

Adds a constant stored in the model to one activation. The constant may be a scalar, a per-channel vector or a full tensor.

```text
out[0] = constant + in[0]   if is_const_first
         in[0] + constant   otherwise
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `constant` | `Weight` | required | Constant operand. Broadcasts against in\[0\]. |
| `is_const_first` | `bool` | `false` | `true` if the constant is the left operand. |
| `shape` | `int32[]` | `[]` | Shape of the constant operand before the stored `constant` was reduced (e.g. to a per-channel vector). The output shape is broadcast(shape(in\[0\]), shape). The default `[]` means the tensor's shape was not recorded; the field is optional. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `broadcast(in[0], shape)` | Output activation. |

### Batchnorm

Per-channel normalization with stored mean and variance, then scale and shift.

Inference-form batch normalization. Each channel is normalized with the constant `moving_mean` and `moving_var` stored in the model, then scaled by `gamma` and shifted by `beta`.

```text
out[0][n, h, w, c] = (in[0][n, h, w, c] - moving_mean[c]) / sqrt(moving_var[c] + epsilon)
                     * gamma[c] + beta[c]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `gamma` | `Weight` | required | Per-channel scale. |
| `beta` | `Weight` | required | Per-channel shift. |
| `moving_mean` | `Weight` | required | Per-channel mean subtracted from the input. |
| `moving_var` | `Weight` | required | Per-channel variance the input is divided by (after adding `epsilon` and taking the square root). |
| `epsilon` | `Weight` | required | Scalar added to the variance for numerical stability. |
| `channel_last` | `bool` | `true` | Must be `true`; the NPU uses the NHWC layout. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Celu

Continuously differentiable ELU with parameter `alpha`.

Passes positive inputs through and maps negative inputs to an exponential that saturates at `-alpha`.

```text
out[0] = max(0, in[0]) + min(0, alpha * (exp(in[0] / alpha) - 1))
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `alpha` | `float32` | required | Saturation level of the negative branch and divisor of the exponent. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Clip

Clamps each element to `[clip_min_value, clip_max_value]`.

Limits every element to the given bounds. Either bound may be absent, and that side is then left unlimited.

```text
out[0] = min(max(in[0], clip_min_value), clip_max_value)
an absent bound is skipped
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `clip_max_value` | `float32` | optional | Upper bound. Absent means no upper bound. |
| `clip_min_value` | `float32` | optional | Lower bound. Absent means no lower bound. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Concatenate

Joins activations along one axis.

Concatenates in\[0..K-1\], in operand order, along `axis`. All operands have the same rank and agree on every other axis.

```text
out[0] = concat(in[0], in[1], ..., in[K-1]) along axis
out[0].shape[axis] = in[0].shape[axis] + ... + in[K-1].shape[axis]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0..K-1]` | equal except along axis | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `original_key` | `str` | optional | Deprecated and no longer used. The key is always present, but its value is always `None`. |
| `axis` | `int32` | optional | Axis to join along; negative values count from the last axis. It indexes the tensor as laid out, so `3` is the channel axis of an NHWC activation. |
| `original_axis` | `int32` | optional | Deprecated and no longer used. The key is always present, but its value is always `None`. |
| `original_inputs` | `int32[]` | optional | Deprecated and no longer used. The key is always present, but its value is always `None`. |
| `input_index` | `int32[]` | optional | Always `[0, 1, ..., K-1]`. |
| `input_names` | `str[]` | optional | Always the names of this operation's input activations. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | in\[0\] with the axis extent summed over operands | Output activation. |

### Convolution

2-D convolution of an NHWC activation with a constant filter.

Slides `weight` over the spatial axes of in\[0\] with the given stride, dilation and zero padding, and adds `bias`. The weight is stored output-channel first, as `[out_channels, in_channels / group_number, h_kernel, w_kernel]`.

```text
G  = group_number,  Cg = in_channels / G,  Og = out_channels / G   (// : integer division)
xp = zero_pad(in[0], top=north_padding, bottom=south_padding,
                     left=west_padding, right=east_padding)
out[0][n, y, x, o] = bias[o] + sum_{ky, kx, c < Cg}
    weight[o, c, ky, kx]
    * xp[n, y*h_stride + ky*h_dilation, x*w_stride + kx*w_dilation, (o // Og)*Cg + c]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, in_channels]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `in_channels` | `int32` | required | Number of input channels. |
| `out_channels` | `int32` | required | Number of output channels. |
| `h_kernel` | `int32` | required | Kernel height. |
| `w_kernel` | `int32` | required | Kernel width. |
| `h_stride` | `int32` | required | Stride along H. |
| `w_stride` | `int32` | required | Stride along W. |
| `h_dilation` | `int32` | required | Dilation along H. |
| `w_dilation` | `int32` | required | Dilation along W. |
| `west_padding` | `int32` | required | Zero padding at the start of W (left). |
| `east_padding` | `int32` | required | Zero padding at the end of W (right). |
| `north_padding` | `int32` | required | Zero padding at the start of H (top). |
| `south_padding` | `int32` | required | Zero padding at the end of H (bottom). |
| `padding_list` | `int32[]` | `[]` | The original 2-D padding as `[top, left, bottom, right]`, kept for reference. The computation uses the four `*_padding` attributes. |
| `group_number` | `int32` | required | Number of channel groups; must be 1. A convolution with more than one group must be a `GroupConvolution` or `DepthwiseConvolution`. |
| `weight` | `Weight` | required | Filter, `[out_channels, in_channels / group_number, h_kernel, w_kernel]`. |
| `bias` | `Weight` | optional | Bias, `[out_channels]`. Absent means zero. |
| `channel_last` | `bool` | `true` | Must be `true`; the NPU uses the NHWC layout. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `[1, H_out, W_out, out_channels]` | Output activation. |

### Cos

Element-wise cosine.

```text
out[0] = cos(in[0])
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### DepthwiseConvolution

Depthwise 2-D convolution; each input channel has its own filters.

A `Convolution` with one group per input channel (`group_number = in_channels`). Each input channel is convolved with its own `M = out_channels / in_channels` filters and is never mixed with other channels; output channel `o` reads input channel `o / M`. The weight is stored as `[out_channels, 1, h_kernel, w_kernel]`.

```text
M  = out_channels / in_channels
xp = zero_pad(in[0], top=north_padding, bottom=south_padding,
                     left=west_padding, right=east_padding)
out[0][n, y, x, o] = bias[o] + sum_{ky, kx}
    weight[o, 0, ky, kx]
    * xp[n, y*h_stride + ky*h_dilation, x*w_stride + kx*w_dilation, o / M]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, in_channels]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `in_channels` | `int32` | required | Number of input channels. |
| `out_channels` | `int32` | required | Number of output channels, `in_channels * M`. |
| `h_kernel` | `int32` | required | Kernel height. |
| `w_kernel` | `int32` | required | Kernel width. |
| `h_stride` | `int32` | required | Stride along H. |
| `w_stride` | `int32` | required | Stride along W. |
| `h_dilation` | `int32` | required | Dilation along H. |
| `w_dilation` | `int32` | required | Dilation along W. |
| `west_padding` | `int32` | required | Zero padding at the start of W (left). |
| `east_padding` | `int32` | required | Zero padding at the end of W (right). |
| `north_padding` | `int32` | required | Zero padding at the start of H (top). |
| `south_padding` | `int32` | required | Zero padding at the end of H (bottom). |
| `padding_list` | `int32[]` | `[]` | The original 2-D padding as `[top, left, bottom, right]`, kept for reference. The computation uses the four `*_padding` attributes. |
| `group_number` | `int32` | required | Number of channel groups; equals `in_channels`, one group per input channel. |
| `weight` | `Weight` | required | Filter, `[out_channels, 1, h_kernel, w_kernel]`. |
| `bias` | `Weight` | optional | Bias, `[out_channels]`. Absent means zero. |
| `channel_last` | `bool` | `true` | Must be `true`; the NPU uses the NHWC layout. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `[1, H_out, W_out, out_channels]` | Output activation. |

### Div

Element-wise quotient of two activations.

Divides in\[0\] by in\[1\] element by element (true division).

```text
out[0] = in[0] / in[1]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |
| `in[1]` | broadcastable to in\[0\] | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `broadcast(in[0], in[1])` | Output activation. |

### DivConstant

Divides an activation by a constant, or a constant by an activation.

Divides with one operand taken from a constant stored in the model. The constant may be a scalar, a per-channel vector or a full tensor. With `is_const_first` true the result is `constant / in[0]`, otherwise `in[0] / constant`.

```text
out[0] = constant / in[0]   if is_const_first
         in[0] / constant   otherwise
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `constant` | `Weight` | required | Constant operand. Broadcasts against in\[0\]. |
| `is_const_first` | `bool` | required | `true` if the constant is the dividend (left operand), `false` if it is the divisor. |
| `shape` | `int32[]` | `[]` | Shape of the constant operand before the stored `constant` was reduced (e.g. to a per-channel vector). The output shape is broadcast(shape(in\[0\]), shape). The default `[]` means the tensor's shape was not recorded; the field is optional. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `broadcast(in[0], shape)` | Output activation. |

### Elu

Exponential linear unit with negative-side scale `leaky_alpha`.

Passes positive inputs through and maps the rest to `leaky_alpha * (exp(x) - 1)`, which saturates at `-leaky_alpha`.

```text
out[0] = in[0]                            if in[0] > 0
         leaky_alpha * (exp(in[0]) - 1)   otherwise
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `leaky_alpha` | `float32` | required | ELU alpha, the scale of the negative branch. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Erf

Element-wise Gauss error function.

```text
out[0] = erf(in[0]) = 2 / sqrt(pi) * integral_0^in[0] exp(-t^2) dt
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Exp

Element-wise natural exponential.

```text
out[0] = exp(in[0])
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Gelu

Gaussian error linear unit.

Applies GELU element by element. `approximate` selects between the `erf` form and the tanh approximation.

```text
approximate = false:  out[0] = 0.5 * x * (1 + erf(x / sqrt(2)))
approximate = true:   out[0] = 0.5 * x * (1 + tanh(sqrt(2/pi) * (x + 0.044715 * x^3)))
where x = in[0]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `approximate` | `bool` | `false` | `true` for the tanh approximation, `false` for the exact erf form. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### GroupConvolution

Grouped 2-D convolution of an NHWC activation with a constant filter.

Computes the same grouped convolution as `Convolution`. The channels are split into `group_number` groups, and each group's `out_channels / group_number` outputs see only that group's `in_channels / group_number` inputs.

```text
G  = group_number,  Cg = in_channels / G,  Og = out_channels / G   (// : integer division)
xp = zero_pad(in[0], top=north_padding, bottom=south_padding,
                     left=west_padding, right=east_padding)
out[0][n, y, x, o] = bias[o] + sum_{ky, kx, c < Cg}
    weight[o, c, ky, kx]
    * xp[n, y*h_stride + ky*h_dilation, x*w_stride + kx*w_dilation, (o // Og)*Cg + c]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, in_channels]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `in_channels` | `int32` | required | Number of input channels. |
| `out_channels` | `int32` | required | Number of output channels. |
| `h_kernel` | `int32` | required | Kernel height. |
| `w_kernel` | `int32` | required | Kernel width. |
| `h_stride` | `int32` | required | Stride along H. |
| `w_stride` | `int32` | required | Stride along W. |
| `h_dilation` | `int32` | required | Dilation along H. |
| `w_dilation` | `int32` | required | Dilation along W. |
| `west_padding` | `int32` | required | Zero padding at the start of W (left). |
| `east_padding` | `int32` | required | Zero padding at the end of W (right). |
| `north_padding` | `int32` | required | Zero padding at the start of H (top). |
| `south_padding` | `int32` | required | Zero padding at the end of H (bottom). |
| `padding_list` | `int32[]` | `[]` | The original 2-D padding as `[top, left, bottom, right]`, kept for reference. The computation uses the four `*_padding` attributes. |
| `group_number` | `int32` | required | Number of channel groups. Divides both `in_channels` and `out_channels`. |
| `weight` | `Weight` | required | Filter, `[out_channels, in_channels / group_number, h_kernel, w_kernel]`. |
| `bias` | `Weight` | optional | Bias, `[out_channels]`. Absent means zero. |
| `channel_last` | `bool` | `true` | Must be `true`; the NPU uses the NHWC layout. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `[1, H_out, W_out, out_channels]` | Output activation. |

### GroupNormalization

Normalizes each group of channels over H, W and the group, then scales and shifts.

Splits the C channels into `num_groups` contiguous groups and normalizes each group of each batch item to zero mean and unit variance over H, W and the group's channels. A per-channel `scale` and `bias` follow. The variance is the population variance.

```text
G = num_groups,  Cg = C / G,  g(c) = floor(c / Cg)
S(n, g)   = in[0][n, :, :, g*Cg : (g+1)*Cg]
mu[n, g]  = mean(S(n, g))
var[n, g] = mean((S(n, g) - mu[n, g])^2)
out[0][n, h, w, c] = (in[0][n, h, w, c] - mu[n, g(c)]) / sqrt(var[n, g(c)] + epsilon)
                     * scale[c] + bias[c]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `num_groups` | `int32` | required | Number of channel groups. Divides C; each group is `C / num_groups` consecutive channels. |
| `scale` | `Weight` | required | Per-channel scale (gamma). |
| `bias` | `Weight` | optional | Per-channel shift (beta). Absent means zero. |
| `epsilon` | `Weight` | required | Scalar added to the variance for numerical stability. |
| `channel_last` | `bool` | `true` | Must be `true`; the NPU uses the NHWC layout. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### HardSigmoid

Piecewise-linear sigmoid, `clamp(alpha * x + beta, 0, 1)`.

A linear ramp clamped to \[0, 1\]. An absent `alpha` or `beta` takes 1/6 or 0.5.

```text
out[0] = clamp(alpha * in[0] + beta, 0, 1)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `alpha` | `float32` | optional | Slope of the ramp. Absent means 1/6. |
| `beta` | `float32` | optional | Offset of the ramp. Absent means 0.5. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### HardSwish

Piecewise swish, `x * clamp(x + 3, 0, 6) / 6`.

```text
out[0] = in[0] * clamp(in[0] + 3, 0, 6) / 6
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### HeaderView

Splits the channel axis into attention heads and stacks them along H.

One operation for the reshape-transpose-reshape that turns a `[1, H, W, heads*D]` projection into per-head rows of `D` channels. The rows of all heads are stacked along H.

```text
D = C / num_heads
batch_order = 0 (head-minor rows, row = h*num_heads + head):
  [1, H, W, C] -reshape-> [1, H, W, num_heads, D]
               -transpose(0,1,3,2,4)-> [1, H, num_heads, W, D]
               -reshape-> [1, H*num_heads, W, D]
batch_order = 1 (head-major rows, row = head*H + h):
  [1, H, W, C] -reshape-> [1, H, W, num_heads, D]
               -transpose(0,3,1,2,4)-> [1, num_heads, H, W, D]
               -reshape-> [1, num_heads*H, W, D]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `view_shape` | `int32[]` | `[]` | Deprecated; ignored. Use `num_heads`. |
| `num_heads` | `int32` | `-1` | Number of heads. Divides `C`. |
| `batch_order` | `int32` | `0` | Row order of the stacked heads: `0` interleaves heads within each row of H, `1` places each head's H rows contiguously. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `[1, H*num_heads, W, C/num_heads]` | Output activation. |

### HeaderViewRevert

Unstacks attention heads from H and merges them back into channels.

Takes per-head rows of `D` channels stacked along H, `[1, num_heads*H, W, D]`, and concatenates the heads along the channel axis into `[1, H, W, num_heads*D]`. `batch_order` must be the value the rows were stacked with.

```text
in[0] is [1, num_heads*H, W, D]
batch_order = 0 (input row = h*num_heads + head):
  -reshape-> [1, H, num_heads, W, D]
  -transpose(0,1,3,2,4)-> [1, H, W, num_heads, D]
  -reshape-> [1, H, W, num_heads*D]
batch_order = 1 (input row = head*H + h):
  -reshape-> [1, num_heads, H, W, D]
  -transpose(0,2,3,1,4)-> [1, H, W, num_heads, D]
  -reshape-> [1, H, W, num_heads*D]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, num_heads*H, W, D]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `revert_shape` | `int32[]` | `[]` | Deprecated; ignored. Use `num_heads`. |
| `num_heads` | `int32` | `-1` | Number of heads. Divides the H extent of in\[0\]. |
| `batch_order` | `int32` | `0` | Row order of the stacked heads in in\[0\]: `0` if heads alternate within each row of H, `1` if each head's H rows are contiguous. Same meaning as in `HeaderView`. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `[1, H, W, num_heads*D]` | Output activation. |

### Input

Marks an activation as a graph input.

Declares that in\[0\] is supplied from outside the graph. The operand and the result are the same activation, so the operation computes nothing; it only fixes the activation as a graph input.

```text
out[0] = in[0]      (the same activation)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `orig_name` | `str` | `""` | User-facing name of this input in the source model, kept so the compiled model exposes the same input names. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### InputConstant

Provides a constant stored in the model as an activation.

Has no operands; its result is the stored `constant`, with element type `dtype` and shape `shape`.

```text
out[0] = constant
```

#### Inputs

None.

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `constant` | `Weight` | required | The constant value. |
| `dtype` | `str` | `""` | Element type name of the constant, e.g. `float32` or `int64`. |
| `shape` | `int32[]` | `[]` | Shape of the constant, which is the shape of out\[0\]. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `shape` | Output activation. |

### InstanceNormalization

Normalizes each channel over H and W.

Normalizes every channel of every batch item to zero mean and unit variance over H and W, then applies per-channel `scale` and `bias`. The variance is the population variance. Normalization that shares statistics across several channels (`num_groups` other than `-1`) is expressed with `GroupNormalization`.

```text
mu[n, c]  = mean(in[0][n, :, :, c])
var[n, c] = mean((in[0][n, :, :, c] - mu[n, c])^2)
out[0][n, h, w, c] = (in[0][n, h, w, c] - mu[n, c]) / sqrt(var[n, c] + epsilon)
                     * scale[c] + bias[c]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `scale` | `Weight` | required | Scale (gamma). |
| `bias` | `Weight` | optional | Shift (beta), shaped like `scale`. Absent means zero. |
| `epsilon` | `Weight` | required | Scalar added to the variance for numerical stability. |
| `channel_last` | `bool` | `true` | Must be `true`; the NPU uses the NHWC layout. |
| `num_groups` | `int32` | `-1` | Always `-1`. Grouped normalization is `GroupNormalization`. |
| `group_order` | `int32` | `-1` | Always `-1`; unused. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### InverseSigmoid

Inverse of the sigmoid, `log(x / (1 - x))`, kept finite by `eps`.

Clamps in\[0\] to \[0, 1\], then keeps the numerator and the denominator of `x / (1 - x)` at or above `eps`, so inputs at or beyond 0 and 1 still give finite results.

```text
x1 = max(clamp(in[0], 0, 1), eps)
x2 = max(1 - x1, eps)
out[0] = log(x1 / x2)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `eps` | `float32` | `1e-05` | Lower bound applied to the numerator and the denominator. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### L1Normalization

Divides by the sum of absolute values over the given axes.

Scales in\[0\] so that its absolute values along `norm_axis` sum to 1, per position of the other axes. The divisor is bounded below by 1e-12.

```text
s      = sum(abs(in[0]), norm_axis, keep_dims=true)
out[0] = in[0] / max(s, 1e-12)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `norm_axis` | `int32[]` | required | Axes to normalize over, as indices into in\[0\] as laid out. Negative values count from the last axis. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### L2Normalization

Divides by the Euclidean norm over the given axes.

Scales in\[0\] to unit L2 norm along `norm_axis`, per position of the other axes. The norm is bounded below by `epsilon` to avoid division by zero.

```text
r      = sqrt(sum(in[0]^2, norm_axis, keep_dims=true))
out[0] = in[0] / max(r, epsilon)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `norm_axis` | `int32[]` | required | Axes to normalize over, as indices into in\[0\] as laid out. Negative values count from the last axis. |
| `epsilon` | `Weight` | optional | Lower bound on the norm. Absent means 1e-12. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### LayerNormalization

Normalizes over the trailing axes, then scales and shifts.

Normalizes in\[0\] to zero mean and unit variance over its last `len(normalized_shape)` axes, then applies the element-wise affine `scale` and `bias`. Mean and variance are taken per position of the remaining leading axes; the variance is the biased (population) variance.

```text
axes   = the last len(normalized_shape) axes of in[0]
mu     = mean(in[0], axes)
var    = mean((in[0] - mu)^2, axes)
out[0] = (in[0] - mu) / sqrt(var + epsilon) * scale + bias
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[..., *normalized_shape]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `normalized_shape` | `int64[]` | `[]` | Shape of the normalized trailing axes, e.g. `[C]` to normalize over the channel axis of an NHWC activation. |
| `scale` | `Weight` | required | Per-element scale (gamma), shape `normalized_shape`. |
| `bias` | `Weight` | optional | Per-element shift (beta), shape `normalized_shape`. Absent means zero. |
| `epsilon` | `Weight` | required | Scalar added to the variance for numerical stability. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### LeakyRelu

ReLU with the scalar slope `leaky_alpha` for negative inputs.

Multiplies negative elements by one scalar slope.

```text
out[0] = in[0]                 if in[0] >= 0
         leaky_alpha * in[0]   otherwise
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `leaky_alpha` | `float32` | required | Slope for negative inputs. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Log

Element-wise natural logarithm.

```text
out[0] = ln(in[0])
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### MatMul

Batched matrix product of two activations over their last two axes.

Multiplies in\[0\] by in\[1\] as matrices over their last two axes; all leading axes are batch axes and broadcast. For 4-D NHWC activations the matrices are `[W, C]` and N, H are batch axes.

```text
a = in[0],  b = in[1]
repeat = [0, R]:  a = repeat_interleave(a, R, axis=1)
repeat = [1, R]:  b = repeat_interleave(b, R, axis=1)
  where repeat_interleave(x, R, axis=1)[n, h, ...] = x[n, floor(h / R), ...]
out[0][..., m, p] = sum_k a[..., m, k] * b[..., k, p]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[..., M, K]` | Input activation. |
| `in[1]` | `[..., K, P]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `constant` | `Weight` | optional | Unused; always `-1`. A constant matrix is passed as an activation from `InputConstant`. |
| `is_const_first` | `bool` | `false` | Unused; always `false`. |
| `repeat` | `int32[2]` | `[-1, -1]` | `[operand, R]`: before the product, repeat operand `operand` (`0` = in\[0\], `1` = in\[1\]) `R` times along axis 1 (H), each H slice `R` times in place (`h0 h0 h1 h1 ...`, not `h0 h1 h0 h1 ...`). `[-1, -1]`: no repeat. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | \[broadcast of the leading axes, M, P\] | Output activation. |

### Mish

Self-gated activation `x * tanh(softplus(x))`.

```text
out[0] = in[0] * tanh(log(1 + exp(in[0])))
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Multiply

Element-wise product of two activations.

Multiplies two activations element by element.

```text
out[0] = in[0] * in[1]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |
| `in[1]` | broadcastable to in\[0\] | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `broadcast(in[0], in[1])` | Output activation. |

### MultiplyConstant

Multiplies an activation by a constant tensor.

Multiplies one activation by a constant stored in the model. The constant may be a scalar, a per-channel vector or a full tensor.

```text
out[0] = constant * in[0]   if is_const_first
         in[0] * constant   otherwise
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `constant` | `Weight` | required | Constant operand. Broadcasts against in\[0\]. |
| `is_const_first` | `bool` | `false` | `true` if the constant is the left operand. |
| `shape` | `int32[]` | `[]` | Shape of the constant operand before the stored `constant` was reduced (e.g. to a per-channel vector). The output shape is broadcast(shape(in\[0\]), shape). The default `[]` means the tensor's shape was not recorded; the field is optional. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `broadcast(in[0], shape)` | Output activation. |

### OrderedPatchify

Partitions H x W into windows and lays the pixels out window by window.

One operation for the reshape-transpose-reshape that cuts a `[1, H, W, C]` map into a `gh x gw` grid of `h_kernel x w_kernel` windows and flattens it into a 2-D layout on H and W. `order=0` gives one row per window, holding its pixels in raster order; `order=1` is its H/W transpose, one row per pixel offset inside the window; `order=2` undoes `order=1`. Channels are moved as a whole and never mixed.

```text
gh = H / h_kernel,  gw = W / w_kernel
x = reshape(in[0], [1, gh, h_kernel, gw, w_kernel, C])
order 0: out[0] = reshape(transpose(x, 0,1,3,2,4,5), [1, gh*gw, h_kernel*w_kernel, C])
order 1: out[0] = reshape(transpose(x, 0,2,4,1,3,5), [1, h_kernel*w_kernel, gh*gw, C])
order 2: out[0] = reshape(transpose(x, 0,3,1,4,2,5), [1, gw*gh, w_kernel*h_kernel, C])
Element map, p < gh, q < h_kernel, u < gw, v < w_kernel:
  in[0][0, p*h_kernel + q, u*w_kernel + v, c] =
    order 0: out[0][0, p*gw + u, q*w_kernel + v, c]
    order 1: out[0][0, q*w_kernel + v, p*gw + u, c]
    order 2: out[0][0, u*gh + p, v*h_kernel + q, c]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `h_kernel` | `int32` | required | Window height. Divides H. |
| `w_kernel` | `int32` | required | Window width. Divides W. |
| `order` | `int32` | `0` | Layout: `0` one row per window, `1` one row per in-window offset (the transpose of `0`), `2` the inverse of `1`. See the formula. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | order 0, 2: \[1, gh*gw, h\_kernel*w\_kernel, C\]; order 1: \[1, h\_kernel*w\_kernel, gh*gw, C\] | Output activation. |

### Output

Marks an activation as a graph output.

Declares that in\[0\] is returned from the graph. The operand and the result are the same activation, so the operation computes nothing; it only fixes the activation as a graph output.

```text
out[0] = in[0]      (the same activation)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `orig_name` | `str` | `""` | User-facing name of this output in the source model, kept so the compiled model exposes the same output names. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Pad

Pads the H and W axes of an NHWC activation with a constant or reflection.

Pads in\[0\] by `north_padding` above, `south_padding` below, `west_padding` to its left and `east_padding` to its right. The batch and channel axes are not padded.

```text
H_out = north_padding + H + south_padding
W_out = west_padding  + W + east_padding
out[0][n, north_padding + i, west_padding + j, c] = in[0][n, i, j, c]
other cells:
  constant:  constant
  reflect:   in[0] mirrored about its first/last row and column, edge not
             repeated (the row above row 0 is row 1)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `west_padding` | `int32` | `-1` | Columns added at the start of W (left). |
| `east_padding` | `int32` | `-1` | Columns added at the end of W (right). |
| `south_padding` | `int32` | `-1` | Rows added at the end of H (bottom). |
| `north_padding` | `int32` | `-1` | Rows added at the start of H (top). |
| `pad_mode` | `str` | required | `constant` fills the new cells with `constant`; `reflect` mirrors in\[0\] about its border without repeating the edge row or column. |
| `constant` | `float32` | `0.0` | Fill value for `pad_mode = constant`. |
| `padding_list` | `int32[]` | optional | The same padding written for every axis as `[x1_begin, x2_begin, ..., x1_end, x2_end, ...]`, for NHWC `[0, top, left, 0, 0, bottom, right, 0]`, kept for reference. The computation uses the four `*_padding` attributes. |
| `channel_last` | `bool` | `true` | Must be `true`; the NPU uses the NHWC layout. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `[1, H_out, W_out, C]` | Output activation. |

### Pooling

2-D max or average pooling over sliding windows of an NHWC activation.

Pads in\[0\] by the `*_padding` attributes with the fill chosen by `pad_mode`, then takes the maximum (`MaxPool`) or the average (`AvgPool`) of each `h_kernel` x `w_kernel` window, moved by the stride and spread by the dilation. Channels are pooled independently.

```text
xp = pad(in[0], top=north_padding, bottom=south_padding,
                left=west_padding, right=east_padding)    (fill per pad_mode)
win(n, y, x, c) = { xp[n, y*h_stride + ky*h_dilation, x*w_stride + kx*w_dilation, c]
                    : ky < h_kernel, kx < w_kernel }
MaxPool:  out[0][n, y, x, c] = max(win(n, y, x, c))
AvgPool:  out[0][n, y, x, c] = sum(win(n, y, x, c)) / cnt
          cnt = h_kernel * w_kernel                    if is_include_pad
                number of window cells inside in[0]    otherwise
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `h_kernel` | `int32` | required | Window height. |
| `w_kernel` | `int32` | required | Window width. |
| `h_stride` | `int32` | required | Stride along H. |
| `w_stride` | `int32` | required | Stride along W. |
| `h_dilation` | `int32` | `1` | Dilation along H. |
| `w_dilation` | `int32` | `1` | Dilation along W. |
| `west_padding` | `int32` | required | Padding at the start of W (left). |
| `east_padding` | `int32` | required | Padding at the end of W (right). |
| `north_padding` | `int32` | required | Padding at the start of H (top). |
| `south_padding` | `int32` | required | Padding at the end of H (bottom). |
| `padding_list` | `int32[]` | `[]` | The original 2-D padding as `[top, left, bottom, right]`, kept for reference; may be empty. The computation uses the four `*_padding` attributes. |
| `pool_type` | `str` | required | `MaxPool` for the window maximum, `AvgPool` for the window average. |
| `pad_mode` | `str` | required | Fill of the padded border: `constant` (zeros), `-inf` (negative infinity, so padding never wins a maximum) or `reflect` (mirror image of the border, edge not repeated). `-inf` is not combined with `AvgPool` and `is_include_pad = true`. |
| `is_ceil_mode` | `bool` | `false` | Records that the source pooling rounded the output size up. Informational; the output size always follows the floor formula below, so a rounded-up size appears as extra padding in `south_padding` (bottom) and `east_padding` (right). |
| `is_include_pad` | `bool` | `false` | `AvgPool` only. `true` divides by the full window size `h_kernel * w_kernel`, counting padded cells; `false` divides by the number of window cells that lie inside in\[0\]. |
| `channel_last` | `bool` | `true` | Must be `true`; the NPU uses the NHWC layout. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `[1, H_out, W_out, C]` | Output activation. |

### Pow

Raises each element to a constant scalar power.

Raises every element of in\[0\] to the same exponent.

```text
out[0] = in[0] ^ exponent
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `exponent` | `float32` | required | Scalar exponent. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### PRelu

Leaky ReLU whose negative slope is a constant tensor.

Multiplies negative elements by the slope `leaky_alpha`, a tensor stored in the model. The slope broadcasts against in\[0\], aligned from the last axis.

```text
out[0] = in[0]                 if in[0] >= 0
         leaky_alpha * in[0]   otherwise   (leaky_alpha broadcast to in[0])
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `leaky_alpha` | `Weight` | required | Slope for negative inputs; broadcastable to in\[0\]. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### QuickGelu

Sigmoid approximation of GELU, `x * sigmoid(1.702 * x)`.

Approximates `Gelu` by gating with `sigmoid(1.702 * x)`.

```text
out[0] = in[0] * sigmoid(1.702 * in[0])
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### ReduceMax

Maximum over the given axes.

Takes the largest element of in\[0\] over `reduce_axes`, separately for each position of the remaining axes.

```text
out[0] = max(in[0], reduce_axes, keep_dims)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `keep_dims` | `bool` | required | `true` keeps each reduced axis with size 1; `false` removes it. |
| `reduce_axes` | `int32[]` | required | Axes to reduce, as indices into in\[0\] as laid out (for an NHWC activation `3` is the channel axis). Negative values count from the last axis. |
| `noop_with_empty_axes` | `int32` | required | Only matters when `reduce_axes` is empty - `1` returns in\[0\] unchanged, `0` reduces over every axis. The IR normally lists the reduced axes explicitly. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | in\[0\] with the reduced axes set to 1 or removed | Output activation. |

### ReduceMean

Arithmetic mean over the given axes.

Averages in\[0\] over `reduce_axes`, separately for each position of the remaining axes.

```text
out[0] = sum(in[0], reduce_axes, keep_dims) / (product of in[0]'s sizes on reduce_axes)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `keep_dims` | `bool` | required | `true` keeps each reduced axis with size 1; `false` removes it. |
| `reduce_axes` | `int32[]` | required | Axes to reduce, as indices into in\[0\] as laid out (for an NHWC activation `3` is the channel axis). Negative values count from the last axis. |
| `noop_with_empty_axes` | `int32` | required | Only matters when `reduce_axes` is empty - `1` returns in\[0\] unchanged, `0` reduces over every axis. The IR normally lists the reduced axes explicitly. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | in\[0\] with the reduced axes set to 1 or removed | Output activation. |

### ReduceMin

Minimum over the given axes.

Takes the smallest element of in\[0\] over `reduce_axes`, separately for each position of the remaining axes.

```text
out[0] = min(in[0], reduce_axes, keep_dims)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `keep_dims` | `bool` | required | `true` keeps each reduced axis with size 1; `false` removes it. |
| `reduce_axes` | `int32[]` | required | Axes to reduce, as indices into in\[0\] as laid out (for an NHWC activation `3` is the channel axis). Negative values count from the last axis. |
| `noop_with_empty_axes` | `int32` | required | Only matters when `reduce_axes` is empty - `1` returns in\[0\] unchanged, `0` reduces over every axis. The IR normally lists the reduced axes explicitly. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | in\[0\] with the reduced axes set to 1 or removed | Output activation. |

### ReduceSum

Sum over the given axes.

Adds up the elements of in\[0\] over `reduce_axes`, separately for each position of the remaining axes.

```text
out[0] = sum(in[0], reduce_axes, keep_dims)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `keep_dims` | `bool` | required | `true` keeps each reduced axis with size 1; `false` removes it. |
| `reduce_axes` | `int32[]` | required | Axes to reduce, as indices into in\[0\] as laid out (for an NHWC activation `3` is the channel axis). Negative values count from the last axis. |
| `noop_with_empty_axes` | `int32` | required | Only matters when `reduce_axes` is empty - `1` returns in\[0\] unchanged, `0` reduces over every axis. The IR normally lists the reduced axes explicitly. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | in\[0\] with the reduced axes set to 1 or removed | Output activation. |

### RelShift

Re-indexes relative-offset attention scores by absolute key position.

Relative-position shift of Transformer-XL style attention. Row `i` of in\[0\] holds the scores of query `i` for `2Q - 1` relative offsets, with column `k` meaning query minus key `= Q - 1 - k`. The op shifts row `i` left by `Q - 1 - i` and keeps `Q` columns, so column `j` of the result is the score for key `j`.

```text
out[0][n, h, i, j] = in[0][n, h, i, Q - 1 - i + j],   i, j < Q
Equivalent sequence on [1, H, Q, P], P = 2Q - 1:
  pad one zero at the start of axis 3   -> [1, H, Q, P+1]
  reshape                               -> [1, H, P+1, Q]
  drop the first row of axis 2          -> [1, H, P, Q]
  reshape                               -> [1, H, Q, P]
  slice axis 3 to [0, Q)                -> [1, H, Q, Q]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, Q, 2Q-1]` | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `[1, H, Q, Q]` | Output activation. |

### Relu

Rectified linear unit, `max(x, 0)`.

```text
out[0] = max(in[0], 0)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Reshape

Changes the shape of an activation without moving its elements.

Returns in\[0\] with the shape `new_shape`. Elements keep their row-major order (last axis fastest).

```text
out[0] = reshape(in[0], new_shape)
flatten(out[0]) == flatten(in[0])   (same elements, row-major order)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `new_shape` | `int32[]` | optional | Target shape. One entry may be `-1`; it is inferred so that the element count stays the same. A `0` entry is read as `allowzero` says; graph builders write explicit extents, so `0` normally does not appear. |
| `allowzero` | `int32` | `0` | How a `0` in `new_shape` is read: `0` copies the extent of in\[0\] at the same index, `1` makes a zero-length axis. |
| `is_const_first` | `bool` | `false` | Not used by the computation; in\[0\] is always the data. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | new\_shape, with -1 resolved | Output activation. |

### Resize

Resizes the spatial axes by nearest, bilinear or bicubic interpolation.

Resamples in\[0\] to a new H x W. `sizes` and `scales` hold one entry per axis of in\[0\], in its own axis order, and only the two spatial axes may differ from the input.

```text
shape(out[0]) = sizes                     if sizes is given
                shape(in[0]) * scales     otherwise, axis by axis
s_h = H_out / H,  s_w = W_out / W         (or the matching scales entries)
nearest:  out[0][n, y, x, c] = in[0][n, floor(y / s_h), floor(x / s_w), c]
linear:   bilinear blend of the 2 x 2 input pixels around (ys, xs)
cubic:    bicubic blend of the 4 x 4 input pixels around (ys, xs)
where the source row ys of output row y follows transform_mode
(columns likewise with x, W, W_out, s_w):
  half_pixel:     ys = (y + 0.5) / s_h - 0.5
  align_corners:  ys = y * (H - 1) / (H_out - 1)
  asymmetric:     ys = y / s_h
source positions outside the input are clamped to the border
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `resize_type` | `str` | optional | Interpolation: `nearest`, `linear` (bilinear over H and W) or `cubic` (bicubic). |
| `resize_mode` | `str` | optional | The interpolation mode as recorded when the graph was built; same values as `resize_type`, may be absent. The computation reads `resize_type`. |
| `nearest_mode` | `str` | optional | Rounding rule of the source model for `nearest` (`round_prefer_floor`, `round_prefer_ceil`, `floor` or `ceil`). Recorded only; `nearest` always rounds down, as in the formula. |
| `transform_mode` | `str` | optional | Mapping from output to input coordinates for `linear` and `cubic`: `half_pixel` (pixel centres aligned; also applies to any other recorded value), `align_corners` (corner pixels aligned) or `asymmetric` (output index divided by the scale); see the formula. `nearest` always uses the asymmetric mapping. |
| `roi` | `int32[]` | optional | Region of interest of a crop-and-resize. Recorded only; the computation does not read it. |
| `scales` | `float32[]` | optional | Scale factor per axis of in\[0\], `1.0` on axes that keep their size; for example `[1, 2, 2, 1]` doubles H and W of an NHWC tensor. |
| `sizes` | `int32[]` | optional | Output shape, one entry per axis of in\[0\]. |
| `cubic_coeff_a` | `float32` | optional | Coefficient `a` of the cubic kernel for `cubic`. Recorded only; the computation does not read it. |
| `exclude_outside` | `int32` | optional | For `cubic`, `1` drops kernel taps outside the input and renormalizes the rest. Recorded only; the computation does not read it. |
| `extrapolation_value` | `float32` | optional | Value for output pixels whose source lies outside the input in a crop-and-resize. Recorded only; the computation does not read it. |
| `antialias` | `int32` | optional | `1` applies a low-pass (antialiasing) filter when downscaling with `linear` or `cubic`; `0` or absent does not. |
| `axes` | `int32[]` | optional | Axes the source resize named for its scales or sizes. Recorded only; in the IR, `scales` and `sizes` always cover every axis. |
| `keep_aspect_ratio_policy` | `str` | optional | How the source fitted its target size to keep the aspect ratio (`stretch`, `not_larger`, `not_smaller`). Recorded only; the computation does not read it. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `[1, H_out, W_out, C]` | Output activation. |

### RmsNormalization

Divides by the root mean square over the trailing axes, then scales.

Divides in\[0\] by its root mean square over the last `len(normalized_shape)` axes and multiplies by `scale`. With `rms_center`, the RMS is taken of `in[0] - rms_center` while the numerator stays in\[0\].

```text
axes   = the last len(normalized_shape) axes of in[0]
center = rms_center if present, else 0
rms    = sqrt(mean((in[0] - center)^2, axes, keep_dims=true) + epsilon)
out[0] = in[0] / rms * scale
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[..., *normalized_shape]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `normalized_shape` | `int64[]` | `[]` | Shape of the trailing axes to normalize over, normally `[C]`, the channel axis of an NHWC activation. |
| `epsilon` | `Weight` | required | Scalar added to the mean square for numerical stability. |
| `scale` | `Weight` | required | Per-element scale (gamma), shape `normalized_shape`. |
| `rms_center` | `Weight` | optional | Per-channel offset subtracted from in\[0\] inside the RMS only. Absent means 0. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Sigmoid

Logistic sigmoid with an optional input scale `alpha`.

Computes `1 / (1 + exp(-alpha * x))`.

```text
out[0] = 1 / (1 + exp(-alpha * in[0]))
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `alpha` | `float32` | `1.0` | Scale applied to the input before the sigmoid. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Sin

Element-wise sine.

```text
out[0] = sin(in[0])
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Slice

Extracts a strided sub-range along one or more axes.

On each listed axis, keeps the elements from `start` up to but not including `end`, every `step`; the other axes stay whole. The lists are read position by position, so entry k of `start`, `end` and `step` applies to axis `axis[k]`.

```text
for each k, along axis a = axis[k] with length D = in[0].shape[a]:
    s = start[k] (+ D if negative), clamped to [0, D]
    e = end[k]   (+ D if negative), clamped to [0, D]
    keep indices s, s + step[k], s + 2*step[k], ... while < e
out[0] = in[0] restricted to the kept indices on every listed axis
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `axis` | `int32[]` | required | Axes to slice; negative values count from the last axis. |
| `start` | `int64[]` | required | First index kept on each axis (inclusive). |
| `end` | `int64[]` | required | End index on each axis (exclusive). A value past the axis length is clamped, so a large value such as `2147483647` means "to the end". |
| `step` | `int64[]` | optional | Stride on each axis. Absent means 1 on every axis. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | in\[0\] with each sliced axis shortened | Output activation. |

### Softmax

Normalizes exponentials along one axis so they sum to 1.

Computes softmax over the axis `axis` of in\[0\], optionally scaling the input by `beta` first.

```text
z = beta * in[0]
out[0] = exp(z) / sum(exp(z), axis)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `beta` | `float32` | `1.0` | Scale applied to the input before the exponential. Must not be 0. |
| `axis` | `int32` | required | Axis to normalize over. A negative value counts from the last axis. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Softplus

Smooth ReLU `log(1 + exp(beta * x)) / beta`, linear above `threshold`.

Where `beta * x` exceeds `threshold` the output is `x` itself.

```text
out[0] = in[0]                               if beta * in[0] > threshold
         log(1 + exp(beta * in[0])) / beta   otherwise
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `beta` | `float32` | `1.0` | Sharpness of the curve; larger values approach ReLU. |
| `threshold` | `float32` | `20.0` | Value of `beta * x` above which the output is linear. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Sqrt

Element-wise square root.

```text
out[0] = sqrt(in[0])
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### StaggeredPadding

Shifts row r of each matrix by r columns with zero fill, then crops.

Treats the last two axes of `[1, H, W, C]` as `W` rows of length `C` and shifts each row by its row index into a zero-filled row, giving a staircase layout. `reverse=false` shifts row `r` right by `r` and drops `window_size` columns at each side; `reverse=true` reads, for each row `r`, the `2*window_size + 1` columns centered on column `r`.

```text
x[n, h, r, k] = in[0][n, h, r, k] for 0 <= k < C, and 0 for any other k
reverse = false:
  o = 0 if window_size = -1, else window_size
  out[0][n, h, r, j] = x[n, h, r, j + o - r],        j < W + C - 1 - 2*o
reverse = true:
  out[0][n, h, r, j] = x[n, h, r, r + j - window_size],   j < 2*window_size + 1
Equivalent sequence for reverse = false:
  [1, H, W, C] -pad axis 3 end by W-> [1, H, W, C+W] -reshape-> [1, H, W*(C+W)]
  -drop the last W-> -reshape-> [1, H, W, W+C-1] -slice axis 3 [o, W+C-1-o)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `axis` | `int32` | `3` | Axis whose entries are shifted. Only `3` (the last axis) is defined; rows run along axis 2. |
| `reverse` | `bool` | `false` | `false`: row `r` starts at column `r` (staircase toward the lower right). `true`: row `r` starts at column `W - 1 - r` (toward the lower left), and the result is cropped to a `2*window_size + 1` band. |
| `window_size` | `int32` | `-1` | `reverse=false`: columns dropped at each side, `-1` to keep the full width `W + C - 1`. `reverse=true`: half-width of the band; must be given (not `-1`), and `W` and `C` are each at least `window_size + 1`. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | reverse=false: \[1, H, W, W + C - 1 - 2*o\]; reverse=true: \[1, H, W, 2*window\_size + 1\] | Output activation. |

### StatefulAttentionMaskWrapper

Adds a causal, sliding-window or supplied mask to attention scores.

Applies the attention mask to raw attention scores. With `causal_mask` the bias is derived from the score shape alone, taking the `Q` queries as the last `Q` of the `K` key positions. With `custom_mask` the bias is a second input.

```text
custom_mask:
  out[0] = in[0] + in[1]
causal_mask:
  qp(i) = K - Q + i                        key position of query i
  masked(i, j) = j > qp(i)  or  (is_sliding and j <= qp(i) - sliding_window)
  out[0][..., i, j] = -inf if masked(i, j) else in[0][..., i, j]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, heads, Q, K]` | Input activation. |
| `in[1]` | broadcastable to in\[0\] | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `past_seqlen` | `int32` | `0` | Number of cached positions in front of the queries when the graph was traced. Informational; the mask follows the runtime `Q` and `K`. |
| `is_sliding` | `bool` | `false` | `true` also masks keys `sliding_window` or more positions behind the query (`causal_mask` only). |
| `sliding_window` | `int32` | `1024` | Number of most recent key positions, the query's own included, that a query may attend to when `is_sliding`. |
| `mask_type` | `str` | `"causal_mask"` | `causal_mask`: mask synthesized from positions, one input. `custom_mask`: adds in\[1\], two inputs. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### StatefulCachedStack

Prepends the last entries kept from previous calls along an axis.

Keeps the last `cache_len` entries along `axis` across calls and prepends them to each new input. The first call sees zeros; later calls see the tail of the earlier inputs. The kept window is state held by the op, not an operand.

```text
state: entries of shape in[0] with axis extent cache_len, initially zeros
out[0] = concat(state, in[0], axis)
state  = last cache_len entries of out[0] along axis
cache_len = 0:  out[0] = in[0]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `axis` | `int32` | `2` | Axis to stack along, by default 2 (the sequence axis W of NHWC). Negative counts from the last axis. |
| `cache_len` | `int32` | `0` | Number of entries kept and prepended. `0` makes the op an identity. |
| `cache_name` | `str` | required | Name of the kept window. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | in\[0\] with axis extent + cache\_len | Output activation. |

### StatefulGRUWrapper

GRU over a sequence, returning per-step outputs and the final hidden state.

Runs a gated recurrent unit recurrence along the time axis of in\[0\] with the constant weights `W`, `R`, `B`, in one or both directions. `input_mask` names the operands that are present; the results are always out\[0\] (hidden state at every step) and out\[1\] (final hidden state).

```text
batch_first = true; T = steps; @ = matrix product, * = element-wise product
for each direction d (0 = forward, 1 = backward):
  act_f, act_g = activations[2d .. 2d+1]          (activations[0..1] if only 2)
  Wz, Wr, Wh = W[d] split into 3 row blocks;  Rz, Rr, Rh = R[d] likewise
  Wbz, Wbr, Wbh, Rbz, Rbr, Rbh = B[d] split into 6   (zero if no B)
  h = in[h][:, 0, d, :]                              (zero if absent)
  for t = 0 .. T-1 (forward) or t = T-1 .. 0 (backward):
    x = in[0][:, 0, t, :]
    z = act_f(x @ Wz^T + h @ Rz^T + Wbz + Rbz)
    r = act_f(x @ Wr^T + h @ Rr^T + Wbr + Rbr)
    n = act_g(x @ Wh^T + Wbh + (r * h) @ Rh^T + Rbh)   if not linear_before_reset
        act_g(x @ Wh^T + Wbh + r * (h @ Rh^T + Rbh))   if linear_before_reset
    h = (1 - z) * n + z * h
    out[0][:, d, t, :] = h
  out[1][:, 0, d, :] = h
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, 1, T, input_size]` | Input activation. |
| `in[s]` | `any` | Input activation. |
| `in[h]` | `[1, 1, num_directions, hidden_size]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `W` | `Weight` | required | Input weights, `[num_directions, 3*hidden_size, input_size]`, gate blocks in the order z (update), r (reset), h (candidate). |
| `R` | `Weight` | required | Recurrent weights, `[num_directions, 3*hidden_size, hidden_size]`, gate order z, r, h. |
| `B` | `Weight` | optional | Biases, `[num_directions, 6*hidden_size]`: the input biases (z, r, h) followed by the recurrent biases (z, r, h). Absent means zero. |
| `input_mask` | `str` | required | Letters naming the operands present, in operand order. `i` input sequence (always first), `s` sequence lengths, `h` initial hidden state; for example `ih` means in\[0\] is the sequence and in\[1\] the hidden state. |
| `output_mask` | `str` | required | Letters among `o`, `h` naming the results that are used (out\[0\], out\[1\]). The positions of the two results do not change. |
| `direction` | `str` | `"forward"` | `forward`, `backward` (`reverse` is treated the same) or `bidirectional`. `bidirectional` runs both and stacks forward (index 0) and backward (index 1) on the direction axis, so `num_directions` is 2; otherwise it is 1. |
| `batch_first` | `bool` | `false` | `true` for the batch-first layout used in the shapes here. `false` is time-major: in\[0\] is `[T, 1, input_size]`, the state is `[num_directions, 1, hidden_size]` and out\[0\] is `[T, num_directions, 1, hidden_size]`. |
| `hidden_size` | `int32` | required | Number of hidden units per direction. |
| `hidden_state_name` | `str` | required | Name of out\[1\] and of the state output of the cell `subgraph`. |
| `subgraph` | `str` | required | Name of the cell subgraph that computes one time step from the step input and the previous hidden state. |
| `activation_alpha` | `float32[]` | optional | Alpha coefficients for the functions in `activations`, in the same order. `sigmoid` and `tanh` take none. |
| `activation_beta` | `float32[]` | optional | Beta coefficients for the functions in `activations`, in the same order. `sigmoid` and `tanh` take none. |
| `activations` | `str[]` | optional | Gate functions `[act_f, act_g]`, usually `[sigmoid, tanh]`. With `bidirectional`, four names give the forward then the backward functions; two names apply to both. The reference computation defines `sigmoid` and `tanh`. |
| `clip` | `float32` | optional | Cell clip threshold. When set, the gate inputs are limited to `[-clip, clip]` before the activation; absent means no clipping. |
| `linear_before_reset` | `bool` | `false` | `false` applies the reset gate to the previous state before the recurrent product; `true` applies it to the product `h @ Rh^T + Rbh`. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `[1, num_directions, T, hidden_size]` | Output activation. |
| `out[1]` | `[1, 1, num_directions, hidden_size]` | Output activation. |

### StatefulKVCacheWrapper

Appends new key or value states to a persistent cache and returns it.

Keeps one layer's key or value cache across calls. Each call writes the `S` new states of in\[0\] at their absolute token positions and returns every state the current attention step can see, ordered by position. The cache and the token positions are runtime state, not IR operands.

```text
pos = absolute positions of the S new tokens (from the runtime), p0 = pos[0]
the cache holds the states of earlier calls by position; in[0] is stored at pos
is_sliding = false:
  out[0] = states of positions 0 .. pos[S-1], concatenated on axis 2
is_sliding = true:
  out[0] = states of positions max(0, p0 - (sliding_window - 1)) .. pos[S-1]
  only the last sliding_window positions stay stored for later calls
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, heads, S, D]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `cache_name` | `str` | required | Name of the cache state this op reads and writes. |
| `past_seqlen` | `int32` | `0` | Number of tokens already in the cache when the graph was traced. The static output length is `S + past_seqlen` (a dynamic extent when non-zero); at run time the length follows the token positions. At most `sliding_window - 1` when `is_sliding`. |
| `is_sliding` | `bool` | `false` | `true` keeps only the most recent `sliding_window` positions (sliding-window attention). |
| `sliding_window` | `int32` | `1024` | Window length in tokens when `is_sliding`. Positive and at most `max_cache_len`. |
| `max_cache_len` | `int32` | `8192` | Capacity of the cache in tokens. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | \[1, heads, P + S, D\], P = past tokens visible (p0, or min(p0, sliding\_window - 1) when sliding) | Output activation. |

### StatefulLSTMWrapper

LSTM over a sequence, returning per-step outputs and final hidden/cell states.

Runs a long short-term memory recurrence along the time axis of in\[0\] with the constant weights `W`, `R`, `B` and optional peepholes `P`, in one or both directions. `input_mask` names the operands that are present; the results are always out\[0\] (hidden state at every step), out\[1\] (final hidden state) and out\[2\] (final cell state).

```text
batch_first = true; T = steps; @ = matrix product, * = element-wise product
for each direction d (0 = forward, 1 = backward):
  act_f, act_g, act_h = activations[3d .. 3d+2]   (activations[0..2] if only 3)
  Wi, Wo, Wf, Wc = W[d] split into 4 row blocks;  Ri, Ro, Rf, Rc = R[d] likewise
  Wbi, Wbo, Wbf, Wbc, Rbi, Rbo, Rbf, Rbc = B[d] split into 8   (zero if no B)
  Pi, Po, Pf = P[d] split into 3                                (zero if no P)
  h = in[h][:, 0, d, :],  c = in[c][:, 0, d, :]                 (zero if absent)
  for t = 0 .. T-1 (forward) or t = T-1 .. 0 (backward):
    x = in[0][:, 0, t, :]
    i = act_f(x @ Wi^T + h @ Ri^T + Wbi + Rbi + Pi * c)
    f = act_f(x @ Wf^T + h @ Rf^T + Wbf + Rbf + Pf * c)
    g = act_g(x @ Wc^T + h @ Rc^T + Wbc + Rbc)
    c = f * c + i * g
    o = act_f(x @ Wo^T + h @ Ro^T + Wbo + Rbo + Po * c)
    h = o * act_h(c)
    out[0][:, d, t, :] = h
  out[1][:, 0, d, :] = h
  out[2][:, 0, d, :] = c
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, 1, T, input_size]` | Input activation. |
| `in[s]` | `any` | Input activation. |
| `in[h]` | `[1, 1, num_directions, hidden_size]` | Input activation. |
| `in[c]` | `[1, 1, num_directions, hidden_size]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `W` | `Weight` | required | Input weights, `[num_directions, 4*hidden_size, input_size]`, gate blocks in the order i, o, f, c. |
| `R` | `Weight` | required | Recurrent weights, `[num_directions, 4*hidden_size, hidden_size]`, gate order i, o, f, c. |
| `B` | `Weight` | optional | Biases, `[num_directions, 8*hidden_size]`: the input biases (i, o, f, c) followed by the recurrent biases (i, o, f, c). Absent means zero. |
| `input_mask` | `str` | required | Letters naming the operands present, in operand order. `i` input sequence (always first), `s` sequence lengths, `h` initial hidden state, `c` initial cell state; for example `ihc` means in\[0..2\] are the sequence, the hidden state and the cell state. |
| `output_mask` | `str` | required | Letters among `o`, `h`, `c` naming the results that are used (out\[0\], out\[1\], out\[2\]). The positions of the three results do not change. |
| `P` | `Weight` | optional | Peephole weights, `[num_directions, 3*hidden_size]`, in the order i, o, f. Absent means no peephole. |
| `direction` | `str` | `"forward"` | `forward`, `backward` (`reverse` is treated the same) or `bidirectional`. `bidirectional` runs both and stacks forward (index 0) and backward (index 1) on the direction axis, so `num_directions` is 2; otherwise it is 1. |
| `batch_first` | `bool` | `false` | `true` for the batch-first layout used in the shapes here. `false` is time-major: in\[0\] is `[T, 1, input_size]`, the states are `[num_directions, 1, hidden_size]` and out\[0\] is `[T, num_directions, 1, hidden_size]`. |
| `hidden_size` | `int32` | required | Number of hidden units per direction. |
| `hidden_state_name` | `str` | required | Name of out\[1\] and of the hidden-state output of the cell `subgraph`. |
| `cell_state_name` | `str` | required | Name of out\[2\] and of the cell-state output of the cell `subgraph`. |
| `subgraph` | `str` | required | Name of the cell subgraph that computes one time step from the step input and the previous hidden and cell states. |
| `activation_alpha` | `float32[]` | optional | Alpha coefficients for the functions in `activations`, in the same order. `sigmoid` and `tanh` take none. |
| `activation_beta` | `float32[]` | optional | Beta coefficients for the functions in `activations`, in the same order. `sigmoid` and `tanh` take none. |
| `activations` | `str[]` | optional | Gate functions `[act_f, act_g, act_h]`, usually `[sigmoid, tanh, tanh]`. With `bidirectional`, six names give the forward then the backward functions; three names apply to both. The reference computation defines `sigmoid` and `tanh`. |
| `clip` | `float32` | optional | Cell clip threshold. When set, the gate inputs are limited to `[-clip, clip]` before the activation; absent means no clipping. |
| `input_forget` | `bool` | `false` | `true` couples the input and forget gates. The formula above is the `false` case. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `[1, num_directions, T, hidden_size]` | Output activation. |
| `out[1]` | `[1, 1, num_directions, hidden_size]` | Output activation. |
| `out[2]` | `[1, 1, num_directions, hidden_size]` | Output activation. |

### StatefulRNNWrapper

Simple RNN over a sequence, returning per-step outputs and the final state.

Runs the recurrence `h = act_f(x W^T + h R^T + b)` along the time axis of in\[0\] with the constant weights `W`, `R`, `B`, in one or both directions. `input_mask` names the operands that are present; the results are always out\[0\] (hidden state at every step) and out\[1\] (final hidden state).

```text
batch_first = true; T = steps; @ = matrix product
for each direction d (0 = forward, 1 = backward):
  act_f = activations[d]            (activations[0] if only 1)
  Wb, Rb = B[d] split into 2        (zero if no B)
  h = in[h][:, 0, d, :]             (zero if absent)
  for t = 0 .. T-1 (forward) or t = T-1 .. 0 (backward):
    x = in[0][:, 0, t, :]
    h = act_f(x @ W[d]^T + h @ R[d]^T + Wb + Rb)
    out[0][:, d, t, :] = h
  out[1][:, 0, d, :] = h
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, 1, T, input_size]` | Input activation. |
| `in[s]` | `any` | Input activation. |
| `in[h]` | `[1, 1, num_directions, hidden_size]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `W` | `Weight` | required | Input weights, `[num_directions, hidden_size, input_size]`. |
| `R` | `Weight` | required | Recurrent weights, `[num_directions, hidden_size, hidden_size]`. |
| `B` | `Weight` | optional | Biases, `[num_directions, 2*hidden_size]`: the input bias followed by the recurrent bias. Absent means zero. |
| `input_mask` | `str` | required | Letters naming the operands present, in operand order. `i` input sequence (always first), `s` sequence lengths, `h` initial hidden state; for example `ih` means in\[0\] is the sequence and in\[1\] the hidden state. |
| `output_mask` | `str` | required | Letters among `o`, `h` naming the results that are used (out\[0\], out\[1\]). The positions of the two results do not change. |
| `direction` | `str` | `"forward"` | `forward`, `backward` (`reverse` is treated the same) or `bidirectional`. `bidirectional` runs both and stacks forward (index 0) and backward (index 1) on the direction axis, so `num_directions` is 2; otherwise it is 1. |
| `batch_first` | `bool` | `false` | `true` for the batch-first layout used in the shapes here. `false` is time-major: in\[0\] is `[T, 1, input_size]`, the state is `[num_directions, 1, hidden_size]` and out\[0\] is `[T, num_directions, 1, hidden_size]`. |
| `hidden_size` | `int32` | required | Number of hidden units per direction. |
| `hidden_state_name` | `str` | required | Name of out\[1\] and of the state output of the cell `subgraph`. |
| `subgraph` | `str` | required | Name of the cell subgraph that computes one time step from the step input and the previous hidden state. |
| `activation_alpha` | `float32[]` | optional | Alpha coefficients for the functions in `activations`, in the same order. `sigmoid` and `tanh` take none. |
| `activation_beta` | `float32[]` | optional | Beta coefficients for the functions in `activations`, in the same order. `sigmoid` and `tanh` take none. |
| `activations` | `str[]` | optional | Recurrence function `[act_f]`, usually `[tanh]`. With `bidirectional`, two names give the forward then the backward function; one name applies to both. The reference computation defines `sigmoid` and `tanh`. |
| `clip` | `float32` | optional | Cell clip threshold. When set, the input of the activation is limited to `[-clip, clip]`; absent means no clipping. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `[1, num_directions, T, hidden_size]` | Output activation. |
| `out[1]` | `[1, 1, num_directions, hidden_size]` | Output activation. |

### StatefulRoPEWrapper

Applies rotary position embedding to queries or keys at their positions.

Rotates channel pairs of a query or key tensor by position-dependent angles, using precomputed cos/sin tables whose row `p` belongs to position `p`. The rows are picked by the absolute positions of the tokens in the current call.

```text
x = in[0];  c, s = rows of in[1], in[2] for the S current positions
rot0(y) = concat(-y[..., h:], y[..., :h]),   h = width(y) / 2
rot1(y)[..., 2i] = -y[..., 2i+1],  rot1(y)[..., 2i+1] = y[..., 2i]
rope_type 0:  R = rotary_emb_dim if > 0, else width(c)
              y = x[..., :R]
              out[0] = concat(y * c[..., :R] + rot0(y) * s[..., :R], x[..., R:])
rope_type 1:  out[0] = x * c + rot1(x) * s
rope_type 2:  rope_type 0 with R = width(c)
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, heads, S, D]` | Input activation. |
| `in[1]` | `[1, 1, L, R]` | Input activation. |
| `in[2]` | `[1, 1, L, R]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `rope_type` | `int32` | `0` | Channel pairing: `0` pairs channel `i` with `i + R/2` (rotary channels split in halves); `1` pairs adjacent channels `2i`, `2i+1`; `2` is `0` with the rotary width taken from the cos table. |
| `seqlen` | `int32` | `0` | Sequence length `S` of in\[0\] when the graph was built. Informational; the computation uses the runtime shape. |
| `rotary_emb_dim` | `int32` | `0` | Number of leading channels of `D` that are rotated (partial rotary); the rest pass through unchanged. `0` means the width of the cos table. Used with `rope_type=0` only. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Sub

Element-wise difference of two activations.

Subtracts in\[1\] from in\[0\] element by element.

```text
out[0] = in[0] - in[1]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |
| `in[1]` | broadcastable to in\[0\] | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `broadcast(in[0], in[1])` | Output activation. |

### SubConstant

Subtracts a constant from an activation, or an activation from a constant.

Subtracts with one operand taken from a constant stored in the model. The constant may be a scalar, a per-channel vector or a full tensor. With `is_const_first` true the result is `constant - in[0]`, otherwise `in[0] - constant`.

```text
out[0] = constant - in[0]   if is_const_first
         in[0] - constant   otherwise
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `constant` | `Weight` | required | Constant operand. Broadcasts against in\[0\]. |
| `is_const_first` | `bool` | required | `true` if the constant is the minuend (left operand), `false` if it is subtracted. |
| `shape` | `int32[]` | `[]` | Shape of the constant operand before the stored `constant` was reduced (e.g. to a per-channel vector). The output shape is broadcast(shape(in\[0\]), shape). The default `[]` means the tensor's shape was not recorded; the field is optional. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `broadcast(in[0], shape)` | Output activation. |

### Swish

Self-gated activation `x * sigmoid(x)`.

```text
out[0] = in[0] * sigmoid(in[0]) = in[0] / (1 + exp(-in[0]))
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Tan

Element-wise tangent.

```text
out[0] = tan(in[0])
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### Tanh

Element-wise hyperbolic tangent.

```text
out[0] = tanh(in[0])
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

None.

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | same as in\[0\] | Output activation. |

### TensorCmp

Element-wise comparison of two activations, giving a boolean tensor.

Compares in\[0\] with in\[1\] element by element, after broadcasting, using the relation named by `kind`. The output is boolean - true where the relation holds.

```text
kind = "eq":   out[0] = (in[0] == in[1])
kind = "neq":  out[0] = (in[0] != in[1])
kind = "lt":   out[0] = (in[0] <  in[1])
kind = "leq":  out[0] = (in[0] <= in[1])
kind = "gt":   out[0] = (in[0] >  in[1])
kind = "geq":  out[0] = (in[0] >= in[1])
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |
| `in[1]` | broadcastable to in\[0\] | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `kind` | `str` | `"eq"` | Relation: `eq`, `neq`, `lt`, `leq`, `gt` or `geq`, read as `in[0] <kind> in[1]`. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `broadcast(in[0], in[1])` | Output activation. |

### Transpose

Permutes the axes of an activation.

Reorders the axes of in\[0\] so that axis i of out\[0\] is axis `transpose_axes[i]` of in\[0\].

```text
P = transpose_axes
out[0].shape[i] = in[0].shape[P[i]]
out[0][i_0, ..., i_{r-1}] = in[0][j]   where j[P[k]] = i_k
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, C]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `transpose_axes` | `int64[]` | required | Permutation of `0..rank-1`; entry i names the input axis that becomes output axis i. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `[in[0].shape[transpose_axes[i]] for each i]` | Output activation. |

### TransposeConvolution

Transposed 2-D convolution, stored as zero insertion plus a convolution.

Upsamples in\[0\] by inserting `h_stride - 1` zero rows and `w_stride - 1` zero columns between neighbouring pixels, zero-pads that tensor by the `*_padding` attributes, and convolves it with `weight` at stride 1 and the given dilation, adding `bias`. The weight is therefore stored in convolution layout, `[out_channels, in_channels / group_number, h_kernel, w_kernel]`, holding the transposed-convolution kernel flipped along H and W, and the paddings apply to the zero-inserted tensor rather than being cropped from the output.

```text
G  = group_number,  Cg = in_channels / G,  Og = out_channels / G   (// : integer division)
z  = zeros([1, (H-1)*h_stride + 1, (W-1)*w_stride + 1, in_channels])
z[n, i*h_stride, j*w_stride, c] = in[0][n, i, j, c]
zp = zero_pad(z, top=north_padding, bottom=south_padding,
                 left=west_padding, right=east_padding)
out[0][n, y, x, o] = bias[o] + sum_{ky, kx, c < Cg}
    weight[o, c, ky, kx]
    * zp[n, y + ky*h_dilation, x + kx*w_dilation, (o // Og)*Cg + c]
```

#### Inputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `in[0]` | `[1, H, W, in_channels]` | Input activation. |

#### Attributes

| Name | Type | Default | Description |
| :--- | :--- | :--- | :--- |
| `in_channels` | `int32` | required | Number of input channels. |
| `out_channels` | `int32` | required | Number of output channels. |
| `h_kernel` | `int32` | required | Kernel height. |
| `w_kernel` | `int32` | required | Kernel width. |
| `h_stride` | `int32` | required | Upsampling stride along H; `h_stride - 1` zero rows go between input rows. |
| `w_stride` | `int32` | required | Upsampling stride along W; `w_stride - 1` zero columns go between input columns. |
| `h_dilation` | `int32` | required | Dilation along H. |
| `w_dilation` | `int32` | required | Dilation along W. |
| `west_padding` | `int32` | required | Zero padding at the start of W (left) of the zero-inserted tensor. |
| `east_padding` | `int32` | required | Zero padding at the end of W (right) of the zero-inserted tensor. |
| `north_padding` | `int32` | required | Zero padding at the start of H (top) of the zero-inserted tensor. |
| `south_padding` | `int32` | required | Zero padding at the end of H (bottom) of the zero-inserted tensor. |
| `padding_list` | `int32[]` | `[]` | The original transposed-convolution padding as `[top, left, bottom, right]` (amounts removed from the output), kept for reference. The computation uses the four `*_padding` attributes. |
| `og_padding_list` | `int32[]` | `[]` | Slot for the original padding of the transposed convolution. Informational; the computation does not read it, and it is normally empty. |
| `og_out_padding_list` | `int32[]` | `[]` | Slot for the original output padding of the transposed convolution (extra rows and columns at the end of the output). Informational; the computation does not read it, and it is normally empty. |
| `group_number` | `int32` | required | Number of channel groups. Divides both `in_channels` and `out_channels`. |
| `weight` | `Weight` | required | Filter in convolution layout, `[out_channels, in_channels / group_number, h_kernel, w_kernel]`: the transposed-convolution kernel flipped along H and W. |
| `bias` | `Weight` | optional | Bias, `[out_channels]`. Absent means zero. |
| `channel_last` | `bool` | `true` | Must be `true`; the NPU uses the NHWC layout. |

#### Outputs

| Name | Shape | Description |
| :--- | :--- | :--- |
| `out[0]` | `[1, H_out, W_out, out_channels]` | Output activation. |

## Other Mobilint IR operations

The remaining operations have input and attribute metadata in the table below. Their presence in this table does not imply NPU or CPU Offload support.

| Operation | Notes |
| :--- | :--- |
| Abs | num inputs: 1<br>num outputs: 1 |
| AddPartial | num inputs: 2<br>num outputs: 1<br>attributes:<br>\- is\_input0\_larger: bool |
| Cast | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- dtype: str |
| Ceil | num inputs: 1<br>num outputs: 1 |
| Convolution3d | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- in\_channels: int32<br>\- out\_channels: int32<br>\- h\_kernel: int32<br>\- w\_kernel: int32<br>\- d\_kernel: int32<br>\- h\_stride: int32<br>\- w\_stride: int32<br>\- d\_stride: int32<br>\- h\_dilation: int32<br>\- w\_dilation: int32<br>\- d\_dilation: int32<br>\- west\_padding: int32<br>\- east\_padding: int32<br>\- north\_padding: int32<br>\- south\_padding: int32<br>\- front\_padding: int32<br>\- back\_padding: int32<br>\- padding\_list: int32\[\] (default=\[\])<br>\- group\_number: int32<br>\- weight: Weight<br>\- bias: Weight (optional)<br>\- channel\_last: bool (default=true) |
| Crop | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- w\_offset: int32 (default=0)<br>\- h\_offset: int32 (default=0)<br>\- crop\_width: int32<br>\- crop\_height: int32 |
| DeviceBridge | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- squeeze\_axes: int32\[\]<br>\- unsqueeze\_axes: int32\[\]<br>\- permute\_axes: int64\[\]<br>\- device: str (optional) |
| Expand | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- constant: Weight (optional)<br>\- is\_const\_first: bool |
| GLU | num inputs: 1<br>num outputs: 1 |
| GridSample | num inputs: 2<br>num outputs: 1<br>attributes:<br>\- align\_corners: int32 (default=0)<br>\- mode: str (default="linear")<br>\- padding\_mode: str (default="zeros")<br><br>description:<br>inputs: X, grid<br>X is in channel-last format.<br>X: (n, h, w, c)<br>grid: (n, h\_out, w\_out, num\_spatial\_dims). in this case, num\_spatial\_dims=2. |
| Hardtanh | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- min\_val: float32 (default=-1.0)<br>\- max\_val: float32 (default=1.0) |
| Hardshrink | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- lambd: float32 (default=0.5) |
| Softshrink | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- lambd: float32 (default=0.5) |
| Identity | num inputs: 1<br>num outputs: 1 |
| SumNormalization | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- norm\_axis: int32\[\]<br>\- epsilon: float32 (optional) |
| LSTM | num inputs: variable<br>num outputs: 1<br>attributes:<br>\- W: Weight<br>\- R: Weight<br>\- B: Weight (optional)<br>\- lstm\_group: str<br>\- input\_mask: str<br>\- output\_mask: str<br>\- P: Weight (optional)<br>\- direction: str (default="forward")<br>\- batch\_first: bool (default=false)<br>\- hidden\_size: int32<br><br>description:<br>inputs:\[input, sequence\_lens, initial\_h, initial\_c\]<br>outputs:\[output\] or \[hidden\_out\] or \[cell\_out\]<br>input\_mask: "ishc" or "i", etc,...<br>output\_mask: "o" or "h" or "c" |
| Logit | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- eps: float32 (optional)<br><br>description:<br>Compute y = ln(z / (1 - z)). If eps is given, we clamp input z = clamp(z, min=eps, max=1-eps) |
| RNN | num inputs: variable<br>num outputs: 1<br>attributes:<br>\- W: Weight<br>\- R: Weight<br>\- B: Weight (optional)<br>\- rnn\_group: str<br>\- input\_mask: str<br>\- output\_mask: str<br>\- direction: str (default="forward")<br>\- batch\_first: bool (default=false)<br>\- hidden\_size: int32<br><br>description:<br>inputs:\[input, sequence\_lens, initial\_h\]<br>outputs:\[output\] or \[hidden\_out\]<br>input\_mask: "ish" or "i", etc,...<br>output\_mask: "o" or "h" |
| GRU | num inputs: variable<br>num outputs: 1<br>attributes:<br>\- W: Weight<br>\- R: Weight<br>\- B: Weight (optional)<br>\- gru\_group: str<br>\- input\_mask: str<br>\- output\_mask: str<br>\- direction: str (default="forward")<br>\- batch\_first: bool (default=false)<br>\- linear\_before\_reset: bool (default=false)<br>\- hidden\_size: int32<br><br>description:<br>inputs:\[input, sequence\_lens, initial\_h\]<br>outputs:\[output\] or \[hidden\_out\]<br>input\_mask: "ish" or "i", etc,...<br>output\_mask: "o" or "h" |
| LogSigmoid | num inputs: 1<br>num outputs: 1 |
| MaskSoftmax | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- beta: float32<br>\- axis: int32 |
| NonMaxSuppression | num inputs: 2<br>num outputs: variable<br>attributes:<br>\- max\_output\_size: int64<br>\- iou\_threshold: float32<br>\- score\_threshold: float32<br>\- soft\_nms\_sigma: float32<br>\- pad\_to\_max\_output\_size: bool<br>\- center\_point\_box: int32 (default=0) |
| GlobalAveragePooling | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- channel\_last: bool (default=true) |
| GlobalResponseNormalization | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- scale: Weight<br>\- bias: Weight<br>\- epsilon: float32<br>\- channel\_last: bool (default=true) |
| ReduceL2 | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- keep\_dims: bool<br>\- reduce\_axes: int32\[\]<br>\- noop\_with\_empty\_axes: int32 |
| ReduceProd | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- keep\_dims: bool<br>\- reduce\_axes: int32\[\]<br>\- noop\_with\_empty\_axes: int32 |
| ReduceRMS | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- keep\_dims: bool<br>\- reduce\_axes: int32\[\]<br>\- epsilon: float32 (optional)<br><br>description:<br>reduce rms with epsilon if exists. i.e.,sqrt(mean\*\*2 + eps)<br>we let eps to be optional so that it has more general fusing opportunities. |
| MeanCentering | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- axes: int32\[\]<br><br>description:<br>compute x - mean(x, axes) |
| ReshapeV2 | num inputs: 2<br>num outputs: 1<br>attributes:<br>\- allowzero: int32 (default=0)<br>\- is\_const\_first: bool (default=false)<br><br>description:<br>Reshape with variable shape argument. inputs\[1\] will be used as a shape. |
| RotaryEmbedding | num inputs: 1<br>num outputs: 2<br>attributes:<br>\- inv\_freq: Weight<br>\- attention\_scaling: float32<br><br>description:<br>following the LlamaRotaryEmbedding of the Huggingface Transformer library.<br>inputs =\> (position\_ids,), outputs =\> (cos, sin) |
| Rsqrt | num inputs: 1<br>num outputs: 1 |
| Rstd | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- reduce\_axes: int32\[\]<br>\- epsilon: Weight<br>\- num\_groups: int32 (default=1)<br><br>description:<br>computes 1/std(x). if gruops \> 1, then activation is grouped along channel axis. |
| Gather | num inputs: 2<br>num outputs: 1<br>attributes:<br>\- axis: int32 (default=0) |
| GatherElements | num inputs: 2<br>num outputs: 1<br>attributes:<br>\- axis: int32 (default=0) |
| ScatterND | num inputs: variable<br>num outputs: 1<br>attributes:<br>\- constant\_data: Weight (optional)<br>\- constant\_indices: Weight (optional) |
| Selu | num inputs: 1<br>num outputs: 1 |
| Shape | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- start: int32 (optional)<br>\- end: int32 (optional) |
| LogSoftmax | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- axis: int32 |
| Softsign | num inputs: 1<br>num outputs: 1 |
| Unsqueeze | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- axes: int32\[\] |
| DynamicSliceInputConst | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- mod: int32<br>\- start: int64\[\]<br>\- end: int64\[\]<br>\- axis: int64\[\]<br>\- step: int64\[\] (optional) |
| MeloTTSPiecewiseRationalQuadraticTransform | num inputs: 2<br>num outputs: 1<br>attributes:<br>\- num\_bins: int32<br>\- filter\_channels: int32<br>\- tail\_bound: float32<br>\- reverse: bool<br><br>description:<br>helper function for parsing melotts model. |
| Neg | num inputs: 1<br>num outputs: 1 |
| Einsum | num inputs: variable<br>num outputs: 1<br>attributes:<br>\- equation: str<br>\- constant: Weight (optional)<br>\- is\_const\_first: bool (default=false) |
| Where | num inputs: 3<br>num outputs: 1<br><br><br>description:<br>condition,X,Y, if cond == True: else y |
| Logical | num inputs: variable<br>num outputs: 1<br>attributes:<br>\- kind: str (default="and")<br><br>description:<br>this function represents the logical operation of two inputs.<br>single input if kind is "not" else two inputs<br>kind: and, or, xor, not |
| MaskedFill | num inputs: 2<br>num outputs: 1<br>attributes:<br>\- value: float32<br><br>description:<br>inputs: (x, mask) |
| SubgraphCall | num inputs: variable<br>num outputs: variable<br>attributes:<br>\- subgraph: str<br>\- scope: str<br>\- kind: str |
| MoeDispatcher | num inputs: 2<br>num outputs: 2<br>attributes:<br>\- top\_k: int32<br>\- norm\_top\_k\_prob: bool (default=true)<br><br>description:<br>inputs: (hidden\_states, router\_logits)<br>outputs: (hidden\_states, top\_k\_logits)<br>if norm\_top\_k\_prob is true<br>topk\_vals, topk\_idx = torch.topk(<br>routing\_weights, self.top\_k, dim=-1<br>) \# (18,8), (18,8)<br>weight\_new = torch.zeros\_like(routing\_weights)<br>weight\_new.scatter\_(1, topk\_idx, topk\_vals)<br>routing\_weights = weight\_new<br>routing\_weights /= routing\_weights.sum(dim=1, keepdim=True) |
| MoeAggregator | num inputs: variable<br>num outputs: 1<br><br><br>description:<br>inputs: (router\_weights, experts\_output0, expert\_output1, ...) |
| TopKValues | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- axis: int32<br>\- k: int32 (default=1)<br>\- largest: bool (default=true)<br>\- sorted: bool (default=true) |
| TopKIndices | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- axis: int32<br>\- k: int32 (default=1)<br>\- largest: bool (default=true)<br>\- sorted: bool (default=true) |
