# Mobilint IR 연산 형식

아래 표는 Mobilint IR에 나타날 수 있는 연산의 형식을 설명합니다. 목록에 있다는 사실만으로 NPU 또는 CPU 오프로딩 지원이 보장되지는 않습니다. 지원 범위는 [NPU 연산 지원 범위](<https://docs.mobilint.com/compiler/kr/v1.3/operation_support/index.md>)와 [CPU 오프로딩](<https://docs.mobilint.com/compiler/kr/v1.3/cpu_offloading/index.md>)을 확인하세요.

| 연산 | 비고 |
| :--- | :--- |
| Abs | num inputs: 1<br>num outputs: 1 |
| AddPartial | num inputs: 2<br>num outputs: 1<br>attributes:<br>\- is\_input0\_larger: bool |
| AddingConstant | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- constant: Weight<br>\- is\_const\_first: bool (default=false)<br>\- shape: int32\[\] (default=\[\]) |
| Adding | num inputs: 2<br>num outputs: 1 |
| Batchnorm | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- gamma: Weight<br>\- beta: Weight<br>\- moving\_mean: Weight<br>\- moving\_var: Weight<br>\- epsilon: Weight<br>\- channel\_last: bool (default=true) |
| Cast | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- dtype: str |
| Ceil | num inputs: 1<br>num outputs: 1 |
| Celu | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- alpha: float32 |
| Clip | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- clip\_max\_value: float32 (optional)<br>\- clip\_min\_value: float32 (optional) |
| Concatenate | num inputs: variable<br>num outputs: 1<br>attributes:<br>\- original\_key: str (optional)<br>\- axis: int32 (optional)<br>\- original\_axis: int32 (optional)<br>\- original\_inputs: int32\[\] (optional)<br>\- input\_index: int32\[\] (optional)<br>\- input\_names: str\[\] (optional)<br><br>description:<br>input\_names: input activation names in original order. input\_index: input index in computation order. |
| Convolution | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- in\_channels: int32<br>\- out\_channels: int32<br>\- h\_kernel: int32<br>\- w\_kernel: int32<br>\- h\_stride: int32<br>\- w\_stride: int32<br>\- h\_dilation: int32<br>\- w\_dilation: int32<br>\- west\_padding: int32<br>\- east\_padding: int32<br>\- north\_padding: int32<br>\- south\_padding: int32<br>\- padding\_list: int32\[\] (default=\[\])<br>\- group\_number: int32<br>\- weight: Weight<br>\- bias: Weight (optional)<br>\- channel\_last: bool (default=true) |
| Convolution3d | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- in\_channels: int32<br>\- out\_channels: int32<br>\- h\_kernel: int32<br>\- w\_kernel: int32<br>\- d\_kernel: int32<br>\- h\_stride: int32<br>\- w\_stride: int32<br>\- d\_stride: int32<br>\- h\_dilation: int32<br>\- w\_dilation: int32<br>\- d\_dilation: int32<br>\- west\_padding: int32<br>\- east\_padding: int32<br>\- north\_padding: int32<br>\- south\_padding: int32<br>\- front\_padding: int32<br>\- back\_padding: int32<br>\- padding\_list: int32\[\] (default=\[\])<br>\- group\_number: int32<br>\- weight: Weight<br>\- bias: Weight (optional)<br>\- channel\_last: bool (default=true) |
| Crop | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- w\_offset: int32 (default=0)<br>\- h\_offset: int32 (default=0)<br>\- crop\_width: int32<br>\- crop\_height: int32 |
| DepthwiseConvolution | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- in\_channels: int32<br>\- out\_channels: int32<br>\- h\_kernel: int32<br>\- w\_kernel: int32<br>\- h\_stride: int32<br>\- w\_stride: int32<br>\- h\_dilation: int32<br>\- w\_dilation: int32<br>\- west\_padding: int32<br>\- east\_padding: int32<br>\- north\_padding: int32<br>\- south\_padding: int32<br>\- padding\_list: int32\[\] (default=\[\])<br>\- group\_number: int32<br>\- weight: Weight<br>\- bias: Weight (optional)<br>\- channel\_last: bool (default=true) |
| DeviceBridge | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- squeeze\_axes: int32\[\]<br>\- unsqueeze\_axes: int32\[\]<br>\- permute\_axes: int64\[\]<br>\- device: str (optional) |
| DivConstant | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- constant: Weight<br>\- is\_const\_first: bool<br>\- shape: int32\[\] (default=\[\]) |
| Div | num inputs: 2<br>num outputs: 1 |
| Elu | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- leaky\_alpha: float32 |
| Erf | num inputs: 1<br>num outputs: 1 |
| Exp | num inputs: 1<br>num outputs: 1 |
| Expand | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- constant: Weight (optional)<br>\- is\_const\_first: bool |
| Gelu | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- approximate: bool (default=false) |
| GLU | num inputs: 1<br>num outputs: 1 |
| GridSample | num inputs: 2<br>num outputs: 1<br>attributes:<br>\- align\_corners: int32 (default=0)<br>\- mode: str (default="linear")<br>\- padding\_mode: str (default="zeros")<br><br>description:<br>inputs: X, grid<br>X is in channel-last format.<br>X: (n, h, w, c)<br>grid: (n, h\_out, w\_out, num\_spatial\_dims). in this case, num\_spatial\_dims=2. |
| GroupConvolution | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- in\_channels: int32<br>\- out\_channels: int32<br>\- h\_kernel: int32<br>\- w\_kernel: int32<br>\- h\_stride: int32<br>\- w\_stride: int32<br>\- h\_dilation: int32<br>\- w\_dilation: int32<br>\- west\_padding: int32<br>\- east\_padding: int32<br>\- north\_padding: int32<br>\- south\_padding: int32<br>\- padding\_list: int32\[\] (default=\[\])<br>\- group\_number: int32<br>\- weight: Weight<br>\- bias: Weight (optional)<br>\- channel\_last: bool (default=true) |
| GroupNormalization | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- num\_groups: int32<br>\- scale: Weight<br>\- bias: Weight (optional)<br>\- epsilon: Weight<br>\- channel\_last: bool (default=true) |
| HardSigmoid | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- alpha: float32 (optional)<br>\- beta: float32 (optional) |
| Hardtanh | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- min\_val: float32 (default=-1.0)<br>\- max\_val: float32 (default=1.0) |
| HardSwish | num inputs: 1<br>num outputs: 1 |
| Hardshrink | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- lambd: float32 (default=0.5) |
| Softshrink | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- lambd: float32 (default=0.5) |
| HeaderView | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- view\_shape: int32\[\] (default=\[\])<br>\- num\_heads: int32 (default=-1)<br>\- batch\_order: int32 (default=0)<br><br>description:<br>HeaderView simplifies the following sequence of operations.<br>If batch\_order == 0<br>(n,h,w,c)-\[reshape\]-\>(1, h, w, num\_heads c/num\_heads)<br>\-\[transpose01324\]-\>(1, h, num\_heads, w, c/num\_heads)<br>\-\[reshape\]-\>(1, h*num\_heads, w, c/num\_heads)<br>If batch\_order == 1<br>(n,h,w,c)-\[reshape\]-\>(1, h, w, num\_heads, c/num\_heads)<br>\-\[transpose03124\]-\>(1, num\_heads, h, w, c/num\_heads)<br>\-\[reshape\]-\>(1, num\_heads*h, w, c/num\_heads) |
| HeaderViewRevert | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- revert\_shape: int32\[\] (default=\[\])<br>\- num\_heads: int32 (default=-1)<br>\- batch\_order: int32 (default=0)<br><br>description:<br>HeaderViewRevert simplifies the following sequence of operations.<br>If batch\_order == 0<br>(n,h,w,c)-\[reshape\]-\>(1, h/num\_heads, num\_heads, w, c)<br>\-\[transpose01324\]-\>(1, h/num\_heads, w, num\_heads, c)<br>\-\[reshape\]-\>(1, h/num\_heads, w, num\_heads \* c)<br>If batch\_order == 1<br>(n,h,w,c)-\[reshape\]-\>(1, num\_heads, h/num\_heads, w, c)<br>\-\[transpose02314\]-\>(1, h/num\_heads, w, num\_heads, c)<br>\-\[reshape\]-\>(1, h/num\_heads, w, num\_heads\*c) |
| Identity | num inputs: 1<br>num outputs: 1 |
| InputConstant | num inputs: 0<br>num outputs: 1<br>attributes:<br>\- constant: Weight<br>\- dtype: str (default="")<br>\- shape: int32\[\] (default=\[\]) |
| Input | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- orig\_name: str (default="") |
| InstanceNormalization | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- scale: Weight<br>\- bias: Weight (optional)<br>\- epsilon: Weight<br>\- channel\_last: bool (default=true)<br>\- num\_groups: int32 (default=-1)<br>\- group\_order: int32 (default=-1)<br><br>description:<br>If `num_groups` and `order` are specified, the channel dimension is grouped and rearranged before normalization.<br>For example, the following sequence:<br>(n, 1, w, c*2)-\[reshape\]-\>(n, w, c, 2)<br>\-\[transpose0312\]-\>(n, 2, w, c)<br>\-\[InstanceNorm, num\_features=c\]-\>(n, 2, w, c)<br>\-\[transpose0231\]-\>(n, w, c, 2)<br>\-\[reshape\]-\>(n, 1, w, c*2)<br>is equivalent to a single operation:<br>(n, 1, w, c*2)-\[InstanceNormalization, num\_features=c, num\_groups=2, group\_order=1\]-\>(n, 1, w, c*2)<br>The follosing sequence:<br>(n, 1, w, 2*c)-\[reshape\]-\>(n,w,2,c)<br>\-\[InstanceNorm, num\_features=c\]-\>(n, w, 2, c)<br>\-\[reshape\]v(n, 1, w, c*2)<br>is equivalent to a single operation:<br>(n, 1, w, 2*c)-\[InstanceNormalization, num\_features=c, num\_groups=2, group\_order=0\]-\>(n, 1, w, 2*c) |
| L1Normalization | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- norm\_axis: int32\[\] |
| L2Normalization | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- norm\_axis: int32\[\]<br>\- epsilon: Weight (optional) |
| SumNormalization | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- norm\_axis: int32\[\]<br>\- epsilon: float32 (optional) |
| LSTM | num inputs: variable<br>num outputs: 1<br>attributes:<br>\- W: Weight<br>\- R: Weight<br>\- B: Weight (optional)<br>\- lstm\_group: str<br>\- input\_mask: str<br>\- output\_mask: str<br>\- P: Weight (optional)<br>\- direction: str (default="forward")<br>\- batch\_first: bool (default=false)<br>\- hidden\_size: int32<br><br>description:<br>inputs:\[input, sequence\_lens, initial\_h, initial\_c\]<br>outputs:\[output\] or \[hidden\_out\] or \[cell\_out\]<br>input\_mask: "ishc" or "i", etc,...<br>output\_mask: "o" or "h" or "c" |
| LayerNormalization | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- normalized\_shape: int64\[\] (default=\[\])<br>\- scale: Weight<br>\- bias: Weight (optional)<br>\- epsilon: Weight |
| Logit | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- eps: float32 (optional)<br><br>description:<br>Compute y = ln(z / (1 - z)). If eps is given, we clamp input z = clamp(z, min=eps, max=1-eps) |
| RNN | num inputs: variable<br>num outputs: 1<br>attributes:<br>\- W: Weight<br>\- R: Weight<br>\- B: Weight (optional)<br>\- rnn\_group: str<br>\- input\_mask: str<br>\- output\_mask: str<br>\- direction: str (default="forward")<br>\- batch\_first: bool (default=false)<br>\- hidden\_size: int32<br><br>description:<br>inputs:\[input, sequence\_lens, initial\_h\]<br>outputs:\[output\] or \[hidden\_out\]<br>input\_mask: "ish" or "i", etc,...<br>output\_mask: "o" or "h" |
| GRU | num inputs: variable<br>num outputs: 1<br>attributes:<br>\- W: Weight<br>\- R: Weight<br>\- B: Weight (optional)<br>\- gru\_group: str<br>\- input\_mask: str<br>\- output\_mask: str<br>\- direction: str (default="forward")<br>\- batch\_first: bool (default=false)<br>\- linear\_before\_reset: bool (default=false)<br>\- hidden\_size: int32<br><br>description:<br>inputs:\[input, sequence\_lens, initial\_h\]<br>outputs:\[output\] or \[hidden\_out\]<br>input\_mask: "ish" or "i", etc,...<br>output\_mask: "o" or "h" |
| LeakyRelu | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- leaky\_alpha: float32 |
| Log | num inputs: 1<br>num outputs: 1 |
| LogSigmoid | num inputs: 1<br>num outputs: 1 |
| MaskSoftmax | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- beta: float32<br>\- axis: int32 |
| MatMul | num inputs: variable<br>num outputs: 1<br>attributes:<br>\- constant: Weight (optional)<br>\- is\_const\_first: bool (default=false)<br>\- repeat: int32\[2\] (default=\[-1, -1\])<br><br>description:<br>If repeat=(0, 4), then input0 is repeated 4 times along the h-direction.<br>For example, you can perform a matrix multiplication with input0 of shape (1, 8, 52, 64) and<br>input1 of shape (1, 32, 52, 64) by concatenating input0 4 times, resulting in a input0 shape of (1, 32, 52, 64). |
| Mish | num inputs: 1<br>num outputs: 1 |
| MultiplyConstant | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- constant: Weight<br>\- is\_const\_first: bool (default=false)<br>\- shape: int32\[\] (default=\[\]) |
| Multiply | num inputs: 2<br>num outputs: 1 |
| NonMaxSuppression | num inputs: 2<br>num outputs: variable<br>attributes:<br>\- max\_output\_size: int64<br>\- iou\_threshold: float32<br>\- score\_threshold: float32<br>\- soft\_nms\_sigma: float32<br>\- pad\_to\_max\_output\_size: bool<br>\- center\_point\_box: int32 (default=0) |
| Output | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- orig\_name: str (default="") |
| PRelu | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- leaky\_alpha: Weight |
| Pad | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- west\_padding: int32 (default=-1)<br>\- east\_padding: int32 (default=-1)<br>\- south\_padding: int32 (default=-1)<br>\- north\_padding: int32 (default=-1)<br>\- pad\_mode: str<br>\- constant: float32 (default=0.0)<br>\- padding\_list: int32\[\] (optional)<br>\- channel\_last: bool (default=true)<br><br>description:<br>padding\_list format should be \[x1\_begin, x2\_begin, …, x1\_end, x2\_end,…\],<br>where xi\_begin is the number of pad values added at the beginning of axis |
| Pooling | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- h\_kernel: int32<br>\- w\_kernel: int32<br>\- h\_stride: int32<br>\- w\_stride: int32<br>\- h\_dilation: int32 (default=1)<br>\- w\_dilation: int32 (default=1)<br>\- west\_padding: int32<br>\- east\_padding: int32<br>\- north\_padding: int32<br>\- south\_padding: int32<br>\- padding\_list: int32\[\] (default=\[\])<br>\- pool\_type: str<br>\- pad\_mode: str<br>\- is\_ceil\_mode: bool (default=false)<br>\- is\_include\_pad: bool (default=false)<br>\- channel\_last: bool (default=true) |
| GlobalAveragePooling | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- channel\_last: bool (default=true) |
| GlobalResponseNormalization | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- scale: Weight<br>\- bias: Weight<br>\- epsilon: float32<br>\- channel\_last: bool (default=true) |
| Pow | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- exponent: float32 |
| QuickGelu | num inputs: 1<br>num outputs: 1 |
| ReduceL2 | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- keep\_dims: bool<br>\- reduce\_axes: int32\[\]<br>\- noop\_with\_empty\_axes: int32 |
| ReduceMax | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- keep\_dims: bool<br>\- reduce\_axes: int32\[\]<br>\- noop\_with\_empty\_axes: int32 |
| ReduceMin | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- keep\_dims: bool<br>\- reduce\_axes: int32\[\]<br>\- noop\_with\_empty\_axes: int32 |
| ReduceMean | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- keep\_dims: bool<br>\- reduce\_axes: int32\[\]<br>\- noop\_with\_empty\_axes: int32 |
| ReduceProd | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- keep\_dims: bool<br>\- reduce\_axes: int32\[\]<br>\- noop\_with\_empty\_axes: int32 |
| ReduceSum | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- keep\_dims: bool<br>\- reduce\_axes: int32\[\]<br>\- noop\_with\_empty\_axes: int32 |
| ReduceRMS | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- keep\_dims: bool<br>\- reduce\_axes: int32\[\]<br>\- epsilon: float32 (optional)<br><br>description:<br>reduce rms with epsilon if exists. i.e.,sqrt(mean\*\*2 + eps)<br>we let eps to be optional so that it has more general fusing opportunities. |
| MeanCentering | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- axes: int32\[\]<br><br>description:<br>compute x - mean(x, axes) |
| Relu | num inputs: 1<br>num outputs: 1 |
| Reshape | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- new\_shape: int32\[\] (optional)<br>\- allowzero: int32 (default=0)<br>\- is\_const\_first: bool (default=false) |
| ReshapeV2 | num inputs: 2<br>num outputs: 1<br>attributes:<br>\- allowzero: int32 (default=0)<br>\- is\_const\_first: bool (default=false)<br><br>description:<br>Reshape with variable shape argument. inputs\[1\] will be used as a shape. |
| Resize | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- resize\_type: str (optional)<br>\- resize\_mode: str (optional)<br>\- nearest\_mode: str (optional)<br>\- transform\_mode: str (optional)<br>\- roi: int32\[\] (optional)<br>\- scales: float32\[\] (optional)<br>\- sizes: int32\[\] (optional)<br>\- cubic\_coeff\_a: float32 (optional)<br>\- exclude\_outside: int32 (optional)<br>\- extrapolation\_value: float32 (optional)<br>\- antialias: int32 (optional)<br>\- axes: int32\[\] (optional)<br>\- keep\_aspect\_ratio\_policy: str (optional) |
| RmsNormalization | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- normalized\_shape: int64\[\] (default=\[\])<br>\- epsilon: Weight<br>\- scale: Weight<br>\- rms\_center: Weight (optional) |
| RotaryEmbedding | num inputs: 1<br>num outputs: 2<br>attributes:<br>\- inv\_freq: Weight<br>\- attention\_scaling: float32<br><br>description:<br>following the LlamaRotaryEmbedding of the Huggingface Transformer library.<br>inputs =\> (position\_ids,), outputs =\> (cos, sin) |
| Rsqrt | num inputs: 1<br>num outputs: 1 |
| Rstd | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- reduce\_axes: int32\[\]<br>\- epsilon: Weight<br>\- num\_groups: int32 (default=1)<br><br>description:<br>computes 1/std(x). if gruops \> 1, then activation is grouped along channel axis. |
| Gather | num inputs: 2<br>num outputs: 1<br>attributes:<br>\- axis: int32 (default=0) |
| GatherElements | num inputs: 2<br>num outputs: 1<br>attributes:<br>\- axis: int32 (default=0) |
| ScatterND | num inputs: variable<br>num outputs: 1<br>attributes:<br>\- constant\_data: Weight (optional)<br>\- constant\_indices: Weight (optional) |
| Selu | num inputs: 1<br>num outputs: 1 |
| Shape | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- start: int32 (optional)<br>\- end: int32 (optional) |
| Sigmoid | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- alpha: float32 (default=1.0)<br><br>description:<br>alpha is added to deal with more general case like 1 / (1 + exp(-alpha \* x)) |
| InverseSigmoid | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- eps: float32 (default=1e-05)<br><br>description:<br>x = x.clamp(x, min=0, max=1)<br>x1 = x.clamp(min=eps)<br>x2 = (1 - x).clamp(min=eps)<br>out = log(x1 / x2) |
| Slice | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- axis: int32\[\]<br>\- start: int64\[\]<br>\- end: int64\[\]<br>\- step: int64\[\] (optional) |
| Softmax | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- beta: float32 (default=1.0)<br>\- axis: int32 |
| LogSoftmax | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- axis: int32 |
| Softplus | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- beta: float32 (default=1.0)<br>\- threshold: float32 (default=20.0) |
| Softsign | num inputs: 1<br>num outputs: 1 |
| Sqrt | num inputs: 1<br>num outputs: 1 |
| StaggeredPadding | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- axis: int32 (default=3)<br>\- reverse: bool (default=false)<br>\- window\_size: int32 (default=-1)<br><br>description:<br>StaggeredPadding does the following:<br>\[\[\[\[1,1,1\],<br>\[2,2,2\],<br>\[3,3,3\],<br>\[4,4,4\]\]<br>\[5,5,5\]\]\]\]<br>\=\> \[\[\[\[1,1,1,0,0,0,0\],<br>\[0,2,2,2,0,0,0\],<br>\[0,0,3,3,3,0,0\],<br>\[0,0,0,4,4,4,0\],<br>\[0,0,0,0,5,5,5\]\]\]\]<br>That is, each original row is shifted to the right so that its non-zero block lies on a “diagonal,” and all other elements are zero-padded.<br>Then slice last dim by (2*window\_size + 1). For example, if window\_size=1 in the above, 5=2*1+1 elements are sliced.<br>\=\> \[\[\[\[1,1,0,0,0\],<br>\[2,2,2,0,0\],<br>\[0,3,3,3,0\],<br>\[0,0,4,4,4\],<br>\[0,0,0,5,5\]\]\]\]<br>If reverse<br>\[\[\[\[1,1,1\],<br>\[2,2,2\],<br>\[3,3,3\],<br>\[4,4,4\]\]<br>\[5,5,5\]\]\]\]<br>\=\> \[\[\[\[0,0,0,0,1,1,1\],<br>\[0,0,0,2,2,2,0\],<br>\[0,0,3,3,3,0,0\],<br>\[0,4,4,4,0,0,0\],<br>\[5,5,5,0,0,0,0\]\]\]\] |
| StatefulAttentionMaskWrapper | num inputs: 1 or 2<br>num outputs: 1<br>attributes:<br>\- past\_seqlen: int32 (default=0)<br>\- is\_sliding: bool (default=false)<br>\- sliding\_window: int32 (default=1024)<br>\- mask\_type: str (default="causal\_mask")<br><br>description:<br>mask\_type: "causal\_mask", "custom\_mask" — "custom\_mask" takes the mask as a second input |
| StatefulKVCacheWrapper | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- cache\_name: str<br>\- past\_seqlen: int32 (default=0)<br>\- is\_sliding: bool (default=false)<br>\- sliding\_window: int32 (default=1024)<br>\- max\_cache\_len: int32 (default=8192) |
| StatefulCachedStack | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- axis: int32 (default=2)<br>\- cache\_len: int32 (default=0)<br>\- cache\_name: str |
| StatefulLSTMWrapper | num inputs: variable<br>num outputs: 3<br>attributes:<br>\- W: Weight<br>\- R: Weight<br>\- B: Weight (optional)<br>\- input\_mask: str<br>\- output\_mask: str<br>\- P: Weight (optional)<br>\- direction: str (default="forward")<br>\- batch\_first: bool (default=false)<br>\- hidden\_size: int32<br>\- hidden\_state\_name: str<br>\- cell\_state\_name: str<br>\- subgraph: str<br>\- activation\_alpha: float32\[\] (optional)<br>\- activation\_beta: float32\[\] (optional)<br>\- activations: str\[\] (optional)<br>\- clip: float32 (optional)<br>\- input\_forget: bool (default=false)<br><br>description:<br>When input\_mask is `ishc`, then inputs are \[input, sequence\_lens, initial\_h, initial\_c\],<br>and if input\_mask is `ihc`, then inputs are \[input, initial\_h, initial\_c\].<br>The output\_mask is `h` means that among \[output, hidden\_out, cell\_out\], only the hidden\_out is being used. |
| StatefulRNNWrapper | num inputs: variable<br>num outputs: 2<br>attributes:<br>\- W: Weight<br>\- R: Weight<br>\- B: Weight (optional)<br>\- input\_mask: str<br>\- output\_mask: str<br>\- direction: str (default="forward")<br>\- batch\_first: bool (default=false)<br>\- hidden\_size: int32<br>\- hidden\_state\_name: str<br>\- subgraph: str<br>\- activation\_alpha: float32\[\] (optional)<br>\- activation\_beta: float32\[\] (optional)<br>\- activations: str\[\] (optional)<br>\- clip: float32 (optional)<br><br>description:<br>When input\_mask is `ish`, then inputs are \[input, sequence\_lens, initial\_h\],<br>and if input\_mask is `ih`, then inputs are \[input, initial\_h\].<br>The output\_mask is `h` means that among \[output, hidden\_out\], only the hidden\_out is being used. |
| StatefulGRUWrapper | num inputs: variable<br>num outputs: 2<br>attributes:<br>\- W: Weight<br>\- R: Weight<br>\- B: Weight (optional)<br>\- input\_mask: str<br>\- output\_mask: str<br>\- direction: str (default="forward")<br>\- batch\_first: bool (default=false)<br>\- hidden\_size: int32<br>\- hidden\_state\_name: str<br>\- subgraph: str<br>\- activation\_alpha: float32\[\] (optional)<br>\- activation\_beta: float32\[\] (optional)<br>\- activations: str\[\] (optional)<br>\- clip: float32 (optional)<br>\- linear\_before\_reset: bool (default=false)<br><br>description:<br>When input\_mask is `ish`, then inputs are \[input, sequence\_lens, initial\_h\],<br>and if input\_mask is `ih`, then inputs are \[input, initial\_h\].<br>The output\_mask is `h` means that among \[output, hidden\_out\], only the hidden\_out is being used. |
| StatefulRoPEWrapper | num inputs: 3<br>num outputs: 1<br>attributes:<br>\- rope\_type: int32 (default=0)<br>\- seqlen: int32 (default=0)<br>\- rotary\_emb\_dim: int32 (default=0)<br><br>description:<br>StatefulRoPEWrapper<br>inputs: (q\_or\_k, con, sin)<br>outputs: (q\_or\_k\_embed)<br>rope\_type=\> 0: llama, 1: cohere2(aya-vision) |
| SubConstant | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- constant: Weight<br>\- is\_const\_first: bool<br>\- shape: int32\[\] (default=\[\]) |
| Sub | num inputs: 2<br>num outputs: 1 |
| Swish | num inputs: 1<br>num outputs: 1 |
| Tanh | num inputs: 1<br>num outputs: 1 |
| Sin | num inputs: 1<br>num outputs: 1 |
| Cos | num inputs: 1<br>num outputs: 1 |
| TransposeConvolution | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- in\_channels: int32<br>\- out\_channels: int32<br>\- h\_kernel: int32<br>\- w\_kernel: int32<br>\- h\_stride: int32<br>\- w\_stride: int32<br>\- h\_dilation: int32<br>\- w\_dilation: int32<br>\- west\_padding: int32<br>\- east\_padding: int32<br>\- north\_padding: int32<br>\- south\_padding: int32<br>\- padding\_list: int32\[\] (default=\[\])<br>\- og\_padding\_list: int32\[\] (default=\[\])<br>\- og\_out\_padding\_list: int32\[\] (default=\[\])<br>\- group\_number: int32<br>\- weight: Weight<br>\- bias: Weight (optional)<br>\- channel\_last: bool (default=true) |
| Transpose | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- transpose\_axes: int64\[\] |
| Unsqueeze | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- axes: int32\[\] |
| OrderedPatchify | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- h\_kernel: int32<br>\- w\_kernel: int32<br>\- order: int32 (default=0)<br><br>description:<br>This operation splits a 4D input tensor into non-overlapping patches and rearranges them into a sequence<br>according to the specified order. Supports row-major (order=0) and column-major (order=1) layouts.<br>If order=0 (row-major), the equivalent is: 256 patches, each 12x12, (h\_kernel=w\_kernel=12)<br>(1,192,192,64)-\[reshape\]-\>(1,16,12,16,12,64)-\[transpose 013245\]-\>(1,16,16,12,12,64)-\[reshape\]-\>(1,256,144,64)<br>Note that order0 is its own inverse.<br>If order=1 (column-major), the equivalent is: 144 patches, each 16x16, (h\_kernel=w\_kernel=12)<br>(1,192,192,64)-\[reshape\]-\>(1,16,12,16,12,64)-\[transpose 024135\]-\>(1,12,12,16,16,64)-\[reshape\]-\>(1,144,256,64)<br>if order=2 (reversing order=1), h\_kernel=12, w\_kernel=16<br>(1,144,256,64)-\[reshape\]-\>(1,12,12,16,16,64)-\[transpose 031425\]-\>(1,16,12,16,12,64)-\[reshape\]-\>(1,192,192,64) |
| DynamicSliceInputConst | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- mod: int32<br>\- start: int64\[\]<br>\- end: int64\[\]<br>\- axis: int64\[\]<br>\- step: int64\[\] (optional) |
| RelShift | num inputs: 1<br>num outputs: 1 |
| MeloTTSPiecewiseRationalQuadraticTransform | num inputs: 2<br>num outputs: 1<br>attributes:<br>\- num\_bins: int32<br>\- filter\_channels: int32<br>\- tail\_bound: float32<br>\- reverse: bool<br><br>description:<br>helper function for parsing melotts model. |
| Neg | num inputs: 1<br>num outputs: 1 |
| Einsum | num inputs: variable<br>num outputs: 1<br>attributes:<br>\- equation: str<br>\- constant: Weight (optional)<br>\- is\_const\_first: bool (default=false) |
| Where | num inputs: 3<br>num outputs: 1<br><br><br>description:<br>condition,X,Y, if cond == True: else y |
| TensorCmp | num inputs: 2<br>num outputs: 1<br>attributes:<br>\- kind: str (default="eq")<br><br>description:<br>kind: eq, neq, lt, le, gt, ge |
| Logical | num inputs: variable<br>num outputs: 1<br>attributes:<br>\- kind: str (default="and")<br><br>description:<br>this function represents the logical operation of two inputs.<br>single input if kind is "not" else two inputs<br>kind: and, or, xor, not |
| MaskedFill | num inputs: 2<br>num outputs: 1<br>attributes:<br>\- value: float32<br><br>description:<br>inputs: (x, mask) |
| SubgraphCall | num inputs: variable<br>num outputs: variable<br>attributes:<br>\- subgraph: str<br>\- scope: str<br>\- kind: str |
| MoeDispatcher | num inputs: 2<br>num outputs: 2<br>attributes:<br>\- top\_k: int32<br>\- norm\_top\_k\_prob: bool (default=true)<br><br>description:<br>inputs: (hidden\_states, router\_logits)<br>outputs: (hidden\_states, top\_k\_logits)<br>if norm\_top\_k\_prob is true<br>topk\_vals, topk\_idx = torch.topk(<br>routing\_weights, self.top\_k, dim=-1<br>) \# (18,8), (18,8)<br>weight\_new = torch.zeros\_like(routing\_weights)<br>weight\_new.scatter\_(1, topk\_idx, topk\_vals)<br>routing\_weights = weight\_new<br>routing\_weights /= routing\_weights.sum(dim=1, keepdim=True) |
| MoeAggregator | num inputs: variable<br>num outputs: 1<br><br><br>description:<br>inputs: (router\_weights, experts\_output0, expert\_output1, ...) |
| TopKValues | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- axis: int32<br>\- k: int32 (default=1)<br>\- largest: bool (default=true)<br>\- sorted: bool (default=true) |
| TopKIndices | num inputs: 1<br>num outputs: 1<br>attributes:<br>\- axis: int32<br>\- k: int32 (default=1)<br>\- largest: bool (default=true)<br>\- sorted: bool (default=true) |
