# ModelVariantHandle

```cpp
#include <qbruntime/model_variant_handle.h>

class mobilint::ModelVariantHandle
```

Handle to a specific variant of a loaded model.

This class provides access to variant-specific information such as input/output shapes, buffer information, and quantization scales. It also offers APIs for managing inference buffers, consistent with the interface of the [`Model`](<https://docs.mobilint.com/runtime/kr/v1.5/api/cpp/model/index.md>) class.

Objects of this class are obtained via [`Model::getModelVariantHandle`](<https://docs.mobilint.com/runtime/kr/v1.5/api/cpp/model/#getmodelvarianthandle>).

## Buffer Management APIs

These APIs are used when performing inference with `Model::inferBuffer` or `Model::inferBufferToFloat`, using this variant’s input and output shapes.

Buffers are acquired using:

- `acquireInputBuffer`
- `acquireOutputBuffer`

Any acquired buffer must be released using:

- `releaseBuffer`
- `releaseBuffers`

Repositioning is handled by:

- `repositionInputs`
- `repositionOutputs`

> [!NOTE]
>
> These APIs are intended for advanced use and follow the same buffer management interface as the [`Model`](<https://docs.mobilint.com/runtime/kr/v1.5/api/cpp/model/index.md>) class.

| Member | Description |
| --- | --- |
| [`acquireInputBuffer`](<#acquireinputbuffer>) |  |
| [`acquireOutputBuffer`](<#acquireoutputbuffer>) |  |
| [`acquireInputBuffers`](<#acquireinputbuffers>) |  |
| [`acquireOutputBuffers`](<#acquireoutputbuffers>) |  |
| [`acquireInputNPUData`](<#acquireinputnpudata>) | Acquires an [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.5/api/cpp/npudata/index.md>) for this variant's input at the given index. |
| [`acquireOutputNPUData`](<#acquireoutputnpudata>) | Acquires an [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.5/api/cpp/npudata/index.md>) for this variant's output at the given index. |
| [`releaseBuffer`](<#releasebuffer>) |  |
| [`releaseBuffers`](<#releasebuffers>) |  |
| [`repositionInputs`](<#repositioninputs>) ×4 |  |
| [`repositionOutputs`](<#repositionoutputs>) ×4 |  |

### acquireInputBuffer

```cpp
std::vector<Buffer> acquireInputBuffer(const std::vector<std::vector<int>> &seqlens={}) const
```

### acquireOutputBuffer

```cpp
std::vector<Buffer> acquireOutputBuffer(const std::vector<std::vector<int>> &seqlens={}) const
```

### acquireInputBuffers

```cpp
std::vector<std::vector<Buffer>> acquireInputBuffers(int batch_size, const std::vector<std::vector<int>> &seqlens={}) const
```

### acquireOutputBuffers

```cpp
std::vector<std::vector<Buffer>> acquireOutputBuffers(int batch_size, const std::vector<std::vector<int>> &seqlens={}) const
```

### acquireInputNPUData

```cpp
NPUData acquireInputNPUData(const std::vector<int64_t> &shape, int idx, bool upload, DataType dtype, StatusCode &sc) const
```

Acquires an [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.5/api/cpp/npudata/index.md>) for this variant's input at the given index.

Equivalent to [`Model::acquireInputNPUData`](<https://docs.mobilint.com/runtime/kr/v1.5/api/cpp/model/#acquireinputnpudata>), except that the tensor layout comes from this variant instead of variant 0. Use this when running inference on a variant other than the first one: [`Model::infer`](<https://docs.mobilint.com/runtime/kr/v1.5/api/cpp/model/#infer>) selects the variant from the input shapes, so the [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.5/api/cpp/npudata/index.md>) must be bound to that same variant.

> [!WARNING]
>
> This API is in beta: it may still contain bugs, and its behavior may change in a future release.

**Parameters**

- `shape` *\[in\]* — The shape of the input tensor, in NHWC/HWC or NCHW/CHW layout.
- `idx` *\[in\]* — The index of the model input.
- `upload` *\[in\]* — If true, NPU memory is allocated and the data is placed on the NPU; otherwise the data is kept on the host (CPU).
- `dtype` *\[in\]* — The element type of the data. Must be one of the types the model accepts for this input; otherwise `sc` is set to `Model_DtypeMismatched`.
- `sc` *\[out\]* — A reference to a status code that will be updated to indicate whether the operation was successful or encountered an error.

**Returns**

The acquired [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.5/api/cpp/npudata/index.md>). Its contents are valid only when `sc` is OK.

### acquireOutputNPUData

```cpp
NPUData acquireOutputNPUData(const std::vector<int64_t> &shape, int idx, bool upload, DataType dtype, StatusCode &sc) const
```

Acquires an [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.5/api/cpp/npudata/index.md>) for this variant's output at the given index.

Equivalent to [`Model::acquireOutputNPUData`](<https://docs.mobilint.com/runtime/kr/v1.5/api/cpp/model/#acquireoutputnpudata>), except that the tensor layout comes from this variant instead of variant 0. See `acquireInputNPUData`.

> [!WARNING]
>
> This API is in beta: it may still contain bugs, and its behavior may change in a future release.

**Parameters**

- `shape` *\[in\]* — The shape of the output tensor, in NHWC/HWC or NCHW/CHW layout.
- `idx` *\[in\]* — The index of the model output.
- `upload` *\[in\]* — If true, NPU memory is allocated and the data is placed on the NPU; otherwise the data is kept on the host (CPU).
- `dtype` *\[in\]* — The element type of the data. Must be one of the types the model produces for this output; otherwise `sc` is set to `Model_DtypeMismatched`.
- `sc` *\[out\]* — A reference to a status code that will be updated to indicate whether the operation was successful or encountered an error.

**Returns**

The acquired [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.5/api/cpp/npudata/index.md>). Its contents are valid only when `sc` is OK.

### releaseBuffer

```cpp
StatusCode releaseBuffer(std::vector<Buffer> &buffer) const
```

### releaseBuffers

```cpp
StatusCode releaseBuffers(std::vector<std::vector<Buffer>> &buffers) const
```

### repositionInputs

```cpp
StatusCode repositionInputs(const std::vector<float *> &input, std::vector<Buffer> &input_buf, const std::vector<std::vector<int>> &seqlens={}) const
```

```cpp
StatusCode repositionInputs(const std::vector<uint8_t *> &input, std::vector<Buffer> &input_buf, const std::vector<std::vector<int>> &seqlens={}) const
```

```cpp
StatusCode repositionInputs(const std::vector<float *> &input, std::vector<std::vector<Buffer>> &input_buf, const std::vector<std::vector<int>> &seqlens={}) const
```

```cpp
StatusCode repositionInputs(const std::vector<uint8_t *> &input, std::vector<std::vector<Buffer>> &input_buf, const std::vector<std::vector<int>> &seqlens={}) const
```

### repositionOutputs

```cpp
StatusCode repositionOutputs(const std::vector<Buffer> &output_buf, std::vector<float *> &output, const std::vector<std::vector<int>> &seqlens={}) const
```

```cpp
StatusCode repositionOutputs(const std::vector<Buffer> &output_buf, std::vector<std::vector<float>> &output, const std::vector<std::vector<int>> &seqlens={}) const
```

```cpp
StatusCode repositionOutputs(const std::vector<std::vector<Buffer>> &output_buf, std::vector<float *> &output, const std::vector<std::vector<int>> &seqlens={}) const
```

```cpp
StatusCode repositionOutputs(const std::vector<std::vector<Buffer>> &output_buf, std::vector<std::vector<float>> &output, const std::vector<std::vector<int>> &seqlens={}) const
```

## Member functions

| Member | Description |
| --- | --- |
| [`ModelVariantHandle`](<#modelvarianthandle>) ×2 |  |
| [`operator=`](<#operator-assign>) ×2 |  |
| [`getVariantIdx`](<#getvariantidx>) | Returns the index of this model variant. |
| [`getModelInputShape`](<#getmodelinputshape>) | Returns the input shape for this model variant. |
| [`getModelOutputShape`](<#getmodeloutputshape>) | Returns the output shape for this model variant. |
| [`getInputBufferInfo`](<#getinputbufferinfo>) | Returns the input buffer information for this variant. |
| [`getOutputBufferInfo`](<#getoutputbufferinfo>) | Returns the output buffer information for this variant. |
| [`getInputScale`](<#getinputscale>) | Returns the input quantization scale(s) for this variant. |
| [`getOutputScale`](<#getoutputscale>) | Returns the output quantization scale(s) for this variant. |
| [`getModelInputDataType`](<#getmodelinputdatatype>) | Returns a data type for model inputs. |
| [`getModelOutputDataType`](<#getmodeloutputdatatype>) | Returns a data type for model outputs. |

### ModelVariantHandle

```cpp
ModelVariantHandle(const ModelVariantHandle &other)=delete
```

```cpp
ModelVariantHandle(ModelVariantHandle &&other)=delete
```

### operator=

```cpp
ModelVariantHandle & operator=(const ModelVariantHandle &rhs)=delete
```

```cpp
ModelVariantHandle & operator=(ModelVariantHandle &&rhs) noexcept=delete
```

### getVariantIdx

```cpp
int getVariantIdx() const
```

Returns the index of this model variant.

**Returns**

Index of the model variant.

### getModelInputShape

```cpp
const std::vector<std::vector<int64_t>> & getModelInputShape() const
```

Returns the input shape for this model variant.

**Returns**

Reference to the input shape.

### getModelOutputShape

```cpp
const std::vector<std::vector<int64_t>> & getModelOutputShape() const
```

Returns the output shape for this model variant.

**Returns**

Reference to the output shape.

### getInputBufferInfo

```cpp
const std::vector<BufferInfo> & getInputBufferInfo() const
```

Returns the input buffer information for this variant.

**Returns**

Reference to a vector of input buffer information.

### getOutputBufferInfo

```cpp
const std::vector<BufferInfo> & getOutputBufferInfo() const
```

Returns the output buffer information for this variant.

**Returns**

Reference to a vector of output buffer information.

### getInputScale

```cpp
std::vector<Scale> getInputScale() const
```

Returns the input quantization scale(s) for this variant.

**Returns**

Vector of input scales.

### getOutputScale

```cpp
std::vector<Scale> getOutputScale() const
```

Returns the output quantization scale(s) for this variant.

**Returns**

Vector of output scales.

### getModelInputDataType

```cpp
DataType getModelInputDataType() const
```

Returns a data type for model inputs.

**Returns**

An input data type.

### getModelOutputDataType

```cpp
DataType getModelOutputDataType() const
```

Returns a data type for model outputs.

**Returns**

An output data type.
