Skip to content

ModelVariantHandle

#include <qbruntime/model_variant_handle.h>

class mobilint::ModelVariantHandle

Handle to a specific variant of a loaded model.

This class provides access to variant-specific information such as input/output shapes, buffer information, and quantization scales. It also offers APIs for managing inference buffers, consistent with the interface of the Model class.

Objects of this class are obtained via Model::getModelVariantHandle.

Buffer Management APIs

These APIs are used when performing inference with Model::inferBuffer or Model::inferBufferToFloat, using this variant’s input and output shapes.

Buffers are acquired using:

  • acquireInputBuffer
  • acquireOutputBuffer

Any acquired buffer must be released using:

  • releaseBuffer
  • releaseBuffers

Repositioning is handled by:

  • repositionInputs
  • repositionOutputs

Note

These APIs are intended for advanced use and follow the same buffer management interface as the Model class.

Member Description
acquireInputBuffer
acquireOutputBuffer
acquireInputBuffers
acquireOutputBuffers
releaseBuffer
releaseBuffers
repositionInputs ×4
repositionOutputs ×4

acquireInputBuffer

std::vector<Buffer> acquireInputBuffer(const std::vector<std::vector<int>> &seqlens={}) const

acquireOutputBuffer

std::vector<Buffer> acquireOutputBuffer(const std::vector<std::vector<int>> &seqlens={}) const

acquireInputBuffers

std::vector<std::vector<Buffer>> acquireInputBuffers(int batch_size, const std::vector<std::vector<int>> &seqlens={}) const

acquireOutputBuffers

std::vector<std::vector<Buffer>> acquireOutputBuffers(int batch_size, const std::vector<std::vector<int>> &seqlens={}) const

releaseBuffer

StatusCode releaseBuffer(std::vector<Buffer> &buffer) const

releaseBuffers

StatusCode releaseBuffers(std::vector<std::vector<Buffer>> &buffers) const

repositionInputs

StatusCode repositionInputs(const std::vector<float *> &input, std::vector<Buffer> &input_buf, const std::vector<std::vector<int>> &seqlens={}) const
StatusCode repositionInputs(const std::vector<uint8_t *> &input, std::vector<Buffer> &input_buf, const std::vector<std::vector<int>> &seqlens={}) const
StatusCode repositionInputs(const std::vector<float *> &input, std::vector<std::vector<Buffer>> &input_buf, const std::vector<std::vector<int>> &seqlens={}) const
StatusCode repositionInputs(const std::vector<uint8_t *> &input, std::vector<std::vector<Buffer>> &input_buf, const std::vector<std::vector<int>> &seqlens={}) const

repositionOutputs

StatusCode repositionOutputs(const std::vector<Buffer> &output_buf, std::vector<float *> &output, const std::vector<std::vector<int>> &seqlens={}) const
StatusCode repositionOutputs(const std::vector<Buffer> &output_buf, std::vector<std::vector<float>> &output, const std::vector<std::vector<int>> &seqlens={}) const
StatusCode repositionOutputs(const std::vector<std::vector<Buffer>> &output_buf, std::vector<float *> &output, const std::vector<std::vector<int>> &seqlens={}) const
StatusCode repositionOutputs(const std::vector<std::vector<Buffer>> &output_buf, std::vector<std::vector<float>> &output, const std::vector<std::vector<int>> &seqlens={}) const

Member functions

Member Description
ModelVariantHandle ×2
operator= ×2
getVariantIdx Returns the index of this model variant.
getModelInputShape Returns the input shape for this model variant.
getModelOutputShape Returns the output shape for this model variant.
getInputBufferInfo Returns the input buffer information for this variant.
getOutputBufferInfo Returns the output buffer information for this variant.
getInputScale Returns the input quantization scale(s) for this variant.
getOutputScale Returns the output quantization scale(s) for this variant.
getModelInputDataType Returns a data type for model inputs.
getModelOutputDataType Returns a data type for model outputs.

ModelVariantHandle

ModelVariantHandle(const ModelVariantHandle &other)=delete
ModelVariantHandle(ModelVariantHandle &&other)=delete

operator=

ModelVariantHandle & operator=(const ModelVariantHandle &rhs)=delete
ModelVariantHandle & operator=(ModelVariantHandle &&rhs) noexcept=delete

getVariantIdx

int getVariantIdx() const

Returns the index of this model variant.

Returns

Index of the model variant.

getModelInputShape

const std::vector<std::vector<int64_t>> & getModelInputShape() const

Returns the input shape for this model variant.

Returns

Reference to the input shape.

getModelOutputShape

const std::vector<std::vector<int64_t>> & getModelOutputShape() const

Returns the output shape for this model variant.

Returns

Reference to the output shape.

getInputBufferInfo

const std::vector<BufferInfo> & getInputBufferInfo() const

Returns the input buffer information for this variant.

Returns

Reference to a vector of input buffer information.

getOutputBufferInfo

const std::vector<BufferInfo> & getOutputBufferInfo() const

Returns the output buffer information for this variant.

Returns

Reference to a vector of output buffer information.

getInputScale

std::vector<Scale> getInputScale() const

Returns the input quantization scale(s) for this variant.

Returns

Vector of input scales.

getOutputScale

std::vector<Scale> getOutputScale() const

Returns the output quantization scale(s) for this variant.

Returns

Vector of output scales.

getModelInputDataType

DataType getModelInputDataType() const

Returns a data type for model inputs.

Returns

An input data type.

getModelOutputDataType

DataType getModelOutputDataType() const

Returns a data type for model outputs.

Returns

An output data type.