ModelVariantHandle Class Reference

SDK qb Runtime Library v1.4 MCS001-EN

• mobilint
• ModelVariantHandle

Public Member Functions | Friends | List of all members

mobilint::ModelVariantHandle Class Reference

C++ API Reference

Handle to a specific variant of a loaded model. More...

#include <model_variant_handle.h>

Public Member Functions
	ModelVariantHandle (const ModelVariantHandle &other)=delete
	ModelVariantHandle (ModelVariantHandle &&other)=delete
ModelVariantHandle &	operator= (const ModelVariantHandle &rhs)=delete
ModelVariantHandle &	operator= (ModelVariantHandle &&rhs) noexcept=delete
int	getVariantIdx () const
	Returns the index of this model variant.
const std::vector< std::vector< int64_t > > &	getModelInputShape () const
	Returns the input shape for this model variant.
const std::vector< std::vector< int64_t > > &	getModelOutputShape () const
	Returns the output shape for this model variant.
const std::vector< BufferInfo > &	getInputBufferInfo () const
	Returns the input buffer information for this variant.
const std::vector< BufferInfo > &	getOutputBufferInfo () const
	Returns the output buffer information for this variant.
std::vector< Scale >	getInputScale () const
	Returns the input quantization scale(s) for this variant.
std::vector< Scale >	getOutputScale () const
	Returns the output quantization scale(s) for this variant.
DataType	getModelInputDataType () const
	Returns a data type for model inputs.
DataType	getModelOutputDataType () const
	Returns a data type for model outputs.
Buffer Management APIs
These APIs are used when performing inference with Model::inferBuffer or Model::inferBufferToFloat, using this variant’s input and output shapes. Buffers are acquired using: acquireInputBuffer acquireOutputBuffer Any acquired buffer must be released using: releaseBuffer releaseBuffers Repositioning is handled by: repositionInputs repositionOutputs Note These APIs are intended for advanced use and follow the same buffer management interface as the Model class.
std::vector< Buffer >	acquireInputBuffer (const std::vector< std::vector< int > > &seqlens={}) const
	Acquires an NPUData for this variant's input at the given index.
std::vector< Buffer >	acquireOutputBuffer (const std::vector< std::vector< int > > &seqlens={}) const
	Acquires an NPUData for this variant's input at the given index.
std::vector< std::vector< Buffer > >	acquireInputBuffers (int batch_size, const std::vector< std::vector< int > > &seqlens={}) const
	Acquires an NPUData for this variant's input at the given index.
std::vector< std::vector< Buffer > >	acquireOutputBuffers (int batch_size, const std::vector< std::vector< int > > &seqlens={}) const
	Acquires an NPUData for this variant's input at the given index.
NPUData	acquireInputNPUData (const std::vector< int64_t > &shape, int idx, bool upload, DataType dtype, StatusCode &sc) const
	Acquires an NPUData for this variant's input at the given index.
NPUData	acquireOutputNPUData (const std::vector< int64_t > &shape, int idx, bool upload, DataType dtype, StatusCode &sc) const
	Acquires an NPUData for this variant's output at the given index.
StatusCode	releaseBuffer (std::vector< Buffer > &buffer) const
	Acquires an NPUData for this variant's input at the given index.
StatusCode	releaseBuffers (std::vector< std::vector< Buffer > > &buffers) const
	Acquires an NPUData for this variant's input at the given index.
StatusCode	repositionInputs (const std::vector< float * > &input, std::vector< Buffer > &input_buf, const std::vector< std::vector< int > > &seqlens={}) const
	Acquires an NPUData for this variant's input at the given index.
StatusCode	repositionOutputs (const std::vector< Buffer > &output_buf, std::vector< float * > &output, const std::vector< std::vector< int > > &seqlens={}) const
	Acquires an NPUData for this variant's input at the given index.
StatusCode	repositionOutputs (const std::vector< Buffer > &output_buf, std::vector< std::vector< float > > &output, const std::vector< std::vector< int > > &seqlens={}) const
	Acquires an NPUData for this variant's input at the given index.
StatusCode	repositionInputs (const std::vector< uint8_t * > &input, std::vector< Buffer > &input_buf, const std::vector< std::vector< int > > &seqlens={}) const
	Acquires an NPUData for this variant's input at the given index.
StatusCode	repositionInputs (const std::vector< float * > &input, std::vector< std::vector< Buffer > > &input_buf, const std::vector< std::vector< int > > &seqlens={}) const
	Acquires an NPUData for this variant's input at the given index.
StatusCode	repositionOutputs (const std::vector< std::vector< Buffer > > &output_buf, std::vector< float * > &output, const std::vector< std::vector< int > > &seqlens={}) const
	Acquires an NPUData for this variant's input at the given index.
StatusCode	repositionOutputs (const std::vector< std::vector< Buffer > > &output_buf, std::vector< std::vector< float > > &output, const std::vector< std::vector< int > > &seqlens={}) const
	Acquires an NPUData for this variant's input at the given index.
StatusCode	repositionInputs (const std::vector< uint8_t * > &input, std::vector< std::vector< Buffer > > &input_buf, const std::vector< std::vector< int > > &seqlens={}) const
	Acquires an NPUData for this variant's input at the given index.

Friends
class	ModelImpl

Detailed Description

Handle to a specific variant of a loaded model.

This class provides access to variant-specific information such as input/output shapes, buffer information, and quantization scales. It also offers APIs for managing inference buffers, consistent with the interface of the Model class.

Objects of this class are obtained via Model::getModelVariantHandle.

Definition at line 35 of file model_variant_handle.h.

---

The documentation for this class was generated from the following file:

• model_variant_handle.h

---

Generated by  1.14.0