qbruntime.model_variant_handle.ModelVariantHandle Class Reference

qbruntime.model_variant_handle.ModelVariantHandle Class Reference#

SDK qb Runtime Library: qbruntime.model_variant_handle.ModelVariantHandle Class Reference

SDK qb Runtime Library v1.4

MCS001-KR

qbruntime
model_variant_handle
ModelVariantHandle

Handle to a specific variant of a loaded model. More...

Public Member Functions
	__init__ (self, _cQbRuntime.ModelVariantHandle _model_variant_handle)
	from_cpp (cls, _cQbRuntime.ModelVariantHandle _model_variant_handle)
int	get_variant_idx (self)
	Returns the index of this model variant.
List[_Shape]	get_model_input_shape (self)
	Returns the input shape for this model variant.
List[_Shape]	get_model_output_shape (self)
	Returns the output shape for this model variant.
List[BufferInfo]	get_input_buffer_info (self)
	Returns the input buffer information for this variant.
List[BufferInfo]	get_output_buffer_info (self)
	Returns the output buffer information for this variant.
List[Scale]	get_input_scale (self)
	Returns the input quantization scale(s) for this variant.
List[Scale]	get_output_scale (self)
	Returns the output quantization scale(s) for this variant.
DataType	get_model_input_data_type (self)
	Returns the input buffer information for the model.
DataType	get_model_output_data_type (self)
	Returns the output buffer information of the model.
List[Buffer]	acquire_input_buffer (self, List[List[int]] seqlens=[])
	Buffer Management API.
List[Buffer]	acquire_output_buffer (self, List[List[int]] seqlens=[])
	Buffer Management API.
NPUData	acquire_input_npu_data (self, List[int] shape, int idx, bool upload, DataType dtype)
	Acquires an NPUData for this variant's input at the given index.
NPUData	acquire_output_npu_data (self, List[int] shape, int idx, bool upload, DataType dtype)
	Acquires an NPUData for this variant's output at the given index.
None	release_buffer (self, List[Buffer] buffer)
	Buffer Management API.
None	reposition_inputs (self, List[np.ndarray] inputs, List[Buffer] input_bufs, List[List[int]] seqlens=[])
	Reposition input.
None	reposition_outputs (self, List[Buffer] output_bufs, List[np.ndarray] outputs, List[List[int]] seqlens=[])
	Reposition output.

Protected Attributes
	_model_variant_handle = _model_variant_handle
List[_Shape]	_output_shape = self.get_model_output_shape()

Detailed Description

Handle to a specific variant of a loaded model.

This class provides access to variant-specific information such as input/output shapes, buffer information, and quantization scales. It also offers APIs for managing inference buffers, consistent with the interface of the Model class.

Objects of this class are obtained via Model.get_model_variant_handle().

Definition at line 18 of file model_variant_handle.py.

Constructor & Destructor Documentation

◆ init()

qbruntime.model_variant_handle.ModelVariantHandle.__init__	(		self,
		_cQbRuntime.ModelVariantHandle	_model_variant_handle )

Definition at line 29 of file model_variant_handle.py.

Member Function Documentation

◆ from_cpp()

qbruntime.model_variant_handle.ModelVariantHandle.from_cpp	(		cls,
		_cQbRuntime.ModelVariantHandle	_model_variant_handle )

Definition at line 34 of file model_variant_handle.py.

◆ get_variant_idx()

int qbruntime.model_variant_handle.ModelVariantHandle.get_variant_idx ( self )

Returns the index of this model variant.

Returns: Index of the model variant.

Definition at line 37 of file model_variant_handle.py.

◆ get_model_input_shape()

List[_Shape] qbruntime.model_variant_handle.ModelVariantHandle.get_model_input_shape ( self )

Returns the input shape for this model variant.

Returns: model variant's input shape.

Definition at line 45 of file model_variant_handle.py.

◆ get_model_output_shape()

List[_Shape] qbruntime.model_variant_handle.ModelVariantHandle.get_model_output_shape ( self )

Returns the output shape for this model variant.

Returns: model variant's output shape.

Definition at line 53 of file model_variant_handle.py.

◆ get_input_buffer_info()

List[BufferInfo] qbruntime.model_variant_handle.ModelVariantHandle.get_input_buffer_info ( self )

Returns the input buffer information for this variant.

Returns: A list of input buffer information.

Definition at line 61 of file model_variant_handle.py.

◆ get_output_buffer_info()

List[BufferInfo] qbruntime.model_variant_handle.ModelVariantHandle.get_output_buffer_info ( self )

Returns the output buffer information for this variant.

Returns: A list of output buffer information.

Definition at line 72 of file model_variant_handle.py.

◆ get_input_scale()

List[Scale] qbruntime.model_variant_handle.ModelVariantHandle.get_input_scale ( self )

Returns the input quantization scale(s) for this variant.

Returns: A list of input scales.

Definition at line 83 of file model_variant_handle.py.

◆ get_output_scale()

List[Scale] qbruntime.model_variant_handle.ModelVariantHandle.get_output_scale ( self )

Returns the output quantization scale(s) for this variant.

Returns: A list of output scales.

Definition at line 91 of file model_variant_handle.py.

◆ get_model_input_data_type()

DataType qbruntime.model_variant_handle.ModelVariantHandle.get_model_input_data_type ( self )

Returns the input buffer information for the model.

Returns: A list of input buffer information.

Definition at line 101 of file model_variant_handle.py.

◆ get_model_output_data_type()

DataType qbruntime.model_variant_handle.ModelVariantHandle.get_model_output_data_type ( self )

Returns the output buffer information of the model.

Returns: A list of output buffer information.

Definition at line 109 of file model_variant_handle.py.

◆ acquire_input_buffer()

List[Buffer] qbruntime.model_variant_handle.ModelVariantHandle.acquire_input_buffer	(		self,
		List[List[int]]	seqlens = [] )

Buffer Management API.

Acquires list of Buffer for input. These API is required when calling Model.infer_buffer().

Note: These APIs are intended for advanced use and follow the same buffer management interface as the Model class.

Definition at line 117 of file model_variant_handle.py.

◆ acquire_output_buffer()

List[Buffer] qbruntime.model_variant_handle.ModelVariantHandle.acquire_output_buffer	(		self,
		List[List[int]]	seqlens = [] )

Buffer Management API.

Acquires list of Buffer for output. These API is required when calling Model.infer_buffer().

Note: These APIs are intended for advanced use and follow the same buffer management interface as the Model class.

Definition at line 131 of file model_variant_handle.py.

◆ acquire_input_npu_data()

NPUData qbruntime.model_variant_handle.ModelVariantHandle.acquire_input_npu_data	(		self,
		List[int]	shape,
		int	idx,
		bool	upload,
		DataType	dtype )

Acquires an NPUData for this variant's input at the given index.

Equivalent to Model.acquire_input_npu_data, except that the tensor layout comes from this variant instead of variant 0.

Note: This is an advanced API rather than a typical usage.

Warning: This API is in beta: it may still contain bugs, and its behavior may change in a future release.

Definition at line 145 of file model_variant_handle.py.

◆ acquire_output_npu_data()

NPUData qbruntime.model_variant_handle.ModelVariantHandle.acquire_output_npu_data	(		self,
		List[int]	shape,
		int	idx,
		bool	upload,
		DataType	dtype )

Acquires an NPUData for this variant's output at the given index.

Equivalent to Model.acquire_output_npu_data, except that the tensor layout comes from this variant instead of variant 0.

Note: This is an advanced API rather than a typical usage.

Warning: This API is in beta: it may still contain bugs, and its behavior may change in a future release.

Definition at line 164 of file model_variant_handle.py.

◆ release_buffer()

None qbruntime.model_variant_handle.ModelVariantHandle.release_buffer	(		self,
		List[Buffer]	buffer )

Buffer Management API.

Deallocate acquired Input/Output buffer

Note: These APIs are intended for advanced use and follow the same buffer management interface as the Model class.

Definition at line 183 of file model_variant_handle.py.

◆ reposition_inputs()

None qbruntime.model_variant_handle.ModelVariantHandle.reposition_inputs	(		self,
		List[np.ndarray]	inputs,
		List[Buffer]	input_bufs,
		List[List[int]]	seqlens = [] )

Reposition input.

Definition at line 194 of file model_variant_handle.py.

◆ reposition_outputs()

None qbruntime.model_variant_handle.ModelVariantHandle.reposition_outputs	(		self,
		List[Buffer]	output_bufs,
		List[np.ndarray]	outputs,
		List[List[int]]	seqlens = [] )

Reposition output.

Definition at line 206 of file model_variant_handle.py.

Member Data Documentation

◆ _model_variant_handle

qbruntime.model_variant_handle.ModelVariantHandle._model_variant_handle = _model_variant_handle

protected

Definition at line 30 of file model_variant_handle.py.

◆ _output_shape

List[_Shape] qbruntime.model_variant_handle.ModelVariantHandle._output_shape = self.get_model_output_shape()

protected

Definition at line 31 of file model_variant_handle.py.

The documentation for this class was generated from the following file:

model_variant_handle.py

qbruntime.model_variant_handle.ModelVariantHandle Class Reference

qbruntime.model_variant_handle.ModelVariantHandle Class Reference#

Public Member Functions

Protected Attributes

Detailed Description

Constructor & Destructor Documentation

◆ __init__()

Member Function Documentation

◆ from_cpp()

◆ get_variant_idx()

◆ get_model_input_shape()

◆ get_model_output_shape()

◆ get_input_buffer_info()

◆ get_output_buffer_info()

◆ get_input_scale()

◆ get_output_scale()

◆ get_model_input_data_type()

◆ get_model_output_data_type()

◆ acquire_input_buffer()

◆ acquire_output_buffer()

◆ acquire_input_npu_data()

◆ acquire_output_npu_data()

◆ release_buffer()

◆ reposition_inputs()

◆ reposition_outputs()

Member Data Documentation

◆ _model_variant_handle

◆ _output_shape

◆ init()