콘텐츠로 이동

ModelVariantHandle

class ModelVariantHandle

Handle to a specific variant of a loaded model.

This class provides access to variant-specific information such as input/output shapes, buffer information, and quantization scales. It also offers APIs for managing inference buffers, consistent with the interface of the Model class.

Objects of this class are obtained via Model.get_model_variant_handle().

Methods

Name Description
__init__
from_cpp
get_variant_idx Returns the index of this model variant.
get_model_input_shape Returns the input shape for this model variant.
get_model_output_shape Returns the output shape for this model variant.
get_input_buffer_info Returns the input buffer information for this variant.
get_output_buffer_info Returns the output buffer information for this variant.
get_input_scale Returns the input quantization scale(s) for this variant.
get_output_scale Returns the output quantization scale(s) for this variant.
get_model_input_data_type Returns the input buffer information for the model.
get_model_output_data_type Returns the output buffer information of the model.
acquire_input_buffer Buffer Management API
acquire_output_buffer Buffer Management API
acquire_input_npu_data Acquires an NPUData for this variant's input at the given index.
acquire_output_npu_data Acquires an NPUData for this variant's output at the given index.
release_buffer Buffer Management API
reposition_inputs Reposition input
reposition_outputs Reposition output

__init__

def __init__(self, _model_variant_handle: _cQbRuntime.ModelVariantHandle)

from_cpp

def from_cpp(cls, _model_variant_handle: _cQbRuntime.ModelVariantHandle)

get_variant_idx

def get_variant_idx(self) -> int

Returns the index of this model variant.

Returns

Index of the model variant.

get_model_input_shape

def get_model_input_shape(self) -> List[_Shape]

Returns the input shape for this model variant.

Returns

model variant's input shape.

get_model_output_shape

def get_model_output_shape(self) -> List[_Shape]

Returns the output shape for this model variant.

Returns

model variant's output shape.

get_input_buffer_info

def get_input_buffer_info(self) -> List[BufferInfo]

Returns the input buffer information for this variant.

Returns

A list of input buffer information.

get_output_buffer_info

def get_output_buffer_info(self) -> List[BufferInfo]

Returns the output buffer information for this variant.

Returns

A list of output buffer information.

get_input_scale

def get_input_scale(self) -> List[Scale]

Returns the input quantization scale(s) for this variant.

Returns

A list of input scales.

get_output_scale

def get_output_scale(self) -> List[Scale]

Returns the output quantization scale(s) for this variant.

Returns

A list of output scales.

get_model_input_data_type

def get_model_input_data_type(self) -> DataType

Returns the input buffer information for the model.

Returns

A list of input buffer information.

get_model_output_data_type

def get_model_output_data_type(self) -> DataType

Returns the output buffer information of the model.

Returns

A list of output buffer information.

acquire_input_buffer

def acquire_input_buffer(self, seqlens: List[List[int]] = []) -> List[Buffer]

Buffer Management API

Acquires list of Buffer for input. These API is required when calling Model.infer_buffer().

Note

These APIs are intended for advanced use and follow the same buffer management interface as the Model class.

acquire_output_buffer

def acquire_output_buffer(self, seqlens: List[List[int]] = []) -> List[Buffer]

Buffer Management API

Acquires list of Buffer for output. These API is required when calling Model.infer_buffer().

Note

These APIs are intended for advanced use and follow the same buffer management interface as the Model class.

acquire_input_npu_data

def acquire_input_npu_data(
    self, shape: List[int], idx: int, upload: bool, dtype: DataType
) -> NPUData

Acquires an NPUData for this variant's input at the given index.

Equivalent to Model.acquire_input_npu_data, except that the tensor layout comes from this variant instead of variant 0.

Note

This is an advanced API rather than a typical usage.

Warning

This API is in beta: it may still contain bugs, and its behavior may change in a future release.

acquire_output_npu_data

def acquire_output_npu_data(
    self, shape: List[int], idx: int, upload: bool, dtype: DataType
) -> NPUData

Acquires an NPUData for this variant's output at the given index.

Equivalent to Model.acquire_output_npu_data, except that the tensor layout comes from this variant instead of variant 0.

Note

This is an advanced API rather than a typical usage.

Warning

This API is in beta: it may still contain bugs, and its behavior may change in a future release.

release_buffer

def release_buffer(self, buffer: List[Buffer]) -> None

Buffer Management API

Deallocate acquired Input/Output buffer

Note

These APIs are intended for advanced use and follow the same buffer management interface as the Model class.

reposition_inputs

def reposition_inputs(
    self,
    inputs: List[np.ndarray],
    input_bufs: List[Buffer],
    seqlens: List[List[int]] = [],
) -> None

Reposition input

reposition_outputs

def reposition_outputs(
    self,
    output_bufs: List[Buffer],
    outputs: List[np.ndarray],
    seqlens: List[List[int]] = [],
) -> None

Reposition output