ModelVariantHandle¶
Handle to a specific variant of a loaded model.
This class provides access to variant-specific information such as input/output shapes, buffer information, and quantization scales. It also offers APIs for managing inference buffers, consistent with the interface of the Model class.
Objects of this class are obtained via Model.get_model_variant_handle().
Methods¶
| Name | Description |
|---|---|
__init__ |
|
from_cpp |
|
get_variant_idx |
Returns the index of this model variant. |
get_model_input_shape |
Returns the input shape for this model variant. |
get_model_output_shape |
Returns the output shape for this model variant. |
get_input_buffer_info |
Returns the input buffer information for this variant. |
get_output_buffer_info |
Returns the output buffer information for this variant. |
get_input_scale |
Returns the input quantization scale(s) for this variant. |
get_output_scale |
Returns the output quantization scale(s) for this variant. |
acquire_input_buffer |
Buffer Management API |
acquire_output_buffer |
Buffer Management API |
release_buffer |
Buffer Management API |
reposition_inputs |
Reposition input |
reposition_outputs |
Reposition output |
__init__¶
from_cpp¶
get_variant_idx¶
Returns the index of this model variant.
Returns
Index of the model variant.
get_model_input_shape¶
Returns the input shape for this model variant.
Returns
model variant's input shape.
get_model_output_shape¶
Returns the output shape for this model variant.
Returns
model variant's output shape.
get_input_buffer_info¶
Returns the input buffer information for this variant.
Returns
A list of input buffer information.
get_output_buffer_info¶
Returns the output buffer information for this variant.
Returns
A list of output buffer information.
get_input_scale¶
Returns the input quantization scale(s) for this variant.
Returns
A list of input scales.
get_output_scale¶
Returns the output quantization scale(s) for this variant.
Returns
A list of output scales.
acquire_input_buffer¶
Buffer Management API
Acquires list of Buffer for input. These API is required when calling Model.infer_buffer().
Note
These APIs are intended for advanced use and follow the same buffer management interface as the Model class.
acquire_output_buffer¶
Buffer Management API
Acquires list of Buffer for output. These API is required when calling Model.infer_buffer().
Note
These APIs are intended for advanced use and follow the same buffer management interface as the Model class.
release_buffer¶
Buffer Management API
Deallocate acquired Input/Output buffer
Note
These APIs are intended for advanced use and follow the same buffer management interface as the Model class.
reposition_inputs¶
def reposition_inputs(
self,
inputs: List[np.ndarray],
input_bufs: List[Buffer],
seqlens: List[List[int]] = [],
) -> None
Reposition input
reposition_outputs¶
def reposition_outputs(
self,
output_bufs: List[Buffer],
outputs: List[np.ndarray],
seqlens: List[List[int]] = [],
) -> None
Reposition output