Python API¶
Classes¶
| Name | Description |
|---|---|
Accelerator |
Represents an accelerator, i.e., an NPU, used for executing models. |
BatchParam |
Struct containing BatchLLM parameters. |
Buffer |
A simple byte-sized buffer. |
BufferInfo |
Struct representing input/output buffer information. |
CacheInfo |
Struct representing KV-cache information. |
CacheType |
CacheType |
Cluster |
Enumerates clusters in the ARIES NPU. |
Core |
Enumerates cores within a cluster in the ARIES NPU. |
CoreAllocationPolicy |
Core allocation policy |
CoreId |
Represents a unique identifier for an NPU core. |
CoreMode |
Defines the core mode for NPU execution. |
DataType |
DataType |
Future |
Represents a future for retrieving the result of asynchronous inference. |
LogLevel |
LogLevel |
Model |
Represents an AI model loaded from an MXQ file. |
ModelConfig |
Configures a core mode and core allocation of a model for NPU inference. The ModelConfig class provides methods for setting a core mode and allocating cores for NPU inference. Supported core modes are single-core, multi-core, global4-core, and global8-core. Users can also specify which cores to allocate for the model. Additionally, the configuration offers an option to enforce the use of a specific NPU bundle. |
ModelVariantHandle |
Handle to a specific variant of a loaded model. |
PinnedMemory |
An NPU-accessible pinned (physically contiguous) memory buffer. |
Scale |
Struct for scale values. |
Functions¶
| Name | Description |
|---|---|
load |
Single-step inference API. Creates model and uploads the model into NPU immediately. |
set_log_level |
|
start_tracing_events |
Starts event tracing and prepares to save the trace log to a specified file. |
stop_tracing_events |
Stops event tracing and writes the recorded trace log. |
get_model_summary |
Generates a structured summary of the specified MXQ model. |
get_available_device_numbers |
Get the number of available NPU devices. |
load¶
Single-step inference API. Creates model and uploads the model into NPU immediately.
This operation performs the Accelerator declaration, Model declaration, and launch in a single step.
set_log_level¶
start_tracing_events¶
Starts event tracing and prepares to save the trace log to a specified file.
The trace log is recorded in "Chrome Tracing JSON format," which can be viewed at https://ui.perfetto.dev/.
The trace log is not written immediately; it is saved only when stop_tracing_events() is called.
Parameters
path[in] — The file path where the trace log should be stored.
Returns
True if tracing starts successfully, False otherwise.
stop_tracing_events¶
Stops event tracing and writes the recorded trace log.
This function finalizes tracing and saves the collected trace data to the file specified when start_tracing_events() was called.
get_model_summary¶
Generates a structured summary of the specified MXQ model.
Returns an overview of the model contained in the MXQ file, including:
- Target NPU hardware
- Supported core modes and their associated cores
- The total number of model variants
- For each variant:
- Input and output tensor shapes
- A list of layers with their types, output shapes, and input layer indices
The summary is returned as a human-readable string in a table and is useful for inspecting model compatibility, structure, and input/output shapes.
Parameters
mxq_path[in] — Path to the MXQ model file.
Returns
A formatted string containing the model summary.
get_available_device_numbers¶
Get the number of available NPU devices.
Returns
The number of available NPU devices.