콘텐츠로 이동

Python API

Classes

Name Description
Accelerator Represents an accelerator, i.e., an NPU, used for executing models.
Buffer A simple byte-sized buffer.
BufferInfo Struct representing input/output buffer information.
CacheInfo Struct representing KV-cache information.
CacheType CacheType
Cluster Enumerates clusters in the ARIES NPU.
Core Enumerates cores within a cluster in the ARIES NPU.
CoreAllocationPolicy Core allocation policy
CoreId Represents a unique identifier for an NPU core.
CoreMode Defines the core mode for NPU execution.
Future Represents a future for retrieving the result of asynchronous inference.
LogLevel LogLevel
Model Represents an AI model loaded from an MXQ file.
ModelConfig Configures a core mode and core allocation of a model for NPU inference. The ModelConfig class provides methods for setting a core mode and allocating cores for NPU inference. Supported core modes are single-core, multi-core, global4-core, and global8-core. Users can also specify which cores to allocate for the model. Additionally, the configuration offers an option to enforce the use of a specific NPU bundle.
ModelVariantHandle Handle to a specific variant of a loaded model.
Scale Struct for scale values.

Functions

Name Description
load Single-step inference API. Creates model and uploads the model into NPU immediately.
set_log_level
start_tracing_events Starts event tracing and prepares to save the trace log to a specified file.
stop_tracing_events Stops event tracing and writes the recorded trace log.
get_model_summary Generates a structured summary of the specified MXQ model.

load

def load(path: str, model_config: Optional[ModelConfig] = None) -> Model

Single-step inference API. Creates model and uploads the model into NPU immediately.

This operation performs the Accelerator declaration, Model declaration, and launch in a single step.

set_log_level

def set_log_level(level: LogLevel)

start_tracing_events

def start_tracing_events(path: str) -> bool

Starts event tracing and prepares to save the trace log to a specified file.

The trace log is recorded in "Chrome Tracing JSON format," which can be viewed at https://ui.perfetto.dev/.

The trace log is not written immediately; it is saved only when stop_tracing_events() is called.

Parameters

  • path [in] — The file path where the trace log should be stored.

Returns

True if tracing starts successfully, False otherwise.

stop_tracing_events

def stop_tracing_events()

Stops event tracing and writes the recorded trace log.

This function finalizes tracing and saves the collected trace data to the file specified when start_tracing_events() was called.

get_model_summary

def get_model_summary(mxq_path: str) -> str

Generates a structured summary of the specified MXQ model.

Returns an overview of the model contained in the MXQ file, including:

  • Target NPU hardware
  • Supported core modes and their associated cores
  • The total number of model variants
  • For each variant:
    • Input and output tensor shapes
    • A list of layers with their types, output shapes, and input layer indices

The summary is returned as a human-readable string in a table and is useful for inspecting model compatibility, structure, and input/output shapes.

Parameters

  • mxq_path [in] — Path to the MXQ model file.

Returns

A formatted string containing the model summary.