콘텐츠로 이동

Python API

Classes

Name Description
Accelerator Represents an accelerator, i.e., an NPU, used for executing models.
BatchParam Struct containing BatchLLM parameters.
Buffer A simple byte-sized buffer.
BufferInfo Struct representing input/output buffer information.
CacheInfo Struct representing KV-cache information.
CacheType CacheType
Cluster Enumerates clusters in the ARIES NPU.
Core Enumerates cores within a cluster in the ARIES NPU.
CoreAllocationPolicy Core allocation policy
CoreId Represents a unique identifier for an NPU core.
CoreMode Defines the core mode for NPU execution.
DataType DataType
Future Represents a future for retrieving the result of asynchronous inference.
LogLevel LogLevel
Model Represents an AI model loaded from an MXQ file.
ModelConfig Configures a core mode and core allocation of a model for NPU inference. The ModelConfig class provides methods for setting a core mode and allocating cores for NPU inference. Supported core modes are single-core, multi-core, global4-core, and global8-core. Users can also specify which cores to allocate for the model. Additionally, the configuration offers an option to enforce the use of a specific NPU bundle.
ModelPackage Represents an MXQ package (.mxqpkg) that packs multiple MXQ files into a single file.
ModelVariantHandle Handle to a specific variant of a loaded model.
NPUData A model input or output tensor that can reside on the host (CPU) or NPU.
PinnedMemory An NPU-accessible pinned (physically contiguous) memory buffer.
Scale Struct for scale values.

Functions

Name Description
load Single-step inference API. Creates model and uploads the model into NPU immediately.
set_log_level
start_tracing_events Starts event tracing and prepares to save the trace log to a specified file.
stop_tracing_events Stops event tracing and writes the recorded trace log.
get_model_summary Generates a structured summary of the specified MXQ model.
get_available_devices Get the device names of the NPU devices detected on the system.
get_available_device_numbers Get the available device numbers of the given device.

load

def load(path: str, model_config: Optional[ModelConfig] = None) -> Model
def load(
    path: str, entry_name: str, model_config: Optional[ModelConfig] = None
) -> Model

Single-step inference API. Creates model and uploads the model into NPU immediately.

This operation performs the Accelerator declaration, Model declaration, and launch in a single step.

set_log_level

def set_log_level(level: LogLevel)

start_tracing_events

def start_tracing_events(path: str) -> bool

Starts event tracing and prepares to save the trace log to a specified file.

The trace log is recorded in "Chrome Tracing JSON format," which can be viewed at https://ui.perfetto.dev/.

The trace log is not written immediately; it is saved only when stop_tracing_events() is called.

Parameters

  • path [in] — The file path where the trace log should be stored.

Returns

True if tracing starts successfully, False otherwise.

stop_tracing_events

def stop_tracing_events()

Stops event tracing and writes the recorded trace log.

This function finalizes tracing and saves the collected trace data to the file specified when start_tracing_events() was called.

get_model_summary

def get_model_summary(mxq_path: str, entry_name: Optional[str] = None) -> str

Generates a structured summary of the specified MXQ model.

Returns an overview of the model contained in the MXQ file, including:

  • Target NPU device
  • Supported core modes and their associated cores
  • The total number of model variants
  • For each variant:
    • Input and output tensor shapes
    • A list of layers with their types, output shapes, and input layer indices

The summary is returned as a human-readable string in a table and is useful for inspecting model compatibility, structure, and input/output shapes.

Parameters

  • mxq_path [in] — The path to the MXQ model file, or to the MXQ package file if entry_name is given.
  • entry_name [in] — The name of the MXQ entry in the package. If None, mxq_path is loaded as a plain MXQ file.

Returns

A formatted string containing the model summary.

get_available_devices

def get_available_devices() -> List[str]

Get the device names of the NPU devices detected on the system.

Returns

The detected device names (e.g. "aries-rb", "regulus-ra"). Empty if no NPU device is detected.

get_available_device_numbers

def get_available_device_numbers(device_name: str = "auto") -> List[int]

Get the available device numbers of the given device.

Parameters

  • device_name — The device name ("auto" or a name such as "aries", "aries-rb", "regulus", "regulus-ra"; case-insensitive).

Returns

The available device numbers. Empty if device_name cannot be resolved.