콘텐츠로 이동

Accelerator

class Accelerator

Represents an accelerator, i.e., an NPU, used for executing models.

Attention

For the regulus-usb device family, do not unplug the USB device while the runtime is running. Disconnecting a device that is in use may crash the program or cause unexpected misbehavior.

Methods

Name Description
__init__ Creates an Accelerator object for a specific target device and device number.
get_device_name Returns the resolved device name of this accelerator.
allocate_pinned_memory Allocates an NPU-accessible pinned memory buffer.

__init__

def __init__(self, dev_no: int = 0)
def __init__(self, device_name: str = "auto")
def __init__(self, device_name: str = "auto", dev_no: int = 0)

Creates an Accelerator object for a specific target device and device number.

The device_name parameter selects target device to open; case-insensitive. Supported values:

  • "auto"
  • "aries", "aries-rb"
  • "regulus", "regulus-usb"
  • "regulus-ra", "regulus-ra-usb"
  • "regulus-rb", "regulus-rb-usb"

Automatic resolution:

  • "auto" resolves the target device automatically, but only when exactly one supported kind of target device is present in the system.
  • A base name ("aries", "regulus", "regulus-usb", ...) is a shorthand for every target device whose name begins with it (e.g. "aries" matches "aries-rb"). It resolves to the concrete target device automatically, but only when exactly one such matching target device is present.

The dev_no parameter is the device number. For example, on Linux an ARIES NPU attached as /dev/aries0 has device number 0.

Parameters

  • device_name [in] — Device name to open (case-insensitive). See the list above.
  • dev_no [in] — The device number to associate with the Accelerator.

Throws

  • QbRuntimeError — If device_name is unknown, if auto-resolution is ambiguous (more than one candidate), or if no matching device exists.

Note

Supported constructor forms:

  • Accelerator(): auto-select the only attached NPU.
  • Accelerator(dev_no): auto-detect device, open target device dev_no.
  • Accelerator(device_name): open device_name at device 0.
  • Accelerator(device_name, dev_no): open device_name at device dev_no.

get_device_name

def get_device_name(self) -> str

Returns the resolved device name of this accelerator.

When created with a automatic resolve such as "auto", "aries", this returns the real target device name that was resolved (e.g. "aries-rb", "regulus-ra-usb").

Returns

The resolved device name.

allocate_pinned_memory

def allocate_pinned_memory(
    self, shape: Sequence[int], dtype: np.dtype
) -> PinnedMemory

Allocates an NPU-accessible pinned memory buffer.

The returned buffer can be used as input or output for zero-copy inference via Model.infer_pinned_memory.

Note

This is an experimental API and is only supported on REGULUS device.

Parameters

  • shape — The shape of the buffer to allocate.
  • dtype — The element type of the buffer. Must be numpy.float32 or numpy.uint8.

Returns

A PinnedMemory wrapping the allocated buffer.