Accelerator¶
Represents an accelerator, i.e., an NPU, used for executing models.
Attention
For the regulus-usb device family, do not unplug the USB device while the runtime is running. Disconnecting a device that is in use may crash the program or cause unexpected misbehavior.
Methods¶
| Name | Description |
|---|---|
__init__ |
Creates an Accelerator object for a specific target device and device number. |
get_device_name |
Returns the resolved device name of this accelerator. |
allocate_pinned_memory |
Allocates an NPU-accessible pinned memory buffer. |
__init__¶
Creates an Accelerator object for a specific target device and device number.
The device_name parameter selects target device to open; case-insensitive. Supported values:
"auto""aries","aries-rb""regulus","regulus-usb""regulus-ra","regulus-ra-usb""regulus-rb","regulus-rb-usb"
Automatic resolution:
"auto"resolves the target device automatically, but only when exactly one supported kind of target device is present in the system.- A base name (
"aries","regulus","regulus-usb", ...) is a shorthand for every target device whose name begins with it (e.g."aries"matches"aries-rb"). It resolves to the concrete target device automatically, but only when exactly one such matching target device is present.
The dev_no parameter is the device number. For example, on Linux an ARIES NPU attached as /dev/aries0 has device number 0.
Parameters
device_name[in] — Device name to open (case-insensitive). See the list above.dev_no[in] — The device number to associate with the Accelerator.
Throws
QbRuntimeError— Ifdevice_nameis unknown, if auto-resolution is ambiguous (more than one candidate), or if no matching device exists.
Note
Supported constructor forms:
Accelerator(): auto-select the only attached NPU.Accelerator(dev_no): auto-detect device, open target devicedev_no.Accelerator(device_name): opendevice_nameat device 0.Accelerator(device_name, dev_no): opendevice_nameat devicedev_no.
get_device_name¶
Returns the resolved device name of this accelerator.
When created with a automatic resolve such as "auto", "aries", this returns the real target device name that was resolved (e.g. "aries-rb", "regulus-ra-usb").
Returns
The resolved device name.
allocate_pinned_memory¶
Allocates an NPU-accessible pinned memory buffer.
The returned buffer can be used as input or output for zero-copy inference via Model.infer_pinned_memory.
Note
This is an experimental API and is only supported on REGULUS device.
Parameters
shape— The shape of the buffer to allocate.dtype— The element type of the buffer. Must benumpy.float32ornumpy.uint8.
Returns
A PinnedMemory wrapping the allocated buffer.