Accelerator¶
Represents an accelerator, i.e., an NPU, used for executing models.
Attention
For the regulus-usb device family, do not unplug the USB device while the runtime is running. Disconnecting a device that is in use may crash the program or cause unexpected misbehavior.
Static functions¶
| Member | Description |
|---|---|
create ×5 |
Creates an Accelerator object for executing models, using device number 0. |
create¶
Creates an Accelerator object for executing models, using device number 0.
This function is useful when only one device is attached, in which case the default device (device 0) will be used.
Parameters
sc[out] — Reference to a StatusCode that will be updated with the result of the creation.
Returns
A unique pointer to the created Accelerator object.
Creates an Accelerator object for a specific device number.
The dev_no parameter represents the device number. For example, on Linux, if an ARIES NPU is attached as /dev/aries0, the device number is 0.
This overload resolves the target device automatically (as if "auto" were passed), so creation fails when more than one kind of target device is present in the system (auto-resolution is ambiguous), or when no device numbered dev_no exists. On failure the function returns nullptr and writes the corresponding error code to sc.
Parameters
dev_no[in] — The device number to associate with the Accelerator.sc[out] — Reference to a StatusCode that will be updated with the result of the creation.
Returns
A unique pointer to the created Accelerator object.
Creates an Accelerator object for a specific target device using device number 0.
The device_name string selects target device to open; case-insensitive. Supported values:
"auto""aries","aries-rb""regulus","regulus-usb""regulus-ra","regulus-ra-usb""regulus-rb","regulus-rb-usb"
Automatic resolution:
"auto"resolves the target device automatically, but only when exactly one supported kind of target device is present in the system.- A base name (
"aries","regulus","regulus-usb", ...) is a shorthand for every target device whose name begins with it (e.g."aries"matches"aries-rb"). It resolves to the concrete target device automatically, but only when exactly one such matching target device is present.
Creation fails when device_name is unknown, when auto-resolution is ambiguous (more than one candidate), or when no matching device exists. On failure the function returns nullptr and writes the corresponding error code to sc.
Parameters
device_name[in] — Device name to open (case-insensitive). See the list above.sc[out] — Reference to a StatusCode that will be updated with the result of the creation.
Returns
A unique pointer to the created Accelerator object.
static std::unique_ptr<Accelerator> create(const std::string &device_name, int dev_no, StatusCode &sc)
Creates an Accelerator object for a specific target device and device number.
The device_name string selects target device to open; case-insensitive. Supported values:
"auto""aries","aries-rb""regulus","regulus-usb""regulus-ra","regulus-ra-usb""regulus-rb","regulus-rb-usb"
Automatic resolution:
"auto"resolves the target device automatically, but only when exactly one supported kind of target device is present in the system.- A base name (
"aries","regulus","regulus-usb", ...) is a shorthand for every target device whose name begins with it (e.g."aries"matches"aries-rb"). It resolves to the concrete target device automatically, but only when exactly one such matching target device is present.
Creation fails when device_name is unknown, when auto-resolution is ambiguous (more than one candidate), or when no matching device exists. On failure the function returns nullptr and writes the corresponding error code to sc.
Parameters
device_name[in] — Device name to open (case-insensitive). See the list above.dev_no[in] — The device number to associate with the Accelerator.sc[out] — Reference to a StatusCode updated with the result.
Returns
A unique pointer to the created Accelerator object.
Deprecated
Use std::unique_ptr<Accelerator> create(int dev_no, StatusCode& sc) instead.
Member functions¶
| Member | Description |
|---|---|
Accelerator ×2 |
|
operator= ×2 |
|
getDeviceName |
Returns the resolved device name of this accelerator. |
allocatePinnedMemory |
Allocates a pinned (zero-copy) memory buffer on the NPU device. |
Accelerator¶
operator=¶
getDeviceName¶
Returns the resolved device name of this accelerator.
When created with a automatic resolve such as "auto", "aries", this returns the real target device name that was resolved (e.g. "aries-rb", "regulus-ra-usb").
Returns
The resolved device name.
allocatePinnedMemory¶
template <typename T>
PinnedMemory<T> allocatePinnedMemory(const std::vector<int64_t> &shape, StatusCode &sc)
Allocates a pinned (zero-copy) memory buffer on the NPU device.
The returned buffer is mapped into both host and NPU address space so that inference can read/write it directly without an intermediate copy. Only supported on REGULUS SoC device (excluding USB); other target devices return Model_IsNotSupportedHardware.
Template parameters
T— Element type of the buffer. Currently onlyuint8_tandfloatare supported.
Parameters
shape[in] — Buffer shape. All dimensions must be positive.sc[out] — Reference to a StatusCode that will be updated with the result of the allocation.
Returns
The allocated PinnedMemory handle. On failure the handle is empty and sc holds the error code.