Accelerator Class Reference#
|
SDK qb Runtime Library v1.4
MCS001-EN
|
Represents an accelerator, i.e., an NPU, used for executing models. More...
#include <acc.h>
Public Member Functions | |
| Accelerator (const Accelerator &other)=delete | |
| Accelerator (Accelerator &&other) noexcept | |
| Accelerator & | operator= (const Accelerator &rhs)=delete |
| Accelerator & | operator= (Accelerator &&rhs) noexcept |
| std::string | getDeviceName () const |
| Returns the resolved device name of this accelerator. | |
| template<typename T> | |
| PinnedMemory< T > | allocatePinnedMemory (const std::vector< int64_t > &shape, StatusCode &sc) |
| Allocates a pinned (zero-copy) memory buffer on the NPU device. | |
| ~Accelerator () | |
| Destroys the Accelerator object and releases resources. | |
Static Public Member Functions | |
| static std::unique_ptr< Accelerator > | create (StatusCode &sc) |
| Creates an Accelerator object for executing models, using device number 0. | |
| static std::unique_ptr< Accelerator > | create (int dev_no, StatusCode &sc) |
| Creates an Accelerator object for a specific device number. | |
| static std::unique_ptr< Accelerator > | create (const std::string &device_name, StatusCode &sc) |
| Creates an Accelerator object for a specific target device using device number 0. | |
| static std::unique_ptr< Accelerator > | create (const std::string &device_name, int dev_no, StatusCode &sc) |
| Creates an Accelerator object for a specific target device and device number. | |
| static std::unique_ptr< Accelerator > | create (int dev_no, bool verbose, StatusCode &sc) |
Friends | |
| class | Model |
| class | NPUData |
Detailed Description
Represents an accelerator, i.e., an NPU, used for executing models.
- Attention
- For the regulus-usb device family, do not unplug the USB device while the runtime is running. Disconnecting a device that is in use may crash the program or cause unexpected misbehavior.
Member Function Documentation
◆ create() [1/5]
|
static |
Creates an Accelerator object for executing models, using device number 0.
This function is useful when only one device is attached, in which case the default device (device 0) will be used.
- Parameters
-
[out] sc Reference to a StatusCode that will be updated with the result of the creation.
- Returns
- A unique pointer to the created Accelerator object.
◆ create() [2/5]
|
static |
Creates an Accelerator object for a specific device number.
The dev_no parameter represents the device number. For example, on Linux, if an ARIES NPU is attached as /dev/aries0, the device number is 0.
This overload resolves the target device automatically (as if "auto" were passed), so creation fails when more than one kind of target device is present in the system (auto-resolution is ambiguous), or when no device numbered dev_no exists. On failure the function returns nullptr and writes the corresponding error code to sc.
- Parameters
-
[in] dev_no The device number to associate with the Accelerator. [out] sc Reference to a StatusCode that will be updated with the result of the creation.
- Returns
- A unique pointer to the created Accelerator object.
◆ create() [3/5]
|
static |
Creates an Accelerator object for a specific target device using device number 0.
The device_name string selects target device to open; case-insensitive. Supported values:
- "auto"
- "aries", "aries-rb"
- "regulus", "regulus-usb"
- "regulus-ra", "regulus-ra-usb"
- "regulus-rb", "regulus-rb-usb"
Automatic resolution:
- "auto" resolves the target device automatically, but only when exactly one supported kind of target device is present in the system.
- A base name ("aries", "regulus", "regulus-usb", ...) is a shorthand for every target device whose name begins with it (e.g. "aries" matches "aries-rb"). It resolves to the concrete target device automatically, but only when exactly one such matching target device is present.
Creation fails when device_name is unknown, when auto-resolution is ambiguous (more than one candidate), or when no matching device exists. On failure the function returns nullptr and writes the corresponding error code to sc.
- Parameters
-
[in] device_name Device name to open (case-insensitive). See the list above. [out] sc Reference to a StatusCode that will be updated with the result of the creation.
- Returns
- A unique pointer to the created Accelerator object.
◆ create() [4/5]
|
static |
Creates an Accelerator object for a specific target device and device number.
The device_name string selects target device to open; case-insensitive. Supported values:
- "auto"
- "aries", "aries-rb"
- "regulus", "regulus-usb"
- "regulus-ra", "regulus-ra-usb"
- "regulus-rb", "regulus-rb-usb"
Automatic resolution:
- "auto" resolves the target device automatically, but only when exactly one supported kind of target device is present in the system.
- A base name ("aries", "regulus", "regulus-usb", ...) is a shorthand for every target device whose name begins with it (e.g. "aries" matches "aries-rb"). It resolves to the concrete target device automatically, but only when exactly one such matching target device is present.
Creation fails when device_name is unknown, when auto-resolution is ambiguous (more than one candidate), or when no matching device exists. On failure the function returns nullptr and writes the corresponding error code to sc.
- Parameters
-
[in] device_name Device name to open (case-insensitive). See the list above. [in] dev_no The device number to associate with the Accelerator. [out] sc Reference to a StatusCode updated with the result.
- Returns
- A unique pointer to the created Accelerator object.
◆ create() [5/5]
|
static |
- Deprecated
- Use std::unique_ptr<Accelerator> create(int dev_no, StatusCode& sc) instead.
◆ getDeviceName()
| std::string mobilint::Accelerator::getDeviceName | ( | ) | const |
Returns the resolved device name of this accelerator.
When created with a automatic resolve such as "auto", "aries", this returns the real target device name that was resolved (e.g. "aries-rb", "regulus-ra-usb").
- Returns
- The resolved device name.
◆ allocatePinnedMemory()
| PinnedMemory< T > mobilint::Accelerator::allocatePinnedMemory | ( | const std::vector< int64_t > & | shape, |
| StatusCode & | sc ) |
Allocates a pinned (zero-copy) memory buffer on the NPU device.
The returned buffer is mapped into both host and NPU address space so that inference can read/write it directly without an intermediate copy. Only supported on REGULUS SoC device (excluding USB); other target devices return Model_IsNotSupportedHardware.
- Template Parameters
-
T Element type of the buffer. Currently only uint8_t and float are supported.
- Parameters
-
[in] shape Buffer shape. All dimensions must be positive. [out] sc Reference to a StatusCode that will be updated with the result of the allocation.
- Returns
- The allocated PinnedMemory handle. On failure the handle is empty and sc holds the error code.
Friends And Related Symbol Documentation
◆ Model
◆ NPUData
The documentation for this class was generated from the following file:
Generated by
1.14.0