NPUData Class Reference#
|
SDK qb Runtime Library v1.4
MCS001-EN
|
Represents a model input or output tensor that can reside either on the host (CPU) or on the NPU. More...
#include <npu_data.h>
Public Member Functions | |
| NPUData (const NPUData &other)=default | |
| NPUData & | operator= (const NPUData &rhs)=default |
| NPUData (NPUData &&other) noexcept=default | |
| NPUData & | operator= (NPUData &&rhs) noexcept=default |
| StatusCode | launch (Accelerator &acc) |
| Uploads the data to the NPU memory associated with the given accelerator. | |
| StatusCode | cpu () |
| Brings the data back to host (CPU) memory. | |
| StatusCode | copyFrom (NPUData &src) |
| Copies the data from another NPUData. | |
| template<typename T> | |
| T * | data (StatusCode &sc) const |
| Returns a typed pointer to the host (CPU) data. | |
| const std::vector< int64_t > & | shape () const |
| Returns the shape of the data. | |
| DataType | dataType () const |
| Returns the element data type of the data. | |
| int | devNo () const |
| Returns the device number of the accelerator the data resides on. | |
| std::string | deviceName () const |
| Returns the hardware name (device type) of the accelerator the data resides on. | |
Friends | |
| class | Model |
| class | ModelImpl |
| class | ModelVariantHandle |
Detailed Description
Represents a model input or output tensor that can reside either on the host (CPU) or on the NPU.
An NPUData object is obtained from a model via Model::acquireInputNPUData or Model::acquireOutputNPUData. It owns the underlying storage (host buffer and/or NPU memory) and lets the user move the data between CPU and NPU.
An NPU-resident NPUData may be passed to any model that expects a tensor of the same shape and element type on the same accelerator, so several models can share one NPU-resident tensor without copying it back to the host in between.
- Warning
- This API is in beta: it may still contain bugs, and its behavior may change in a future release.
- Note
- These APIs are intended for advanced use rather than typical usage.
- An NPUData is tracked by the Model it came from: if that model is destroyed first, its still-outstanding NPUData are reset (their backing NPU and host memory is released), so an NPUData may safely outlive the Model it came from – it simply becomes empty.
Definition at line 51 of file npu_data.h.
Member Function Documentation
◆ launch()
| StatusCode mobilint::NPUData::launch | ( | Accelerator & | acc | ) |
Uploads the data to the NPU memory associated with the given accelerator.
- Parameters
-
[in] acc The accelerator on which to place the data.
- Returns
- A status code indicating success or the specific error condition.
◆ cpu()
| StatusCode mobilint::NPUData::cpu | ( | ) |
Brings the data back to host (CPU) memory.
- Returns
- A status code indicating success or the specific error condition.
◆ copyFrom()
| StatusCode mobilint::NPUData::copyFrom | ( | NPUData & | src | ) |
Copies the data from another NPUData.
Both tensors must have the same shape and element type; the source may reside on either side (a CPU-to-NPU or NPU-to-CPU copy is staged through the host).
- Parameters
-
[in] src The source NPUData to copy from.
- Returns
- A status code indicating success or the specific error condition.
◆ data()
|
inline |
Returns a typed pointer to the host (CPU) data.
- Note
- The data must reside on the CPU (see NPUData::cpu). Otherwise,
scis set to an error and nullptr is returned.
- Template Parameters
-
T The element type to reinterpret the data as.
- Parameters
-
[out] sc A reference to a status code that will be updated to indicate success or the specific error condition.
- Returns
- A pointer to the host data, or nullptr on error.
Definition at line 97 of file npu_data.h.
◆ shape()
| const std::vector< int64_t > & mobilint::NPUData::shape | ( | ) | const |
Returns the shape of the data.
- Returns
- A reference to the shape vector.
◆ dataType()
| DataType mobilint::NPUData::dataType | ( | ) | const |
Returns the element data type of the data.
- Returns
- The DataType of the elements.
◆ devNo()
| int mobilint::NPUData::devNo | ( | ) | const |
Returns the device number of the accelerator the data resides on.
- Returns
- The device number of the connected accelerator when the data is on the NPU, or -1 when the data is on the host (CPU).
◆ deviceName()
| std::string mobilint::NPUData::deviceName | ( | ) | const |
Returns the hardware name (device type) of the accelerator the data resides on.
Device numbers are assigned per hardware type (e.g. an "aries-rb" and a "regulus-ra" can both be device #0), so devNo() alone does not identify a physical device; pair it with this name to distinguish devices.
- Returns
- The hardware name (e.g. "aries-rb") when the data is on the NPU, or an empty string when the data is on the host (CPU).
Friends And Related Symbol Documentation
◆ Model
|
friend |
Definition at line 143 of file npu_data.h.
◆ ModelImpl
|
friend |
Definition at line 144 of file npu_data.h.
◆ ModelVariantHandle
|
friend |
Definition at line 145 of file npu_data.h.
The documentation for this class was generated from the following file:
Generated by
1.14.0