NPUData Class Reference#
|
SDK qb Runtime Library v1.5
MCS001-KR
|
Represents a model input or output tensor that can reside either on the host (CPU) or on the NPU. More...
#include <npu_data.h>
Public Member Functions | |
| NPUData (const NPUData &other)=default | |
| NPUData & | operator= (const NPUData &rhs)=default |
| NPUData (NPUData &&other) noexcept=default | |
| NPUData & | operator= (NPUData &&rhs) noexcept=default |
| StatusCode | launch (Accelerator &acc) |
| Uploads the data to the NPU memory associated with the given accelerator. | |
| StatusCode | cpu () |
| Brings the data back to host (CPU) memory. | |
| StatusCode | copyFrom (NPUData &src) |
| Copies the data from another NPUData. | |
| template<typename T> | |
| T * | data (StatusCode &sc) const |
| Returns a typed pointer to the host (CPU) data. | |
| const std::vector< int64_t > & | shape () const |
| Returns the shape of the data. | |
| DataType | dataType () const |
| Returns the element data type of the data. | |
| int | devNo () const |
| Returns the device number of the accelerator the data resides on. | |
| std::string | deviceName () const |
| Returns the hardware name (device type) of the accelerator the data resides on. | |
Friends | |
| class | Model |
| class | ModelImpl |
| class | ModelVariantHandle |
Detailed Description
Represents a model input or output tensor that can reside either on the host (CPU) or on the NPU.
An NPUData object is obtained from a model via Model::acquireInputNPUData or Model::acquireOutputNPUData. It owns its host buffer, holds NPU memory that belongs to the Accelerator while the data is on the NPU, and lets the user move the data between CPU and NPU.
An NPU-resident NPUData may be passed to any model that expects a tensor of the same shape and element type on the same accelerator, so several models can share one NPU-resident tensor without copying it back to the host in between.
- Warning
- This API is in beta: it may still contain bugs, and its behavior may change in a future release.
- NPUData objects are not fully thread-safe; the client must serialize access to an NPUData from different threads. In particular, launch() and cpu() change where the data resides and must not run concurrently with each other.
- Note
- These APIs are intended for advanced use rather than typical usage.
- An NPUData is not owned by the Model it came from and may safely outlive it: destroying the model leaves the data intact and usable. NPU memory instead belongs to the Accelerator the data resides on. If that accelerator is destroyed while the data is still on the NPU, the NPU memory is reclaimed and the NPUData becomes invalid: cpu(), launch(), copyFrom() and data() fail, and devNo() returns -1. A CPU-resident NPUData holds no accelerator resource and is unaffected by the accelerator's destruction.
Definition at line 59 of file npu_data.h.
Member Function Documentation
◆ launch()
| StatusCode mobilint::NPUData::launch | ( | Accelerator & | acc | ) |
Uploads the data to the NPU memory associated with the given accelerator.
- Warning
- Not thread-safe: see the class-level note.
- Parameters
-
[in] acc The accelerator on which to place the data.
- Returns
- A status code indicating success or the specific error condition.
◆ cpu()
| StatusCode mobilint::NPUData::cpu | ( | ) |
Brings the data back to host (CPU) memory.
- Warning
- Not thread-safe: see the class-level note.
- Returns
- A status code indicating success or the specific error condition.
◆ copyFrom()
| StatusCode mobilint::NPUData::copyFrom | ( | NPUData & | src | ) |
Copies the data from another NPUData.
Both tensors must have the same shape and element type; the source may reside on either side (a CPU-to-NPU or NPU-to-CPU copy is staged through the host).
- Parameters
-
[in] src The source NPUData to copy from.
- Returns
- A status code indicating success or the specific error condition.
◆ data()
|
inline |
Returns a typed pointer to the host (CPU) data.
- Note
- The data must reside on the CPU (see NPUData::cpu). Otherwise,
scis set to an error and nullptr is returned.
- Template Parameters
-
T The element type to reinterpret the data as.
- Parameters
-
[out] sc A reference to a status code that will be updated to indicate success or the specific error condition.
- Returns
- A pointer to the host data, or nullptr on error.
Definition at line 109 of file npu_data.h.
◆ shape()
| const std::vector< int64_t > & mobilint::NPUData::shape | ( | ) | const |
Returns the shape of the data.
- Returns
- A reference to the shape vector.
◆ dataType()
| DataType mobilint::NPUData::dataType | ( | ) | const |
Returns the element data type of the data.
- Returns
- The DataType of the elements.
◆ devNo()
| int mobilint::NPUData::devNo | ( | ) | const |
Returns the device number of the accelerator the data resides on.
- Returns
- The device number of the connected accelerator when the data is on the NPU, or -1 when the data is on the host (CPU) or has been invalidated (see the class-level note on accelerator destruction).
◆ deviceName()
| std::string mobilint::NPUData::deviceName | ( | ) | const |
Returns the hardware name (device type) of the accelerator the data resides on.
Device numbers are assigned per hardware type (e.g. an "aries-rb" and a "regulus-ra" can both be device #0), so devNo() alone does not identify a physical device; pair it with this name to distinguish devices.
- Returns
- The hardware name (e.g. "aries-rb") when the data is on the NPU, or an empty string when the data is on the host (CPU) or has been invalidated (see the class-level note on accelerator destruction).
Friends And Related Symbol Documentation
◆ Model
|
friend |
Definition at line 157 of file npu_data.h.
◆ ModelImpl
|
friend |
Definition at line 158 of file npu_data.h.
◆ ModelVariantHandle
|
friend |
Definition at line 159 of file npu_data.h.
The documentation for this class was generated from the following file:
Generated by
1.14.0