콘텐츠로 이동

NPUData

#include <qbruntime/npu_data.h>

class mobilint::NPUData

Represents a model input or output tensor that can reside either on the host (CPU) or on the NPU.

An NPUData object is obtained from a model via Model::acquireInputNPUData or Model::acquireOutputNPUData. It owns the underlying storage (host buffer and/or NPU memory) and lets the user move the data between CPU and NPU.

An NPU-resident NPUData may be passed to any model that expects a tensor of the same shape and element type on the same accelerator, so several models can share one NPU-resident tensor without copying it back to the host in between.

Warning

This API is in beta: it may still contain bugs, and its behavior may change in a future release.

Note

These APIs are intended for advanced use rather than typical usage.

Note

An NPUData is tracked by the Model it came from: if that model is destroyed first, its still-outstanding NPUData are reset (their backing NPU and host memory is released), so an NPUData may safely outlive the Model it came from – it simply becomes empty.

Member functions

Member Description
NPUData ×2
operator= ×2
launch Uploads the data to the NPU memory associated with the given accelerator.
cpu Brings the data back to host (CPU) memory.
copyFrom Copies the data from another NPUData.
data Returns a typed pointer to the host (CPU) data.
shape Returns the shape of the data.
dataType Returns the element data type of the data.
devNo Returns the device number of the accelerator the data resides on.
deviceName Returns the hardware name (device type) of the accelerator the data resides on.

NPUData

NPUData(const NPUData &other)=default
NPUData(NPUData &&other) noexcept=default

operator=

NPUData & operator=(const NPUData &rhs)=default
NPUData & operator=(NPUData &&rhs) noexcept=default

launch

StatusCode launch(Accelerator &acc)

Uploads the data to the NPU memory associated with the given accelerator.

Parameters

  • acc [in] — The accelerator on which to place the data.

Returns

A status code indicating success or the specific error condition.

cpu

StatusCode cpu()

Brings the data back to host (CPU) memory.

Returns

A status code indicating success or the specific error condition.

copyFrom

StatusCode copyFrom(NPUData &src)

Copies the data from another NPUData.

Both tensors must have the same shape and element type; the source may reside on either side (a CPU-to-NPU or NPU-to-CPU copy is staged through the host).

Parameters

  • src [in] — The source NPUData to copy from.

Returns

A status code indicating success or the specific error condition.

data

template <typename T>
T * data(StatusCode &sc) const

Returns a typed pointer to the host (CPU) data.

Note

The data must reside on the CPU (see NPUData::cpu). Otherwise, sc is set to an error and nullptr is returned.

Template parameters

  • T — The element type to reinterpret the data as.

Parameters

  • sc [out] — A reference to a status code that will be updated to indicate success or the specific error condition.

Returns

A pointer to the host data, or nullptr on error.

shape

const std::vector<int64_t> & shape() const

Returns the shape of the data.

Returns

A reference to the shape vector.

dataType

DataType dataType() const

Returns the element data type of the data.

Returns

The DataType of the elements.

devNo

int devNo() const

Returns the device number of the accelerator the data resides on.

Returns

The device number of the connected accelerator when the data is on the NPU, or -1 when the data is on the host (CPU).

deviceName

std::string deviceName() const

Returns the hardware name (device type) of the accelerator the data resides on.

Device numbers are assigned per hardware type (e.g. an "aries-rb" and a "regulus-ra" can both be device #0), so devNo() alone does not identify a physical device; pair it with this name to distinguish devices.

Returns

The hardware name (e.g. "aries-rb") when the data is on the NPU, or an empty string when the data is on the host (CPU).