# NPUData

```cpp
#include <qbruntime/npu_data.h>

class mobilint::NPUData
```

Represents a model input or output tensor that can reside either on the host (CPU) or on the NPU.

An [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/index.md>) object is obtained from a model via [`Model::acquireInputNPUData`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/model/#acquireinputnpudata>) or [`Model::acquireOutputNPUData`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/model/#acquireoutputnpudata>). It owns its host buffer, holds NPU memory that belongs to the [`Accelerator`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/accelerator/index.md>) while the data is on the NPU, and lets the user move the data between CPU and NPU.

An NPU-resident [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/index.md>) may be passed to any model that expects a tensor of the same shape and element type on the same accelerator, so several models can share one NPU-resident tensor without copying it back to the host in between.

> [!WARNING]
>
> This API is in beta: it may still contain bugs, and its behavior may change in a future release.

> [!WARNING]
>
> [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/index.md>) objects are not fully thread-safe; the client must serialize access to an [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/index.md>) from different threads. In particular, [`launch()`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/#launch>) and [`cpu()`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/#cpu>) change where the data resides and must not run concurrently with each other.

> [!NOTE]
>
> These APIs are intended for advanced use rather than typical usage.

> [!NOTE]
>
> An [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/index.md>) is not owned by the [`Model`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/model/index.md>) it came from and may safely outlive it: destroying the model leaves the data intact and usable. NPU memory instead belongs to the [`Accelerator`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/accelerator/index.md>) the data resides on. If that accelerator is destroyed while the data is still on the NPU, the NPU memory is reclaimed and the [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/index.md>) becomes invalid: [`cpu()`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/#cpu>), [`launch()`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/#launch>), [`copyFrom()`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/#copyfrom>) and [`data()`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/#data>) fail, and [`devNo()`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/#devno>) returns -1. A CPU-resident [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/index.md>) holds no accelerator resource and is unaffected by the accelerator's destruction.

## Member functions

| Member | Description |
| --- | --- |
| [`NPUData`](<#npudata>) ×2 |  |
| [`operator=`](<#operator-assign>) ×2 |  |
| [`launch`](<#launch>) | Uploads the data to the NPU memory associated with the given accelerator. |
| [`cpu`](<#cpu>) | Brings the data back to host (CPU) memory. |
| [`copyFrom`](<#copyfrom>) | Copies the data from another [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/index.md>). |
| [`data`](<#data>) | Returns a typed pointer to the host (CPU) data. |
| [`shape`](<#shape>) | Returns the shape of the data. |
| [`dataType`](<#datatype>) | Returns the element data type of the data. |
| [`devNo`](<#devno>) | Returns the device number of the accelerator the data resides on. |
| [`deviceName`](<#devicename>) | Returns the hardware name (device type) of the accelerator the data resides on. |

### NPUData

```cpp
NPUData(const NPUData &other)=default
```

```cpp
NPUData(NPUData &&other) noexcept=default
```

### operator=

```cpp
NPUData & operator=(const NPUData &rhs)=default
```

```cpp
NPUData & operator=(NPUData &&rhs) noexcept=default
```

### launch

```cpp
StatusCode launch(Accelerator &acc)
```

Uploads the data to the NPU memory associated with the given accelerator.

> [!WARNING]
>
> Not thread-safe: see the class-level note.

**Parameters**

- `acc` *\[in\]* — The accelerator on which to place the data.

**Returns**

A status code indicating success or the specific error condition.

### cpu

```cpp
StatusCode cpu()
```

Brings the data back to host (CPU) memory.

> [!WARNING]
>
> Not thread-safe: see the class-level note.

**Returns**

A status code indicating success or the specific error condition.

### copyFrom

```cpp
StatusCode copyFrom(NPUData &src)
```

Copies the data from another [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/index.md>).

Both tensors must have the same shape and element type; the source may reside on either side (a CPU-to-NPU or NPU-to-CPU copy is staged through the host).

**Parameters**

- `src` *\[in\]* — The source [`NPUData`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/index.md>) to copy from.

**Returns**

A status code indicating success or the specific error condition.

### data

```cpp
template <typename T>
T * data(StatusCode &sc) const
```

Returns a typed pointer to the host (CPU) data.

> [!NOTE]
>
> The data must reside on the CPU (see [`NPUData::cpu`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/#cpu>)). Otherwise, `sc` is set to an error and `nullptr` is returned.

**Template parameters**

- `T` — The element type to reinterpret the data as.

**Parameters**

- `sc` *\[out\]* — A reference to a status code that will be updated to indicate success or the specific error condition.

**Returns**

A pointer to the host data, or `nullptr` on error.

### shape

```cpp
const std::vector<int64_t> & shape() const
```

Returns the shape of the data.

**Returns**

A reference to the shape vector.

### dataType

```cpp
DataType dataType() const
```

Returns the element data type of the data.

**Returns**

The [`DataType`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/#datatype>) of the elements.

### devNo

```cpp
int devNo() const
```

Returns the device number of the accelerator the data resides on.

**Returns**

The device number of the connected accelerator when the data is on the NPU, or -1 when the data is on the host (CPU) or has been invalidated (see the class-level note on accelerator destruction).

### deviceName

```cpp
std::string deviceName() const
```

Returns the hardware name (device type) of the accelerator the data resides on.

Device numbers are assigned per hardware type (e.g. an "aries-rb" and a "regulus-ra" can both be device \#0), so [`devNo()`](<https://docs.mobilint.com/runtime/kr/v1.6/api/cpp/npudata/#devno>) alone does not identify a physical device; pair it with this name to distinguish devices.

**Returns**

The hardware name (e.g. "aries-rb") when the data is on the NPU, or an empty string when the data is on the host (CPU) or has been invalidated (see the class-level note on accelerator destruction).
