Skip to content

NPUData

class NPUData

A model input or output tensor that can reside on the host (CPU) or NPU.

An NPUData owns the storage backing a single model input/output tensor on the host and/or the NPU, and lets the data be moved between CPU and NPU. Instances are created through Model.acquire_input_npu_data / Model.acquire_output_npu_data, not constructed directly.

The data is accessible as a numpy array (via indexing, e.g. nd[...]) only while it resides on the CPU. Bring it back with cpu before reading results that were produced on the NPU.

An NPU-resident NPUData may be passed to any model that expects a tensor of the same shape and dtype on the same accelerator, so several models can share one NPU-resident tensor without copying it back to the host in between.

Note

This is an advanced API rather than a typical usage.

Warning

This API is in beta: it may still contain bugs, and its behavior may change in a future release.

Attributes

Name Description
shape Returns the shape of the data.
dtype Returns the numpy dtype of the data's elements.
dev_no Device number of the accelerator the data resides on.
hardware_name Hardware name (device type) of the accelerator the data resides on.

shape

property shape: _Shape

Returns the shape of the data.

dtype

property dtype: np.dtype

Returns the numpy dtype of the data's elements.

dev_no

property dev_no: int

Device number of the accelerator the data resides on.

Returns

The device number when the data is on the NPU, or -1 when it is on the host (CPU).

hardware_name

property hardware_name: str

Hardware name (device type) of the accelerator the data resides on.

Device numbers are assigned per hardware type (e.g. an "aries-rb" and a "regulus-ra" can both be device #0), so dev_no alone does not identify a physical device; pair it with this name to distinguish devices.

Returns

The hardware name (e.g. "aries-rb") when the data is on the NPU, or an empty string when it is on the host (CPU).

Methods

Name Description
__init__ Internal constructor.
launch Uploads the data to the NPU memory of the given accelerator.
cpu Brings the data back to host (CPU) memory.
copy_from Copies the data from another NPUData.
__getitem__ Reads element(s) from the host buffer via numpy indexing.
__setitem__ Writes value(s) into the host buffer via numpy indexing.
__len__ Returns the length of the buffer's first dimension.

__init__

def __init__(self, _npu_data: _cQbRuntime.NPUData)

Internal constructor.

Parameters

launch

def launch(self, acc: Accelerator) -> None

Uploads the data to the NPU memory of the given accelerator.

Parameters

  • acc — The accelerator on which to place the data.

cpu

def cpu(self) -> None

Brings the data back to host (CPU) memory.

copy_from

def copy_from(self, src: "NPUData") -> None

Copies the data from another NPUData.

Both tensors must have the same shape and dtype; the source may reside on either side (a CPU-to-NPU or NPU-to-CPU copy is staged through the host).

Parameters

  • src — The source NPUData to copy from.

__getitem__

def __getitem__(self, key)

Reads element(s) from the host buffer via numpy indexing.

__setitem__

def __setitem__(self, key, value)

Writes value(s) into the host buffer via numpy indexing.

__len__

def __len__(self) -> int

Returns the length of the buffer's first dimension.