콘텐츠로 이동

NPUData

class NPUData

A model input or output tensor that can reside on the host (CPU) or NPU.

An NPUData backs a single model input/output tensor: it owns its host buffer, holds NPU memory that belongs to the Accelerator while the data is on the NPU, and lets the data be moved between CPU and NPU. Instances are created through Model.acquire_input_npu_data / Model.acquire_output_npu_data, not constructed directly.

The data is accessible as a numpy array (via indexing, e.g. nd[...]) only while it resides on the CPU. Bring it back with cpu before reading results that were produced on the NPU.

An NPU-resident NPUData may be passed to any model that expects a tensor of the same shape and dtype on the same accelerator, so several models can share one NPU-resident tensor without copying it back to the host in between.

An NPUData is not owned by the Model it came from and may outlive it. Its NPU memory belongs to the Accelerator it resides on: if that accelerator is destroyed while the data is still on the NPU, the memory is reclaimed and the NPUData becomes invalid (cpu, launch, copy_from and indexing raise, and dev_no returns -1). A CPU-resident NPUData is unaffected.

Note

This is an advanced API rather than a typical usage.

Warning

This API is in beta: it may still contain bugs, and its behavior may change in a future release.

Warning

NPUData objects are not fully thread-safe; the client must serialize access to an NPUData from different threads. In particular, launch and cpu change where the data resides and must not run concurrently with each other.

Attributes

Name Description
shape Returns the shape of the data.
dtype Returns the numpy dtype of the data's elements.
dev_no Device number of the accelerator the data resides on.
hardware_name Hardware name (device type) of the accelerator the data resides on.

shape

property shape: _Shape

Returns the shape of the data.

dtype

property dtype: np.dtype

Returns the numpy dtype of the data's elements.

dev_no

property dev_no: int

Device number of the accelerator the data resides on.

Returns

The device number when the data is on the NPU, or -1 when it is on the host (CPU) or has been invalidated (see the class-level note on accelerator destruction).

hardware_name

property hardware_name: str

Hardware name (device type) of the accelerator the data resides on.

Device numbers are assigned per hardware type (e.g. an "aries-rb" and a "regulus-ra" can both be device #0), so dev_no alone does not identify a physical device; pair it with this name to distinguish devices.

Returns

The hardware name (e.g. "aries-rb") when the data is on the NPU, or an empty string when it is on the host (CPU) or has been invalidated (see the class-level note on accelerator destruction).

Methods

Name Description
__init__ Internal constructor.
launch Uploads the data to the NPU memory of the given accelerator.
cpu Brings the data back to host (CPU) memory.
copy_from Copies the data from another NPUData.
__getitem__ Reads element(s) from the host buffer via numpy indexing.
__setitem__ Writes value(s) into the host buffer via numpy indexing.
__len__ Returns the length of the buffer's first dimension.

__init__

def __init__(self, _npu_data: _cQbRuntime.NPUData)

Internal constructor.

Parameters

launch

def launch(self, acc: Accelerator) -> None

Uploads the data to the NPU memory of the given accelerator.

Warning

Not thread-safe: see the class-level note.

Parameters

  • acc — The accelerator on which to place the data.

cpu

def cpu(self) -> None

Brings the data back to host (CPU) memory.

Warning

Not thread-safe: see the class-level note.

copy_from

def copy_from(self, src: "NPUData") -> None

Copies the data from another NPUData.

Both tensors must have the same shape and dtype; the source may reside on either side (a CPU-to-NPU or NPU-to-CPU copy is staged through the host).

Parameters

  • src — The source NPUData to copy from.

__getitem__

def __getitem__(self, key)

Reads element(s) from the host buffer via numpy indexing.

__setitem__

def __setitem__(self, key, value)

Writes value(s) into the host buffer via numpy indexing.

__len__

def __len__(self) -> int

Returns the length of the buffer's first dimension.