qbruntime.npu_data.NPUData Class Reference#
|
SDK qb Runtime Library v1.5
MCS001-KR
|
A model input or output tensor that can reside on the host (CPU) or NPU. More...
Public Member Functions | |
| __init__ (self, _cQbRuntime.NPUData _npu_data) | |
| Internal constructor. | |
| None | launch (self, Accelerator acc) |
| Uploads the data to the NPU memory of the given accelerator. | |
| None | cpu (self) |
| Brings the data back to host (CPU) memory. | |
| None | copy_from (self, "NPUData" src) |
| Copies the data from another NPUData. | |
| __getitem__ (self, key) | |
| Reads element(s) from the host buffer via numpy indexing. | |
| __setitem__ (self, key, value) | |
| Writes value(s) into the host buffer via numpy indexing. | |
| int | __len__ (self) |
| Returns the length of the buffer's first dimension. | |
| _Shape | shape (self) |
| Returns the shape of the data. | |
| np.dtype | dtype (self) |
| Returns the numpy dtype of the data's elements. | |
| int | dev_no (self) |
| Device number of the accelerator the data resides on. | |
| str | hardware_name (self) |
| Hardware name (device type) of the accelerator the data resides on. | |
Protected Member Functions | |
| _make_array (self) | |
| Builds the numpy view, or returns None if not CPU-resident. | |
Protected Attributes | |
| _npu_data = _npu_data | |
| _array = self._make_array() | |
Detailed Description
A model input or output tensor that can reside on the host (CPU) or NPU.
An NPUData backs a single model input/output tensor: it owns its host buffer, holds NPU memory that belongs to the Accelerator while the data is on the NPU, and lets the data be moved between CPU and NPU. Instances are created through Model.acquire_input_npu_data / Model.acquire_output_npu_data, not constructed directly.
The data is accessible as a numpy array (via indexing, e.g. nd[...]) only while it resides on the CPU. Bring it back with cpu before reading results that were produced on the NPU.
An NPU-resident NPUData may be passed to any model that expects a tensor of the same shape and dtype on the same accelerator, so several models can share one NPU-resident tensor without copying it back to the host in between.
An NPUData is not owned by the Model it came from and may outlive it. Its NPU memory belongs to the Accelerator it resides on: if that accelerator is destroyed while the data is still on the NPU, the memory is reclaimed and the NPUData becomes invalid (cpu, launch, copy_from and indexing raise, and dev_no returns -1). A CPU-resident NPUData is unaffected.
- Note
- This is an advanced API rather than a typical usage.
- Warning
- This API is in beta: it may still contain bugs, and its behavior may change in a future release.
- NPUData objects are not fully thread-safe; the client must serialize access to an NPUData from different threads. In particular, launch and cpu change where the data resides and must not run concurrently with each other.
Definition at line 29 of file npu_data.py.
Constructor & Destructor Documentation
◆ __init__()
| qbruntime.npu_data.NPUData.__init__ | ( | self, | |
| _cQbRuntime.NPUData | _npu_data ) |
Internal constructor.
- Parameters
-
_npu_data The underlying C++ NPUData object. Use Model.acquire_input_npu_data / Model.acquire_output_npu_data to create instances.
Definition at line 61 of file npu_data.py.
Member Function Documentation
◆ _make_array()
|
protected |
Builds the numpy view, or returns None if not CPU-resident.
The residency is checked first (dev_no() == -1 means the data is on the host): requesting the buffer protocol while the data is on the NPU would propagate a C++ exception out of pybind's extern "C" buffer slot and call std::terminate, so it must not be attempted.
Definition at line 77 of file npu_data.py.
◆ launch()
| None qbruntime.npu_data.NPUData.launch | ( | self, | |
| Accelerator | acc ) |
Uploads the data to the NPU memory of the given accelerator.
- Warning
- Not thread-safe: see the class-level note.
- Parameters
-
acc The accelerator on which to place the data.
Definition at line 89 of file npu_data.py.
◆ cpu()
| None qbruntime.npu_data.NPUData.cpu | ( | self | ) |
Brings the data back to host (CPU) memory.
- Warning
- Not thread-safe: see the class-level note.
Definition at line 100 of file npu_data.py.
◆ copy_from()
| None qbruntime.npu_data.NPUData.copy_from | ( | self, | |
| "NPUData" | src ) |
Copies the data from another NPUData.
Both tensors must have the same shape and dtype; the source may reside on either side (a CPU-to-NPU or NPU-to-CPU copy is staged through the host).
- Parameters
-
src The source NPUData to copy from.
Definition at line 109 of file npu_data.py.
◆ __getitem__()
| qbruntime.npu_data.NPUData.__getitem__ | ( | self, | |
| key ) |
Reads element(s) from the host buffer via numpy indexing.
Definition at line 119 of file npu_data.py.
◆ __setitem__()
| qbruntime.npu_data.NPUData.__setitem__ | ( | self, | |
| key, | |||
| value ) |
Writes value(s) into the host buffer via numpy indexing.
Definition at line 125 of file npu_data.py.
◆ __len__()
| int qbruntime.npu_data.NPUData.__len__ | ( | self | ) |
Returns the length of the buffer's first dimension.
Definition at line 131 of file npu_data.py.
◆ shape()
| _Shape qbruntime.npu_data.NPUData.shape | ( | self | ) |
Returns the shape of the data.
Definition at line 136 of file npu_data.py.
◆ dtype()
| np.dtype qbruntime.npu_data.NPUData.dtype | ( | self | ) |
Returns the numpy dtype of the data's elements.
Definition at line 141 of file npu_data.py.
◆ dev_no()
| int qbruntime.npu_data.NPUData.dev_no | ( | self | ) |
Device number of the accelerator the data resides on.
- Returns
- The device number when the data is on the NPU, or -1 when it is on the host (CPU) or has been invalidated (see the class-level note on accelerator destruction).
Definition at line 151 of file npu_data.py.
◆ hardware_name()
| str qbruntime.npu_data.NPUData.hardware_name | ( | self | ) |
Hardware name (device type) of the accelerator the data resides on.
Device numbers are assigned per hardware type (e.g. an "aries-rb" and a "regulus-ra" can both be device #0), so dev_no alone does not identify a physical device; pair it with this name to distinguish devices.
- Returns
- The hardware name (e.g. "aries-rb") when the data is on the NPU, or an empty string when it is on the host (CPU) or has been invalidated (see the class-level note on accelerator destruction).
Definition at line 161 of file npu_data.py.
Member Data Documentation
◆ _npu_data
|
protected |
Definition at line 68 of file npu_data.py.
◆ _array
|
protected |
Definition at line 75 of file npu_data.py.
The documentation for this class was generated from the following file:
Generated by
1.14.0