qbruntime.npu_data.NPUData Class Reference#
|
SDK qb Runtime Library v1.4
MCS001-KR
|
A model input or output tensor that can reside on the host (CPU) or NPU. More...
Public Member Functions | |
| __init__ (self, _cQbRuntime.NPUData _npu_data) | |
| Internal constructor. | |
| None | launch (self, Accelerator acc) |
| Uploads the data to the NPU memory of the given accelerator. | |
| None | cpu (self) |
| Brings the data back to host (CPU) memory. | |
| None | copy_from (self, "NPUData" src) |
| Copies the data from another NPUData. | |
| __getitem__ (self, key) | |
| Reads element(s) from the host buffer via numpy indexing. | |
| __setitem__ (self, key, value) | |
| Writes value(s) into the host buffer via numpy indexing. | |
| int | __len__ (self) |
| Returns the length of the buffer's first dimension. | |
| _Shape | shape (self) |
| Returns the shape of the data. | |
| np.dtype | dtype (self) |
| Returns the numpy dtype of the data's elements. | |
| int | dev_no (self) |
| Device number of the accelerator the data resides on. | |
| str | hardware_name (self) |
| Hardware name (device type) of the accelerator the data resides on. | |
Protected Member Functions | |
| _make_array (self) | |
| Builds the numpy view, or returns None if not CPU-resident. | |
Protected Attributes | |
| _npu_data = _npu_data | |
| _array = self._make_array() | |
Detailed Description
A model input or output tensor that can reside on the host (CPU) or NPU.
An NPUData owns the storage backing a single model input/output tensor on the host and/or the NPU, and lets the data be moved between CPU and NPU. Instances are created through Model.acquire_input_npu_data / Model.acquire_output_npu_data, not constructed directly.
The data is accessible as a numpy array (via indexing, e.g. nd[...]) only while it resides on the CPU. Bring it back with cpu before reading results that were produced on the NPU.
An NPU-resident NPUData may be passed to any model that expects a tensor of the same shape and dtype on the same accelerator, so several models can share one NPU-resident tensor without copying it back to the host in between.
- Note
- This is an advanced API rather than a typical usage.
- Warning
- This API is in beta: it may still contain bugs, and its behavior may change in a future release.
Definition at line 29 of file npu_data.py.
Constructor & Destructor Documentation
◆ __init__()
| qbruntime.npu_data.NPUData.__init__ | ( | self, | |
| _cQbRuntime.NPUData | _npu_data ) |
Internal constructor.
- Parameters
-
_npu_data The underlying C++ NPUData object. Use Model.acquire_input_npu_data / Model.acquire_output_npu_data to create instances.
Definition at line 50 of file npu_data.py.
Member Function Documentation
◆ _make_array()
|
protected |
Builds the numpy view, or returns None if not CPU-resident.
The residency is checked first (dev_no() == -1 means the data is on the host): requesting the buffer protocol while the data is on the NPU would propagate a C++ exception out of pybind's extern "C" buffer slot and call std::terminate, so it must not be attempted.
Definition at line 66 of file npu_data.py.
◆ launch()
| None qbruntime.npu_data.NPUData.launch | ( | self, | |
| Accelerator | acc ) |
Uploads the data to the NPU memory of the given accelerator.
- Parameters
-
acc The accelerator on which to place the data.
Definition at line 78 of file npu_data.py.
◆ cpu()
| None qbruntime.npu_data.NPUData.cpu | ( | self | ) |
Brings the data back to host (CPU) memory.
Definition at line 87 of file npu_data.py.
◆ copy_from()
| None qbruntime.npu_data.NPUData.copy_from | ( | self, | |
| "NPUData" | src ) |
Copies the data from another NPUData.
Both tensors must have the same shape and dtype; the source may reside on either side (a CPU-to-NPU or NPU-to-CPU copy is staged through the host).
- Parameters
-
src The source NPUData to copy from.
Definition at line 93 of file npu_data.py.
◆ __getitem__()
| qbruntime.npu_data.NPUData.__getitem__ | ( | self, | |
| key ) |
Reads element(s) from the host buffer via numpy indexing.
Definition at line 103 of file npu_data.py.
◆ __setitem__()
| qbruntime.npu_data.NPUData.__setitem__ | ( | self, | |
| key, | |||
| value ) |
Writes value(s) into the host buffer via numpy indexing.
Definition at line 109 of file npu_data.py.
◆ __len__()
| int qbruntime.npu_data.NPUData.__len__ | ( | self | ) |
Returns the length of the buffer's first dimension.
Definition at line 115 of file npu_data.py.
◆ shape()
| _Shape qbruntime.npu_data.NPUData.shape | ( | self | ) |
Returns the shape of the data.
Definition at line 120 of file npu_data.py.
◆ dtype()
| np.dtype qbruntime.npu_data.NPUData.dtype | ( | self | ) |
Returns the numpy dtype of the data's elements.
Definition at line 125 of file npu_data.py.
◆ dev_no()
| int qbruntime.npu_data.NPUData.dev_no | ( | self | ) |
Device number of the accelerator the data resides on.
- Returns
- The device number when the data is on the NPU, or -1 when it is on the host (CPU).
Definition at line 135 of file npu_data.py.
◆ hardware_name()
| str qbruntime.npu_data.NPUData.hardware_name | ( | self | ) |
Hardware name (device type) of the accelerator the data resides on.
Device numbers are assigned per hardware type (e.g. an "aries-rb" and a "regulus-ra" can both be device #0), so dev_no alone does not identify a physical device; pair it with this name to distinguish devices.
- Returns
- The hardware name (e.g. "aries-rb") when the data is on the NPU, or an empty string when it is on the host (CPU).
Definition at line 144 of file npu_data.py.
Member Data Documentation
◆ _npu_data
|
protected |
Definition at line 57 of file npu_data.py.
◆ _array
|
protected |
Definition at line 64 of file npu_data.py.
The documentation for this class was generated from the following file:
Generated by
1.14.0