PinnedMemory¶
A buffer of NPU-accessible pinned (physically contiguous) memory.
A PinnedMemory wraps a host-mapped buffer that is pinned in device memory so the NPU can read from / write to it directly, avoiding extra copies during inference. Instances are created through Accelerator::allocatePinnedMemory and consumed by Model::inferPinnedMemory.
Note
This is an experimental API and is only supported on REGULUS device.
Note
Destroying the last handle to a buffer (or destroying the owning Accelerator) frees the buffer and the relocation state derived from it. This must not happen while a Model::inferPinnedMemory call that uses the buffer is in flight on another thread; doing so is undefined behavior.
Warning
Model::inferPinnedMemory does not take ownership of the buffers passed to it. The caller must keep every PinnedMemory handle given as an input or output alive for the entire duration of the call. Do not destroy the last handle to such a buffer, and do not destroy the owning Accelerator, from another thread while the call is running — the NPU may still be reading from / writing to the buffer, so freeing it mid-flight is undefined behavior. This is the caller's responsibility; the runtime does not synchronize the buffer's lifetime against an in-flight inference.
The object has shared ownership of the underlying buffer: copying produces another handle to the same buffer (no data is duplicated), and the buffer is released only once the last handle is destroyed, or when the owning Accelerator is destroyed first (in which case the buffer is reclaimed and the handles become unavailable). Use isAvailable() to check whether the buffer is still valid before use.
Template parameters
T— The element type of the buffer (uint8_torfloat).
Member functions¶
| Member | Description |
|---|---|
PinnedMemory ×3 |
|
operator= ×2 |
|
data |
Returns a pointer to the host-mapped buffer. |
shape |
Returns the shape of the buffer. |
size |
Returns the size of the buffer in bytes. |
isAvailable |
Checks whether the buffer is allocated and still usable. |
getDmabufFd |
Returns a Linux dma-buf fd that refers to this buffer. |
PinnedMemory¶
operator=¶
data¶
Returns a pointer to the host-mapped buffer.
Returns
A pointer to the buffer, or nullptr if the buffer is not available.
shape¶
Returns the shape of the buffer.
Returns
The shape used at allocation, or an empty vector if the buffer is not available.
size¶
Returns the size of the buffer in bytes.
Returns
The buffer size in bytes, or 0 if the buffer is not available.
isAvailable¶
Checks whether the buffer is allocated and still usable.
A buffer is unavailable when allocation failed, when it was moved from, or when the owning Accelerator was destroyed and reclaimed the memory.
Returns
true if the buffer is available, false otherwise.
getDmabufFd¶
Returns a Linux dma-buf fd that refers to this buffer.
The fd lets external hardware write directly into this buffer, with no CPU copy. Some devices (e.g. a DMA350 V4L2 M2M device doing compose/copy) accept buffers only as dma-buf fds (V4L2_MEMORY_DMABUF); pass this fd to such a device as the buffer it writes into.
Note
Supported on REGULUS only, except USB devices (regulus-*-usb).
Note
The fd is created on the first successful call, and later calls return the same fd.
Warning
This PinnedMemory owns the fd and closes it when the buffer becomes unavailable. Do not close it yourself; dup() it if you need your own copy.
Returns
A dma-buf fd on success, or -1 if the buffer is unavailable, the device doesn't support exporting (non-REGULUS, or regulus-*-usb), or the export fails.