PinnedMemory¶
An NPU-accessible pinned (physically contiguous) memory buffer.
A PinnedMemory wraps a buffer that is pinned in device memory so the NPU can read from / write to it directly, enabling zero-copy inference via Model.infer_pinned_memory. Instances are created through Accelerator.allocate_pinned_memory, not constructed directly.
Note
This is an experimental API and is only supported on Regulus hardware.
Attributes¶
| Name | Description |
|---|---|
shape |
Returns the shape of the buffer. |
dtype |
Returns the numpy dtype of the buffer's elements. |
size |
Returns the size of the buffer in bytes. |
is_available |
Returns whether the buffer is allocated and still usable. |
shape¶
Returns the shape of the buffer.
dtype¶
Returns the numpy dtype of the buffer's elements.
size¶
Returns the size of the buffer in bytes.
is_available¶
Returns whether the buffer is allocated and still usable.
Methods¶
| Name | Description |
|---|---|
__init__ |
Internal constructor. |
__getitem__ |
Reads element(s) from the pinned buffer via indexing. |
__setitem__ |
Writes value(s) into the pinned buffer via indexing. |
__len__ |
Returns the length of the buffer's first dimension. |
__init__¶
def __init__(
self,
_pinned_memory: Union[
_cQbRuntime.PinnedMemoryFloat, _cQbRuntime.PinnedMemoryUint8
],
)
Internal constructor.
Parameters
_pinned_memory— The underlying C++ pinned-memory object. UseAccelerator.allocate_pinned_memoryto create instances.
__getitem__¶
Reads element(s) from the pinned buffer via indexing.
Supports the same indexing as numpy (integers, slices, tuples, etc.), e.g. pm[0], pm[1, 2], pm[:10]. The data is read directly from the pinned buffer without an intermediate copy.
Parameters
key— The index or slice.
Returns
The selected element or a view into the pinned buffer.
__setitem__¶
Writes value(s) into the pinned buffer via indexing.
Writes directly into the pinned buffer, e.g. pm[0] = 1.0, pm[:10] = src. This is the recommended way to fill an input buffer before Model.infer_pinned_memory.
Parameters
key— The index or slice.value— The value(s) to write.
__len__¶
Returns the length of the buffer's first dimension.