Skip to content

PinnedMemory

class PinnedMemory

An NPU-accessible pinned (physically contiguous) memory buffer.

A PinnedMemory wraps a buffer that is pinned in device memory so the NPU can read from / write to it directly, enabling zero-copy inference via Model.infer_pinned_memory. Instances are created through Accelerator.allocate_pinned_memory, not constructed directly.

Note

This is an experimental API and is only supported on REGULUS device.

Attributes

Name Description
shape Returns the shape of the buffer.
dtype Returns the numpy dtype of the buffer's elements.
size Returns the size of the buffer in bytes.
is_available Returns whether the buffer is allocated and still usable.

shape

property shape: _Shape

Returns the shape of the buffer.

dtype

property dtype: np.dtype

Returns the numpy dtype of the buffer's elements.

size

property size: int

Returns the size of the buffer in bytes.

is_available

property is_available: bool

Returns whether the buffer is allocated and still usable.

Methods

Name Description
__init__ Internal constructor.
__getitem__ Reads element(s) from the pinned buffer via indexing.
__setitem__ Writes value(s) into the pinned buffer via indexing.
__len__ Returns the length of the buffer's first dimension.

__init__

def __init__(
    self,
    _pinned_memory: Union[
        _cQbRuntime.PinnedMemoryFloat, _cQbRuntime.PinnedMemoryUint8
    ],
)

Internal constructor.

Parameters

__getitem__

def __getitem__(self, key)

Reads element(s) from the pinned buffer via indexing.

Supports the same indexing as numpy (integers, slices, tuples, etc.), e.g. pm[0], pm[1, 2], pm[:10]. The data is read directly from the pinned buffer without an intermediate copy.

Parameters

  • key — The index or slice.

Returns

The selected element or a view into the pinned buffer.

__setitem__

def __setitem__(self, key, value)

Writes value(s) into the pinned buffer via indexing.

Writes directly into the pinned buffer, e.g. pm[0] = 1.0, pm[:10] = src. This is the recommended way to fill an input buffer before Model.infer_pinned_memory.

Parameters

  • key — The index or slice.
  • value — The value(s) to write.

__len__

def __len__(self) -> int

Returns the length of the buffer's first dimension.