PinnedMemory< T > Class Template Reference

SDK qb Runtime Library v1.3 MCS001-EN

• mobilint
• PinnedMemory

Public Member Functions | Friends | List of all members

mobilint::PinnedMemory< T > Class Template Reference

C++ API Reference

A buffer of NPU-accessible pinned (physically contiguous) memory. More...

#include <pinned_memory.h>

Public Member Functions
	PinnedMemory (const PinnedMemory &other)
PinnedMemory &	operator= (const PinnedMemory &other)
	PinnedMemory (PinnedMemory &&other) noexcept
PinnedMemory &	operator= (PinnedMemory &&other) noexcept
T *	data () const
	Returns a pointer to the host-mapped buffer.
const std::vector< int64_t > &	shape () const
	Returns the shape of the buffer.
uint64_t	size () const
	Returns the size of the buffer in bytes.
bool	isAvailable () const
	Checks whether the buffer is allocated and still usable.

Friends
class	PinnedMemoryImpl
class	ModelImpl
class	AcceleratorImpl

Detailed Description

template<typename T>
class mobilint::PinnedMemory< T >

A buffer of NPU-accessible pinned (physically contiguous) memory.

A PinnedMemory wraps a host-mapped buffer that is pinned in device memory so the NPU can read from / write to it directly, avoiding extra copies during inference. Instances are created through Accelerator::allocatePinnedMemory and consumed by Model::inferPinnedMemory.

Note

This is an experimental API and is only supported on Regulus hardware.

Destroying the last handle to a buffer (or destroying the owning Accelerator) frees the buffer and the relocation state derived from it. This must not happen while a Model::inferPinnedMemory call that uses the buffer is in flight on another thread; doing so is undefined behavior.

Warning

Model::inferPinnedMemory does not take ownership of the buffers passed to it. The caller must keep every PinnedMemory handle given as an input or output alive for the entire duration of the call. Do not destroy the last handle to such a buffer, and do not destroy the owning Accelerator, from another thread while the call is running — the NPU may still be reading from / writing to the buffer, so freeing it mid-flight is undefined behavior. This is the caller's responsibility; the runtime does not synchronize the buffer's lifetime against an in-flight inference.

The object has shared ownership of the underlying buffer: copying produces another handle to the same buffer (no data is duplicated), and the buffer is released only once the last handle is destroyed, or when the owning Accelerator is destroyed first (in which case the buffer is reclaimed and the handles become unavailable). Use isAvailable() to check whether the buffer is still valid before use.

Template Parameters

T	The element type of the buffer (uint8_t or float).

Definition at line 56 of file pinned_memory.h.

---

The documentation for this class was generated from the following file:

• pinned_memory.h

---

Generated by  1.14.0