C++ API¶
C++ API provides core functionalities for the NPU.
Classes¶
| Name | Description |
|---|---|
Accelerator |
Represents an accelerator, i.e., an NPU, used for executing models. |
Buffer |
A simple byte-sized buffer. |
BufferInfo |
Struct representing input/output buffer information. |
CacheInfo |
Struct representing KV-cache information. |
CoreId |
Represents a unique identifier for an NPU core. |
Future |
Represents a future for retrieving the result of asynchronous inference. |
FutureImpl |
|
Model |
Represents an AI model loaded from an MXQ file. |
ModelConfig |
Configures a core mode and core allocation of a model for NPU inference. |
ModelVariantHandle |
Handle to a specific variant of a loaded model. |
NDArray |
A class representing an N-dimensional array (NDArray). |
internal::NDArrayData |
A class representing the data underlying an NDArray. |
Scale |
Struct for scale values. |
Enumerations¶
StatusCode¶
Enumerates status codes for the qbruntime.
This enumeration defines return codes used in the qbruntime to indicate success or specific error conditions. A value of StatusCode::OK (0) represents success, while any other value indicates a particular error type.
| Value | Description |
|---|---|
OK = 0 |
OK |
InternalError = 23 |
Should never be reached, but reached anyway |
NotImplemented = 18 |
Not implemented |
BadAlloc = 39 |
Bad allocation |
Acc_CoreAlreadyInUse = 11 |
Core already in use. |
Acc_NPUTimeout = 42 |
NPU timeout |
Acc_NoIMemInitFound = 43 |
No imem initialization found |
Acc_NoSuchModel = 12 |
No such model |
Acc_TaskQueueNotFound = 28 |
Task queue not found |
Driver_FailedToAllocateHostMemory = 13 |
Failed to allocate host memory |
Driver_FailedToAllocateModelMemory = 44 |
Failed to allocate model memory |
Driver_FailedToBuildCmaIoReq = 45 |
Failed to build CMA IO request |
Driver_FailedToClaimCores = 24 |
Failed to claim cores |
Driver_FailedToFreeModelMemory = 46 |
Failed to free model memory |
Driver_FailedToLockCore = 49 |
Failed to lock core |
Driver_FailedToPostInfer = 33 |
Failed to post infer |
Driver_FailedToReadMemoryBuffer = 15 |
Failed to read memory buffer |
Driver_FailedToUnclaimCores = 25 |
Failed to unclaim cores |
Driver_FailedToUnlockCore = 50 |
Failed to unlock core |
Driver_FailedToWaitDone = 34 |
Failed to wait done |
Driver_FailedToWriteMemoryBuffer = 14 |
Failed to write memory buffer |
Driver_NotInitialized = 1 |
Not implemented |
Driver_WaitDoneTimeout = 35 |
Wait done timeout |
Driver_WrongBaseAddress = 47 |
Wrong base address |
MemoryPool_AllocatorNotSet = 30 |
Allocator not set |
MemoryPool_BufNotFound = 29 |
Buffer not found |
Model_AlreadyLaunched = 65 |
Model has been launched already |
Model_AsyncPipelineCheckFailed = 55 |
Asynchronous pipeline check failed |
Model_AsyncPipelineNotAlive = 56 |
Asynchronous pipeline not alive |
Model_AsyncPipelineTimeout = 57 |
Asynchronous pipeline Timeout |
Model_BrokenMXQ = 20 |
Broken mxq |
Model_BufferSizeMismatched = 63 |
Size of buffer is mismatched |
Model_CacheOverflow = 53 |
KV-cache overflow |
Model_DtypeMismatched = 22 |
dtype mismatched |
Model_FailedToAllocMemory = 32 |
Failed to allocate memory |
Model_FailedToFindDirectory = 61 |
Failed to find specified directory |
Model_FailedToLoadMXQ = 3 |
Failed to load mxq |
Model_FailedToOpenCacheFile = 62 |
Failed to open cache file |
Model_FailedToOpenModelDescFile = 19 |
Failed to open model description file |
Model_FailedToOpenOutputScale = 6 |
Deprecated: Scale file doesn't exist any more. |
Model_FailedToOpenScaleFile = 4 |
Deprecated: Scale file doesn't exist any more. |
Model_FailedToOpenSectionBinary = 8 |
Failed to open section binary |
Model_FailedToSaveTensor = 37 |
Failed to save tensor |
Model_NumCacheMismatched = 60 |
Size of KV-cache mismatched |
Model_InvalidNPUDtype = 51 |
Invalid NPU dtype |
Model_InvalidOutputScaleValue = 7 |
Deprecated: Scale file doesn't exist any more. |
Model_InvalidRmemType = 52 |
Invalid RMEM type |
Model_InvalidScaleValue = 5 |
Invalid scale value |
Model_InvalidSupplementary = 59 |
Invalid supplementary inference |
Model_InvalidVariantIdx = 64 |
Invalid model variant idx |
Model_IsNotPacked = 27 |
Deprecated: Packed logic doesn't exist any more. |
Model_IsNotSupportedHardware = 48 |
Is not supported hardware |
Model_IsPacked = 26 |
Deprecated: Packed logic doesn't exist any more. |
Model_MXQAndModelConfigNotMatch = 16 |
mxq and model config not match |
Model_NoCache = 54 |
Model does not support KV-cache |
Model_NoGlobalCoreWithGlobalMultiMode = 17 |
No global core with global multi mode |
Model_NoTargetCores = 2 |
No target cores |
Model_NotAlive = 10 |
Not alive |
Model_NotLaunched = 9 |
Not launched |
Model_PredictError = 36 |
Predict error |
Model_ShapeMismatched = 21 |
Shape mismatched |
Model_TaskQueueClosed = 40 |
Task queue closed |
Model_TaskQueueTimeout = 41 |
Task queue timeout |
Model_UnexpectedMemoryFormat = 38 |
Deprecated: |
Future_NotValid = 58 |
Future is not valid |
Cluster¶
Enumerates clusters in the ARIES NPU.
Note
The ARIES NPU consists of two clusters, each containing one global core and four local cores, totaling eight local cores. REGULUS has only a single cluster (Cluster0) with one local core (Core0).
| Value | Description |
|---|---|
Cluster0 = 1 << 16 |
Cluster 0 |
Cluster1 = 2 << 16 |
Cluster 1 |
Error = 0x7FFF'0000 |
Represents an invalid or uninitialized state. |
Core¶
Enumerates cores within a cluster in the ARIES NPU.
Note
The ARIES NPU consists of two clusters, each containing one global core and four local cores, totaling eight local cores. REGULUS has only a single cluster (Cluster0) with one local core (Core0).
| Value | Description |
|---|---|
Core0 = 1 |
Local core 0 |
Core1 = 2 |
Local core 1 |
Core2 = 3 |
Local core 2 |
Core3 = 4 |
Local core 3 |
All = 0x0000'FFFC |
Deprecated |
GlobalCore = 0x0000'FFFE |
Global core |
Error = 0x0000'FFFF |
Represents an invalid or uninitialized state. |
CoreAllocationPolicy¶
Core allocation policy.
| Value | Description |
|---|---|
Auto |
Auto |
Manual |
Manual |
CoreMode¶
Defines the core mode for NPU execution.
Supported core modes include single-core, multi-core, global4-core, and global8-core. For detailed explanations of each mode, refer to the following functions:
ModelConfig::setSingleCoreModeModelConfig::setMultiCoreModeModelConfig::setGlobal4CoreModeModelConfig::setGlobal8CoreMode
| Value | Description |
|---|---|
Single = 0 |
Single-core mode |
Multi = 1 |
Multi-core mode |
Global = 2 |
Deprecated |
Global4 = 3 |
Global4-core mode |
Global8 = 4 |
Global8-core mode |
Error = 0xF |
Represents an invalid or uninitialized state. |
LogLevel¶
| Value | Description |
|---|---|
DEBUG = 1 |
|
INFO = 2 |
|
WARN = 3 |
|
ERR = 4 |
|
FATAL = 5 |
|
OFF = 6 |
CacheType¶
| Value | Description |
|---|---|
Default = 0 |
|
Batch |
|
Error = 0x0F |
Functions¶
statusCodeToString¶
Convert StatusCode into string.
Parameters
status_code[in] — The StatusCode to convert.
Returns
String of each StatusCode if valid, else nullptr.
operator!¶
Checks whether the given StatusCode represents an error.
This operator enables StatusCode to be used in conditional statements for error checking. It returns false if sc is StatusCode::OK (0) and true otherwise.
StatusCode sc = someFunction();
if (!sc) { // `!sc` is true when `sc` is not `StatusCode::OK`.
// Handle error
}
Parameters
sc[in] — The StatusCode to evaluate.
Returns
True if sc is an error (nonzero), false if it is StatusCode::OK.
getQbRuntimeVersion¶
Retrieves the version of the qbruntime.
Returns
A string representing the runtime version.
getQbRuntimeGitVersion¶
Retrieves the Git commit hash of the qbruntime.
Returns
A string containing the Git hash.
getQbRuntimeVendor¶
Retrieves the vendor name of the qbruntime.
Typically, this function returns "mobilint."
Returns
A string containing the vendor name.
getQbRuntimeProduct¶
Retrieves product information of the qbruntime.
This function indicates the product for which the qbruntime is built. For example, it may return values such as "aries2-v4" or "regulus-v4."
Returns
A string containing product details.
setLogLevel¶
startTracingEvents¶
Starts event tracing and prepares to save the trace log to a specified file.
The trace log is recorded in "Chrome Tracing JSON format," which can be viewed at https://ui.perfetto.dev/.
The trace log is not written immediately; it is saved only when stopTracingEvents() is called.
Parameters
path[in] — The file path where the trace log should be stored.
Returns
True if tracing starts successfully, false otherwise.
stopTracingEvents¶
Stops event tracing and writes the recorded trace log.
This function finalizes tracing and saves the collected trace data to the file specified when startTracingEvents() was called.
getModelSummary¶
Generates a structured summary of the specified MXQ model.
Returns an overview of the model contained in the MXQ file, including:
- Target NPU hardware
- Supported core modes and their associated cores
- The total number of model variants
- For each variant:
- Input and output tensor shapes
- A list of layers with their types, output shapes, and input layer indices
The summary is returned as a human-readable string in a table and is useful for inspecting model compatibility, structure, and input/output shapes.
Parameters
mxq_path[in] — Path to the MXQ model file.
Returns
A formatted string containing the model summary.