# C++ API

C++ API provides core functionalities for the NPU.

## Classes

| Name | Description |
| --- | --- |
| [`Accelerator`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/accelerator/index.md>) | Represents an accelerator, i.e., an NPU, used for executing models. |
| [`BatchParam`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/batchparam/index.md>) | Struct containing BatchLLM parameters. |
| [`Buffer`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/buffer/index.md>) | A simple byte-sized buffer. |
| [`BufferInfo`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/bufferinfo/index.md>) | Struct representing input/output buffer information. |
| [`CacheInfo`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/cacheinfo/index.md>) | Struct representing KV-cache information. |
| [`CoreId`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/coreid/index.md>) | Represents a unique identifier for an NPU core. |
| [`Future`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/future/index.md>) | Represents a future for retrieving the result of asynchronous inference. |
| [`FutureImpl`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/futureimpl/index.md>) |  |
| [`Model`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/model/index.md>) | Represents an AI model loaded from an MXQ file. |
| [`ModelConfig`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/modelconfig/index.md>) | Configures a core mode and core allocation of a model for NPU inference. |
| [`ModelVariantHandle`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/modelvarianthandle/index.md>) | Handle to a specific variant of a loaded model. |
| [`NDArray`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/ndarray/index.md>) | A class representing an N-dimensional array ([NDArray](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/ndarray/index.md>)). |
| [`internal::NDArrayData`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/internal-ndarraydata/index.md>) | A class representing the data underlying an [NDArray](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/ndarray/index.md>). |
| [`NPUData`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/npudata/index.md>) | Represents a model input or output tensor that can reside either on the host (CPU) or on the NPU. |
| [`PinnedMemory`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/pinnedmemory/index.md>) | A buffer of NPU-accessible pinned (physically contiguous) memory. |
| [`Scale`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/scale/index.md>) | Struct for scale values. |

## Enumerations

### StatusCode

```cpp
enum class StatusCode
```

Enumerates status codes for the qbruntime.

This enumeration defines return codes used in the qbruntime to indicate success or specific error conditions. A value of [`StatusCode::OK`](<#statuscode>) (0) represents success, while any other value indicates a particular error type.

| Value | Description |
| --- | --- |
| `OK = 0` | OK |
| `InternalError = 23` | Should never be reached, but reached anyway |
| `NotImplemented = 18` | Not implemented |
| `BadAlloc = 39` | Bad allocation |
| `Acc_AmbiguousDevice = 72` | Multiple devices found; select one explicitly |
| `Acc_CoreAlreadyInUse = 11` | [Core](<#core>) already in use. |
| `Acc_DeviceNotFound = 71` | Requested device not found in the system |
| `Acc_NPUTimeout = 42` | NPU timeout |
| `Acc_NoIMemInitFound = 43` | No imem initialization found |
| `Acc_NoSuchModel = 12` | No such model |
| `Acc_TaskQueueNotFound = 28` | Task queue not found |
| `Acc_UnknownDevice = 73` | Unknown or unsupported device name |
| `Driver_FailedToAllocateHostMemory = 13` | Failed to allocate host memory |
| `Driver_FailedToAllocateModelMemory = 44` | Failed to allocate model memory |
| `Driver_FailedToBuildCmaIoReq = 45` | Failed to build CMA IO request |
| `Driver_FailedToClaimCores = 24` | Failed to claim cores |
| `Driver_FailedToFreeHostMemory = 67` | Failed to free host memory |
| `Driver_FailedToFreeModelMemory = 46` | Failed to free model memory |
| `Driver_FailedToLockCore = 49` | Failed to lock core |
| `Driver_FailedToPostInfer = 33` | Failed to post infer |
| `Driver_FailedToReadMemoryBuffer = 15` | Failed to read memory buffer |
| `Driver_FailedToSyncMappedMemory = 68` | Failed to sync mmaped memory |
| `Driver_FailedToUnclaimCores = 25` | Failed to unclaim cores |
| `Driver_FailedToUnlockCore = 50` | Failed to unlock core |
| `Driver_FailedToWaitDone = 34` | Failed to wait done |
| `Driver_FailedToWriteMemoryBuffer = 14` | Failed to write memory buffer |
| `Driver_NotInitialized = 1` | Not implemented |
| `Driver_WaitDoneTimeout = 35` | Wait done timeout |
| `Driver_WrongBaseAddress = 47` | Wrong base address |
| `MemoryPool_AllocatorNotSet = 30` | Allocator not set |
| `MemoryPool_BufNotFound = 29` | [Buffer](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/buffer/index.md>) not found |
| `Model_AlreadyLaunched = 65` | [Model](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/model/index.md>) has been launched already |
| `Model_AsyncPipelineCheckFailed = 55` | Asynchronous pipeline check failed |
| `Model_AsyncPipelineNotAlive = 56` | Asynchronous pipeline not alive |
| `Model_AsyncPipelineTimeout = 57` | Asynchronous pipeline Timeout |
| `Model_BrokenMXQ = 20` | Broken mxq |
| `Model_BufferSizeMismatched = 63` | Size of buffer is mismatched |
| `Model_CacheNotFound = 74` | KV-cache with given name or layer hash not found |
| `Model_CacheOverflow = 53` | KV-cache overflow |
| `Model_DtypeMismatched = 22` | dtype mismatched |
| `Model_FailedToAllocMemory = 32` | Failed to allocate memory |
| `Model_FailedToFindDirectory = 61` | Failed to find specified directory |
| `Model_FailedToLoadMXQ = 3` | Failed to load mxq |
| `Model_FailedToOpenCacheFile = 62` | Failed to open cache file |
| `Model_FailedToOpenModelDescFile = 19` | Failed to open model description file |
| `Model_FailedToOpenOutputScale = 6` | **Deprecated:** [Scale](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/scale/index.md>) file doesn't exist any more. |
| `Model_FailedToOpenScaleFile = 4` | **Deprecated:** [Scale](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/scale/index.md>) file doesn't exist any more. |
| `Model_FailedToOpenSectionBinary = 8` | Failed to open section binary |
| `Model_FailedToSaveTensor = 37` | Failed to save tensor |
| `Model_NumCacheMismatched = 60` | Size of KV-cache mismatched |
| `Model_InvalidBatchParam = 66` | Invalid [BatchParam](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/batchparam/index.md>) |
| `Model_InvalidNPUDtype = 51` | Invalid NPU dtype |
| `Model_InvalidOutputScaleValue = 7` | **Deprecated:** [Scale](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/scale/index.md>) file doesn't exist any more. |
| `Model_InvalidRmemType = 52` | Invalid RMEM type |
| `Model_InvalidScaleValue = 5` | Invalid scale value |
| `Model_InvalidSupplementary = 59` | Invalid supplementary inference |
| `Model_InvalidVariantIdx = 64` | Invalid model variant idx |
| `Model_IsNotPacked = 27` | **Deprecated:** `Packed` logic doesn't exist any more. |
| `Model_IsNotSupportedHardware = 48` | [Model](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/model/index.md>) is not supported on this target device |
| `Model_IsPacked = 26` | **Deprecated:** `Packed` logic doesn't exist any more. |
| `Model_MXQAndModelConfigNotMatch = 16` | mxq and model config not match |
| `Model_NoCache = 54` | [Model](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/model/index.md>) does not support KV-cache |
| `Model_NoGlobalCoreWithGlobalMultiMode = 17` | No global core with global multi mode |
| `Model_NoTargetCores = 2` | No target cores |
| `Model_NotAlive = 10` | Not alive |
| `Model_NotLaunched = 9` | Not launched |
| `Model_NotSupported = 69` | Not supported api |
| `Model_PredictError = 36` | Predict error |
| `Model_ShapeMismatched = 21` | Shape mismatched |
| `Model_TaskQueueClosed = 40` | Task queue closed |
| `Model_TaskQueueTimeout = 41` | Task queue timeout |
| `Model_UnexpectedMemoryFormat = 38` | **Deprecated:** |
| `PinnedMemory_NotAvailable = 70` | [PinnedMemory](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/pinnedmemory/index.md>) buffer is not available |
| `Future_NotValid = 58` | [Future](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/future/index.md>) is not valid |

### Cluster

```cpp
enum class Cluster
```

Enumerates clusters in the ARIES NPU.

> [!NOTE]
>
> The ARIES NPU consists of two clusters, each containing one global core and four local cores, totaling eight local cores. REGULUS has only a single cluster (Cluster0) with one local core (Core0).

| Value | Description |
| --- | --- |
| `Cluster0 = 1 << 16` | [Cluster](<#cluster>) 0 |
| `Cluster1 = 2 << 16` | [Cluster](<#cluster>) 1 |
| `Error = 0x7FFF'0000` | Represents an invalid or uninitialized state. |

### Core

```cpp
enum class Core
```

Enumerates cores within a cluster in the ARIES NPU.

> [!NOTE]
>
> The ARIES NPU consists of two clusters, each containing one global core and four local cores, totaling eight local cores. REGULUS has only a single cluster (Cluster0) with one local core (Core0).

| Value | Description |
| --- | --- |
| `Core0 = 1` | Local core 0 |
| `Core1 = 2` | Local core 1 |
| `Core2 = 3` | Local core 2 |
| `Core3 = 4` | Local core 3 |
| `All = 0x0000'FFFC` | Deprecated |
| `GlobalCore = 0x0000'FFFE` | Global core |
| `Error = 0x0000'FFFF` | Represents an invalid or uninitialized state. |

### CoreAllocationPolicy

```cpp
enum class CoreAllocationPolicy
```

[Core](<#core>) allocation policy.

| Value | Description |
| --- | --- |
| `Auto` | Auto |
| `Manual` | Manual |

### CoreMode

```cpp
enum class CoreMode
```

Defines the core mode for NPU execution.

Supported core modes include single-core, multi-core, global4-core, and global8-core. For detailed explanations of each mode, refer to the following functions:

- [`ModelConfig::setAutoCoreMode`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/modelconfig/#setautocoremode>)
- [`ModelConfig::setSingleCoreMode`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/modelconfig/#setsinglecoremode>)
- [`ModelConfig::setMultiCoreMode`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/modelconfig/#setmulticoremode>)
- [`ModelConfig::setGlobal4CoreMode`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/modelconfig/#setglobal4coremode>)
- [`ModelConfig::setGlobal8CoreMode`](<https://docs.mobilint.com/runtime/en/v1.5/api/cpp/modelconfig/#setglobal8coremode>)

| Value | Description |
| --- | --- |
| `Single = 0` | Single-core mode |
| `Multi = 1` | Multi-core mode |
| `Global = 2` | Deprecated |
| `Global4 = 3` | Global4-core mode |
| `Global8 = 4` | Global8-core mode |
| `Auto = 0xE` | Detects model's [CoreMode](<#coremode>) automatically. |
| `Error = 0xF` | Represents an invalid or uninitialized state. |

### LogLevel

```cpp
enum class LogLevel
```

[LogLevel](<#loglevel>).

| Value | Description |
| --- | --- |
| `Debug = 1` |  |
| `Info = 2` |  |
| `Warn = 3` |  |
| `Err = 4` |  |
| `Fatal = 5` |  |
| `Off = 6` |  |

### CacheType

```cpp
enum class CacheType
```

[CacheType](<#cachetype>).

| Value | Description |
| --- | --- |
| `Default = 0` |  |
| `Batch` |  |
| `FixedTailCache` |  |
| `Error = 0x0F` |  |

### DataType

```cpp
enum class DataType
```

[DataType](<#datatype>).

| Value | Description |
| --- | --- |
| `Float32 = 0` |  |
| `Float16` |  |
| `Int8` |  |
| `Uint8` |  |
| `BInt16` |  |
| `Error = 0x0F` |  |

## Functions

### statusCodeToString

```cpp
const std::string statusCodeToString(const StatusCode status_code)
```

Convert [StatusCode](<#statuscode>) into string.

**Parameters**

- `status_code` *\[in\]* — The [StatusCode](<#statuscode>) to convert.

**Returns**

String of each [StatusCode](<#statuscode>) if valid, else nullptr.

### operator!

```cpp
bool operator!(StatusCode sc)
```

Checks whether the given [StatusCode](<#statuscode>) represents an error.

This operator enables [`StatusCode`](<#statuscode>) to be used in conditional statements for error checking. It returns `false` if `sc` is [`StatusCode::OK`](<#statuscode>) (0) and `true` otherwise.

```cpp
StatusCode sc = someFunction();
if (!sc) { // `!sc` is true when `sc` is not `StatusCode::OK`.
    // Handle error
}
```

**Parameters**

- `sc` *\[in\]* — The [StatusCode](<#statuscode>) to evaluate.

**Returns**

True if `sc` is an error (nonzero), false if it is [`StatusCode::OK`](<#statuscode>).

### getQbRuntimeVersion

```cpp
std::string getQbRuntimeVersion()
```

Retrieves the version of the qbruntime.

**Returns**

A string representing the runtime version.

### getQbRuntimeGitVersion

```cpp
std::string getQbRuntimeGitVersion()
```

Retrieves the Git commit hash of the qbruntime.

**Returns**

A string containing the Git hash.

### getQbRuntimeVendor

```cpp
std::string getQbRuntimeVendor()
```

> [!WARNING]
>
> **Deprecated**
>
> This function returns nothing.

### getQbRuntimeProduct

```cpp
std::string getQbRuntimeProduct()
```

> [!WARNING]
>
> **Deprecated**
>
> This function returns nothing.

### setLogLevel

```cpp
void setLogLevel(LogLevel level)
```

### startTracingEvents

```cpp
bool startTracingEvents(const char *path)
```

Starts event tracing and prepares to save the trace log to a specified file.

The trace log is recorded in "Chrome Tracing JSON format," which can be viewed at [https://ui.perfetto.dev/](<https://ui.perfetto.dev/>).

The trace log is not written immediately; it is saved only when [stopTracingEvents()](<#stoptracingevents>) is called.

**Parameters**

- `path` *\[in\]* — The file path where the trace log should be stored.

**Returns**

True if tracing starts successfully, false otherwise.

### stopTracingEvents

```cpp
void stopTracingEvents()
```

Stops event tracing and writes the recorded trace log.

This function finalizes tracing and saves the collected trace data to the file specified when [startTracingEvents()](<#starttracingevents>) was called.

### getModelSummary

```cpp
std::string getModelSummary(const std::string &mxq_path)
```

Generates a structured summary of the specified MXQ model.

Returns an overview of the model contained in the MXQ file, including:

- Target NPU device
- Supported core modes and their associated cores
- The total number of model variants
- For each variant:

  - Input and output tensor shapes
  - A list of layers with their types, output shapes, and input layer indices

The summary is returned as a human-readable string in a table and is useful for inspecting model compatibility, structure, and input/output shapes.

**Parameters**

- `mxq_path` *\[in\]* — Path to the MXQ model file.

**Returns**

A formatted string containing the model summary.

### getAvailableDevices

```cpp
std::vector<std::string> getAvailableDevices()
```

Get the device names of the NPU devices detected on the system.

**Returns**

The detected device names (e.g. `"aries-rb"`, `"regulus-rb"`). Empty if no NPU device is detected.

### getAvailableDeviceNumbers

```cpp
std::vector<int> getAvailableDeviceNumbers()
```

Get the number of available NPU devices.

Equivalent to `getAvailableDeviceNumbers("auto")`.

**Returns**

The number of available NPU devices.

```cpp
std::vector<int> getAvailableDeviceNumbers(const std::string &device_name)
```

Get the numbers of available devices of the given device.

**Parameters**

- `device_name` *\[in\]* — The device name (`"auto"` or a name such as `"aries"`, `"aries-rb"`, `"regulus-rb"`, ...; case-insensitive).

**Returns**

The available device numbers. Empty if `device_name` cannot be resolved.

### internal::numel

```cpp
int64_t internal::numel(const std::vector<int64_t> &shape)
```
