# Overview

Build AI inference on Mobilint NPUs. qb Runtime gives developers, engineers, and researchers the building blocks to run AI inference on Mobilint NPUs.

- [Get Started](<#get-started>)
- [API Reference](<https://docs.mobilint.com/runtime/en/v1.2/api/cpp/index.md>)

> [!TIP]
>
> **qb Runtime v1.5.0** released — individual KV cache control APIs, more accurate input quantization for asymmetrically quantized models, and `NPUData` fixes. See the [release notes](<https://docs.mobilint.com/runtime/en/v1.2/release_note/index.md>).

> [!NOTE]
>
> **Important**
>
> Driver installation, firmware update, and NPU core configuration guides for ARIES devices have moved to the [ARIES manual](</aries/en/latest/>). This manual covers the qb Runtime and its utilities.

## What is qb Runtime?

**qb Runtime** is the inference engine of SDK qb, Mobilint's software development kit for its Neural Processing Unit (NPU). It loads models compiled into Mobilint's executable format (**MXQ**) by the [qb Compiler](</compiler/en/latest/>) and runs them on the NPU using internal optimized logic.

![Runtime Execution](<https://docs.mobilint.com/runtime/images/SDK_Runtime_Execution.png>)

- **C++ and Python interfaces**: Integrate NPU inference into your application with the runtime library's C++ and Python APIs.
- **Linux and Windows support**: Deploy on either operating system for easier system integration.

## Precompiled Models

Mobilint publishes NPU-ready model files on [Hugging Face](<https://huggingface.co/mobilint>). Inference code for vision, Transformers, and MeloTTS models is maintained in separate packages.

- [Find model files and inference packages](<https://docs.mobilint.com/runtime/en/v1.2/model_zoo/index.md>).

## Get Started

Go from installing the product to maximizing its performance with advanced functions.

## Installation

> [!NOTE]
>
> **Important**
>
> - The installation steps below apply to **ARIES-based devices only**. Driver, firmware, and hardware setup for ARIES products live in the [ARIES manual](</aries/en/latest/>).
> - **REGULUS** devices ship with the essential SDK stack (driver, runtime library) and utilities pre-installed. Proceed directly to the **Tutorials** section below.

Install the SDK software required to use an ARIES NPU by following these documents in order.

1. [Driver Installation (ARIES manual)](</aries/en/latest/driver-installation/>): The kernel driver that exposes the NPU to your operating system.
2. [Installing Runtime Library](<https://docs.mobilint.com/runtime/en/v1.2/installing_runtime_library/index.md>): The C++ and Python runtime for loading models and running inference.
3. [Checking Compatibility](<https://docs.mobilint.com/runtime/en/v1.2/compatibility/index.md>): Verify that your runtime and MXQ model versions match.

> [!NOTE]
>
> You need a [Download Center](<https://dl.mobilint.com>) account to access and download some files and modules. Please contact contact@mobilint.com for more information.

### Optional Setup

These steps are not required for a standard setup — perform them only when needed.

- [Firmware Update (ARIES manual)](</aries/en/latest/firmware-update/>): Needed only when the compatibility check reports a device firmware older than the version the SDK requires.

> [!NOTE]
>
> Compiling your own models into MXQ is covered in the separate [qb Compiler documentation](</compiler/en/latest/>) and is not part of this manual. To run models right away, find available model files and inference packages in the [precompiled models guide](<https://docs.mobilint.com/runtime/en/v1.2/model_zoo/index.md>).

## Tutorials

After installing all essential SDK components required for NPU operation, refer to the following tutorials to learn how to successfully run AI inference on a Mobilint NPU.

- [Running Pre-Compiled ResNet50](<https://docs.mobilint.com/runtime/en/v1.2/tutorial_resnet50/index.md>): Run your first inference with a ready-made model.
- [NPU Programming Guide](<https://docs.mobilint.com/runtime/en/v1.2/programming_guide/index.md>): The core runtime API and the inference workflow.

## Utilities

Explore the utility tools provided by Mobilint. These help you monitor the NPU's status or run tests. If you want to use an open-source pre-trained model off the shelf, visit the [Mobilint, Inc. Hugging Face page](<https://huggingface.co/mobilint>) to check available model files.

- [Installing Utility](<https://docs.mobilint.com/runtime/en/v1.2/installing_utility/index.md>): Install Mobilint's command-line tools.
- [Utility Usage](<https://docs.mobilint.com/runtime/en/v1.2/utility_usage/index.md>): Monitor NPU status and run diagnostics.
- [Precompiled models](<https://docs.mobilint.com/runtime/en/v1.2/model_zoo/index.md>): Model files on Hugging Face and task-specific inference packages.

## Advanced

Once you get the hang of the system, follow the Advanced Usage sections to further optimize your NPU usage or refer to the advanced tutorials.

- [Advanced Usage](<https://docs.mobilint.com/runtime/en/v1.2/advanced_usage/index.md>): Optimize throughput, batching, and memory.
- [Utilizing Multi-Core Modes (ARIES manual)](</aries/en/latest/core-mode/>): Distribute inference across NPU cores.

## Deployment

To deploy NPU workloads on a Kubernetes cluster, install the device plugin so that Pods can request NPUs as a schedulable resource.

- [Mobilint Device Plugin](<https://docs.mobilint.com/runtime/en/v1.2/kubernetes_device_plugin/index.md>): Schedule NPUs as resources on a Kubernetes cluster.

> [!TIP]
>
> Have questions or need custom deployment support? Contact us at [https://discuss.mobilint.com](<https://discuss.mobilint.com>).
