Build AI inference on Mobilint NPUs

Build AI inference on Mobilint NPUs#

qb Runtime gives developers, engineers, and researchers the building blocks to run AI inference on Mobilint NPUs

Tip

qb Runtime v1.5.0 released — individual KV cache control APIs, more accurate input quantization for asymmetrically quantized models, and NPUData fixes. See the release notes.

Important

Driver installation, firmware update, and NPU core configuration guides for ARIES devices have moved to the ARIES manual. This manual covers the qb Runtime and its utilities.

What is qb Runtime?#

qb Runtime is the inference engine of SDK qb, Mobilint’s software development kit for its Neural Processing Unit (NPU). It loads models compiled into Mobilint’s executable format (MXQ) by the qb Compiler and runs them on the NPU using internal optimized logic.

Runtime Execution

C++ and Python interfaces: Integrate NPU inference into your application with the runtime library’s C++ and Python APIs.
Linux and Windows support: Deploy on either operating system for easier system integration.

Precompiled Models#

Mobilint publishes NPU-ready model files on Hugging Face. Inference code for vision, Transformers, and MeloTTS models is maintained in separate packages.

Find model files and inference packages.

Get Started#

Go from installing the product to maximizing its performance with advanced functions.

Installation and first inference

Tip

Have questions or need custom deployment support? Contact us at https://discuss.mobilint.com.