Release Notes

Release Notes#

v1.3.1#

Release date: July 9, 2026 Type: Minor

16-bit integer support, SIMD level selection, a configurable NPU timeout, and reliability and performance improvements.

Added#

  • 16-bit integer support — Runtime now supports 16-bit integers as an internal data type. Model input and output data types are not affected.

  • SIMD level selection — Scale and transpose operations now support AVX-512. By default, qb Runtime selects the fastest SIMD level the system supports; set the QBRUNTIME_SIMD_LEVEL environment variable (auto, avx512, avx2, or sse2) to override it.

  • Configurable NPU timeout — Set the QBRUNTIME_NPU_TIMEOUT_MS environment variable to control how long qb Runtime waits for the NPU before reporting a timeout.

Revised#

  • inferSpeedrun no longer crashes when used with models that accept variable-length input.

  • Fixed issues affecting the inferAsync API.

  • Model::dispose() no longer waits 3 seconds when Model::releaseBuffer() was not called.

  • Improved inference and data-transfer performance on Linux.

Known Issues#

  • Large models on Windows — Some large models, including 7B LLMs, may fail to run on Windows. A fix is in progress and planned for v1.3.2.

v1.2.0#

Release date: April 2, 2026 Type: Minor

Adds Batch LLM support.

Added#

  • BatchParam — a new struct BatchParam for Batch LLM inference. It holds the per-batch information needed during inference:

    • sequence_length : the sequence length for each batch.

    • cache_size : the cache size each batch will use.

    • cache_id : the cache identifier for each batch. All inputs in the same context must share one cache ID, and the value must be within the model’s maximum batch count.

    To run Batch LLM, concatenate multiple inputs into a single input — along the seq_len dimension when the shape is (1, seq_len, hidden_dim) — then pass a BatchParam for each input:

    import qbruntime
import numpy as np

## Check the maximum batch count supported by the model.
print(model.get_cache_infos()[0].num_batches)

## Concatenate inputs along the 2nd dimension (axis=1).
batch_input = np.concatenate([input0, input1], axis=1)

## qbruntime.BatchParam(sequence_length, cache_size, cache_id)
batch_params = [
    qbruntime.BatchParam(10, 0, 0),
    qbruntime.BatchParam(80, 0, 1),
]
res = model.infer([batch_input], params=batch_params)

batch_params2 = [
    qbruntime.BatchParam(1, 10, 0),
    qbruntime.BatchParam(1, 80, 1),
]
res = model.infer(res, params=batch_params2)

Known Issues#

  • Running LLM models on ARM (aarch64) systems may fail with a “Bus Error”. Present since v1.1.0; a driver patch is planned.

v1.1.0#

Release date: March 23, 2026 Type: Minor

Automatic core-mode selection, data-type query APIs, and performance optimizations.

Added#

  • CoreMode::Auto — the runtime auto-selects the available core mode from the MXQ. Set CoreMode::Auto in your ModelConfig (the default constructor already uses it), so non-default modes such as Multi, Global4, and Global8 no longer need manual construction. See setAutoCoreMode().

  • getModelInputDataType() / getModelOutputDataType() — query a model’s input and output data types at runtime.

  • getAvailableDeviceNumbers() — retrieve the list of available NPU device numbers.

Note

If the MXQ was compiled with a flag like scheme="all" that produces multiple core modes, you must still select the core mode manually.

Revised#

  • REGULUS now uses the dynamic-allocation approach introduced in v1.0.0, for a consistent usage pattern.

  • Improved data-transfer performance to NPU devices on Windows.

  • Optimized internal type conversion.

  • Fixed a compile error caused by std::filesystem on GCC versions below 9.

  • Fixed an intermittent deadlock in certain models.

  • [Breaking] The supported REGULUS driver revision changes from REV0 to REV1.

Known Issues#

  • Running LLM models on ARM (aarch64) systems may fail with a “Bus Error”. A driver patch is planned.

See also

For the complete changelog, see the Changelog page.

v1.0.0#

Release date: January 31, 2026 Type: Major

A major release focused on scalability, consistency, and a structural refactor for future expansion. To upgrade, follow the Migration Guide.

Added#

  • uint8 inference — uint8 quantized models can be compiled with qb Compiler and executed by qb Runtime, reducing CPU overhead during preprocessing for models with uint8 inputs.

  • Activation slots — setActivationSlots(int num) (C++) and set_activation_slots(num) (Python) tune pipelining between NPU inference and data transfer. More slots use more NPU memory but improve throughput in multithreaded workloads.

Note

For models that use cache (e.g., LLMs), the activation slot count is currently limited to 1.

Revised#

  • [Highlight] Model-count limit removed — models compiled with the latest qb Compiler (MXQv7) load and run concurrently within available DRAM, regardless of compile-time core mode. This helps multi-model services, mixed core-mode execution, and large models such as LLMs, with no code changes.

  • [Breaking] SDK qb naming unified — runtime library maccel → qb Runtime, compiler qubee → qb Compiler. Packages, headers, and module names changed accordingly.

Removed#

  • Legacy packages (mobilint-npu-runtime, aries-driver) are no longer maintained. See the Migration Guide.