# Release Notes

## v1.3.1

**Release date:** July 9, 2026
**Type:** Minor

16-bit integer support, SIMD level selection, a configurable NPU timeout, and reliability and performance improvements.

### Added

- **16-bit integer support** — Runtime now supports 16-bit integers as an internal data type. Model input and output data types are not affected.
- **SIMD level selection** — Scale and transpose operations now support AVX-512. By default, qb Runtime selects the fastest SIMD level the system supports; set the `QBRUNTIME_SIMD_LEVEL` environment variable (`auto`, `avx512`, `avx2`, or `sse2`) to override it.
- **Configurable NPU timeout** — Set the `QBRUNTIME_NPU_TIMEOUT_MS` environment variable to control how long qb Runtime waits for the NPU before reporting a timeout.

### Revised

- `inferSpeedrun` no longer crashes when used with models that accept variable-length input.
- Fixed issues affecting the `inferAsync` API.
- `Model::dispose()` no longer waits 3 seconds when `Model::releaseBuffer()` was not called.
- Improved inference and data-transfer performance on Linux.

### Known Issues

- **Large models on Windows** — Some large models, including 7B LLMs, may fail to run on Windows. A fix is in progress and planned for v1.3.2.

## v1.2.0

**Release date:** April 2, 2026
**Type:** Minor

Adds Batch LLM support.

### Added

- **BatchParam** — a new struct {doxylink}`BatchParam <mobilint::BatchParam>` for Batch LLM inference. It holds the per-batch information needed during inference:
    - `sequence_length` : the sequence length for each batch.
    - `cache_size` : the cache size each batch will use.
    - `cache_id` : the cache identifier for each batch. All inputs in the same context must share one cache ID, and the value must be within the model's maximum batch count.

  To run Batch LLM, concatenate multiple inputs into a single input — along the `seq_len` dimension when the shape is `(1, seq_len, hidden_dim)` — then pass a `BatchParam` for each input:

  ```python
  import qbruntime
  import numpy as np

  ## Check the maximum batch count supported by the model.
  print(model.get_cache_infos()[0].num_batches)

  ## Concatenate inputs along the 2nd dimension (axis=1).
  batch_input = np.concatenate([input0, input1], axis=1)

  ## qbruntime.BatchParam(sequence_length, cache_size, cache_id)
  batch_params = [
      qbruntime.BatchParam(10, 0, 0),
      qbruntime.BatchParam(80, 0, 1),
  ]
  res = model.infer([batch_input], params=batch_params)

  batch_params2 = [
      qbruntime.BatchParam(1, 10, 0),
      qbruntime.BatchParam(1, 80, 1),
  ]
  res = model.infer(res, params=batch_params2)
  ```

### Known Issues

- Running LLM models on ARM (aarch64) systems may fail with a "Bus Error". Present since v1.1.0; a driver patch is planned.

## v1.1.0

**Release date:** March 23, 2026
**Type:** Minor

Automatic core-mode selection, data-type query APIs, and performance optimizations.

### Added

- **`CoreMode::Auto`** — the runtime auto-selects the available core mode from the MXQ. Set `CoreMode::Auto` in your `ModelConfig` (the default constructor already uses it), so non-default modes such as `Multi`, `Global4`, and `Global8` no longer need manual construction. See {doxylink}`setAutoCoreMode() <mobilint::ModelConfig::setAutoCoreMode()>`.
- `getModelInputDataType()` / `getModelOutputDataType()` — query a model's input and output data types at runtime.
- `getAvailableDeviceNumbers()` — retrieve the list of available NPU device numbers.

```{note}
If the MXQ was compiled with a flag like `scheme="all"` that produces multiple core modes, you must still select the core mode manually.
```

### Revised

- REGULUS now uses the dynamic-allocation approach introduced in v1.0.0, for a consistent usage pattern.
- Improved data-transfer performance to NPU devices on Windows.
- Optimized internal type conversion.
- Fixed a compile error caused by `std::filesystem` on GCC versions below 9.
- Fixed an intermittent deadlock in certain models.
- **[Breaking]** The supported REGULUS driver revision changes from REV0 to REV1.

### Known Issues

- Running LLM models on ARM (aarch64) systems may fail with a "Bus Error". A driver patch is planned.

```{seealso}
For the complete changelog, see the [Changelog](CHANGELOG.md) page.
```

## v1.0.0

**Release date:** January 31, 2026
**Type:** Major

A major release focused on scalability, consistency, and a structural refactor for future expansion. To upgrade, follow the [Migration Guide](migration_guide.md).

### Added

- **uint8 inference** — uint8 quantized models can be compiled with qb Compiler and executed by qb Runtime, reducing CPU overhead during preprocessing for models with uint8 inputs.
- **Activation slots** — `setActivationSlots(int num)` (C++) and `set_activation_slots(num)` (Python) tune pipelining between NPU inference and data transfer. More slots use more NPU memory but improve throughput in multithreaded workloads.

```{note}
For models that use cache (e.g., LLMs), the activation slot count is currently limited to 1.
```

### Revised

- **[Highlight]** Model-count limit removed — models compiled with the latest qb Compiler (MXQv7) load and run concurrently within available DRAM, regardless of compile-time core mode. This helps multi-model services, mixed core-mode execution, and large models such as LLMs, with no code changes.
- **[Breaking]** SDK qb naming unified — runtime library `maccel` → **qb Runtime**, compiler `qubee` → **qb Compiler**. Packages, headers, and module names changed accordingly.

### Removed

- Legacy packages (`mobilint-npu-runtime`, `aries-driver`) are no longer maintained. See the [Migration Guide](migration_guide.md).