# Reference

This chapter is a reference for the current qbcompiler command line, Python API, configuration schema, supported inputs, and related terms. The source of truth is the current qbcompiler implementation.

## CLI Reference

Run the CLI as:

```bash
python -m qbcompiler <command> [options]
```

The current command set is `compile`, `parse`, `quantize`, `dump-config`, `info`, `check`, `presets`, `validate`, and `extract-body`. The table below covers the five this manual builds its examples on; `presets` is described under [Compile Configuration](compile_configuration.md).

| Command | Purpose | Main options |
| --- | --- | --- |
| `compile` | End-to-end model to `.mxq`: parse to temporary `.mblt`, then quantize. | `--model`, `--output`, `--backend`, `--device`, `--target-device`, `--cpu-offload` / `--no-cpu-offload`, `--config-preset`, `--compile-config`, `--calib-data-path`, `--use-random-calib` / `--no-use-random-calib`, `--config-save-path` |
| `parse` | Convert an input model to `.mblt` intermediate IR. | `--model`, `--output`, `--backend`, `--device`, `--target-device`, `--cpu-offload` / `--no-cpu-offload`, `--config-preset`, `--compile-config` |
| `quantize` | Convert a runnable `.mblt` file to `.mxq`. | `--mblt`, `--output`, `--target-device`, `--calib-data-path`, `--use-random-calib` / `--no-use-random-calib`, `--config-preset`, `--compile-config`, `--device`, `--config-save-path` |
| `dump-config` | Write a default or preset-derived `CompileConfig` template. | `--output`, `--preset` |
| `info` | Print the provenance recorded in a `.mblt`. | `--mblt` |

`--backend` defaults to `onnx`. `--calib-data-path` is repeatable. `--compile-config` accepts JSON, YAML, or YML and is the extension point for settings that do not have dedicated CLI flags. `dump-config` chooses JSON or YAML from the output file suffix.

For subprocess integrations, the CLI has an opt-in JSONL protocol. Set `QBCOMPILER_JSONL=1`, `true`, `yes`, or `on` to emit structured `status`, `progress`, `result`, `log`, and `error` messages on stdout. Without that environment variable, normal logs are written to stderr.

## Python API Reference

Public functions are available from `qbcompiler`.

| API | Purpose |
| --- | --- |
| `mxq_compile(...)` | Main Python entry point. Compiles a model or runnable `.mblt` to `.mxq`, delegating to one of the two functions below based on the input. |
| `mxq_compile_from_source(...)` | Parses a source model (ONNX / PyTorch / TensorFlow / TF Lite / TorchScript), then compiles it to `.mxq`. Takes the same arguments as `mxq_compile`. |
| `mxq_compile_from_mblt(...)` | Compiles an existing `.mblt`, or a list of `.mblt` files that are one model at different sizes, to `.mxq`. Its first parameter is `mblt`, not `model`. The graph is already parsed, so the parser-only arguments (`save_subgraph_type`, `output_subgraph_path`, `feed_dict`, `dynamic_axes`, `multi_shape`, `yolo_decode_include`, `exclude_first_subgraph`) are absent; if you pass any of them, a warning names them and they are ignored. `model_part` and `model_part_options` also apply only to parsing: pass them to `mblt_compile()` when producing the `.mblt`. `mxq_compile()` given a `.mblt` together with either of them raises `ValueError`. |
| `mxq_compile_with_callback(...)` | `mxq_compile` with a progress callback. |
| `mblt_compile_with_callback(...)` | `mblt_compile` with a progress callback. Takes its output path before its target device. |
| `read_provenance(mblt_path)` | Returns the provenance record of a `.mblt`. `None` when the file carries no record — written before provenance existed, or with recording off. Both container formats carry one otherwise. |
| `mblt_compile(...)` | Exports a model to `.mblt` without producing `.mxq`. |
| `list_presets()` | Returns preset metadata dictionaries. |
| `dump_default_config(output_path, preset=None)` | Writes a `CompileConfig` template to JSON/YAML. |
| `get_body_subgraph(mblt_path, output_path)` | Writes a `.mblt` containing the largest NPU-runnable subgraph, together with any subgraphs it calls. A subgraph that is only ever called is not chosen as the body. |
| `set_log_level(level)` | Sets the log level of every `qbcompiler` logger at once, as a `logging` level or its name such as `"debug"` or `"warning"`. New in qbcompiler 1.4. |

The main `mxq_compile` parameters are:

```python
from qbcompiler import mxq_compile

mxq_compile(
    model="model.onnx",
    target_device="aries-rb",
    calib_data_path="calib",
    save_path="model.mxq",
    backend="onnx",
    device="gpu",
    config_preset="classification",
    compile_config=None,
)
```

Important parameters include `model`, `target_device`, `calib_data_path`, `save_path`, `backend`, `feed_dict`, `dynamic_axes`, `multi_shape`, `yolo_decode_include`, `exclude_first_subgraph`, `device`, `inference_scheme`, `use_random_calib`, `cpu_offload`, `optimize_option`, `bias_correction`, `buffer_mode`, `force_npu_input_reposition`, `force_npu_output_reposition`, `image_channels`, `split_blocks`, `split_parts`, `config_preset`, `compile_config`, `config_save_path`, `model_part`, `model_part_options`, and sub-config objects such as `calibration_config`, `bit_config`, `llm_config`, `hessian_quant_config`, `bias_correction_config`, `mod_config`, `equivalent_transformation_config`, `search_weight_scale_config`, `uint8_input_config`, `preprocessing_config`, `save_sample_config`, `resource_management_config`, and the keyword-only `extra_output_config`.

`multi_shape` and `extra_output_config` are new in qbcompiler 1.4:

- `multi_shape` compiles one model at several sizes of one input dimension in a single call. It is keyed by input name like `feed_dict`, and each entry names the axis and the sizes it takes, for example `{"x": {"axis": 3, "values": [100, 200, 300]}}` or `{"x": {"axis": (2, 3), "values": [(224, 224), (448, 448)]}}` for axes that move together. The input axis named by `multi_shape.axis` must already be marked as a dynamic axis when the model is exported to ONNX.

  ```python
  widths = (100, 200, 300)

  mxq_compile(
      model="model.onnx",
      backend="onnx",
      target_device="aries-rb",
      feed_dict={"x": example},
      multi_shape={"x": {"axis": 3, "values": widths}},
      calib_data_path=[f"calib/w{w}" for w in widths],
      save_path="model.mxq",
  )
  ```

  `multi_shape` requires `feed_dict`. It resizes the named axis of its tensors to create an example tensor for each input size, and parses the model once per size. Several inputs are walked together by index, so their `values` lists must be the same length. `multi_shape` cannot be combined with the `dynamic_axes` argument. `mblt_compile()` writes one `.mblt` per size: pass a list of `mblt_save_path` values, or one path to get `<stem>_<i>.mblt`. When an original model is passed with `multi_shape`, `mxq_compile()` creates one `.mblt` per input size in a temporary directory, runs quantization once, and packs the results into one `.mxq`. Alternatively, pass the paths of existing `.mblt` files as `model`. Because those files are already parsed, parser arguments such as `feed_dict` and `multi_shape` do not apply and produce a warning if passed. Before quantization, the compiler instead compares the source-model information and graph structure recorded in the files, rejecting a set that does not represent the same model and reporting the difference. In both multi-shape cases, `calib_data_path` takes one calibration directory per size, paired by index with `values`.
- `extra_output_config` adds named intermediate layers to the MXQ's outputs. See [Compile Configuration — Exporting intermediate activations](compile_configuration.md#exporting-intermediate-activations).

qbcompiler 1.4 also removes the legacy `Model_Dict` parser and the arguments only it read. `qbcompiler.Model_Dict`, `qbcompiler.compiler.LegacyCompiler` and `qbcompiler.compiler.compiler_legacy` no longer exist; use `mxq_compile()` or `mblt_compile()`. Passing `in_dformats` or `input_shape_dict` raises `ValueError`. Input layouts are inferred from the model, and `multi_shape` replaces `input_shape_dict`. `layer_bias_correction` and `layer_bias_correction_config` are renamed to `bias_correction` and `bias_correction_config`.

`config_save_path`, `model_part` and `model_part_options` are new in qbcompiler 1.3:

- `config_save_path` writes the fully resolved `CompileConfig` — the normalized configuration after every layer of the precedence order below has been applied — before compilation starts. A `.yaml` or `.yml` suffix writes YAML, any other suffix writes JSON, and parent directories are created. Because it is written up front, the record survives a compile that fails after it starts, and the file can be passed straight back as `compile_config=` to reproduce the compile. It is written by the quantize phase, so a `compile` that fails while parsing produces nothing. The CLI exposes it as `--config-save-path` on `quantize` and `compile`.
- `model_part` names which part of a multi-part torch model to parse (`"vision"`, `"language"`, `"encoder"`, and so on), and `model_part_options` passes extra arguments that part needs, such as `{"mel_frames": 100}`. A model declaring exactly one part resolves it from `None`; a model declaring several requires a name. List them with `available_parts(model)` from `qbcompiler.model_dict.parser.patcher.parts`. These replace the removed `hf_config` argument, which now raises `ValueError`.

Configuration resolution priority (highest wins): explicit arguments → sub-config objects → `compile_config` file *or* `config_preset`, whichever is given → defaults. Passing both uses the preset and ignores the file. For details and usage examples, see [Compile Configuration — Config Resolution Priority](compile_configuration.md#config-resolution-priority).

## CompileConfig Schema

`CompileConfig` is a Pydantic model with `extra="forbid"` and aliases enabled. Use alias names in JSON/YAML files. A full template can be generated with:

```bash
python -m qbcompiler dump-config --output compile_config.yaml
python -m qbcompiler dump-config --preset yolo_640 --output yolo_640.yaml
```

Top-level fields:

| Alias | Python field | Default | Meaning |
| --- | --- | --- | --- |
| `modelPaths` | `model_paths` | `[]` | Model path list. Usually supplied by API/CLI instead. |
| `calibDataPaths` | `calib_data_path` | `[]` | Calibration dataset path list. |
| `savePaths` | `save_paths` | `["./tmp.mxq"]` | MXQ output path list. |
| `useRandomCalib` | `use_random_calib` | `false` | Generate random calibration data. |
| `inferenceScheme` | `inference_scheme` | `single` | Compile-time NPU core-assignment scheme used by the generated MXQ. See [inferenceScheme](#inferencescheme). |
| `cpuOffload` | `cpu_offload` | `false` | Enable CPU offloading for unsupported groups. |
| `forceNpuInputReposition` | `force_npu_input_reposition` | `false` | Force input reposition operations onto NPU. |
| `forceNpuOutputReposition` | `force_npu_output_reposition` | `false` | Force output reposition operations onto NPU. |
| `optimizeOption` | `optimize_option` | `1` | Back-end optimization selector. Accepted values are `0`, `1`, `2`, `3`, `4` and `99`. From qbcompiler 1.4 any other value is rejected during back-end compilation instead of being ignored. |
| `bufferMode` | `buffer_mode` | `1` | Buffer serialization mode. |
| `device` | `device` | `gpu` | Compilation compute device: `gpu` or `cpu`. |
| `dtype` | `dtype` | `float` | Computation dtype metadata. |
| `debug` | `debug` | `false` | Enable debug mode. |
| `trace` | `trace` | `false` | Enable trace mode. |
| `imageChannels` | `image_channels` | `0` | Number of image channels; `0` means auto-detect. |
| `configVersion` | `config_version` | `1.0.0` | Config schema version. |
| `splitBlocks` | `split_blocks` | `[]` | LLM multi-MXQ split points by block index. |
| `splitParts` | `split_parts` | `0` | Split LLM transformer blocks into N parts. |

Nested sections:

| Alias | Type | Purpose |
| --- | --- | --- |
| `uint8Input` | `Uint8InputConfig` | Treat model inputs as uint8, optionally by input name. |
| `preprocessing` | `PreprocessingConfig` | Input preprocessing pipeline such as resize, letterbox, crop, normalize, and format conversion. |
| `resourceManagement` | `ResourceManagementConfig` | Weight dtype, the GPU memory budget for weight quantization (`gpuMemoryBudgetMB`, in MiB; `-1` automatic by default, `0` unlimited), and weight-memory method. `useGPUOnlyForCalibration` was removed in 1.4. |
| `calibration` | `CalibrationConfig` | Quantization method, output quantization, calibration mode, clipping/statistics, and layer overrides. |
| `bit` | `BitConfig` | Quantization precision settings. |
| `hessianQuant` | `HessianQuantConfig` | HessianQuant optimization settings, including `hessianDtype`, `solver`, `rescomp` and `attributes.alpha`. `accumulationDevice` was removed in 1.4. See [Model Quantization](model_quantization.md). |
| `biasCorrection` | `BiasCorrectionConfig` | Corrects the per-channel bias that quantization introduces. On by default. Replaces `layerBiasCorrection` from 1.4. |
| `mod` | `ModConfig` | MOD training/optimization settings. |
| `llm` | `LlmConfig` | LLM sequence/cache/runtime/debug settings. If `runtime.batchSize` is greater than `1`, `inferenceScheme` must be `single`, `global4` or `global8` — `multi` and `all` are rejected — and the Batch LLM runtime API uses `BatchParam` metadata; see {external+runtime:doc}`qb Runtime Release Notes <en/release_note>`. |
| `moe` | `MoeConfig` | Sparse MoE calibration expert selection. |
| `equivalentTransformation` | `EquivalentTransformationConfig` | SmoothQuant-like, rotation, and FFN equivalent transformations. |
| `searchWeightScale` | `SearchWeightScaleConfig` | Weight-scale search. |
| `loadScale` | `LoadScaleConfig` | Load externally supplied scale entries. |
| `runtimeOptions` | `RuntimeOptions` | Internal runtime metadata. |
| `saveSample` | `SaveSampleConfig` | Save sample data for inspection/debugging. |
| `extraOutput` | `ExtraOutputConfig` | Add named intermediate layers to the MXQ's outputs (`apply`, `layers`). New in 1.4. |

`CompileConfig.from_file()` accepts `.json`, `.yaml`, and `.yml`. For compatibility, it also flattens grouped keys such as `quantization.calibration`, `quantization.bit`, `advancedQuantization.hessianQuant`, `advancedQuantization.mod`, `advancedQuantization.EquivalentTransformation`, `advancedQuantization.searchWeightScale`, `advancedQuantization.loadScale`, and `advancedQuantization.biasCorrection`, which replaces 1.3's `advancedQuantization.layerBiasCorrection` from 1.4. That grouped spelling is also the one the tool writes — `dump-config` and `--config-save-path` both emit it. Flattening overwrites rather than merges, so a top-level `calibration` block added beside a `quantization.calibration` one is discarded with no message: do not mix the two spellings in one file.

### inferenceScheme

`inferenceScheme` selects how NPU core work is assigned when qb Runtime uses the MXQ for inference. Because the compiled MXQ is prepared for the selected core-assignment scheme, an MXQ built only for one mode cannot later be switched arbitrarily to another mode at runtime. One built with `all` carries every mode the model and target support, but the caller must name one: qb Runtime's default `CoreMode::Auto` accepts only an MXQ with exactly one mode, so an `all` build needs an explicit `setSingleCoreMode()` or `setGlobal4CoreMode()` — see {external+aries:doc}`ARIES ModelConfig Configuration <modelconfig>`.

Available choices are `single`, `multi`, `global4`, `global8`, and `all`. `single` uses independent Local Cores, `multi` is the cluster-level 4-batch mode, and `global4`/`global8` use 4 or 8 Local Cores together for one input. `all` prepares the MXQ for every core mode supported by the selected model and target. Support can vary by model, target, and compiler version. REGULUS targets have a single NPU core, so MXQ files compiled for REGULUS can use only `single` mode. From qbcompiler 1.3 this is enforced rather than left to fail later: `multi`, `global4` and `global8` are rejected outright on a single-core target, and `all` narrows to `single` there. The check runs before quantization regardless of whether the scheme or the target device was set first. `global` is a legacy/compatibility value; use `global4` or `global8` for new settings.

If `llm.attributes.runtime.batchSize` is greater than `1`, LLM/KV-cache transformer compilation accepts `single`, `global4` and `global8`, and rejects `multi` and `all` rather than narrowing them — each segment compiles under exactly one scenario, and guessing which one was meant would build for a core set nobody asked for. Vision batch inference and LLM batch inference do not use the same core-mode rule.

For what each mode means, see {external+aries:doc}`ARIES Core Mode <core-mode>`. For runtime core/cluster selection, see {external+aries:doc}`ARIES ModelConfig Configuration <modelconfig>`.

## RuntimeOptions

`RuntimeOptions` currently contains only:

| Field | Default | Meaning |
| --- | --- | --- |
| `version` | `0.0.0` | Compiler/runtime version metadata. |

This section is runtime metadata rather than a normal user tuning surface. Prefer the compile, calibration, preprocessing, LLM, and optimization sections for user-controlled behavior.

## Preset List

| Preset | Extends | Behavior |
| --- | --- | --- |
| `classification` | none | Image classification defaults: `calibration.mode=1`, `calibration.output=0`. |
| `detection` | none | Object detection defaults: `calibration.mode=1`, `calibration.output=1`. |
| `classification_torchvision` | `classification` | Enables uint8 input and standard Torchvision preprocessing: resize shortest side to 256 with the Pillow backend (`backend: "pil"`), center crop 224x224, normalize ImageNet mean/std, `imageChannels=3`. |
| `yolo_640` | `detection` | Enables uint8 input and 640x640 letterbox preprocessing with pad value 114 and the OpenCV backend (`backend: "opencv"`), `imageChannels=3`. |
| `yolo_1280` | `detection` | Enables uint8 input and 1280x1280 letterbox preprocessing with pad value 114 and the OpenCV backend (`backend: "opencv"`), `imageChannels=3`. |

From qbcompiler 1.4 the three image presets use the Pillow and OpenCV backends to match their reference evaluators, so their calibration tensors and quantization results can differ from 1.3.
| `llm` | none | Enables LLM config with `maxSequenceLength=4096`, `maxCacheLength=4096`, `calibration.mode=0`, `calibration.output=0`, full-sequence-length LLM calibration, and the QK, UD, VO, SpinR1, SpinR2 and OptimizeFFN equivalent transformations. |
| `llm_fast` | `llm` | Turns off those six equivalent transformations and full-sequence-length calibration, trading accuracy for compile time. |
| `vision_transformer` | none | Transformer-oriented calibration/bit settings. |
| `multimodal` | none | Enables LLM handling and uses `calibration.method=3`. |

## Supported Frameworks

The public backend strings are below. Backend names are case-insensitive. From qbcompiler 1.4 every backend uses the current parser; `tf`, `tflite` and `torchscript` no longer go through the legacy parser.

| Backend | Input |
| --- | --- |
| `onnx` | ONNX model path. This is the primary path and is used by `validate_model`. |
| `tf` | TensorFlow SavedModel directory, Keras `.keras` / `.h5` file, or frozen GraphDef `.pb`. `tensorflow` and `keras` are accepted as other names for this backend. |
| `tflite` | TensorFlow Lite `.tflite` model path. |
| `torchscript` | Path of a `torch.jit.save` archive. It is exported to ONNX first, so `feed_dict` is required. |
| `torch` | PyTorch model or HuggingFace/transformers model path/object flows. HuggingFace LLMs use this backend path, not a separate `hf` backend. |

## Supported Target Device

Use these target device strings in CLI and Python API calls:

| target device | Meaning |
| --- | --- |
| `regulus-ra` | REGULUS RA target. |
| `aries-rb` | ARIES RB target. |
| `regulus-rb` | REGULUS RB target. |
| `regulus-rb-usb` | REGULUS RB target connected over USB. From qbcompiler 1.3. |

The validator accepts the public strings above. Internal enum names use underscores, but user-facing values use hyphens.

## Supported Operators

[Mobilint IR Operations List](supported.md#mobilint-ir-operations-list) lists the operations Mobilint hardware runs natively. It is not the whole answer: an operation outside that list may still compile through a graph-level transformation, and whether any given one runs natively also depends on the model, backend, shapes and target device, which the parser and target-device allocation logic settle per compile.

Available reference paths:

- For non-ONNX backends, parse/compile the model and inspect unsupported groups with `.mblt` and CPU offloading workflows.
- If a model contains unsupported groups, `cpuOffload` / `cpu_offload` can partition those groups for CPU execution where the runtime flow supports it.

## Glossary

| Term | Meaning |
| --- | --- |
| `MXQ` | Mobilint executable package produced by quantization/compilation and run on Mobilint NPUs. |
| `MBLT` | Mobilint intermediate model format used between parsing and MXQ generation. |
| `parse` | Convert a source model into `.mblt`. |
| `quantize` | Convert a runnable `.mblt` into `.mxq` using calibration/config settings. |
| `compile` | End-to-end parse plus quantize flow. |
| `backend` | Source model framework identifier such as `onnx`, `torch`, or `tflite`. |
| `target_device` | Mobilint NPU target device string such as `aries-rb`. |
| NPU Chip | Mobilint NPU product name such as ARIES or REGULUS. |
| target device | Compile target such as `aries-rb`, `regulus-ra`, `regulus-rb`, or `regulus-rb-usb`; used as the target device string. |
| `calibration data` | Representative input tensors used to derive quantization scales/statistics. |
| `CPU offloading` | Partitioning unsupported graph groups for CPU execution while supported body subgraphs run on NPU. |
| `body subgraph` | The largest NPU-runnable supported subgraph extracted from a partitioned `.mblt`. |
| `preset` | Built-in partial `CompileConfig` for common model families. |

## License

See [License](license.md) for the open-source license notices included with this manual.