# Reference

This chapter is a reference for the current qbcompiler command line, Python API, configuration schema, supported inputs, and related terms. The source of truth is the current qbcompiler implementation.

## CLI Reference

Run the CLI as:

```bash
python -m qbcompiler <command> [options]
```

The current command set is `compile`, `parse`, `quantize`, and `dump-config`.

| Command | Purpose | Main options |
| --- | --- | --- |
| `compile` | End-to-end model to `.mxq`: parse to temporary `.mblt`, then quantize. | `--model`, `--output`, `--backend`, `--device`, `--target-device`, `--cpu-offload` / `--no-cpu-offload`, `--config-preset`, `--compile-config`, `--calib-data-path`, `--use-random-calib` / `--no-use-random-calib` |
| `parse` | Convert an input model to `.mblt` intermediate IR. | `--model`, `--output`, `--backend`, `--device`, `--target-device`, `--cpu-offload` / `--no-cpu-offload`, `--config-preset`, `--compile-config` |
| `quantize` | Convert a runnable `.mblt` file to `.mxq`. | `--mblt`, `--output`, `--calib-data-path`, `--use-random-calib` / `--no-use-random-calib`, `--config-preset`, `--compile-config`, `--device` |
| `dump-config` | Write a default or preset-derived `CompileConfig` template. | `--output`, `--preset` |

`--backend` defaults to `onnx`. `--calib-data-path` is repeatable. `--compile-config` accepts JSON, YAML, or YML and is the extension point for settings that do not have dedicated CLI flags. `dump-config` chooses JSON or YAML from the output file suffix.

For subprocess integrations, the CLI has an opt-in JSONL protocol. Set `QBCOMPILER_JSONL=1`, `true`, `yes`, or `on` to emit structured `status`, `progress`, `result`, `log`, and `error` messages on stdout. Without that environment variable, normal logs are written to stderr.

## Python API Reference

Public functions are available from `qbcompiler`.

| API | Purpose |
| --- | --- |
| `mxq_compile(...)` | Main Python entry point. Compiles a model or runnable `.mblt` to `.mxq`. |
| `mblt_compile(...)` | Exports a model to `.mblt` without producing `.mxq`. |
| `list_presets()` | Returns preset metadata dictionaries. |
| `dump_default_config(output_path, preset=None)` | Writes a `CompileConfig` template to JSON/YAML. |
| `get_body_subgraph(mblt_path, output_path)` | Writes a body-only `.mblt` containing the largest NPU-runnable subgraph. |
| `run_mblt_model(...)` | Runs `.mblt` inference for debug/inspection flows. |

The main `mxq_compile` parameters are:

```python
from qbcompiler import mxq_compile

mxq_compile(
    model="model.onnx",
    target_device="aries-rb",
    calib_data_path="calib",
    save_path="model.mxq",
    backend="onnx",
    device="gpu",
    config_preset="classification",
    compile_config=None,
)
```

Important parameters include `model`, `target_device`, `calib_data_path`, `save_path`, `backend`, `feed_dict`, `dynamic_axes`, `in_dformats`, `yolo_decode_include`, `exclude_first_subgraph`, `device`, `inference_scheme`, `use_random_calib`, `cpu_offload`, `optimize_option`, `buffer_mode`, `input_shape_dict`, `force_npu_input_reposition`, `force_npu_output_reposition`, `image_channels`, `split_blocks`, `split_parts`, `config_preset`, `compile_config`, and sub-config objects such as `calibration_config`, `bit_config`, `llm_config`, `hessian_quant_config`, `mod_config`, `equivalent_transformation_config`, `search_weight_scale_config`, `uint8_input_config`, `preprocessing_config`, and `save_sample_config`.

Configuration resolution priority (highest wins): explicit arguments → sub-config objects → `compile_config` file → `config_preset` → defaults. For details and usage examples, see [Compile Configuration — Config Resolution Priority](compile_configuration.md#config-resolution-priority).

## CompileConfig Schema

`CompileConfig` is a Pydantic model with `extra="forbid"` and aliases enabled. Use alias names in JSON/YAML files. A full template can be generated with:

```bash
python -m qbcompiler dump-config --output compile_config.yaml
python -m qbcompiler dump-config --preset yolo_640 --output yolo_640.yaml
```

Top-level fields:

| Alias | Python field | Default | Meaning |
| --- | --- | --- | --- |
| `modelPaths` | `model_paths` | `[]` | Model path list. Usually supplied by API/CLI instead. |
| `calibDataPaths` | `calib_data_path` | `[]` | Calibration dataset path list. |
| `savePaths` | `save_paths` | `["./tmp.mxq"]` | MXQ output path list. |
| `useRandomCalib` | `use_random_calib` | `false` | Generate random calibration data. |
| `saveMsgpackName` | `save_msgpack_name` | `null` | Optional msgpack output name. |
| `inferenceScheme` | `inference_scheme` | `single` | Compile-time NPU core-assignment scheme used by the generated MXQ. See [inferenceScheme](#inferencescheme). |
| `cpuOffload` | `cpu_offload` | `false` | Enable CPU offloading for unsupported groups. |
| `forceNpuInputReposition` | `force_npu_input_reposition` | `false` | Force input reposition operations onto NPU. |
| `forceNpuOutputReposition` | `force_npu_output_reposition` | `false` | Force output reposition operations onto NPU. |
| `optimizeOption` | `optimize_option` | `1` | Compiler optimization selector. |
| `bufferMode` | `buffer_mode` | `1` | Buffer serialization mode. |
| `inputShapeDict` | `input_shape_dict` | `{}` | Multi-shape compile specification for supported models. |
| `device` | `device` | `gpu` | Compilation compute device: `gpu` or `cpu`. |
| `dtype` | `dtype` | `float` | Computation dtype metadata. |
| `debug` | `debug` | `false` | Enable debug mode. |
| `trace` | `trace` | `false` | Enable trace mode. |
| `imageChannels` | `image_channels` | `0` | Number of image channels; `0` means auto-detect. |
| `configVersion` | `config_version` | `1.0.0` | Config schema version. |
| `splitBlocks` | `split_blocks` | `[]` | LLM multi-MXQ split points by block index. |
| `splitParts` | `split_parts` | `0` | Split LLM transformer blocks into N parts. |

Nested sections:

| Alias | Type | Purpose |
| --- | --- | --- |
| `uint8Input` | `Uint8InputConfig` | Treat model inputs as uint8, optionally by input name. |
| `preprocessing` | `PreprocessingConfig` | Input preprocessing pipeline such as resize, letterbox, crop, normalize, and format conversion. |
| `resourceManagement` | `ResourceManagementConfig` | Weight dtype, GPU calibration policy, and weight-memory method. |
| `calibration` | `CalibrationConfig` | Quantization method, output quantization, calibration mode, clipping/statistics, and layer overrides. |
| `bit` | `BitConfig` | Quantization precision settings. |
| `hessianQuant` | `HessianQuantConfig` | HessianQuant optimization settings. |
| `mod` | `ModConfig` | MOD training/optimization settings. |
| `llm` | `LlmConfig` | LLM sequence/cache/runtime/debug settings. If `runtime.batchSize` is greater than `1`, `inferenceScheme` is fixed to `single`, and the Batch LLM runtime API uses `BatchParam` metadata; see {external+runtime:doc}`qb Runtime Release Notes <en/release_note>`. |
| `moe` | `MoeConfig` | Sparse MoE calibration expert selection. |
| `equivalentTransformation` | `EquivalentTransformationConfig` | SmoothQuant-like, rotation, and FFN equivalent transformations. |
| `searchWeightScale` | `SearchWeightScaleConfig` | Weight-scale search. |
| `loadScale` | `LoadScaleConfig` | Load externally supplied scale entries. |
| `runtimeOptions` | `RuntimeOptions` | Internal runtime metadata. |
| `saveSample` | `SaveSampleConfig` | Save sample data for inspection/debugging. |

`CompileConfig.from_file()` accepts `.json`, `.yaml`, and `.yml`. For compatibility, it also flattens grouped keys such as `quantization.calibration`, `quantization.bit`, `advancedQuantization.hessianQuant`, `advancedQuantization.mod`, `advancedQuantization.EquivalentTransformation`, `advancedQuantization.searchWeightScale`, and `advancedQuantization.loadScale`.

### inferenceScheme

`inferenceScheme` selects how NPU core work is assigned when qb Runtime uses the MXQ for inference. Because the compiled MXQ is prepared for the selected core-assignment scheme, an MXQ built only for one mode cannot later be switched arbitrarily to another mode at runtime.

Available choices are `single`, `multi`, `global4`, `global8`, and `all`. `single` uses independent Local Cores, `multi` is the cluster-level 4-batch mode, and `global4`/`global8` use 4 or 8 Local Cores together for one input. `all` prepares the MXQ for every core mode supported by the selected model and target. Support can vary by model, target, and compiler version. REGULUS targets have a single NPU core, so MXQ files compiled for REGULUS can use only `single` mode. `global` is a legacy/compatibility value; use `global4` or `global8` for new settings.

If `llm.attributes.runtime.batchSize` is greater than `1`, LLM/KV-cache transformer compilation automatically fixes `inferenceScheme` to `single`. Vision batch inference and LLM batch inference do not use the same core-mode rule.

For what each mode means, see {external+aries:doc}`ARIES Core Mode <core-mode>`. For runtime core/cluster selection, see {external+aries:doc}`ARIES ModelConfig Configuration <modelconfig>`.

## RuntimeOptions

`RuntimeOptions` currently contains only:

| Field | Default | Meaning |
| --- | --- | --- |
| `version` | `0.0.0` | Compiler/runtime version metadata. |

This section is runtime metadata rather than a normal user tuning surface. Prefer the compile, calibration, preprocessing, LLM, and optimization sections for user-controlled behavior.

## Preset List

| Preset | Extends | Behavior |
| --- | --- | --- |
| `classification` | none | Image classification defaults: `calibration.mode=1`, `calibration.output=0`. |
| `detection` | none | Object detection defaults: `calibration.mode=1`, `calibration.output=1`. |
| `classification_torchvision` | `classification` | Enables uint8 input and standard Torchvision preprocessing: resize 256x256, center crop 224x224, normalize ImageNet mean/std, `imageChannels=3`. |
| `yolo_640` | `detection` | Enables uint8 input and 640x640 letterbox preprocessing with pad value 114, `imageChannels=3`. |
| `yolo_1280` | `detection` | Enables uint8 input and 1280x1280 letterbox preprocessing with pad value 114, `imageChannels=3`. |
| `llm` | none | Enables LLM config with `maxSequenceLength=4096`, `maxCacheLength=4096`, `calibration.mode=0`, `calibration.output=0`. |
| `llm_fast` | `llm` | Disables HessianQuant, enables selected equivalent transformations, and uses full sequence length for LLM calibration. |
| `vision_transformer` | none | Transformer-oriented calibration/bit settings. |
| `multimodal` | none | Enables LLM handling and uses `calibration.method=3`. |

## Supported Frameworks

The public backend strings are:

| Backend | Input |
| --- | --- |
| `onnx` | ONNX model path. This is the primary path and is used by `validate_model`. |
| `tf` | TensorFlow SavedModel or supported TensorFlow/Keras model path. |
| `tflite` | TensorFlow Lite model path. |
| `torchscript` | TorchScript module/model. |
| `torch` | PyTorch model or HuggingFace/transformers model path/object flows. HuggingFace LLMs use this backend path, not a separate `hf` backend. |

## Supported Target Device

Use these target device strings in CLI and Python API calls:

- `regulus-ra`
- `aries-rb`
- `regulus-rb`

The validator accepts the public strings above. Internal enum names use underscores, but user-facing values use hyphens.

## Supported Operators

qbcompiler does not currently expose a stable static supported-operator table in the manual source. Operator support is determined by the parser and target-device allocation logic for the specific model, backend, shapes, and target device.

Available reference paths:

- For non-ONNX backends, parse/compile the model and inspect unsupported groups with `.mblt` and CPU offloading workflows.
- If a model contains unsupported groups, `cpuOffload` / `cpu_offload` can partition those groups for CPU execution where the runtime flow supports it.

## Glossary

| Term | Meaning |
| --- | --- |
| `MXQ` | Mobilint executable package produced by quantization/compilation and run on Mobilint NPUs. |
| `MBLT` | Mobilint intermediate model format used between parsing and MXQ generation. |
| `parse` | Convert a source model into `.mblt`. |
| `quantize` | Convert a runnable `.mblt` into `.mxq` using calibration/config settings. |
| `compile` | End-to-end parse plus quantize flow. |
| `backend` | Source model framework identifier such as `onnx`, `torch`, or `tflite`. |
| `target_device` | Mobilint NPU target device string such as `aries-rb`. |
| NPU Chip | Mobilint NPU product name such as ARIES or REGULUS. |
| target device | Compile target such as `aries-rb`, `regulus-ra`, or `regulus-rb`; used as the target device string. |
| `calibration data` | Representative input tensors used to derive quantization scales/statistics. |
| `CPU offloading` | Partitioning unsupported graph groups for CPU execution while supported body subgraphs run on NPU. |
| `body subgraph` | The largest NPU-runnable supported subgraph extracted from a partitioned `.mblt`. |
| `preset` | Built-in partial `CompileConfig` for common model families. |

## License

See [License](license.md) for the open-source license notices included with this manual.