Reference

Reference#

This chapter is a reference for the current qbcompiler command line, Python API, configuration schema, supported inputs, and related terms. The source of truth is the current qbcompiler implementation.

CLI Reference#

Run the CLI as:

python -m qbcompiler <command> [options]

The current command set is compile, parse, quantize, dump-config, info, check, presets, validate, and extract-body. The table below covers the five this manual builds its examples on; presets is described under Compile Configuration.

Command	Purpose	Main options
`compile`	End-to-end model to `.mxq`: parse to temporary `.mblt`, then quantize.	`--model`, `--output`, `--backend`, `--device`, `--target-device`, `--cpu-offload` / `--no-cpu-offload`, `--config-preset`, `--compile-config`, `--calib-data-path`, `--use-random-calib` / `--no-use-random-calib`, `--config-save-path`
`parse`	Convert an input model to `.mblt` intermediate IR.	`--model`, `--output`, `--backend`, `--device`, `--target-device`, `--cpu-offload` / `--no-cpu-offload`, `--config-preset`, `--compile-config`
`quantize`	Convert a runnable `.mblt` file to `.mxq`.	`--mblt`, `--output`, `--target-device`, `--calib-data-path`, `--use-random-calib` / `--no-use-random-calib`, `--config-preset`, `--compile-config`, `--device`, `--config-save-path`
`dump-config`	Write a default or preset-derived `CompileConfig` template.	`--output`, `--preset`
`info`	Print the provenance recorded in a `.mblt`.	`--mblt`

--backend defaults to onnx. --calib-data-path is repeatable. --compile-config accepts JSON, YAML, or YML and is the extension point for settings that do not have dedicated CLI flags. dump-config chooses JSON or YAML from the output file suffix.

For subprocess integrations, the CLI has an opt-in JSONL protocol. Set QBCOMPILER_JSONL=1, true, yes, or on to emit structured status, progress, result, log, and error messages on stdout. Without that environment variable, normal logs are written to stderr.

Python API Reference#

Public functions are available from qbcompiler.

API	Purpose
`mxq_compile(...)`	Main Python entry point. Compiles a model or runnable `.mblt` to `.mxq`, delegating to one of the two functions below based on the input.
`mxq_compile_from_source(...)`	Parses a source model (ONNX / PyTorch / TensorFlow / TF Lite / TorchScript), then compiles it to `.mxq`. Takes the same arguments as `mxq_compile`.
`mxq_compile_from_mblt(...)`	Compiles an existing `.mblt` to `.mxq`. Its first parameter is `mblt`, not `model`. The graph is already parsed, so the parser-only arguments (`save_subgraph_type`, `output_subgraph_path`, `feed_dict`, `dynamic_axes`, `in_dformats`, `yolo_decode_include`, `exclude_first_subgraph`, `model_part`, `model_part_options`) are absent.
`mxq_compile_with_callback(...)`	`mxq_compile` with a progress callback.
`mblt_compile_with_callback(...)`	`mblt_compile` with a progress callback. Takes its output path before its target device.
`read_provenance(mblt_path)`	Returns the provenance record of a `.mblt`. `None` when the file carries no record — written before provenance existed, or with recording off. Both container formats carry one otherwise.
`mblt_compile(...)`	Exports a model to `.mblt` without producing `.mxq`.
`list_presets()`	Returns preset metadata dictionaries.
`dump_default_config(output_path, preset=None)`	Writes a `CompileConfig` template to JSON/YAML.
`get_body_subgraph(mblt_path, output_path)`	Writes a `.mblt` containing the largest NPU-runnable subgraph, together with any subgraphs it calls. A subgraph that is only ever called is not chosen as the body.

The main mxq_compile parameters are:

from qbcompiler import mxq_compile

mxq_compile(
    model="model.onnx",
    target_device="aries-rb",
    calib_data_path="calib",
    save_path="model.mxq",
    backend="onnx",
    device="gpu",
    config_preset="classification",
    compile_config=None,
)

Important parameters include model, target_device, calib_data_path, save_path, backend, feed_dict, dynamic_axes, in_dformats, yolo_decode_include, exclude_first_subgraph, device, inference_scheme, use_random_calib, cpu_offload, layer_bias_correction, buffer_mode, input_shape_dict, force_npu_input_reposition, force_npu_output_reposition, image_channels, split_blocks, split_parts, config_preset, compile_config, config_save_path, model_part, model_part_options, and sub-config objects such as calibration_config, bit_config, llm_config, hessian_quant_config, layer_bias_correction_config, mod_config, equivalent_transformation_config, search_weight_scale_config, uint8_input_config, preprocessing_config, and save_sample_config.

config_save_path, model_part and model_part_options are new in qbcompiler 1.3:

config_save_path writes the fully resolved CompileConfig — the normalized configuration after every layer of the precedence order below has been applied — before compilation starts. A .yaml or .yml suffix writes YAML, any other suffix writes JSON, and parent directories are created. Because it is written up front, the record survives a compile that fails after it starts, and the file can be passed straight back as compile_config= to reproduce the compile. It is written by the quantize phase, so a compile that fails while parsing produces nothing. The CLI exposes it as --config-save-path on quantize and compile.
model_part names which part of a multi-part torch model to parse ("vision", "language", "encoder", and so on), and model_part_options passes extra arguments that part needs, such as {"mel_frames": 100}. A model declaring exactly one part resolves it from None; a model declaring several requires a name. List them with available_parts(model) from qbcompiler.model_dict.parser.patcher.parts. These replace the removed hf_config argument, which now raises ValueError.

Configuration resolution priority (highest wins): explicit arguments → sub-config objects → compile_config file or config_preset, whichever is given → defaults. Passing both uses the preset and ignores the file. For details and usage examples, see Compile Configuration — Config Resolution Priority.

CompileConfig Schema#

CompileConfig is a Pydantic model with extra="forbid" and aliases enabled. Use alias names in JSON/YAML files. A full template can be generated with:

python -m qbcompiler dump-config --output compile_config.yaml
python -m qbcompiler dump-config --preset yolo_640 --output yolo_640.yaml

Top-level fields:

Alias	Python field	Default	Meaning
`modelPaths`	`model_paths`	`[]`	Model path list. Usually supplied by API/CLI instead.
`calibDataPaths`	`calib_data_path`	`[]`	Calibration dataset path list.
`savePaths`	`save_paths`	`["./tmp.mxq"]`	MXQ output path list.
`useRandomCalib`	`use_random_calib`	`false`	Generate random calibration data.
`inferenceScheme`	`inference_scheme`	`single`	Compile-time NPU core-assignment scheme used by the generated MXQ. See inferenceScheme.
`cpuOffload`	`cpu_offload`	`false`	Enable CPU offloading for unsupported groups.
`forceNpuInputReposition`	`force_npu_input_reposition`	`false`	Force input reposition operations onto NPU.
`forceNpuOutputReposition`	`force_npu_output_reposition`	`false`	Force output reposition operations onto NPU.
`bufferMode`	`buffer_mode`	`1`	Buffer serialization mode.
`inputShapeDict`	`input_shape_dict`	`{}`	Multi-shape compile specification for supported models.
`device`	`device`	`gpu`	Compilation compute device: `gpu` or `cpu`.
`dtype`	`dtype`	`float`	Computation dtype metadata.
`debug`	`debug`	`false`	Enable debug mode.
`trace`	`trace`	`false`	Enable trace mode.
`imageChannels`	`image_channels`	`0`	Number of image channels; `0` means auto-detect.
`configVersion`	`config_version`	`1.0.0`	Config schema version.
`splitBlocks`	`split_blocks`	`[]`	LLM multi-MXQ split points by block index.
`splitParts`	`split_parts`	`0`	Split LLM transformer blocks into N parts.

Nested sections:

Alias	Type	Purpose
`uint8Input`	`Uint8InputConfig`	Treat model inputs as uint8, optionally by input name.
`preprocessing`	`PreprocessingConfig`	Input preprocessing pipeline such as resize, letterbox, crop, normalize, and format conversion.
`resourceManagement`	`ResourceManagementConfig`	Weight dtype, GPU calibration policy, and weight-memory method.
`calibration`	`CalibrationConfig`	Quantization method, output quantization, calibration mode, clipping/statistics, and layer overrides.
`bit`	`BitConfig`	Quantization precision settings.
`hessianQuant`	`HessianQuantConfig`	HessianQuant optimization settings, including `hessianDtype` and `accumulationDevice`.
`layerBiasCorrection`	`LayerBiasCorrectionConfig`	Calibration-derived per-channel bias correction folded into integer convolution biases.
`mod`	`ModConfig`	MOD training/optimization settings.
`llm`	`LlmConfig`	LLM sequence/cache/runtime/debug settings. If `runtime.batchSize` is greater than `1`, `inferenceScheme` must be `single`, `global4` or `global8` — `multi` and `all` are rejected — and the Batch LLM runtime API uses `BatchParam` metadata; see qb Runtime Release Notes.
`moe`	`MoeConfig`	Sparse MoE calibration expert selection.
`equivalentTransformation`	`EquivalentTransformationConfig`	SmoothQuant-like, rotation, and FFN equivalent transformations.
`searchWeightScale`	`SearchWeightScaleConfig`	Weight-scale search.
`loadScale`	`LoadScaleConfig`	Load externally supplied scale entries.
`runtimeOptions`	`RuntimeOptions`	Internal runtime metadata.
`saveSample`	`SaveSampleConfig`	Save sample data for inspection/debugging.

CompileConfig.from_file() accepts .json, .yaml, and .yml. For compatibility, it also flattens grouped keys such as quantization.calibration, quantization.bit, advancedQuantization.hessianQuant, advancedQuantization.mod, advancedQuantization.EquivalentTransformation, advancedQuantization.searchWeightScale, advancedQuantization.loadScale, and advancedQuantization.layerBiasCorrection, which 1.3 adds. That grouped spelling is also the one the tool writes — dump-config and --config-save-path both emit it. Flattening overwrites rather than merges, so a top-level calibration block added beside a quantization.calibration one is discarded with no message: do not mix the two spellings in one file.

inferenceScheme#

inferenceScheme selects how NPU core work is assigned when qb Runtime uses the MXQ for inference. Because the compiled MXQ is prepared for the selected core-assignment scheme, an MXQ built only for one mode cannot later be switched arbitrarily to another mode at runtime. One built with all carries every mode the model and target support, but the caller must name one: qb Runtime’s default CoreMode::Auto accepts only an MXQ with exactly one mode, so an all build needs an explicit setSingleCoreMode() or setGlobal4CoreMode() — see ARIES ModelConfig Configuration.

Available choices are single, multi, global4, global8, and all. single uses independent Local Cores, multi is the cluster-level 4-batch mode, and global4/global8 use 4 or 8 Local Cores together for one input. all prepares the MXQ for every core mode supported by the selected model and target. Support can vary by model, target, and compiler version. REGULUS targets have a single NPU core, so MXQ files compiled for REGULUS can use only single mode. From qbcompiler 1.3 this is enforced rather than left to fail later: multi, global4 and global8 are rejected outright on a single-core target, and all narrows to single there. The check runs before quantization regardless of whether the scheme or the target device was set first. global is a legacy/compatibility value; use global4 or global8 for new settings.

If llm.attributes.runtime.batchSize is greater than 1, LLM/KV-cache transformer compilation accepts single, global4 and global8, and rejects multi and all rather than narrowing them — each segment compiles under exactly one scenario, and guessing which one was meant would build for a core set nobody asked for. Vision batch inference and LLM batch inference do not use the same core-mode rule.

For what each mode means, see ARIES Core Mode. For runtime core/cluster selection, see ARIES ModelConfig Configuration.

RuntimeOptions#

RuntimeOptions currently contains only:

Field	Default	Meaning
`version`	`0.0.0`	Compiler/runtime version metadata.

This section is runtime metadata rather than a normal user tuning surface. Prefer the compile, calibration, preprocessing, LLM, and optimization sections for user-controlled behavior.

Preset List#

Preset	Extends	Behavior
`classification`	none	Image classification defaults: `calibration.mode=1`, `calibration.output=0`.
`detection`	none	Object detection defaults: `calibration.mode=1`, `calibration.output=1`.
`classification_torchvision`	`classification`	Enables uint8 input and standard Torchvision preprocessing: resize shortest side to 256, center crop 224x224, normalize ImageNet mean/std, `imageChannels=3`.
`yolo_640`	`detection`	Enables uint8 input and 640x640 letterbox preprocessing with pad value 114, `imageChannels=3`.
`yolo_1280`	`detection`	Enables uint8 input and 1280x1280 letterbox preprocessing with pad value 114, `imageChannels=3`.
`llm`	none	Enables LLM config with `maxSequenceLength=4096`, `maxCacheLength=4096`, `calibration.mode=0`, `calibration.output=0`, full-sequence-length LLM calibration, and the QK, UD, VO, SpinR1, SpinR2 and OptimizeFFN equivalent transformations.
`llm_fast`	`llm`	Turns off those six equivalent transformations and full-sequence-length calibration, trading accuracy for compile time.
`vision_transformer`	none	Transformer-oriented calibration/bit settings.
`multimodal`	none	Enables LLM handling and uses `calibration.method=3`.

Supported Frameworks#

The public backend strings are:

Backend	Input
`onnx`	ONNX model path. This is the primary path and is used by `validate_model`.
`tf`	TensorFlow SavedModel or supported TensorFlow/Keras model path.
`tflite`	TensorFlow Lite model path.
`torchscript`	TorchScript module/model.
`torch`	PyTorch model or HuggingFace/transformers model path/object flows. HuggingFace LLMs use this backend path, not a separate `hf` backend.

Supported Target Device#

Use these target device strings in CLI and Python API calls:

target device	Meaning
`regulus-ra`	REGULUS RA target.
`aries-rb`	ARIES RB target.
`regulus-rb`	REGULUS RB target.
`regulus-rb-usb`	REGULUS RB target connected over USB. From qbcompiler 1.3.

The validator accepts the public strings above. Internal enum names use underscores, but user-facing values use hyphens.

Supported Operators#

Mobilint IR Operations List lists the operations Mobilint hardware runs natively. It is not the whole answer: an operation outside that list may still compile through a graph-level transformation, and whether any given one runs natively also depends on the model, backend, shapes and target device, which the parser and target-device allocation logic settle per compile.

Available reference paths:

For non-ONNX backends, parse/compile the model and inspect unsupported groups with .mblt and CPU offloading workflows.
If a model contains unsupported groups, cpuOffload / cpu_offload can partition those groups for CPU execution where the runtime flow supports it.

Glossary#

Term	Meaning
`MXQ`	Mobilint executable package produced by quantization/compilation and run on Mobilint NPUs.
`MBLT`	Mobilint intermediate model format used between parsing and MXQ generation.
`parse`	Convert a source model into `.mblt`.
`quantize`	Convert a runnable `.mblt` into `.mxq` using calibration/config settings.
`compile`	End-to-end parse plus quantize flow.
`backend`	Source model framework identifier such as `onnx`, `torch`, or `tflite`.
`target_device`	Mobilint NPU target device string such as `aries-rb`.
NPU Chip	Mobilint NPU product name such as ARIES or REGULUS.
target device	Compile target such as `aries-rb`, `regulus-ra`, `regulus-rb`, or `regulus-rb-usb`; used as the target device string.
`calibration data`	Representative input tensors used to derive quantization scales/statistics.
`CPU offloading`	Partitioning unsupported graph groups for CPU execution while supported body subgraphs run on NPU.
`body subgraph`	The largest NPU-runnable supported subgraph extracted from a partitioned `.mblt`.
`preset`	Built-in partial `CompileConfig` for common model families.

License#

See License for the open-source license notices included with this manual.