Skip to content

Reference

This chapter is a reference for the current qbcompiler command line, Python API, configuration schema, supported inputs, and related terms. The source of truth is the current qbcompiler implementation.

CLI Reference

Run the CLI as:

python -m qbcompiler <command> [options]

The current command set is compile, parse, quantize, dump-config, info, check, presets, validate, and extract-body. The table below covers the five this manual builds its examples on; presets is described under Compile Configuration.

Command Purpose Main options
compile End-to-end model to .mxq: parse to temporary .mblt, then quantize. --model, --output, --backend, --device, --target-device, --cpu-offload / --no-cpu-offload, --config-preset, --compile-config, --calib-data-path, --use-random-calib / --no-use-random-calib, --config-save-path
parse Convert an input model to .mblt intermediate IR. --model, --output, --backend, --device, --target-device, --cpu-offload / --no-cpu-offload, --config-preset, --compile-config
quantize Convert a runnable .mblt file to .mxq. --mblt, --output, --target-device, --calib-data-path, --use-random-calib / --no-use-random-calib, --config-preset, --compile-config, --device, --config-save-path
dump-config Write a default or preset-derived CompileConfig template. --output, --preset
info Print the provenance recorded in a .mblt. --mblt

--backend defaults to onnx. --calib-data-path is repeatable. --compile-config accepts JSON, YAML, or YML and is the extension point for settings that do not have dedicated CLI flags. dump-config chooses JSON or YAML from the output file suffix.

For subprocess integrations, the CLI has an opt-in JSONL protocol. Set QBCOMPILER_JSONL=1, true, yes, or on to emit structured status, progress, result, log, and error messages on stdout. Without that environment variable, normal logs are written to stderr.

Python API Reference

Public functions are available from qbcompiler.

API Purpose
mxq_compile(...) Main Python entry point. Compiles a model or runnable .mblt to .mxq, delegating to one of the two functions below based on the input.
mxq_compile_from_source(...) Parses a source model (ONNX / PyTorch / TensorFlow / TF Lite / TorchScript), then compiles it to .mxq. Takes the same arguments as mxq_compile.
mxq_compile_from_mblt(...) Compiles an existing .mblt to .mxq. Its first parameter is mblt, not model. The graph is already parsed, so the parser-only arguments (save_subgraph_type, output_subgraph_path, feed_dict, dynamic_axes, in_dformats, yolo_decode_include, exclude_first_subgraph, model_part, model_part_options) are absent.
mxq_compile_with_callback(...) mxq_compile with a progress callback.
mblt_compile_with_callback(...) mblt_compile with a progress callback. Takes its output path before its target device.
read_provenance(mblt_path) Returns the provenance record of a .mblt. None when the file carries no record — written before provenance existed, or with recording off. Both container formats carry one otherwise.
mblt_compile(...) Exports a model to .mblt without producing .mxq.
list_presets() Returns preset metadata dictionaries.
dump_default_config(output_path, preset=None) Writes a CompileConfig template to JSON/YAML.
get_body_subgraph(mblt_path, output_path) Writes a .mblt containing the largest NPU-runnable subgraph, together with any subgraphs it calls. A subgraph that is only ever called is not chosen as the body.

The main mxq_compile parameters are:

from qbcompiler import mxq_compile

mxq_compile(
    model="model.onnx",
    target_device="aries-rb",
    calib_data_path="calib",
    save_path="model.mxq",
    backend="onnx",
    device="gpu",
    config_preset="classification",
    compile_config=None,
)

Important parameters include model, target_device, calib_data_path, save_path, backend, feed_dict, dynamic_axes, in_dformats, yolo_decode_include, exclude_first_subgraph, device, inference_scheme, use_random_calib, cpu_offload, layer_bias_correction, buffer_mode, input_shape_dict, force_npu_input_reposition, force_npu_output_reposition, image_channels, split_blocks, split_parts, config_preset, compile_config, config_save_path, model_part, model_part_options, and sub-config objects such as calibration_config, bit_config, llm_config, hessian_quant_config, layer_bias_correction_config, mod_config, equivalent_transformation_config, search_weight_scale_config, uint8_input_config, preprocessing_config, and save_sample_config.

config_save_path, model_part and model_part_options are new in qbcompiler 1.3:

  • config_save_path writes the fully resolved CompileConfig — the normalized configuration after every layer of the precedence order below has been applied — before compilation starts. A .yaml or .yml suffix writes YAML, any other suffix writes JSON, and parent directories are created. Because it is written up front, the record survives a compile that fails after it starts, and the file can be passed straight back as compile_config= to reproduce the compile. It is written by the quantize phase, so a compile that fails while parsing produces nothing. The CLI exposes it as --config-save-path on quantize and compile.
  • model_part names which part of a multi-part torch model to parse ("vision", "language", "encoder", and so on), and model_part_options passes extra arguments that part needs, such as {"mel_frames": 100}. A model declaring exactly one part resolves it from None; a model declaring several requires a name. List them with available_parts(model) from qbcompiler.model_dict.parser.patcher.parts. These replace the removed hf_config argument, which now raises ValueError.

Configuration resolution priority (highest wins): explicit arguments → sub-config objects → compile_config file or config_preset, whichever is given → defaults. Passing both uses the preset and ignores the file. For details and usage examples, see Compile Configuration — Config Resolution Priority.

CompileConfig Schema

CompileConfig is a Pydantic model with extra="forbid" and aliases enabled. Use alias names in JSON/YAML files. A full template can be generated with:

python -m qbcompiler dump-config --output compile_config.yaml
python -m qbcompiler dump-config --preset yolo_640 --output yolo_640.yaml

Top-level fields:

Alias Python field Default Meaning
modelPaths model_paths [] Model path list. Usually supplied by API/CLI instead.
calibDataPaths calib_data_path [] Calibration dataset path list.
savePaths save_paths ["./tmp.mxq"] MXQ output path list.
useRandomCalib use_random_calib false Generate random calibration data.
inferenceScheme inference_scheme single Compile-time NPU core-assignment scheme used by the generated MXQ. See inferenceScheme.
cpuOffload cpu_offload false Enable CPU offloading for unsupported groups.
forceNpuInputReposition force_npu_input_reposition false Force input reposition operations onto NPU.
forceNpuOutputReposition force_npu_output_reposition false Force output reposition operations onto NPU.
bufferMode buffer_mode 1 Buffer serialization mode.
inputShapeDict input_shape_dict {} Multi-shape compile specification for supported models.
device device gpu Compilation compute device: gpu or cpu.
dtype dtype float Computation dtype metadata.
debug debug false Enable debug mode.
trace trace false Enable trace mode.
imageChannels image_channels 0 Number of image channels; 0 means auto-detect.
configVersion config_version 1.0.0 Config schema version.
splitBlocks split_blocks [] LLM multi-MXQ split points by block index.
splitParts split_parts 0 Split LLM transformer blocks into N parts.

Nested sections:

Alias Type Purpose
uint8Input Uint8InputConfig Treat model inputs as uint8, optionally by input name.
preprocessing PreprocessingConfig Input preprocessing pipeline such as resize, letterbox, crop, normalize, and format conversion.
resourceManagement ResourceManagementConfig Weight dtype, GPU calibration policy, and weight-memory method.
calibration CalibrationConfig Quantization method, output quantization, calibration mode, clipping/statistics, and layer overrides.
bit BitConfig Quantization precision settings.
hessianQuant HessianQuantConfig HessianQuant optimization settings, including hessianDtype and accumulationDevice.
layerBiasCorrection LayerBiasCorrectionConfig Calibration-derived per-channel bias correction folded into integer convolution biases.
mod ModConfig MOD training/optimization settings.
llm LlmConfig LLM sequence/cache/runtime/debug settings. If runtime.batchSize is greater than 1, inferenceScheme must be single, global4 or global8 — multi and all are rejected — and the Batch LLM runtime API uses BatchParam metadata; see qb Runtime Release Notes.
moe MoeConfig Sparse MoE calibration expert selection.
equivalentTransformation EquivalentTransformationConfig SmoothQuant-like, rotation, and FFN equivalent transformations.
searchWeightScale SearchWeightScaleConfig Weight-scale search.
loadScale LoadScaleConfig Load externally supplied scale entries.
runtimeOptions RuntimeOptions Internal runtime metadata.
saveSample SaveSampleConfig Save sample data for inspection/debugging.

CompileConfig.from_file() accepts .json, .yaml, and .yml. For compatibility, it also flattens grouped keys such as quantization.calibration, quantization.bit, advancedQuantization.hessianQuant, advancedQuantization.mod, advancedQuantization.EquivalentTransformation, advancedQuantization.searchWeightScale, advancedQuantization.loadScale, and advancedQuantization.layerBiasCorrection, which 1.3 adds. That grouped spelling is also the one the tool writes — dump-config and --config-save-path both emit it. Flattening overwrites rather than merges, so a top-level calibration block added beside a quantization.calibration one is discarded with no message: do not mix the two spellings in one file.

inferenceScheme

inferenceScheme selects how NPU core work is assigned when qb Runtime uses the MXQ for inference. Because the compiled MXQ is prepared for the selected core-assignment scheme, an MXQ built only for one mode cannot later be switched arbitrarily to another mode at runtime. One built with all carries every mode the model and target support, but the caller must name one: qb Runtime's default CoreMode::Auto accepts only an MXQ with exactly one mode, so an all build needs an explicit setSingleCoreMode() or setGlobal4CoreMode() — see ARIES ModelConfig Configuration.

Available choices are single, multi, global4, global8, and all. single uses independent Local Cores, multi is the cluster-level 4-batch mode, and global4/global8 use 4 or 8 Local Cores together for one input. all prepares the MXQ for every core mode supported by the selected model and target. Support can vary by model, target, and compiler version. REGULUS targets have a single NPU core, so MXQ files compiled for REGULUS can use only single mode. From qbcompiler 1.3 this is enforced rather than left to fail later: multi, global4 and global8 are rejected outright on a single-core target, and all narrows to single there. The check runs before quantization regardless of whether the scheme or the target device was set first. global is a legacy/compatibility value; use global4 or global8 for new settings.

If llm.attributes.runtime.batchSize is greater than 1, LLM/KV-cache transformer compilation accepts single, global4 and global8, and rejects multi and all rather than narrowing them — each segment compiles under exactly one scenario, and guessing which one was meant would build for a core set nobody asked for. Vision batch inference and LLM batch inference do not use the same core-mode rule.

For what each mode means, see ARIES Core Mode. For runtime core/cluster selection, see ARIES ModelConfig Configuration.

RuntimeOptions

RuntimeOptions currently contains only:

Field Default Meaning
version 0.0.0 Compiler/runtime version metadata.

This section is runtime metadata rather than a normal user tuning surface. Prefer the compile, calibration, preprocessing, LLM, and optimization sections for user-controlled behavior.

Preset List

Preset Extends Behavior
classification none Image classification defaults: calibration.mode=1, calibration.output=0.
detection none Object detection defaults: calibration.mode=1, calibration.output=1.
classification_torchvision classification Enables uint8 input and standard Torchvision preprocessing: resize shortest side to 256, center crop 224x224, normalize ImageNet mean/std, imageChannels=3.
yolo_640 detection Enables uint8 input and 640x640 letterbox preprocessing with pad value 114, imageChannels=3.
yolo_1280 detection Enables uint8 input and 1280x1280 letterbox preprocessing with pad value 114, imageChannels=3.
llm none Enables LLM config with maxSequenceLength=4096, maxCacheLength=4096, calibration.mode=0, calibration.output=0, full-sequence-length LLM calibration, and the QK, UD, VO, SpinR1, SpinR2 and OptimizeFFN equivalent transformations.
llm_fast llm Turns off those six equivalent transformations and full-sequence-length calibration, trading accuracy for compile time.
vision_transformer none Transformer-oriented calibration/bit settings.
multimodal none Enables LLM handling and uses calibration.method=3.

Supported Frameworks

The public backend strings are:

Backend Input
onnx ONNX model path. This is the primary path and is used by validate_model.
tf TensorFlow SavedModel or supported TensorFlow/Keras model path.
tflite TensorFlow Lite model path.
torchscript TorchScript module/model.
torch PyTorch model or HuggingFace/transformers model path/object flows. HuggingFace LLMs use this backend path, not a separate hf backend.

Supported Target Device

Use these target device strings in CLI and Python API calls:

target device Meaning
regulus-ra Target device for regulus-ra.
aries-rb Target device for aries-rb.
regulus-rb Target device for regulus-rb.
regulus-rb-usb USB-connected target device. Available from qb Compiler 1.3.

The validator accepts the public strings above. Internal enum names use underscores, but user-facing values use hyphens.

Supported Operators

NPU Operator Support lists Mobilint IR operations eligible for NPU allocation and their device-specific constraints. Operations outside that list may still compile after graph-level transformation. For input and attribute definitions, see Mobilint IR Operation Formats; for CPU execution, see CPU Offloading.

Available reference paths:

  • For non-ONNX backends, parse/compile the model and inspect unsupported groups with .mblt and CPU offloading workflows.
  • If a model contains unsupported groups, cpuOffload / cpu_offload can partition those groups for CPU execution where the runtime flow supports it.

Glossary

Term Meaning
MXQ Mobilint executable package produced by quantization/compilation and run on Mobilint NPUs.
MBLT Mobilint intermediate model format used between parsing and MXQ generation.
parse Convert a source model into .mblt.
quantize Convert a runnable .mblt into .mxq using calibration/config settings.
compile End-to-end parse plus quantize flow.
backend Source model framework identifier such as onnx, torch, or tflite.
target_device Mobilint NPU target device string such as aries-rb.
NPU Chip Mobilint NPU product name such as ARIES or REGULUS.
target device Compile target such as aries-rb, regulus-ra, regulus-rb, or regulus-rb-usb; used as the target device string.
calibration data Representative input tensors used to derive quantization scales/statistics.
CPU offloading Partitioning unsupported graph groups for CPU execution while supported body subgraphs run on NPU.
body subgraph The largest NPU-runnable supported subgraph extracted from a partitioned .mblt.
preset Built-in partial CompileConfig for common model families.

License

See License for the open-source license notices included with this manual.