Reference

This chapter is a reference for the current qbcompiler command line, Python API, configuration schema, supported inputs, and related terms. The source of truth is the current qbcompiler implementation.

CLI Reference

Run the CLI as:

python -m qbcompiler <command> [options]

The current command set is compile, parse, quantize, and dump-config.

Command	Purpose	Main options
compile	End-to-end model to .mxq: parse to temporary .mblt, then quantize.	--model, --output, --backend, --device, --target-device, --cpu-offload / --no-cpu-offload, --config-preset, --compile-config, --calib-data-path, --use-random-calib / --no-use-random-calib
parse	Convert an input model to .mblt intermediate IR.	--model, --output, --backend, --device, --target-device, --cpu-offload / --no-cpu-offload, --config-preset, --compile-config
quantize	Convert a runnable .mblt file to .mxq.	--mblt, --output, --calib-data-path, --use-random-calib / --no-use-random-calib, --config-preset, --compile-config, --device
dump-config	Write a default or preset-derived CompileConfig template.	--output, --preset

--backend defaults to onnx. --calib-data-path is repeatable. --compile-config accepts JSON, YAML, or YML and is the extension point for settings that do not have dedicated CLI flags. dump-config chooses JSON or YAML from the output file suffix.

For subprocess integrations, the CLI has an opt-in JSONL protocol. Set QBCOMPILER_JSONL=1, true, yes, or on to emit structured status, progress, result, log, and error messages on stdout. Without that environment variable, normal logs are written to stderr.

Python API Reference

Public functions are available from qbcompiler.

API	Purpose
mxq_compile(...)	Main Python entry point. Compiles a model or runnable .mblt to .mxq.
mblt_compile(...)	Exports a model to .mblt without producing .mxq.
list_presets()	Returns preset metadata dictionaries.
dump_default_config(output_path, preset=None)	Writes a CompileConfig template to JSON/YAML.
get_body_subgraph(mblt_path, output_path)	Writes a body-only .mblt containing the largest NPU-runnable subgraph.
run_mblt_model(...)	Runs .mblt inference for debug/inspection flows.

The main mxq_compile parameters are:

from qbcompiler import mxq_compile

mxq_compile(
    model="model.onnx",
    target_device="aries-rb",
    calib_data_path="calib",
    save_path="model.mxq",
    backend="onnx",
    device="gpu",
    config_preset="classification",
    compile_config=None,
)

Important parameters include model, target_device, calib_data_path, save_path, backend, feed_dict, dynamic_axes, in_dformats, yolo_decode_include, exclude_first_subgraph, device, inference_scheme, use_random_calib, cpu_offload, optimize_option, buffer_mode, input_shape_dict, force_npu_input_reposition, force_npu_output_reposition, image_channels, split_blocks, split_parts, config_preset, compile_config, and sub-config objects such as calibration_config, bit_config, llm_config, hessian_quant_config, mod_config, equivalent_transformation_config, search_weight_scale_config, uint8_input_config, preprocessing_config, and save_sample_config.

Configuration resolution priority (highest wins): explicit arguments → sub-config objects → compile_config file → config_preset → defaults. For details and usage examples, see Compile Configuration — Config Resolution Priority.

CompileConfig Schema

CompileConfig is a Pydantic model with extra="forbid" and aliases enabled. Use alias names in JSON/YAML files. A full template can be generated with:

python -m qbcompiler dump-config --output compile_config.yaml
python -m qbcompiler dump-config --preset yolo_640 --output yolo_640.yaml

Top-level fields:

Alias	Python field	Default	Meaning
modelPaths	model_paths	[]	Model path list. Usually supplied by API/CLI instead.
calibDataPaths	calib_data_path	[]	Calibration dataset path list.
savePaths	save_paths	["./tmp.mxq"]	MXQ output path list.
useRandomCalib	use_random_calib	false	Generate random calibration data.
saveMsgpackName	save_msgpack_name	null	Optional msgpack output name.
inferenceScheme	inference_scheme	single	Compile-time NPU core-assignment scheme used by the generated MXQ. See inferenceScheme.
cpuOffload	cpu_offload	false	Enable CPU offloading for unsupported groups.
forceNpuInputReposition	force_npu_input_reposition	false	Force input reposition operations onto NPU.
forceNpuOutputReposition	force_npu_output_reposition	false	Force output reposition operations onto NPU.
optimizeOption	optimize_option	1	Compiler optimization selector.
bufferMode	buffer_mode	1	Buffer serialization mode.
inputShapeDict	input_shape_dict	{}	Multi-shape compile specification for supported models.
device	device	gpu	Compilation compute device: gpu or cpu.
dtype	dtype	float	Computation dtype metadata.
debug	debug	false	Enable debug mode.
trace	trace	false	Enable trace mode.
imageChannels	image_channels	0	Number of image channels; 0 means auto-detect.
configVersion	config_version	1.0.0	Config schema version.
splitBlocks	split_blocks	[]	LLM multi-MXQ split points by block index.
splitParts	split_parts	0	Split LLM transformer blocks into N parts.

Nested sections:

Alias	Type	Purpose
uint8Input	Uint8InputConfig	Treat model inputs as uint8, optionally by input name.
preprocessing	PreprocessingConfig	Input preprocessing pipeline such as resize, letterbox, crop, normalize, and format conversion.
resourceManagement	ResourceManagementConfig	Weight dtype, GPU calibration policy, and weight-memory method.
calibration	CalibrationConfig	Quantization method, output quantization, calibration mode, clipping/statistics, and layer overrides.
bit	BitConfig	Quantization precision settings.
hessianQuant	HessianQuantConfig	HessianQuant optimization settings.
mod	ModConfig	MOD training/optimization settings.
llm	LlmConfig	LLM sequence/cache/runtime/debug settings. If runtime.batchSize is greater than 1, inferenceScheme is fixed to single, and the Batch LLM runtime API uses BatchParam metadata; see qb Runtime Release Notes.
moe	MoeConfig	Sparse MoE calibration expert selection.
equivalentTransformation	EquivalentTransformationConfig	SmoothQuant-like, rotation, and FFN equivalent transformations.
searchWeightScale	SearchWeightScaleConfig	Weight-scale search.
loadScale	LoadScaleConfig	Load externally supplied scale entries.
runtimeOptions	RuntimeOptions	Internal runtime metadata.
saveSample	SaveSampleConfig	Save sample data for inspection/debugging.

CompileConfig.from_file() accepts .json, .yaml, and .yml. For compatibility, it also flattens grouped keys such as quantization.calibration, quantization.bit, advancedQuantization.hessianQuant, advancedQuantization.mod, advancedQuantization.EquivalentTransformation, advancedQuantization.searchWeightScale, and advancedQuantization.loadScale.

inferenceScheme

inferenceScheme selects how NPU core work is assigned when qb Runtime uses the MXQ for inference. Because the compiled MXQ is prepared for the selected core-assignment scheme, an MXQ built only for one mode cannot later be switched arbitrarily to another mode at runtime.

Available choices are single, multi, global4, global8, and all. single uses independent Local Cores, multi is the cluster-level 4-batch mode, and global4/global8 use 4 or 8 Local Cores together for one input. all prepares the MXQ for every core mode supported by the selected model and target. Support can vary by model, target, and compiler version. REGULUS targets have a single NPU core, so MXQ files compiled for REGULUS can use only single mode. global is a legacy/compatibility value; use global4 or global8 for new settings.

If llm.attributes.runtime.batchSize is greater than 1, LLM/KV-cache transformer compilation automatically fixes inferenceScheme to single. Vision batch inference and LLM batch inference do not use the same core-mode rule.

For what each mode means, see ARIES Core Mode. For runtime core/cluster selection, see ARIES ModelConfig Configuration.

RuntimeOptions

RuntimeOptions currently contains only:

Field	Default	Meaning
version	0.0.0	Compiler/runtime version metadata.

This section is runtime metadata rather than a normal user tuning surface. Prefer the compile, calibration, preprocessing, LLM, and optimization sections for user-controlled behavior.

Preset List

Preset	Extends	Behavior
classification	none	Image classification defaults: calibration.mode=1, calibration.output=0.
detection	none	Object detection defaults: calibration.mode=1, calibration.output=1.
classification_torchvision	classification	Enables uint8 input and standard Torchvision preprocessing: resize 256x256, center crop 224x224, normalize ImageNet mean/std, imageChannels=3.
yolo_640	detection	Enables uint8 input and 640x640 letterbox preprocessing with pad value 114, imageChannels=3.
yolo_1280	detection	Enables uint8 input and 1280x1280 letterbox preprocessing with pad value 114, imageChannels=3.
llm	none	Enables LLM config with maxSequenceLength=4096, maxCacheLength=4096, calibration.mode=0, calibration.output=0.
llm_fast	llm	Disables HessianQuant, enables selected equivalent transformations, and uses full sequence length for LLM calibration.
vision_transformer	none	Transformer-oriented calibration/bit settings.
multimodal	none	Enables LLM handling and uses calibration.method=3.

Supported Frameworks

The public backend strings are:

Backend	Input
onnx	ONNX model path. This is the primary path and is used by validate_model.
tf	TensorFlow SavedModel or supported TensorFlow/Keras model path.
tflite	TensorFlow Lite model path.
torchscript	TorchScript module/model.
torch	PyTorch model or HuggingFace/transformers model path/object flows. HuggingFace LLMs use this backend path, not a separate hf backend.

Supported Target Device

Use these target device strings in CLI and Python API calls:

• regulus-ra

• aries-rb

• regulus-rb

The validator accepts the public strings above. Internal enum names use underscores, but user-facing values use hyphens.

Supported Operators

qbcompiler does not currently expose a stable static supported-operator table in the manual source. Operator support is determined by the parser and target-device allocation logic for the specific model, backend, shapes, and target device.

Available reference paths:

• For non-ONNX backends, parse/compile the model and inspect unsupported groups with .mblt and CPU offloading workflows.

• If a model contains unsupported groups, cpuOffload / cpu_offload can partition those groups for CPU execution where the runtime flow supports it.

Glossary

Term	Meaning
MXQ	Mobilint executable package produced by quantization/compilation and run on Mobilint NPUs.
MBLT	Mobilint intermediate model format used between parsing and MXQ generation.
parse	Convert a source model into .mblt.
quantize	Convert a runnable .mblt into .mxq using calibration/config settings.
compile	End-to-end parse plus quantize flow.
backend	Source model framework identifier such as onnx, torch, or tflite.
target_device	Mobilint NPU target device string such as aries-rb.
NPU Chip	Mobilint NPU product name such as ARIES or REGULUS.
target device	Compile target such as aries-rb, regulus-ra, or regulus-rb; used as the target device string.
calibration data	Representative input tensors used to derive quantization scales/statistics.
CPU offloading	Partitioning unsupported graph groups for CPU execution while supported body subgraphs run on NPU.
body subgraph	The largest NPU-runnable supported subgraph extracted from a partitioned .mblt.
preset	Built-in partial CompileConfig for common model families.

License

See License for the open-source license notices included with this manual.