API Reference

API Reference#

Mobilint SDK qb Compiler: API Reference
Mobilint SDK qb Compiler v1.3
MCS002-EN
Classes | Functions | Variables
API Reference

Mobilint qb Compiler API provides functions for compiling models from various framework(onnx, pytorch, transformers, etc.) to MXQ format. More...

Classes

class  qbcompiler.frontend.Model_Dict
 Wrapper around the Mobilint compiler to support compilation and inference workflows. More...
class  qbcompiler.calibration.preprocess.GetNpy
 Load NumPy data from an in-memory array or a .npy file. More...
class  qbcompiler.calibration.preprocess.NpyNormalize
class  qbcompiler.calibration.preprocess.NpyPad
 Pad NumPy arrays to a desired shape. More...
class  qbcompiler.calibration.preprocess.GetImage
 Retrieve an image tensor from a file path or NumPy array using OpenCV. More...
class  qbcompiler.calibration.preprocess.Pad
 Pad images to a target shape or to meet a size divisor. More...
class  qbcompiler.calibration.preprocess.Normalize
 Apply mean/std normalization to images. More...
class  qbcompiler.calibration.preprocess.ResizeTorch
class  qbcompiler.calibration.preprocess.Resize
 Resize images with optional aspect-ratio preservation. More...
class  qbcompiler.calibration.preprocess.CenterCrop
 Perform a centered crop on the image. More...
class  qbcompiler.calibration.preprocess.SetOrder
class  qbcompiler.calibration.preprocess.preproc_builder
 Build a calibration preprocessing pipeline from a YAML or dict configuration. More...
class  GetText
 Extract text strings from raw inputs. More...
class  TokenizeToEmbedding
 Convert tokenized text into embeddings using a Hugging Face model. More...
class  YoloPre
 Apply YOLO-style resize, padding, and normalization to images. More...

Functions

 qbcompiler.frontend.mxq_compile (model, str target_device, Union[str, List[str]] calib_data_path=UNSET, int save_subgraph_type=0, output_subgraph_path="", Union[str, List[str]] save_path=UNSET, backend="onnx", feed_dict=None, dynamic_axes=None, in_dformats=None, yolo_decode_include=False, exclude_first_subgraph=False, device=UNSET, inference_scheme=UNSET, use_random_calib=UNSET, cpu_offload=UNSET, optimize_option=UNSET, buffer_mode=UNSET, input_shape_dict=UNSET, force_npu_input_reposition=UNSET, force_npu_output_reposition=UNSET, image_channels=UNSET, split_blocks=UNSET, split_parts=UNSET, Optional[str] config_preset=None, Optional[CompileConfig] compile_config=None, Optional[ResourceManagementConfig] resource_management_config=None, Optional[CalibrationConfig] calibration_config=None, Optional[BitConfig] bit_config=None, Optional[LlmConfig] llm_config=None, Optional[HessianQuantConfig] hessian_quant_config=None, Optional[ModConfig] mod_config=None, Optional[EquivalentTransformationConfig] equivalent_transformation_config=None, Optional[SearchWeightScaleConfig] search_weight_scale_config=None, Optional[SaveSampleConfig] save_sample_config=None, Optional[Uint8InputConfig] uint8_input_config=None, Optional[PreprocessingConfig] preprocessing_config=None, layer_bias_correction=UNSET, Optional[LayerBiasCorrectionConfig] layer_bias_correction_config=None, Optional[str] model_part=None, Optional[dict] model_part_options=None, Optional[str] config_save_path=None, **kwargs)
 Compile a model into a Mobilint eXeCUtable (MXQ) package for execution on Mobilint NPUs.
 qbcompiler.frontend.mxq_compile_from_source (model, str target_device, Union[str, List[str]] calib_data_path=UNSET, int save_subgraph_type=0, output_subgraph_path="", Union[str, List[str]] save_path=UNSET, backend="onnx", feed_dict=None, dynamic_axes=None, in_dformats=None, yolo_decode_include=False, exclude_first_subgraph=False, device=UNSET, inference_scheme=UNSET, use_random_calib=UNSET, cpu_offload=UNSET, optimize_option=UNSET, buffer_mode=UNSET, input_shape_dict=UNSET, force_npu_input_reposition=UNSET, force_npu_output_reposition=UNSET, image_channels=UNSET, split_blocks=UNSET, split_parts=UNSET, Optional[str] config_preset=None, Optional[CompileConfig] compile_config=None, Optional[ResourceManagementConfig] resource_management_config=None, Optional[CalibrationConfig] calibration_config=None, Optional[BitConfig] bit_config=None, Optional[LlmConfig] llm_config=None, Optional[HessianQuantConfig] hessian_quant_config=None, Optional[ModConfig] mod_config=None, Optional[EquivalentTransformationConfig] equivalent_transformation_config=None, Optional[SearchWeightScaleConfig] search_weight_scale_config=None, Optional[SaveSampleConfig] save_sample_config=None, Optional[Uint8InputConfig] uint8_input_config=None, Optional[PreprocessingConfig] preprocessing_config=None, layer_bias_correction=UNSET, Optional[LayerBiasCorrectionConfig] layer_bias_correction_config=None, Optional[str] model_part=None, Optional[dict] model_part_options=None, Optional[str] config_save_path=None, **kwargs)
 Compile a raw framework model (ONNX / PyTorch / TensorFlow / TF-Lite) into an MXQ package.
None qbcompiler.frontend._mxq_compile_legacy (*, model, str target_device, Union[str, List[str]] calib_data_path=UNSET, int save_subgraph_type=0, output_subgraph_path="", Union[str, List[str]] save_path=UNSET, backend="onnx", feed_dict=None, dynamic_axes=None, in_dformats=None, yolo_decode_include=False, exclude_first_subgraph=False, device=UNSET, inference_scheme=UNSET, use_random_calib=UNSET, cpu_offload=UNSET, optimize_option=UNSET, buffer_mode=UNSET, input_shape_dict=UNSET, force_npu_input_reposition=UNSET, force_npu_output_reposition=UNSET, image_channels=UNSET, split_blocks=UNSET, split_parts=UNSET, Optional[str] config_preset=None, Optional[CompileConfig] compile_config=None, Optional[ResourceManagementConfig] resource_management_config=None, Optional[CalibrationConfig] calibration_config=None, Optional[BitConfig] bit_config=None, Optional[LlmConfig] llm_config=None, Optional[HessianQuantConfig] hessian_quant_config=None, Optional[ModConfig] mod_config=None, Optional[EquivalentTransformationConfig] equivalent_transformation_config=None, Optional[SearchWeightScaleConfig] search_weight_scale_config=None, Optional[SaveSampleConfig] save_sample_config=None, Optional[Uint8InputConfig] uint8_input_config=None, Optional[PreprocessingConfig] preprocessing_config=None, layer_bias_correction=UNSET, Optional[LayerBiasCorrectionConfig] layer_bias_correction_config=None, Optional[str] config_save_path=None, **kwargs)
 Quantize and compile through the legacy Model_Dict parser.
 qbcompiler.frontend.mxq_compile_from_mblt (str mblt, str target_device, Union[str, List[str]] calib_data_path=UNSET, Union[str, List[str]] save_path=UNSET, backend="onnx", device=UNSET, inference_scheme=UNSET, use_random_calib=UNSET, cpu_offload=UNSET, optimize_option=UNSET, buffer_mode=UNSET, input_shape_dict=UNSET, force_npu_input_reposition=UNSET, force_npu_output_reposition=UNSET, image_channels=UNSET, split_blocks=UNSET, split_parts=UNSET, Optional[str] config_preset=None, Optional[CompileConfig] compile_config=None, Optional[ResourceManagementConfig] resource_management_config=None, Optional[CalibrationConfig] calibration_config=None, Optional[BitConfig] bit_config=None, Optional[LlmConfig] llm_config=None, Optional[HessianQuantConfig] hessian_quant_config=None, Optional[ModConfig] mod_config=None, Optional[EquivalentTransformationConfig] equivalent_transformation_config=None, Optional[SearchWeightScaleConfig] search_weight_scale_config=None, Optional[SaveSampleConfig] save_sample_config=None, Optional[Uint8InputConfig] uint8_input_config=None, Optional[PreprocessingConfig] preprocessing_config=None, layer_bias_correction=UNSET, Optional[LayerBiasCorrectionConfig] layer_bias_correction_config=None, Optional[str] config_save_path=None, **kwargs)
 Compile an existing Mobilint IR (.mblt) file into an MXQ package.
 qbcompiler.frontend.mblt_compile (str|Any model, str mblt_save_path, str target_device, backend="onnx", device="cpu", feed_dict=None, dynamic_axes=None, in_dformats=None, yolo_decode_include=False, cpu_offload=False, exclude_first_subgraph=False, Optional[str] model_part=None, Optional[dict] model_part_options=None, **kwargs)
 Export a model to the Mobilint .mblt format without producing an MXQ package.
None qbcompiler.frontend._mblt_compile_dispatch (str|Any model, str mblt_save_path, str target_device, backend="onnx", device=UNSET, feed_dict=None, dynamic_axes=None, in_dformats=None, yolo_decode_include=False, cpu_offload=UNSET, exclude_first_subgraph=False, Optional[str] model_part=None, Optional[dict] model_part_options=None, **kwargs)
 Route a compile-to-mblt call to the pipeline or the legacy parser.
None qbcompiler.frontend._mblt_compile_legacy (*, str|Any model, str mblt_save_path, str target_device, str backend, device=UNSET, feed_dict=None, dynamic_axes=None, in_dformats=None, yolo_decode_include=False, cpu_offload=UNSET, exclude_first_subgraph=False, **kwargs)
 Export to .mblt through the legacy Model_Dict parser.
None qbcompiler.frontend._mxq_compile_pipeline (model, str target_device, Union[str, List[str]] calib_data_path=UNSET, int save_subgraph_type=0, output_subgraph_path="", Union[str, List[str]] save_path=UNSET, backend="onnx", feed_dict=None, dynamic_axes=None, in_dformats=None, yolo_decode_include=False, exclude_first_subgraph=False, device=UNSET, inference_scheme=UNSET, use_random_calib=UNSET, cpu_offload=UNSET, optimize_option=UNSET, buffer_mode=UNSET, input_shape_dict=UNSET, force_npu_input_reposition=UNSET, force_npu_output_reposition=UNSET, image_channels=UNSET, split_blocks=UNSET, split_parts=UNSET, Optional[str] config_preset=None, Optional[CompileConfig|str] compile_config=None, Optional[ResourceManagementConfig] resource_management_config=None, Optional[CalibrationConfig] calibration_config=None, Optional[BitConfig] bit_config=None, Optional[LlmConfig] llm_config=None, Optional[HessianQuantConfig] hessian_quant_config=None, Optional[ModConfig] mod_config=None, Optional[EquivalentTransformationConfig] equivalent_transformation_config=None, Optional[SearchWeightScaleConfig] search_weight_scale_config=None, Optional[SaveSampleConfig] save_sample_config=None, Optional[Uint8InputConfig] uint8_input_config=None, Optional[PreprocessingConfig] preprocessing_config=None, layer_bias_correction=UNSET, Optional[LayerBiasCorrectionConfig] layer_bias_correction_config=None, Optional[str] model_part=None, Optional[dict] model_part_options=None, Optional[str] config_save_path=None, **kwargs)
 Quantize and compile through the ConfigManager-resolved pipeline.
None qbcompiler.frontend._mblt_compile_pipeline (str|object model, str target_device, str mblt_save_path, backend="onnx", object device=UNSET, feed_dict=None, dynamic_axes=None, in_dformats=None, yolo_decode_include=False, object cpu_offload=UNSET, exclude_first_subgraph=False, Optional[str] model_part=None, Optional[dict] model_part_options=None, Optional[str] config_preset=None, Optional[CompileConfig|str] compile_config=None, Optional[str] compilation_mode=None, **kwargs)
 Export to .mblt through the ConfigManager-resolved pipeline.
None qbcompiler.frontend.mblt_compile_with_callback (str model, str mblt_save_path, str target_device, str backend="onnx", Optional[str] device=None, Optional[bool] cpu_offload=None, Optional[str] config_preset=None, Optional[CompileConfig|str] compile_config=None, Optional[str] compilation_mode=None, *, Callable[[int, str], None] progress_callback)
 Compile-to-mblt entry point with progress callbacks.
None qbcompiler.frontend.mxq_compile_with_callback (str model, str target_device, str save_path, str backend="onnx", Optional[str] device=None, Optional[Union[str, List[str]]] calib_data_path=None, Optional[bool] use_random_calib=None, Optional[str] config_preset=None, Optional[CompileConfig|str] compile_config=None, Optional[str] config_save_path=None, *, Callable[[int, str], None] progress_callback)
 Compile/quantize entry point with progress callbacks.
 qbcompiler.calibration.make_calib_data.get_yaml (str yaml_path)
 qbcompiler.calibration.make_calib_data.make_calib_man (pre_ftn, str data_dir, str save_dir, str save_name, str anno_json=None, str file_format="%012d.jpg", int max_size=-1, bool remove_npy=False, int seed=2023, bool save_calib_msg=False, str msg_path=None)
 From images and a user-provided preprocessing function, generate calibration NumPy files and a manifest.
str qbcompiler.calibration.make_calib_data.make_calib (Union[str, dict] args_pre, str data_dir, str save_dir, str save_name=None, str anno_json=None, str file_format="%012d.jpg", int max_size=-1, bool remove_npy=False, int seed=2023, bool save_calib_msg=False, str msg_path=None)
 Create calibration NumPy files and a manifest using a YAML or dict preprocessing configuration.
List[str] qbcompiler.calibration.make_calib_data.make_calib_llm (Union[str, dict] args_pre, str dataset_name, Union[List[str], str] subset_list, str text_field, str split, str save_dir, int max_calib=512, Optional[str] save_prefix=None)
 Generate calibration embeddings for LLM models using Hugging Face datasets.
 qbcompiler.calibration.preprocess.preprocess_fnc (fnc, Datatype="Image")

Variables

dict qbcompiler.calibration.preprocess.DTYPE_GETDATA_MAP = {"Image": "GetImage", "Npy": "GetNpy", "LLM": "GetText"}

Detailed Description

Mobilint qb Compiler API provides functions for compiling models from various framework(onnx, pytorch, transformers, etc.) to MXQ format.

Function Documentation

◆ mxq_compile()

qbcompiler.frontend.mxq_compile ( model,
str target_device,
Union[str, List[str]] calib_data_path = UNSET,
int save_subgraph_type = 0,
output_subgraph_path = "",
Union[str, List[str]] save_path = UNSET,
backend = "onnx",
feed_dict = None,
dynamic_axes = None,
in_dformats = None,
yolo_decode_include = False,
exclude_first_subgraph = False,
device = UNSET,
inference_scheme = UNSET,
use_random_calib = UNSET,
cpu_offload = UNSET,
optimize_option = UNSET,
buffer_mode = UNSET,
input_shape_dict = UNSET,
force_npu_input_reposition = UNSET,
force_npu_output_reposition = UNSET,
image_channels = UNSET,
split_blocks = UNSET,
split_parts = UNSET,
Optional[str] config_preset = None,
Optional[CompileConfig] compile_config = None,
Optional[ResourceManagementConfig] resource_management_config = None,
Optional[CalibrationConfig] calibration_config = None,
Optional[BitConfig] bit_config = None,
Optional[LlmConfig] llm_config = None,
Optional[HessianQuantConfig] hessian_quant_config = None,
Optional[ModConfig] mod_config = None,
Optional[EquivalentTransformationConfig] equivalent_transformation_config = None,
Optional[SearchWeightScaleConfig] search_weight_scale_config = None,
Optional[SaveSampleConfig] save_sample_config = None,
Optional[Uint8InputConfig] uint8_input_config = None,
Optional[PreprocessingConfig] preprocessing_config = None,
layer_bias_correction = UNSET,
Optional[LayerBiasCorrectionConfig] layer_bias_correction_config = None,
Optional[str] model_part = None,
Optional[dict] model_part_options = None,
Optional[str] config_save_path = None,
** kwargs )

Compile a model into a Mobilint eXeCUtable (MXQ) package for execution on Mobilint NPUs.

When no explicit value is provided for a parameter marked with UNSET, the default from CompileConfig is used. This allows a compile_config file or object to supply the value without being overridden by function-level defaults.

Configuration is resolved in priority order (highest to lowest):

  1. Explicitly passed function arguments
  2. Individual sub-config objects (calibration_config, llm_config, etc.)
  3. kwargs partial overrides (quantization_method, weight_dtype, etc.)
  4. compile_config (CompileConfig object or JSON/YAML file) or config_preset
  5. CompileConfig field defaults
Parameters
modelstring or model instance. Model path. When using backend="onnx", this should be the path to an ONNX model file. When backend is "torchscript", provide a TorchScript module; when "torch", provide a standard PyTorch model. For backend="tf", pass the directory that contains the TensorFlow SavedModel graph and assets. For backend="tflite", pass the TF Lite model path.
calib_data_pathstring or list of strings. Path(s) to the calibration dataset. Accepts either a text/json file that lists NumPy files or a directory that contains the pre-processed NumPy files.
save_subgraph_typeint. Controls optional MBLT subgraph exports: 0 disables exports; 1 saves only the graph structure; 2 saves graph structure plus weights; 3 saves the graph structure split into multiple subgraphs; 4 saves both structure and weights split into multiple subgraphs. Defaults to 0.
output_subgraph_pathstring. Destination path for the exported .mblt file when save_subgraph_type is 1–4. The resulting file can be used for visualization. Defaults to "".
save_pathstring or list of strings. Output MXQ filename(s). When omitted, defaults to "{model_name}.mxq" derived from the model path basename.
backendstring. Framework used to generate the Mobilint IR. Case-insensitive, so "ONNX" and "onnx" are the same backend. "onnx" and "torch" are parsed by the current parser; "tf" (also spelled "tensorflow"), "tflite" and "torchscript" go through the legacy parser. Any other value raises ValueError. Defaults to "onnx". When model is a path to an already-compiled .mblt file (the output of mblt_compile()), it is compiled directly with MXQ regardless of the backend value, and the file is validated beforehand; it must be a runnable artifact produced by mblt_compile() rather than a save_subgraph_type preview export.
target_devicestring. Target NPU device for parser/compiler configuration (for example "aries-rb"). Required by execution paths that parse or quantize a model.
devicestring. Compilation and inference device: "cpu" or "gpu". When omitted, uses CompileConfig default.
feed_dictdict. Example input tensors for shape inference and inference validation.
dynamic_axesdict. Marks model axes as dynamic. Keys are input names and values map dimension indices to aliases (for example {"input": {2: "seq_len"}}).
in_dformatsdict. Describes input tensor data formats.
inference_schemestring. NPU inference scheme. One of "single", "multi", "global", "global4", or "global8". When omitted, uses CompileConfig default.
yolo_decode_includebool. Determines whether YOLO decode runs on NPU. Defaults to False.
use_random_calibbool. Generates random calibration data to validate model compilability. When omitted, uses CompileConfig default.
cpu_offloadbool. Enables CPU offloading during NPU inference. When omitted, uses CompileConfig default.
optimize_optionint. Compiler optimization strategy selector. When omitted, uses CompileConfig default.
buffer_modeint. Buffer serialization mode: 0 uses a naive buffer, 1 uses an mmap-backed buffer. When omitted, uses CompileConfig default.
input_shape_dictdict. Multi-shape compilation specification for STT/TTS models only (e.g., Conformer-CTC, MeloTTS). The dynamic axis is defined with respect to each input tensor's shape. During compilation, the same axis is varied across all model inputs using the provided values. Currently, only a single entry key ("multi_shape0") is supported. Example: {"multi_shape0": {"axis": 2, "values": [100, 200, 300]}}
force_npu_input_repositionbool. Force input reposition operations to run on NPU instead of CPU. When omitted, uses CompileConfig default.
force_npu_output_repositionbool. Force output reposition operations to run on NPU instead of CPU. When omitted, uses CompileConfig default.
image_channelsint. Number of image channels (0 for auto-detect). When omitted, uses CompileConfig default.
split_blockslist of int. Multi-MXQ split points by transformer block index. Only supported for LLM models. When omitted, uses CompileConfig default.
split_partsint. Evenly split transformer blocks into N MXQ parts. Only supported for LLM models. When omitted, uses CompileConfig default.
layer_bias_correctionbool. Enables calibration-derived layer bias correction. When omitted, uses layer_bias_correction_config or the CompileConfig value.
exclude_first_subgraphbool. Applies only when CPU offloading is enabled: exclude the first subgraph from the final graph if it is unsupported. Defaults to False.
config_presetstring. Name of a built-in configuration preset. When provided, loads the preset via CompileConfig.from_preset(). Defaults to None (no preset). Available presets:
  • "classification": Image classification models (ResNet, EfficientNet, ViT, etc.)
  • "detection": Object detection models (YOLO, SSD, DETR, etc.)
  • "classification_torchvision": Torchvision classification models with standard preprocessing
  • "yolo_640": YOLO detection models with 640x640 letterbox preprocessing
  • "yolo_1280": YOLO detection models with 1280x1280 letterbox preprocessing
  • "llm": Large Language Models (LLaMA, Qwen, Gemma, etc.)
  • "llm_fast": LLM with faster compilation (less accuracy optimization)
  • "vision_transformer": Vision Transformer models (ViT, DeiT, Swin, etc.)
  • "multimodal": Multimodal models (CLIP, BLIP, LLaVA, etc.)
compile_configCompileConfig or string. CompileConfig object or path to JSON/YAML configuration file containing all compilation settings (resourceManagement, calibration, bit, hessianQuant, mod, llm, etc.).
config_save_pathstring. When provided, the fully resolved configuration - the normalized CompileConfig after every layer of the precedence order above has been applied, including all sub-configurations - is written to this path before compilation starts. The format follows the path suffix: ".yaml"/".yml" produce YAML, anything else produces JSON. Parent directories are created as needed. The saved file can be fed back as compile_config to reproduce the same compilation. Defaults to None (no config file is written).
resource_management_configResourceManagementConfig. Resource management configuration object.
calibration_configCalibrationConfig. Calibration configuration object.
bit_configBitConfig. Bit configuration object for quantization precision settings.
llm_configLlmConfig. LLM configuration object.
hessian_quant_configHessianQuantConfig. HessianQuant (Hessian-based Quantization) configuration object.
layer_bias_correction_configLayerBiasCorrectionConfig. Calibration-derived layer bias correction settings.
model_partstring. Selects a specific part of a torch model to parse ("vision", "language", "encoder", ...). Typically used to support models with complex architectures (e.g. Qwen3-VL) by compiling one part at a time. Parts a model declares: parser.patcher.parts.available_parts(model). A model declaring exactly one part resolves it from None; one declaring several requires a name.
model_part_optionsdict. Extra arguments for the named part (e.g. {"mel_frames": 100} for Qwen3-ASR audio).
mod_configModConfig. MOD (Metric-based Optimization and Distillation) configuration object.
equivalent_transformation_configEquivalentTransformationConfig. Configuration for equivalent transformations like SmoothQuant.
search_weight_scale_configSearchWeightScaleConfig. Configuration for weight scale search.
save_sample_configSaveSampleConfig. Configuration for sample data generation and saving.
uint8_input_configUint8InputConfig. Configuration for uint8 input handling.
preprocessing_configPreprocessingConfig. Preprocessing pipeline configuration.
kwargsdict. Additional compiler arguments. Supports partial config overrides such as quantization_method, quantization_mode, percentile, weight_dtype, ram_usage, max_sequence_length, etc. Also accepts deprecated parameters (quantization_config, advanced_quantization_config, input_process_config, save_sample, sample_dtype) which will emit DeprecationWarning.
Returns
None.
Using configuration
There are three ways to configure quantization settings:
  1. Load all settings from a JSON/YAML config file:
    from qbcompiler import mxq_compile
    mxq_compile(
    model="path/to/model.onnx",
    target_device="aries-rb",
    calib_data_path="path/to/calib",
    compile_config="path/to/config.json", # or config.yaml
    device="gpu",
    )
  2. Pass individual sub-config objects:
    from qbcompiler import mxq_compile
    from qbcompiler.configs import (
    ResourceManagementConfig,
    CalibrationConfig,
    BitConfig,
    HessianQuantConfig,
    ModConfig,
    LlmConfig,
    )
    resource_mgmt = ResourceManagementConfig(weight_dtype="float32")
    calib_cfg = CalibrationConfig(method=1, mode=1)
    bit_cfg = BitConfig(...)
    hessian_quant_cfg = HessianQuantConfig(apply=True)
    mod_cfg = ModConfig(apply=False)
    llm_cfg = LlmConfig(apply=True)
    mxq_compile(
    model="path/to/model.onnx",
    target_device="aries-rb",
    calib_data_path="path/to/calib",
    resource_management_config=resource_mgmt,
    calibration_config=calib_cfg,
    bit_config=bit_cfg,
    hessian_quant_config=hessian_quant_cfg,
    mod_config=mod_cfg,
    llm_config=llm_cfg,
    device="gpu",
    )
    qbcompiler.configs
    Auto-generated config module from config_schema.yaml.
    Definition __init__.py:1
  3. Automatically applied partial configuration overrides:
    from qbcompiler import mxq_compile
    mxq_compile(
    model="path/to/model.onnx",
    target_device="aries-rb",
    calib_data_path="path/to/calib",
    quantization_method=1, # per channel quantization
    quantization_mode=1, # max percentile quantization
    percentile=0.999, # percentile value for max percentile quantization
    quantization_output=0, # per layer quantization for the output layer
    device="gpu",
    )
    Please refer to the mxq_compile function and the quantization configuration section for the meaning of the quantization-related numeric values.
Compiling models with custom inputs (ONNX/Torch/TensorFlow)
Provide NumPy inputs whenever the model omits shape information so that qbcompiler can infer unknown dimensions and data formats.
from qbcompiler import mxq_compile
import numpy as np
example_input = {
"input_node_name_1": np.random.randn(1, 3, 224, 224).astype(np.float32),
"input_node_name_2": np.random.randn(1, 8, 224, 224).astype(np.float32),
"input_node_name_3": np.random.randn(1, 4, 56, 56).astype(np.float32),
}
in_dformats = {
"input_node_name_1": "NCHW",
"input_node_name_2": "NCHW",
"input_node_name_3": "NCHW",
}
onnx_model_path = "path/to/your/model.onnx"
mxq_compile(
model=onnx_model_path,
target_device="aries-rb",
feed_dict=example_input,
in_dformats=in_dformats,
backend="onnx",
compile_config="path/to/config.json",
)

Definition at line 148 of file frontend.py.

◆ mxq_compile_from_source()

qbcompiler.frontend.mxq_compile_from_source ( model,
str target_device,
Union[str, List[str]] calib_data_path = UNSET,
int save_subgraph_type = 0,
output_subgraph_path = "",
Union[str, List[str]] save_path = UNSET,
backend = "onnx",
feed_dict = None,
dynamic_axes = None,
in_dformats = None,
yolo_decode_include = False,
exclude_first_subgraph = False,
device = UNSET,
inference_scheme = UNSET,
use_random_calib = UNSET,
cpu_offload = UNSET,
optimize_option = UNSET,
buffer_mode = UNSET,
input_shape_dict = UNSET,
force_npu_input_reposition = UNSET,
force_npu_output_reposition = UNSET,
image_channels = UNSET,
split_blocks = UNSET,
split_parts = UNSET,
Optional[str] config_preset = None,
Optional[CompileConfig] compile_config = None,
Optional[ResourceManagementConfig] resource_management_config = None,
Optional[CalibrationConfig] calibration_config = None,
Optional[BitConfig] bit_config = None,
Optional[LlmConfig] llm_config = None,
Optional[HessianQuantConfig] hessian_quant_config = None,
Optional[ModConfig] mod_config = None,
Optional[EquivalentTransformationConfig] equivalent_transformation_config = None,
Optional[SearchWeightScaleConfig] search_weight_scale_config = None,
Optional[SaveSampleConfig] save_sample_config = None,
Optional[Uint8InputConfig] uint8_input_config = None,
Optional[PreprocessingConfig] preprocessing_config = None,
layer_bias_correction = UNSET,
Optional[LayerBiasCorrectionConfig] layer_bias_correction_config = None,
Optional[str] model_part = None,
Optional[dict] model_part_options = None,
Optional[str] config_save_path = None,
** kwargs )

Compile a raw framework model (ONNX / PyTorch / TensorFlow / TF-Lite) into an MXQ package.

This is the raw-model half of mxq_compile(): it parses the model into Mobilint IR and then quantizes and compiles it in one pass. The parameters, their defaults, and the configuration precedence rules are identical to mxq_compile() — see that function for the full reference and usage examples.

Passing the path of an existing .mblt file here raises ValueError; use mxq_compile_from_mblt() for that input instead. mxq_compile() routes between the two automatically.

Parameters
modelstring or model instance. Raw model or path to one. Must not be an existing .mblt file.
target_devicestring. Target NPU device (for example "aries-rb").
Returns
None.

Definition at line 499 of file frontend.py.

◆ _mxq_compile_legacy()

None qbcompiler.frontend._mxq_compile_legacy ( * ,
model,
str target_device,
Union[str, List[str]] calib_data_path = UNSET,
int save_subgraph_type = 0,
output_subgraph_path = "",
Union[str, List[str]] save_path = UNSET,
backend = "onnx",
feed_dict = None,
dynamic_axes = None,
in_dformats = None,
yolo_decode_include = False,
exclude_first_subgraph = False,
device = UNSET,
inference_scheme = UNSET,
use_random_calib = UNSET,
cpu_offload = UNSET,
optimize_option = UNSET,
buffer_mode = UNSET,
input_shape_dict = UNSET,
force_npu_input_reposition = UNSET,
force_npu_output_reposition = UNSET,
image_channels = UNSET,
split_blocks = UNSET,
split_parts = UNSET,
Optional[str] config_preset = None,
Optional[CompileConfig] compile_config = None,
Optional[ResourceManagementConfig] resource_management_config = None,
Optional[CalibrationConfig] calibration_config = None,
Optional[BitConfig] bit_config = None,
Optional[LlmConfig] llm_config = None,
Optional[HessianQuantConfig] hessian_quant_config = None,
Optional[ModConfig] mod_config = None,
Optional[EquivalentTransformationConfig] equivalent_transformation_config = None,
Optional[SearchWeightScaleConfig] search_weight_scale_config = None,
Optional[SaveSampleConfig] save_sample_config = None,
Optional[Uint8InputConfig] uint8_input_config = None,
Optional[PreprocessingConfig] preprocessing_config = None,
layer_bias_correction = UNSET,
Optional[LayerBiasCorrectionConfig] layer_bias_correction_config = None,
Optional[str] config_save_path = None,
** kwargs )
protected

Quantize and compile through the legacy Model_Dict parser.

Reached for the backends the pipeline does not implement – tf, tflite, hf, torchscript and the model-specific misc parsers. Configuration resolution goes through the same ConfigManager.resolve_mxq_compile the pipeline path uses, so only the parser differs; the kwargs it hands back are the ones the bindings registry did not consume, which is what the parser gets.

Callers route here through :func:mxq_compile_from_source, which has already validated target_device and normalized backend.

Definition at line 675 of file frontend.py.

◆ mxq_compile_from_mblt()

qbcompiler.frontend.mxq_compile_from_mblt ( str mblt,
str target_device,
Union[str, List[str]] calib_data_path = UNSET,
Union[str, List[str]] save_path = UNSET,
backend = "onnx",
device = UNSET,
inference_scheme = UNSET,
use_random_calib = UNSET,
cpu_offload = UNSET,
optimize_option = UNSET,
buffer_mode = UNSET,
input_shape_dict = UNSET,
force_npu_input_reposition = UNSET,
force_npu_output_reposition = UNSET,
image_channels = UNSET,
split_blocks = UNSET,
split_parts = UNSET,
Optional[str] config_preset = None,
Optional[CompileConfig] compile_config = None,
Optional[ResourceManagementConfig] resource_management_config = None,
Optional[CalibrationConfig] calibration_config = None,
Optional[BitConfig] bit_config = None,
Optional[LlmConfig] llm_config = None,
Optional[HessianQuantConfig] hessian_quant_config = None,
Optional[ModConfig] mod_config = None,
Optional[EquivalentTransformationConfig] equivalent_transformation_config = None,
Optional[SearchWeightScaleConfig] search_weight_scale_config = None,
Optional[SaveSampleConfig] save_sample_config = None,
Optional[Uint8InputConfig] uint8_input_config = None,
Optional[PreprocessingConfig] preprocessing_config = None,
layer_bias_correction = UNSET,
Optional[LayerBiasCorrectionConfig] layer_bias_correction_config = None,
Optional[str] config_save_path = None,
** kwargs )

Compile an existing Mobilint IR (.mblt) file into an MXQ package.

This is the pre-parsed half of mxq_compile(): parsing already happened (via mblt_compile()), so this function only quantizes and compiles. Quantization parameters, their defaults, and the configuration precedence rules are identical to mxq_compile() — see that function for the full reference and usage examples.

Parser-only arguments (save_subgraph_type, output_subgraph_path, feed_dict, dynamic_axes, in_dformats, yolo_decode_include, exclude_first_subgraph) are absent because the graph is already parsed. mxq_compile() accepts them for backward compatibility and drops them when routing here.

Parameters
mbltstring. Path to a runnable .mblt produced by mblt_compile(). The file is validated up front; a save_subgraph_type preview export is rejected.
target_devicestring. Target NPU device (for example "aries-rb").
backendstring. Retained for backward compatibility and ignored: the graph is already parsed, so no parser backend is selected.
Returns
None.
Exceptions
ValueErrormblt is not an existing .mblt file, or it is not a runnable artifact for the requested cpu_offload setting.

Definition at line 800 of file frontend.py.

◆ mblt_compile()

qbcompiler.frontend.mblt_compile ( str | Any model,
str mblt_save_path,
str target_device,
backend = "onnx",
device = "cpu",
feed_dict = None,
dynamic_axes = None,
in_dformats = None,
yolo_decode_include = False,
cpu_offload = False,
exclude_first_subgraph = False,
Optional[str] model_part = None,
Optional[dict] model_part_options = None,
** kwargs )

Export a model to the Mobilint .mblt format without producing an MXQ package.

Parameters
modelstring or model instance. Source model or path to compile.
mblt_save_pathstring. Output path for the .mblt artifact.
backendstring. Framework identifier, case-insensitive. "onnx" and "torch" are parsed by the current parser; "tf" (also spelled "tensorflow"), "tflite" and "torchscript" go through the legacy parser. Any other value raises ValueError. Defaults to "onnx".
devicestring. Compilation device ("cpu" or "gpu"). Defaults to "cpu".
feed_dictdict. Example inputs used for shape inference.
dynamic_axesdict. Declares dynamic axes per input name.
in_dformatsdict. Input dataformat metadata.
yolo_decode_includebool. Runs YOLO decode on NPU when True.
cpu_offloadbool. Enables CPU offloading for unsupported groups.
exclude_first_subgraphbool. Applies only when CPU offloading is enabled: exclude the first subgraph from the final graph if it is unsupported.
model_partstring. Selects a specific part of a torch model to parse ("vision", "language", "encoder", ...). Typically used to support models with complex architectures (e.g. Qwen3-VL) by compiling one part at a time. Parts a model declares: parser.patcher.parts.available_parts(model). A model declaring exactly one part resolves it from None; one declaring several requires a name.
model_part_optionsdict. Extra arguments for the named part (e.g. {"mel_frames": 100} for Qwen3-ASR audio).
kwargsdict. Additional arguments forwarded to the compiler.
Returns
None.

Definition at line 908 of file frontend.py.

◆ _mblt_compile_dispatch()

None qbcompiler.frontend._mblt_compile_dispatch ( str | Any model,
str mblt_save_path,
str target_device,
backend = "onnx",
device = UNSET,
feed_dict = None,
dynamic_axes = None,
in_dformats = None,
yolo_decode_include = False,
cpu_offload = UNSET,
exclude_first_subgraph = False,
Optional[str] model_part = None,
Optional[dict] model_part_options = None,
** kwargs )
protected

Route a compile-to-mblt call to the pipeline or the legacy parser.

Shared by :func:mblt_compile and :func:mblt_compile_with_callback so both validate target_device, normalize backend and pick the parser the same way. device / cpu_offload default to UNSET here rather than to concrete values: that is what lets the callback wrapper forward only the flags its caller actually set and leave the rest to the merged CompileConfig. :func:mblt_compile keeps its own historical "cpu" / False defaults and passes them explicitly.

Definition at line 965 of file frontend.py.

◆ _mblt_compile_legacy()

None qbcompiler.frontend._mblt_compile_legacy ( * ,
str | Any model,
str mblt_save_path,
str target_device,
str backend,
device = UNSET,
feed_dict = None,
dynamic_axes = None,
in_dformats = None,
yolo_decode_include = False,
cpu_offload = UNSET,
exclude_first_subgraph = False,
** kwargs )
protected

Export to .mblt through the legacy Model_Dict parser.

Reached for the backends the pipeline does not implement – tf, tflite, hf, torchscript and the model-specific misc parsers. This parser has no CompileConfig fallback for device / cpu_offload, so an unset value takes :func:mblt_compile's documented default.

Definition at line 1038 of file frontend.py.

◆ _mxq_compile_pipeline()

None qbcompiler.frontend._mxq_compile_pipeline ( model,
str target_device,
Union[str, List[str]] calib_data_path = UNSET,
int save_subgraph_type = 0,
output_subgraph_path = "",
Union[str, List[str]] save_path = UNSET,
backend = "onnx",
feed_dict = None,
dynamic_axes = None,
in_dformats = None,
yolo_decode_include = False,
exclude_first_subgraph = False,
device = UNSET,
inference_scheme = UNSET,
use_random_calib = UNSET,
cpu_offload = UNSET,
optimize_option = UNSET,
buffer_mode = UNSET,
input_shape_dict = UNSET,
force_npu_input_reposition = UNSET,
force_npu_output_reposition = UNSET,
image_channels = UNSET,
split_blocks = UNSET,
split_parts = UNSET,
Optional[str] config_preset = None,
Optional[CompileConfig | str] compile_config = None,
Optional[ResourceManagementConfig] resource_management_config = None,
Optional[CalibrationConfig] calibration_config = None,
Optional[BitConfig] bit_config = None,
Optional[LlmConfig] llm_config = None,
Optional[HessianQuantConfig] hessian_quant_config = None,
Optional[ModConfig] mod_config = None,
Optional[EquivalentTransformationConfig] equivalent_transformation_config = None,
Optional[SearchWeightScaleConfig] search_weight_scale_config = None,
Optional[SaveSampleConfig] save_sample_config = None,
Optional[Uint8InputConfig] uint8_input_config = None,
Optional[PreprocessingConfig] preprocessing_config = None,
layer_bias_correction = UNSET,
Optional[LayerBiasCorrectionConfig] layer_bias_correction_config = None,
Optional[str] model_part = None,
Optional[dict] model_part_options = None,
Optional[str] config_save_path = None,
** kwargs )
protected

Quantize and compile through the ConfigManager-resolved pipeline.

Serves the backends in MODEL_DICT_BACKENDS; the callers (:func:mxq_compile_from_source, :func:mxq_compile_from_mblt) have already validated target_device and normalized backend.

Definition at line 1085 of file frontend.py.

◆ _mblt_compile_pipeline()

None qbcompiler.frontend._mblt_compile_pipeline ( str | object model,
str target_device,
str mblt_save_path,
backend = "onnx",
object device = UNSET,
feed_dict = None,
dynamic_axes = None,
in_dformats = None,
yolo_decode_include = False,
object cpu_offload = UNSET,
exclude_first_subgraph = False,
Optional[str] model_part = None,
Optional[dict] model_part_options = None,
Optional[str] config_preset = None,
Optional[CompileConfig | str] compile_config = None,
Optional[str] compilation_mode = None,
** kwargs )
protected

Export to .mblt through the ConfigManager-resolved pipeline.

compilation_mode is an internal knob: one of {"release", "dev", "debug"} or None (default) to defer to the MBLT_APP_ENV environment variable. When set it overrides the env and drives the parser's inference_validation / device_alloc / log_level cascade — the same preset as MBLT_APP_ENV. Not surfaced in the CLI on purpose; intended for in-process scripts and tests that want scoped dev-mode validation without mutating process-wide environment.

Definition at line 1191 of file frontend.py.

◆ mblt_compile_with_callback()

None qbcompiler.frontend.mblt_compile_with_callback ( str model,
str mblt_save_path,
str target_device,
str backend = "onnx",
Optional[str] device = None,
Optional[bool] cpu_offload = None,
Optional[str] config_preset = None,
Optional[CompileConfig | str] compile_config = None,
Optional[str] compilation_mode = None,
* ,
Callable[[int, str], None] progress_callback )

Compile-to-mblt entry point with progress callbacks.

Only forwards explicitly-set values so the UNSET defaults (and the merged CompileConfig behind them) apply when the caller omits a flag – which is why this goes through _mblt_compile_dispatch rather than :func:mblt_compile, whose own device / cpu_offload defaults would override the config. target_device is required and has no config fallback — callers (e.g. the CLI) validate it via validate_target_device before invoking (QC-228).

Definition at line 1244 of file frontend.py.

◆ mxq_compile_with_callback()

None qbcompiler.frontend.mxq_compile_with_callback ( str model,
str target_device,
str save_path,
str backend = "onnx",
Optional[str] device = None,
Optional[Union[str, List[str]]] calib_data_path = None,
Optional[bool] use_random_calib = None,
Optional[str] config_preset = None,
Optional[CompileConfig | str] compile_config = None,
Optional[str] config_save_path = None,
* ,
Callable[[int, str], None] progress_callback )

Compile/quantize entry point with progress callbacks.

target_device is required and has no config fallback — callers (e.g. the CLI) validate it via validate_target_device before invoking, and the wrapper forwards it unconditionally (QC-228).

Like :func:mblt_compile_with_callback, only explicitly-set values are forwarded, so :func:mxq_compile's UNSET defaults (and the merged CompileConfig behind them) still apply to whatever the caller omits. config_save_path passes straight through, and the resolved config is written there before compilation starts; this is what backs the CLI's --config-save-path on quantize / compile.

Definition at line 1290 of file frontend.py.

◆ get_yaml()

qbcompiler.calibration.make_calib_data.get_yaml ( str yaml_path)

Definition at line 29 of file make_calib_data.py.

◆ make_calib_man()

qbcompiler.calibration.make_calib_data.make_calib_man ( pre_ftn,
str data_dir,
str save_dir,
str save_name,
str anno_json = None,
str file_format = "%012d.jpg",
int max_size = -1,
bool remove_npy = False,
int seed = 2023,
bool save_calib_msg = False,
str msg_path = None )

From images and a user-provided preprocessing function, generate calibration NumPy files and a manifest.

Parameters
pre_ftnCallable. Pre-processing function that accepts an image path and returns a NumPy array.
data_dirstring. Directory containing the raw calibration images.
save_dirstring. Directory in which to store the generated NumPy files and metadata text file.
save_namestring. Basename for the saved assets. NumPy files are written to {save_dir}/{save_name}_npy and the manifest to {save_dir}/{save_name}.txt. Defaults to the basename of data_dir when omitted.
anno_jsonstring. Optional COCO-style annotation file. When provided, samples are randomly selected while maintaining class balance.
file_formatstring. Filename format that uses image_idx (for example "%012d.jpg").
max_sizeint. Maximum number of calibration samples. A value of -1 disables the limit.
remove_npybool. Remove existing NumPy outputs before generating new data.
seedint. Random seed applied to sampling and class-balanced selection. Defaults to 2023.
save_calib_msgbool. Persist the calibration metadata as an MSGpack file in addition to the manifest.
msg_pathstring. Explicit MSGpack output path. When omitted, the path is auto-generated by dataset name and sample count.
Returns
string. Path to the generated text file that lists the preprocessed NumPy samples.
Note
Calibration samples are written as float32 NumPy arrays derived from the pre_ftn output. When max_size is positive and the dataset is larger, a random subset is used. COCO annotations override this sampling to balance categories.

Definition at line 40 of file make_calib_data.py.

◆ make_calib()

str qbcompiler.calibration.make_calib_data.make_calib ( Union[str, dict] args_pre,
str data_dir,
str save_dir,
str save_name = None,
str anno_json = None,
str file_format = "%012d.jpg",
int max_size = -1,
bool remove_npy = False,
int seed = 2023,
bool save_calib_msg = False,
str msg_path = None )

Create calibration NumPy files and a manifest using a YAML or dict preprocessing configuration.

Parameters
args_prestring or dict. Path to a YAML file or an in-memory dictionary describing the preprocessing pipeline. See the preproc_builder class for all supported operators and parameters.
data_dirstring. Directory of calibration images.
save_dirstring. Directory where the generated NumPy files and manifest text file are stored.
save_namestring. Optional basename for outputs. Defaults to the basename of data_dir.
anno_jsonstring. Optional COCO-format annotation file used to sample images with class balance.
file_formatstring. Filename template built from image_idx (defaults to "%012d.jpg").
max_sizeint. Maximum number of calibration samples to generate; -1 keeps all available items.
remove_npybool. Remove existing NumPy outputs before running preprocessing.
seedint. Random seed used for sampling. Defaults to 2023.
save_calib_msgbool. Save calibration metadata as MSGpack alongside NumPy and text outputs.
msg_pathstring. Explicit MSGpack output path. Auto-generated from dataset name and sample count when omitted.
Returns
string. Path to the manifest text file that lists the generated NumPy samples.
Note
The preprocessing pipeline is constructed through preproc_builder and executed via make_calib_man. Use the YAML schema to combine primitives such as GetImage, Resize, Normalize, Pad, CenterCrop, and SetOrder in a Pre-Order list.

Definition at line 139 of file make_calib_data.py.

◆ make_calib_llm()

List[str] qbcompiler.calibration.make_calib_data.make_calib_llm ( Union[str, dict] args_pre,
str dataset_name,
Union[List[str], str] subset_list,
str text_field,
str split,
str save_dir,
int max_calib = 512,
Optional[str] save_prefix = None )

Generate calibration embeddings for LLM models using Hugging Face datasets.

Args: args_pre: YAML path or preprocessing configuration dictionary. Must set Datatype to "LLM". dataset_name: Name of the Hugging Face dataset to load. subset_list: List of dataset subsets (e.g. language splits) to iterate. split: HF dataset split to use. text_field: Key containing text in dataset entries. save_dir: Root directory where calibration samples are stored. max_calib: Maximum number of calibration samples per subset. save_prefix: Override prefix used in output directory naming. If None, derives from tokenizer directory name when available.

Returns: List of paths to generated calibration txt files, one per subset.

Definition at line 200 of file make_calib_data.py.

◆ preprocess_fnc()

qbcompiler.calibration.preprocess.preprocess_fnc ( fnc,
Datatype = "Image" )

Definition at line 28 of file preprocess.py.

Variable Documentation

◆ DTYPE_GETDATA_MAP

dict qbcompiler.calibration.preprocess.DTYPE_GETDATA_MAP = {"Image": "GetImage", "Npy": "GetNpy", "LLM": "GetText"}

Definition at line 24 of file preprocess.py.

Generated by doxygen 1.14.0