API Reference

Mobilint SDK qb Compiler v0.11.0.1 MCS002-EN

Classes | Functions | Variables

API Reference

Mobilint qb Model Compiler API provides functions for compiling models from various framework(onnx, pytorch, transformers, etc.) to MXQ format. More...

Classes
class	qubee.frontend.Model_Dict
	Wrapper around the Mobilint compiler to support compilation and inference workflows. More...
class	qubee.calibration.preprocess.GetNpy
	Load NumPy data from an in-memory array or a .npy file. More...
class	qubee.calibration.preprocess.NpyNormalize
class	qubee.calibration.preprocess.NpyPad
	Pad NumPy arrays to a desired shape. More...
class	qubee.calibration.preprocess.GetImage
	Retrieve an image tensor from a file path or NumPy array using OpenCV. More...
class	qubee.calibration.preprocess.Pad
	Pad images to a target shape or to meet a size divisor. More...
class	qubee.calibration.preprocess.Normalize
	Apply mean/std normalization to images. More...
class	qubee.calibration.preprocess.ResizeTorch
class	qubee.calibration.preprocess.Resize
	Resize images with optional aspect-ratio preservation. More...
class	qubee.calibration.preprocess.CenterCrop
	Perform a centered crop on the image. More...
class	qubee.calibration.preprocess.SetOrder
class	qubee.calibration.preprocess.preproc_builder
	Build a calibration preprocessing pipeline from a YAML or dict configuration. More...
class	GetText
	Extract text strings from raw inputs. More...
class	TokenizeToEmbedding
	Convert tokenized text into embeddings using a Hugging Face model. More...
class	YoloPre
	Apply YOLO-style resize, padding, and normalization to images. More...

Functions
	qubee.frontend.mxq_compile (model, str calib_data_path="", int save_subgraph_type=0, output_subgraph_path="", str model_nickname=None, str save_path=None, backend="onnx", device="cpu", feed_dict=None, dynamic_axes=None, in_dformats=None, quantize_method="maxpercentile", quantize_percentile=0.9999, topk_ratio=0.01, smooth_factor=1.6, min_clip_ratio=-1, act_scale_min=0.0005, is_quant_ch=True, is_asym_quant=False, save_sample=False, inference_scheme="single", yolo_decode_include=False, use_random_calib=False, cpu_offload=False, singlecore_compile=False, optimize_option=0, quant_output="layer", sample_dtype="float", preprocess_dict=None, buffer_mode=1, weight_dtype="float32", use_gpu_only_for_calibration=False, input_shape_dict=dict(), adaq_config=AdaQuantConfig.default_config(), optq_config=OptqConfig.default_config(), mod_config=ModConfig.default_config(), layer_config=LayerConfig.default_config(), llm_config=LlmConfig.default_config(), quantbit_config=QuantBitConfig.default_config(), **kwargs)
	Compile a model into a Mobilint eXeCUtable (MXQ) package for execution on Mobilint NPUs.
	qubee.frontend.mblt_compile (str|Any model, str mblt_save_path, backend="onnx", device="cpu", feed_dict=None, dynamic_axes=None, in_dformats=None, yolo_decode_include=False, cpu_offload=False, **kwargs)
	Export a model to the Mobilint .mblt format without producing an MXQ package.
	qubee.frontend.run_mblt_model (str path_to_mblt, bool save_all_outputs=True, Optional[str] inference_output_path=None, Optional[int] target_subgraph=None)
	qubee.calibration.make_calib_data.make_calib_man (pre_ftn, str data_dir, str save_dir, str save_name, str anno_json=None, str file_format="%012d.jpg", int max_size=-1, bool remove_npy=False, int seed=2023, bool save_calib_msg=False, str msg_path=None)
	From images and a user-provided preprocessing function, generate calibration NumPy files and a manifest.
str	qubee.calibration.make_calib_data.make_calib (Union[str, dict] args_pre, str data_dir, str save_dir, str save_name=None, str anno_json=None, str file_format="%012d.jpg", int max_size=-1, bool remove_npy=False, int seed=2023, bool save_calib_msg=False, str msg_path=None)
	Create calibration NumPy files and a manifest using a YAML or dict preprocessing configuration.
List[str]	qubee.calibration.make_calib_data.make_calib_llm (Union[str, dict] args_pre, str dataset_name, Union[List[str], str] subset_list, str text_field, str split, str save_dir, int max_calib=512, Optional[str] save_prefix=None)
	Generate calibration embeddings for LLM models using Hugging Face datasets.
	qubee.calibration.preprocess.preprocess_fnc (fnc, Datatype="Image")

Variables
dict	qubee.calibration.preprocess.DTYPE_GETDATA_MAP = {"Image": "GetImage", "Npy": "GetNpy", "LLM": "GetText"}

Detailed Description

Mobilint qb Model Compiler API provides functions for compiling models from various framework(onnx, pytorch, transformers, etc.) to MXQ format.

Function Documentation

◆ mxq_compile()

qubee.frontend.mxq_compile	(		model,
		str	calib_data_path = "",
		int	save_subgraph_type = 0,
			output_subgraph_path = "",
		str	model_nickname = None,
		str	save_path = None,
			backend = "onnx",
			device = "cpu",
			feed_dict = None,
			dynamic_axes = None,
			in_dformats = None,
			quantize_method = "maxpercentile",
			quantize_percentile = 0.9999,
			topk_ratio = 0.01,
			smooth_factor = 1.6,
			min_clip_ratio = -1,
			act_scale_min = 0.0005,
			is_quant_ch = True,
			is_asym_quant = False,
			save_sample = False,
			inference_scheme = "single",
			yolo_decode_include = False,
			use_random_calib = False,
			cpu_offload = False,
			singlecore_compile = False,
			optimize_option = 0,
			quant_output = "layer",
			sample_dtype = "float",
			preprocess_dict = None,
			buffer_mode = 1,
			weight_dtype = "float32",
			use_gpu_only_for_calibration = False,
			input_shape_dict = dict(),
			adaq_config = AdaQuantConfig.default_config(),
			optq_config = OptqConfig.default_config(),
			mod_config = ModConfig.default_config(),
			layer_config = LayerConfig.default_config(),
			llm_config = LlmConfig.default_config(),
			quantbit_config = QuantBitConfig.default_config(),
		**	kwargs )

Compile a model into a Mobilint eXeCUtable (MXQ) package for execution on Mobilint NPUs.

Parameters

model	string or model instance. Model path. When using backend="onnx", this should be the path to an ONNX model file. When backend is "torchscript", provide a TorchScript module; when "torch", provide a standard PyTorch model. For backend="tf", pass the directory that contains the TensorFlow SavedModel graph and assets. For backend="tflite", pass the TF Lite model path.
calib_data_path	string. Path to the calibration dataset. Accepts either a text/json file that lists NumPy files or a directory that contains the pre-processed NumPy files.
save_subgraph_type	int. Controls optional MBLT subgraph exports: 0 disables exports; 1 saves only the graph structure; 2 saves graph structure plus weights; 3 saves the graph structure split into multiple subgraphs; 4 saves both structure and weights split into multiple subgraphs.
output_subgraph_path	string. Destination path for the exported .mblt file when save_subgraph_type is 1, 2, 3, or 4. The resulting file can be used for visualization.
model_nickname	string. Friendly name stored in qubee. Defaults to the model basename (for example, the file "/workspace/onnx/resnet50.onnx" becomes "resnet50"). Falls back to "temporary" when it cannot be inferred.
save_path	string. Output MXQ filename. Defaults to "{model_nickname}.mxq" when omitted.
backend	string. Framework used to generate the Mobilint IR. Must be one of "onnx", "tf", "tflite", "torchscript", or "torch". Defaults to "onnx".
device	string. Compilation and inference device: "cpu" or "gpu". Defaults to "cpu".
feed_dict	dict. Example input tensors for shape inference and inference validation.
dynamic_axes	dict. Marks model axes as dynamic. Keys are input names and values map dimension indices to aliases (for example {"input": {2: "seq_len"}}).
in_dformats	dict. Describes input tensor data formats.
quantize_method	string. Quantization method. Supported values are "Max", "Percentile", "MaxPercentile", "FastPercentile", and "KL". Defaults to "Percentile".
quantize_percentile	float. Percentile value for percentile-based quantization; valid range is 0 to 1. Example values: 0.999 or 0.9999. Defaults to 0.9999.
topk_ratio	float. Applicable to "MaxPercentile" quantization; controls how much data is used for calibration. Recommended values are 0.01 or lower; defaults to 0.01.
smooth_factor	float. Gaussian kernel smoothing factor required for KL-divergence estimation. Defaults to 1.6.
min_clip_ratio	float. Ratio applied to minimum clipping of output scales. Defaults to -1 (disabled).
act_scale_min	float. Minimum activation scale used during quantization. Defaults to 0.0005.
is_quant_ch	bool. Enables multi-channel quantization when True; defaults to True.
is_asym_quant	bool. Enables asymmetric quantization when True; defaults to False.
save_sample	bool. Stores input/output tensors in ./sampleInOut when True to aid debugging. Defaults to False.
inference_scheme	string. NPU inference scheme. One of "single", "multi", "global", "global4", or "global8".
yolo_decode_include	bool. Determines whether YOLO decode runs on NPU.
use_random_calib	bool. Generates random calibration data to validate model compilability. Defaults to False.
cpu_offload	bool. Enables CPU offloading during NPU inference. Defaults to False.
singlecore_compile	bool. Uses singlecore compilation to maximize DRAM usage. Defaults to False.
optimize_option	int. Compiler optimization strategy selector.
quant_output	string. Quantization applied to the output layer. Accepts "layer" (per-layer), "ch" (per-channel; only when is_quant_ch is True), or "sigmoid" (use sigmoid-based scaling). Defaults to "layer".
sample_dtype	string. Data type for saved sample inference outputs: "float" or "int8".
preprocess_dict	int. Identifier for additional pre-processing metadata.
buffer_mode	int. Buffer serialization mode: 0 uses a naïve buffer, 1 uses an mmap-backed buffer. Defaults to 1.
weight_dtype	string. Weight dtype for calibration. Defaults to "float32".
use_gpu_only_for_calibration	bool. Restricts GPU usage to calibration passes. Defaults to False.
input_shape_dict	dict. Optional HWC input shape dictionary used for multi-shape compilation (for example {"input0": [[224, 224, 3], [256, 256, 3]]}).
adaq_config	AdaQuantConfig. Configures the AdaQuant algorithm, including learning rates for weights, bias, scales, batch size, and loop epochs. Defaults to disabled values.
optq_config	OptqConfig. Configures OPTQ, such as activation-order sorting, block size, and percentile dampening.
mod_config	ModConfig. Enables Mobilint Optimal Distillation algorithm overrides when provided.
layer_config	LayerConfig. Supplies per-layer overrides for enabling MOD/OPTQ, activation bit precision, and minimum activation scale values.
llm_config	LLMConfig. Large-language-model compilation options such as maximum sequence length, cache length, calibration sequence length, mask usage, starting DRAM offset, global-core utilization, batch size, and NPU core IDs.
quantbit_config	QuantBitConfig. Controls bit precision for Transformer submodules and optional import/export of per-layer precision configurations.
kwargs	dict. Additional compiler arguments propagated to the backend implementation.

Returns

None.

Tips for choosing quantization methods and selecting arguments

Increase the percentile value to widen the quantization interval. "Percentile" samples the calibration histogram and typically starts between 0.9999 and 0.999999. "FastPercentile" relies on reduced, smoothed samples and favors values between 0.99 and 0.99999. "MaxPercentile" operates on min/max stacks and commonly uses 0.9 to 0.9999. Channel-wise quantization (is_quant_ch=True) mitigates per-channel activation variance at the cost of longer compile times. Choose quant_output="ch" when output variance is channel-dependent; otherwise use "layer".

Compiling models with custom inputs (ONNX/Torch/TensorFlow)

Provide NumPy inputs whenever the model omits shape information so that qubee can infer unknown dimensions and data formats.

from qubee import mxq_compile

import numpy as np

example_input = {

"input_node_name_1": np.random.randn(1, 3, 224, 224).astype(np.float32),

"input_node_name_2": np.random.randn(1, 8, 224, 224).astype(np.float32),

"input_node_name_3": np.random.randn(1, 4, 56, 56).astype(np.float32),

}

in_dformats = {

"input_node_name_1": "NCHW",

"input_node_name_2": "NCHW",

"input_node_name_3": "NCHW",

}

onnx_model_path = "path/to/your/model.onnx"

mxq_compile(

model=onnx_model_path,

feed_dict=example_input,

in_dformats=in_dformats,

backend="onnx",

)

Definition at line 127 of file frontend.py.

◆ mblt_compile()

qubee.frontend.mblt_compile	(	str | Any	model,
		str	mblt_save_path,
			backend = "onnx",
			device = "cpu",
			feed_dict = None,
			dynamic_axes = None,
			in_dformats = None,
			yolo_decode_include = False,
			cpu_offload = False,
		**	kwargs )

Export a model to the Mobilint .mblt format without producing an MXQ package.

Parameters

model	string or model instance. Source model or path to compile.
mblt_save_path	string. Output path for the .mblt artifact.
backend	string. Framework identifier ("onnx", "tf", "tflite", "torchscript", or "torch"). Defaults to "onnx".
device	string. Compilation device ("cpu" or "gpu"). Defaults to "cpu".
feed_dict	dict. Example inputs used for shape inference.
dynamic_axes	dict. Declares dynamic axes per input name.
in_dformats	dict. Input dataformat metadata.
yolo_decode_include	bool. Runs YOLO decode on NPU when True.
cpu_offload	bool. Enables CPU offloading for unsupported groups.
kwargs	dict. Additional arguments forwarded to the compiler.

Returns

None.

Definition at line 355 of file frontend.py.

◆ run_mblt_model()

qubee.frontend.run_mblt_model	(	str	path_to_mblt,
		bool	save_all_outputs = True,
		Optional[str]	inference_output_path = None,
		Optional[int]	target_subgraph = None )

Definition at line 403 of file frontend.py.

◆ make_calib_man()

qubee.calibration.make_calib_data.make_calib_man	(		pre_ftn,
		str	data_dir,
		str	save_dir,
		str	save_name,
		str	anno_json = None,
		str	file_format = "%012d.jpg",
		int	max_size = -1,
		bool	remove_npy = False,
		int	seed = 2023,
		bool	save_calib_msg = False,
		str	msg_path = None )

From images and a user-provided preprocessing function, generate calibration NumPy files and a manifest.

Parameters

pre_ftn	Callable. Pre-processing function that accepts an image path and returns a NumPy array.
data_dir	string. Directory containing the raw calibration images.
save_dir	string. Directory in which to store the generated NumPy files and metadata text file.
save_name	string. Basename for the saved assets. NumPy files are written to {save_dir}/{save_name}_npy and the manifest to {save_dir}/{save_name}.txt. Defaults to the basename of data_dir when omitted.
anno_json	string. Optional COCO-style annotation file. When provided, samples are randomly selected while maintaining class balance.
file_format	string. Filename format that uses image_idx (for example "%012d.jpg").
max_size	int. Maximum number of calibration samples. A value of -1 disables the limit.
remove_npy	bool. Remove existing NumPy outputs before generating new data.
seed	int. Random seed applied to sampling and class-balanced selection. Defaults to 2023.
save_calib_msg	bool. Persist the calibration metadata as an MSGpack file in addition to the manifest.
msg_path	string. Explicit MSGpack output path. When omitted, the path is auto-generated by dataset name and sample count.

Returns

string. Path to the generated text file that lists the preprocessed NumPy samples.

Note

Calibration samples are written as float32 NumPy arrays derived from the pre_ftn output. When max_size is positive and the dataset is larger, a random subset is used. COCO annotations override this sampling to balance categories.

Definition at line 29 of file make_calib_data.py.

◆ make_calib()

str qubee.calibration.make_calib_data.make_calib	(	Union[str, dict]	args_pre,
		str	data_dir,
		str	save_dir,
		str	save_name = None,
		str	anno_json = None,
		str	file_format = "%012d.jpg",
		int	max_size = -1,
		bool	remove_npy = False,
		int	seed = 2023,
		bool	save_calib_msg = False,
		str	msg_path = None )

Create calibration NumPy files and a manifest using a YAML or dict preprocessing configuration.

Parameters

args_pre	string or dict. Path to a YAML file or an in-memory dictionary describing the preprocessing pipeline. See the preproc_builder class for all supported operators and parameters.
data_dir	string. Directory of calibration images.
save_dir	string. Directory where the generated NumPy files and manifest text file are stored.
save_name	string. Optional basename for outputs. Defaults to the basename of data_dir.
anno_json	string. Optional COCO-format annotation file used to sample images with class balance.
file_format	string. Filename template built from image_idx (defaults to "%012d.jpg").
max_size	int. Maximum number of calibration samples to generate; -1 keeps all available items.
remove_npy	bool. Remove existing NumPy outputs before running preprocessing.
seed	int. Random seed used for sampling. Defaults to 2023.
save_calib_msg	bool. Save calibration metadata as MSGpack alongside NumPy and text outputs.
msg_path	string. Explicit MSGpack output path. Auto-generated from dataset name and sample count when omitted.

Returns

string. Path to the manifest text file that lists the generated NumPy samples.

Note

The preprocessing pipeline is constructed through preproc_builder and executed via make_calib_man. Use the YAML schema to combine primitives such as GetImage, Resize, Normalize, Pad, CenterCrop, and SetOrder in a Pre-Order list.

Definition at line 128 of file make_calib_data.py.

◆ make_calib_llm()

List[str] qubee.calibration.make_calib_data.make_calib_llm	(	Union[str, dict]	args_pre,
		str	dataset_name,
		Union[List[str], str]	subset_list,
		str	text_field,
		str	split,
		str	save_dir,
		int	max_calib = 512,
		Optional[str]	save_prefix = None )

Generate calibration embeddings for LLM models using Hugging Face datasets.

Args: args_pre: YAML path or preprocessing configuration dictionary. Must set Datatype to "LLM". dataset_name: Name of the Hugging Face dataset to load. subset_list: List of dataset subsets (e.g. language splits) to iterate. split: HF dataset split to use. text_field: Key containing text in dataset entries. save_dir: Root directory where calibration samples are stored. max_calib: Maximum number of calibration samples per subset. save_prefix: Override prefix used in output directory naming. If None, derives from tokenizer directory name when available.

Returns: List of paths to generated calibration txt files, one per subset.

Definition at line 189 of file make_calib_data.py.

◆ preprocess_fnc()

qubee.calibration.preprocess.preprocess_fnc	(		fnc,
			Datatype = "Image" )

Definition at line 29 of file preprocess.py.

Variable Documentation

◆ DTYPE_GETDATA_MAP

dict qubee.calibration.preprocess.DTYPE_GETDATA_MAP = {"Image": "GetImage", "Npy": "GetNpy", "LLM": "GetText"}

Definition at line 25 of file preprocess.py.

---

Generated by  1.14.0