Vision

Vision#

Vision model accuracy depends on the exact input format used at compile time. The model may have been trained with normalized floating-point tensors, but the deployed application may naturally produce raw RGB uint8 images. qb Compiler can support both styles when the compile configuration, calibration data, and inference path use the same input format.

Image Input Format#

Start by identifying what the exported source model expects:

  • color order: RGB or BGR

  • dtype: usually uint8, float32, or framework-specific tensor dtype

  • numeric range: 0..255, 0..1, or normalized values

  • layout: HWC, NHWC, or NCHW

  • spatial size: fixed, dynamic, or preset-specific such as 640 x 640

  • batch dimension: present or omitted in per-sample calibration files

Do not infer these values from the model family alone. A ResNet, YOLO, or ViT export can differ depending on the export script and whether preprocessing was included in the graph.

HWC, NHWC, and NCHW#

HWC means one image without an explicit batch dimension: height, width, channels. NHWC adds batch first. NCHW is the common PyTorch channel-first layout: batch, channels, height, width.

Older image calibration examples often save preprocessed samples as HWC NumPy arrays when compiling without CPU offloading. CPU-offloaded or framework-visible flows may instead require the original model input shape, such as NCHW. Use the shape expected by the compiler.

When accuracy is unexpectedly poor, verify layout first. A tensor with the right total element count but wrong channel order or axis order can compile successfully and still produce incorrect results.

uint8 Input#

uint8 input means the compiled model receives image pixels as integers in the 0..255 range. This is useful when the runtime application captures or decodes images and wants to avoid doing floating-point preprocessing before calling the NPU.

If the compiled model accepts uint8, calibration samples must also use uint8 input in the same format unless the compiler configuration explicitly fuses the conversion and normalization steps. Do not calibrate with normalized float32 samples and then run inference with raw uint8 samples through the same MXQ input.

Raw RGB Input#

Raw RGB input means the application supplies decoded RGB pixels before model-specific normalization. A typical raw RGB image has shape (H, W, 3) and dtype uint8.

Raw RGB is not the same as the tensor used to train many models. For example, TorchVision ImageNet models are commonly trained with RGB images that are converted to floating point, divided by 255, and normalized with ImageNet mean and standard deviation. If the runtime input is raw RGB, the compile configuration must account for that preprocessing so the NPU sees the same effective values as the trained model.

Normalization Fuse#

Normalization fuse moves simple preprocessing operations such as scale, mean subtraction, and standard deviation division into the compiled graph or compile-time input handling. This lets the application pass raw or lightly processed image data while the compiled model still receives normalized values internally.

Use normalization fuse only when the fused operation exactly matches the training and inference preprocessing:

  • same color order

  • same scaling factor, such as division by 255

  • same per-channel mean

  • same per-channel standard deviation

  • same channel order for those mean and standard deviation values

If normalization is fused, calibration must use the input format sent to the compiled model. For example, with raw RGB uint8 input plus fused ImageNet normalization, calibration samples should represent raw RGB uint8 images after resize/crop but before normalization.

Preprocessing Pipeline#

A vision preprocessing pipeline should be written as an ordered contract. For classification, the contract may be:

decode image -> RGB -> resize -> center crop -> uint8 or float conversion -> normalize -> layout conversion

For YOLO-family detection, the contract may be:

decode image -> RGB -> aspect-ratio resize -> letterbox padding -> scale -> layout conversion

Every step changes the numeric distribution seen during calibration. Resize interpolation, padding value, crop policy, and color conversion all matter.

Calibration and Inference Must Match#

The calibration data generator and inference preprocessing should share code or share a single written specification. If they diverge, the compiler calibrates one distribution and the deployed MXQ receives another.

Common mismatches include:

  • calibration uses RGB but inference uses BGR

  • calibration divides by 255 but inference sends 0..255

  • calibration normalizes but inference relies on a fuse that was not enabled

  • calibration uses HWC but inference sends NCHW

  • calibration uses center crop but inference uses resize without crop

  • yolo_640 calibration is reused for a yolo_1280 export

When debugging, save one runtime input tensor and compare it numerically with one calibration tensor produced from the same image.

Classification Preset#

Use classification when the model is a conventional classification network but does not exactly follow the TorchVision recipe. Use classification_torchvision when the model was exported from TorchVision weights and uses the standard ImageNet preprocessing pattern: RGB image decode, resize, crop, scaling by 255, and channel-wise normalization. Keep the compile config, calibration data, and runtime preprocessing aligned with the exported model.

Image classification models are usually the simplest vision models to compile. ResNet, EfficientNet, MobileNet, and similar CNN models often have a single image input and a single logits output.

CLI Examples#

Generic classification preset:

python -m qbcompiler compile \
  --model resnet50.onnx \
  --backend onnx \
  --target-device regulus-rb \
  --config-preset classification \
  --calib-data-path ./calib_resnet50 \
  --output resnet50.mxq

For TorchVision ImageNet weights, use classification_torchvision instead of the generic classification preset:

python -m qbcompiler compile \
  --model resnet50.onnx \
  --backend onnx \
  --target-device regulus-rb \
  --config-preset classification_torchvision \
  --calib-data-path ./calib_resnet50 \
  --output resnet50.mxq

Preprocessing Pipeline#

classification configures the classification calibration defaults only. classification_torchvision extends it and enables a three-channel uint8 image input plus the following preprocessing pipeline:

Step

Preset operation

Values

1

Resize

256 x 256, bilinear interpolation

2

Center crop

224 x 224

3

Scale

Convert uint8 pixel values from [0, 255] to floating-point values in [0.0, 1.0]

4

Normalize

Mean [0.485, 0.456, 0.406]; standard deviation [0.229, 0.224, 0.225]

The preset also enables autoConvertFormat, sets scaleToUint8=true for normalization, and sets fuseIntoFirstLayer=true. Prepare calibration images and runtime inputs for this same RGB, resize, crop, and normalization contract. Use the generic classification preset, or a custom config, when the model uses a different image size, interpolation, color order, scaling rule, or normalization values.

For this recipe, apply resize and center crop before calibration and runtime inference. The fused operation is normalization, so calibration samples should be cropped RGB uint8 images before normalization.

Checklist#

  • Export the model with fixed input shape, commonly 1x3x224x224.

  • Generate calibration tensors from images representative of the deployment data.

  • Match calibration preprocessing with inference preprocessing.

  • Verify whether the model expects NCHW, NHWC, or HWC at the compiled input.

  • After MXQ generation, compare top-k predictions with the original model.

Detection and YOLO Presets#

Use detection for general detection models that do not match a YOLO image size preset. Detection exports often differ in whether decode, NMS, and other postprocessing are inside the graph, so inspect the MBLT before assuming the runtime input and output interface. Use yolo_640 or yolo_1280 when the exported model input size and preprocessing pipeline match that square input size. The preset name must match the exported model shape. Do not compile a 1280-exported model with yolo_640 or a 640-exported model with yolo_1280.

YOLO models usually compile as an NPU body plus application-side or CPU-offloaded postprocessing. The convolutional feature extractor is the main NPU workload, while decode, NMS, slicing, concatenation, or output formatting can contain unsupported or low-value operators.

CLI Example#

python -m qbcompiler compile \
  --model yolov8n.onnx \
  --backend onnx \
  --target-device regulus-rb \
  --config-preset yolo_640 \
  --calib-data-path ./calib_yolo_640 \
  --output yolov8n_body.mxq

For YOLO models with a 1280 x 1280 input size, use yolo_1280.

Preset Details#

Both YOLO presets extend detection, which sets calibration.mode=1 and calibration.output=1. They enable three-channel uint8 input and autoConvertFormat, then apply the following letterbox operation:

Preset

Target size

Padding value

yolo_640

640 x 640

114

yolo_1280

1280 x 1280

114

The preset, exported ONNX or framework model shape, letterbox preprocessing, calibration tensors, and runtime input preparation must all agree on the same image size. Calibration should reproduce the training/inference letterbox path: color conversion, aspect-ratio-preserving resize, padding value, scaling, and layout conversion. If you compile only the NPU body, calibration data should match the body input. If you enable CPU Offloading and keep preprocessing or postprocessing in the compiled graph, calibration data should match the original model input.

Netron Inspection#

Use Netron before final compilation:

  • Open the parsed MBLT.

  • Confirm the main YOLO body is blue, meaning NPU-supported.

  • Identify CPU head, NPU body, and CPU tail regions.

Vision Transformer#

Vision Transformer models are sensitive to shape and layout because patch embedding and attention blocks introduce reshape, transpose, gather, and positional embedding patterns. Patch embedding and normalization behavior can make calibration more sensitive to preprocessing mismatch than in conventional CNNs. Use the vision_transformer preset as the starting point.

CLI Example#

python -m qbcompiler compile \
  --model vit.onnx \
  --backend onnx \
  --target-device regulus-rb \
  --config-preset vision_transformer \
  --calib-data-path ./calib_vit \
  --output vit.mxq

Preset Details#

The vision_transformer preset sets calibration.method=1 and calibration.mode=0. It also sets transformer activation precision to 16 bits for both output and ffn. This preset does not define image preprocessing; provide the resize, crop, scaling, color conversion, and normalization required by the exported model in the calibration and runtime paths.

For quantization configuration options including layer-level overrides, see Model Quantization — Quantization Configuration. For how to use presets, config files, and dump-config, see Compile Configuration. For transformer-architecture specific quantization (attention projections, FFN), see also Transformer.

Checklist#

  • Export with fixed image size and fixed sequence length.

  • Keep batch size fixed unless the model and target flow explicitly support another value.

  • Confirm patch embedding is represented by supported operators.

  • Check position embedding interpolation. Dynamic interpolation may need to be moved outside the compiled graph.

  • Use representative calibration images with the same resize, crop, scaling, and normalization used at inference.

Parse Failure Guidance#

If parse fails around patchify, reshape, or attention blocks, inspect the MBLT in Netron and decide whether to re-export the model, simplify dynamic shape logic, or compile an extracted body.

Multimodal#

Use multimodal when image input is only one part of the model input contract; in that case, image calibration must stay aligned with the text or other modality inputs in the same sample.