콘텐츠로 이동

qbcompiler.calibration.preprocess

Attributes

Name Description
DTYPE_GETDATA_MAP

DTYPE_GETDATA_MAP

DTYPE_GETDATA_MAP = {"Image": "GetImage", "Npy": "GetNpy", "LLM": "GetText"}

GetNpy

class GetNpy

Load NumPy data from an in-memory array or a .npy file.

Methods

Name Description
__init__
__call__ Args: npy_input (Union[np.ndarray, str]): Input .npy path or numpy data.

__init__

def __init__(self)

__call__

def __call__(self, npy_input: Union[np.ndarray, str]) -> np.ndarray

Args: npy_input (Union[np.ndarray, str]): Input .npy path or numpy data.

NpyNormalize

class NpyNormalize

Methods

Name Description
__init__
__call__

__init__

def __init__(
    self,
    mean: Union[List[float], np.ndarray],
    std: Union[List[float], np.ndarray],
    to_float_div255: bool = False,
)

__call__

def __call__(self, img: np.ndarray)

NpyPad

class NpyPad

Pad NumPy arrays to a desired shape.

Parameters

  • shape — list[int]. Expected output shape after padding.
  • padding_type — string. Padding mode, either "constant" or "reflect".
  • pad_val — float. Fill value when padding_mode is 'constant'. Defaults to 0.

Methods

Name Description
__init__
__call__
slice_padded_data

__init__

def __init__(
    self, shape: List[int], padding_type: str, pad_val: Optional[float] = 0
)

__call__

def __call__(self, data: np.ndarray)

slice_padded_data

def slice_padded_data(self, data)

GetImage

class GetImage

Retrieve an image tensor from a file path or NumPy array using OpenCV.

Parameters

  • to_float32 — bool. Cast the output image to float32 when True. Defaults to False.
  • channel_order — string. Channel order to enforce ("bgr", "rgb", "bw", "rgba", or "brga"). Defaults to "bgr".

Methods

Name Description
__init__
__call__ Args: img_input (Union[np.ndarray, str]): Input image path or numpy tensor.
get_image_data

__init__

def __init__(self, to_float32: bool = False, channel_order: str = "bgr")

__call__

def __call__(self, img_input: Union[np.ndarray, str]) -> np.ndarray

Args: img_input (Union[np.ndarray, str]): Input image path or numpy tensor.

Returns: keepdims: Contains image numpy tensor and image meta such as "ori_shape".

get_image_data

def get_image_data(self, img_input: str)

GetText

class GetText

Extract text strings from raw inputs.

Parameters

  • text_key — Optional key to pull from dictionary inputs.
  • strip — Remove leading and trailing whitespace when true.

Methods

Name Description
__init__
__call__

__init__

def __init__(self, text_key: Optional[str] = None, strip: bool = True)

__call__

def __call__(self, text_input: Union[str, Dict[str, str]])

TokenizeToEmbedding

class TokenizeToEmbedding

Convert tokenized text into embeddings using a Hugging Face model.

Parameters

  • tokenizer_dir — Directory containing tokenizer resources.
  • model_dir — Optional directory for the model weights; defaults to tokenizer_dir.
  • device — Device string determining where inference runs.
  • trust_remote_code — Whether to load custom code from the model repo.
  • min_seqlen — Minimum token length required for a sample.
  • max_seqlen — Maximum token length; longer sequences are truncated.
  • pad_to_max — When true, pad sequences up to max_seqlen with zeros.
  • dtype — NumPy dtype used for exported embeddings.

Methods

Name Description
__init__
__call__

__init__

def __init__(
    self,
    tokenizer_dir: str,
    model_dir: Optional[str] = None,
    device: Optional[str] = None,
    trust_remote_code: bool = False,
    min_seqlen: int = 0,
    max_seqlen: Optional[int] = None,
    pad_to_max: bool = False,
    dtype: str = "float32",
)

__call__

def __call__(self, text: str)

Pad

class Pad

Pad images to a target shape or to meet a size divisor.

Parameters

  • shape — tuple[int]. Expected output shape (h, w). Defaults to None.
  • size_divisor — int. Ensures width and height are divisible by this value. Defaults to None.
  • pad_val — float. Fill value used during padding. Defaults to 0.0.
  • right_bottom — bool. When True, pad on the right and bottom only. Defaults to False.

Methods

Name Description
__init__
__call__

__init__

def __init__(
    self,
    shape=None,
    size_divisor: int = None,
    pad_val: float = 0.0,
    right_bottom: bool = False,
)

__call__

def __call__(self, img: np.ndarray)

Normalize

class Normalize

Apply mean/std normalization to images.

Parameters

  • mean — list[float] or np.ndarray. Normalization mean.
  • std — list[float] or np.ndarray. Normalization standard deviation.
  • to_float_div255 — bool. Divide by 255 before normalization when True. Defaults to False.

Methods

Name Description
__init__
__call__

__init__

def __init__(
    self,
    mean: Union[List[float], np.ndarray],
    std: Union[List[float], np.ndarray],
    to_float_div255: bool = False,
)

__call__

def __call__(self, img: np.ndarray)

ResizeTorch

class ResizeTorch

Methods

Name Description
__init__
__call__

__init__

def __init__(self, size: List[int], interpolation: str)

__call__

def __call__(self, img: np.ndarray)

Resize

class Resize

Resize images with optional aspect-ratio preservation.

Parameters

  • img_scale — float or tuple[int, int]. Scaling factor or target size (h, w).
  • keep_ratio — bool. Preserve aspect ratio when True. Defaults to False.
  • interpolation — string. Interpolation mode ("nearest", "bilinear", "bicubic", "area", or "lanczos").

Methods

Name Description
__init__
__call__

__init__

def __init__(
    self,
    img_scale: Union[float, Tuple[int, int]],
    keep_ratio: bool,
    interpolation: str,
)

__call__

def __call__(self, img: np.ndarray)

CenterCrop

class CenterCrop

Perform a centered crop on the image.

Parameters

  • size — list[int]. Target height and width.

Methods

Name Description
__init__
__call__

__init__

def __init__(self, size: List[int])

__call__

def __call__(self, img: np.ndarray):
    # img_tensor = torch.from_numpy(img).permute(2, 0, 1).unsqueeze(0)
    # img_tensor = F.center_crop(img_tensor, self.size)
    # result.img = img_tensor.squeeze(0).permute(1, 2, 0).numpy().astype(np.uint8)

    # h, w = img.shape[:2] # h, w, c
    # h_target = min(self.size[0], h)
    # w_target = min(self.size[1], w)

    # if h_target == h and w_target == w:
    #     return result

    # x_begin = round(w - w_target/2)
    # x_end = x_begin + w_target
    # y_begin = round(h - h_target/2)
    # y_end = y_begin + h_target

    # result.img = img[y_begin:y_end, x_begin:x_end, :]

YoloPre

class YoloPre

Apply YOLO-style resize, padding, and normalization to images.

Parameters

  • size — Target image size provided as [height, width].

Methods

Name Description
__init__
__call__

__init__

def __init__(self, size: List[int])

__call__

def __call__(self, img: np.ndarray)

SetOrder

class SetOrder

Methods

Name Description
__init__
__call__

__init__

def __init__(self, shape: str = "HWC")

__call__

def __call__(self, img: np.ndarray)

preproc_builder

class preproc_builder

Build a calibration preprocessing pipeline from a YAML or dict configuration.

Supported preprocessing types:

  • Datatype: Selects how input data is loaded. Supported value: Image.
  • GetImage: Loads an image tensor via OpenCV or PIL. Must appear before other operators.
  • Pre-Order: Explicit list of preprocessing stages to execute sequentially; entries must match the preprocessing definitions.
  • Pad: Adds padding to the tensor.
  • Normalize: Applies mean/std normalization.
  • ResizeTorch: Resizes images using torchvision.transforms.functional.resize.
  • Resize: Resizes images using cv2.resize.
  • CenterCrop: Performs a center crop.
  • SetOrder: Reorders axes; must be the final operator.

YAML structure

[Pre-processing Type]
    [Parameter]: [Argument]
    ...

# Example
Datatype: Image
GetImage:
  to_float32: false
  channel_order: RGB
Pre-Order: [ResizeTorch, CenterCrop, Normalize, SetOrder]
Pre-processing:
    ResizeTorch:
        size: [256, 256]
        interpolation: blinear
    CenterCrop:
        size: [224, 224]
    Normalize:
        mean: [0.485, 0.456, 0.406]
        std: [0.229, 0.224, 0.225]
        to_float_div255: true
    SetOrder:
        shape: HWC

Operator parameters

  • GetImage: to_float32 (bool, default False); channel_order (string, default "bgr").
  • Pad: shape (tuple[int]); size_divisor (int); pad_val (float, default 0); right_bottom (bool, default False).
  • Normalize: mean (list/ndarray); std (list/ndarray); to_float_div255 (bool, default False).
  • ResizeTorch: size (list[int]); interpolation (nearest|bilinear|bicubic|box|hamming|lanczos).
  • Resize: img_scale (float or tuple[int, int]); keep_ratio (bool, default False); interpolation (nearest|bilinear|bicubic|area|lanczos).
  • CenterCrop: size (list[int]).
  • SetOrder: shape (HWC|CHW|BHWC|BCHW, default HWC).

Parameters

  • args — dict. Parsed preprocessing configuration.
  • to_tensor — bool. When True, converts the resulting NumPy array to a Torch tensor.

Methods

Name Description
__init__
__call__
__repre__
__str__
get_pre_list
check_yaml_pre

__init__

def __init__(self, args, to_tensor=False)

__call__

def __call__(self, img)

__repre__

def __repre__(self)

__str__

def __str__(self)

get_pre_list

def get_pre_list(self, args)

check_yaml_pre

def check_yaml_pre(self, args)

Functions

Name Description
preprocess_fnc

preprocess_fnc

def preprocess_fnc(fnc, Datatype="Image")