Quantization Configuration#
|
Mobilint SDK qb Compiler v0.12.0.0
MCS002-EN
|
Quantization configuration definitions used by Mobilint compiler for model quantization. More...
Classes | |
| class | qubee.configs.quantization_config.CalibrationConfig |
| Configuration for calibration during quantization. More... | |
| class | qubee.configs.quantization_config.BitConfig |
| Configuration for bit precision. More... | |
| class | qubee.configs.quantization_config.QuantizationConfig |
| Unified quantization configuration. More... | |
Functions | |
| qubee.configs.quantization_config.CalibrationConfig.__init__ (self, int quantization_method, int quantization_output, int quantization_mode, numbers.Number percentile, numbers.Number topk_ratio, int max_each, int max_total, int kernel_size, int stack_size, int hist_num_cluster, numbers.Number hist_search_percentile_min, numbers.Number hist_epsilon, numbers.Number hist_adjust_coeff, numbers.Number act_scale_min, numbers.Number weight_scale_min, numbers.Number min_clip_ratio, Dict layer_overrides, bool optional=optional) | |
| Initialize the CalibrationConfig. | |
| qubee.configs.quantization_config.CalibrationConfig.default_config (cls) | |
| qubee.configs.quantization_config.CalibrationConfig.from_kwargs (cls, **kwargs) | |
| qubee.configs.quantization_config.CalibrationConfig.from_dict (cls, dict config_dict) | |
| Create CalibrationConfig from dictionary (JSON structure) | |
| qubee.configs.quantization_config.CalibrationConfig.to_dict (self) | |
| qubee.configs.quantization_config.BitConfig.__init__ (self, int query_act_bits, int key_act_bits, int value_act_bits, int output_act_bits, int ffn_act_bits, int head_act_bits, int query_weight_bits, int key_weight_bits, int value_weight_bits, int output_weight_bits, int ffn_weight_bits, int head_weight_bits, bool mixed_precision_apply, str save_path, str load_path, List[str] activation_16bits, List[str] weight_16bits, bool optional=optional) | |
| Initialize the BitConfig. | |
| qubee.configs.quantization_config.BitConfig.default_config (cls) | |
| qubee.configs.quantization_config.BitConfig.from_kwargs (cls, **kwargs) | |
| qubee.configs.quantization_config.BitConfig.from_dict (cls, dict config_dict) | |
| Create BitConfig from dictionary (JSON structure) | |
| qubee.configs.quantization_config.BitConfig.to_dict (self) | |
| qubee.configs.quantization_config.QuantizationConfig.__init__ (self, bool optional, Optional[CalibrationConfig] calibration=None, Optional[BitConfig] bit=None) | |
| Initialize the QuantizationConfig. | |
| qubee.configs.quantization_config.QuantizationConfig.default_config (cls) | |
| qubee.configs.quantization_config.QuantizationConfig.from_kwargs (cls, **kwargs) | |
| qubee.configs.quantization_config.QuantizationConfig.from_dict (cls, dict config_dict) | |
| qubee.configs.quantization_config.QuantizationConfig.to_dict (self) | |
| CalibrationConfig | qubee.configs.quantization_config.get_calibration_config (**kwargs) |
| Create CalibrationConfig with partial parameters merged with defaults. | |
| BitConfig | qubee.configs.quantization_config.get_bit_config (**kwargs) |
| Create BitConfig with partial parameters merged with defaults. | |
| QuantizationConfig | qubee.configs.quantization_config.get_quantization_config (Optional[CalibrationConfig] calibration=None, Optional[BitConfig] bit=None) |
| Create QuantizationConfig with partial parameters merged with defaults. | |
Detailed Description
Quantization configuration definitions used by Mobilint compiler for model quantization.
Function Documentation
◆ __init__() [1/3]
| qubee.configs.quantization_config.CalibrationConfig.__init__ | ( | self, | |
| int | quantization_method, | ||
| int | quantization_output, | ||
| int | quantization_mode, | ||
| numbers.Number | percentile, | ||
| numbers.Number | topk_ratio, | ||
| int | max_each, | ||
| int | max_total, | ||
| int | kernel_size, | ||
| int | stack_size, | ||
| int | hist_num_cluster, | ||
| numbers.Number | hist_search_percentile_min, | ||
| numbers.Number | hist_epsilon, | ||
| numbers.Number | hist_adjust_coeff, | ||
| numbers.Number | act_scale_min, | ||
| numbers.Number | weight_scale_min, | ||
| numbers.Number | min_clip_ratio, | ||
| Dict | layer_overrides, | ||
| bool | optional = optional ) |
Initialize the CalibrationConfig.
- Parameters
-
quantization_method int. Calibration method index:
0: WChALayer - Weight per-channel, Activation per-layer.
1: WChAMulti - Weight per-channel, Activation multi-layer.
2: WChALayerZeropoint - Weight per-channel, Activation per-layer with zeropoint.
3: WChAMultiZeropoint - Weight per-channel, Activation multi-layer with zeropoint.
quantization_output int. Output quantization type index:
0: Layer - Per-layer quantization.
1: Ch - Per-channel quantization.
2: Sigmoid - Sigmoid-based quantization.
quantization_mode int. Quantization mode index:
0: percentile.
1: max.
2: maxPercentile.
3: fastPercentile.
4: histogramKL.
5: histogramMSE.
percentile float. Percentile value used when quantization_modeis 0 (percentile).topk_ratio float. Top-k ratio used when quantization_modeis 2 (maxPercentile).max_each int. Maximum number of samples processed per iteration. max_total int. Total maximum number of samples processed across all iterations. kernel_size int. Kernel size used for fast distribution estimation (typically for quantization_mode3).stack_size int. Stack size used for fast distribution estimation (typically for quantization_mode3).hist_num_cluster int. Number of clusters/bins used in histogram-based calibration modes. hist_search_percentile_min float. Minimum search percentile for histogram-based calibration. hist_epsilon float. Numerical epsilon used in histogram-based calibration to avoid instability. hist_adjust_coeff float. Adjustment coefficient applied in histogram-based calibration. act_scale_min float. Minimum allowed activation scale (lower bound clamp). weight_scale_min float. Minimum allowed weight scale (lower bound clamp). min_clip_ratio float. Minimum clip ratio constraint applied during calibration. layer_overrides Dict. Layer-specific override settings. Keys are layer identifiers; values override one or more calibration parameters for the specified layer. optional bool. Indicates whether this config is optional. Use the default value defined as a class variable.
Definition at line 92 of file quantization_config.py.
◆ default_config() [1/3]
| qubee.configs.quantization_config.CalibrationConfig.default_config | ( | cls | ) |
Definition at line 170 of file quantization_config.py.
◆ from_kwargs() [1/3]
| qubee.configs.quantization_config.CalibrationConfig.from_kwargs | ( | cls, | |
| ** | kwargs ) |
Definition at line 174 of file quantization_config.py.
◆ from_dict() [1/3]
| qubee.configs.quantization_config.CalibrationConfig.from_dict | ( | cls, | |
| dict | config_dict ) |
Create CalibrationConfig from dictionary (JSON structure)
Definition at line 190 of file quantization_config.py.
◆ to_dict() [1/3]
| qubee.configs.quantization_config.CalibrationConfig.to_dict | ( | self | ) |
Definition at line 248 of file quantization_config.py.
◆ __init__() [2/3]
| qubee.configs.quantization_config.BitConfig.__init__ | ( | self, | |
| int | query_act_bits, | ||
| int | key_act_bits, | ||
| int | value_act_bits, | ||
| int | output_act_bits, | ||
| int | ffn_act_bits, | ||
| int | head_act_bits, | ||
| int | query_weight_bits, | ||
| int | key_weight_bits, | ||
| int | value_weight_bits, | ||
| int | output_weight_bits, | ||
| int | ffn_weight_bits, | ||
| int | head_weight_bits, | ||
| bool | mixed_precision_apply, | ||
| str | save_path, | ||
| str | load_path, | ||
| List[str] | activation_16bits, | ||
| List[str] | weight_16bits, | ||
| bool | optional = optional ) |
Initialize the BitConfig.
- Parameters
-
query_act_bits int. Query activation bit-width. key_act_bits int. Key activation bit-width. value_act_bits int. Value activation bit-width. output_act_bits int. Output activation bit-width. ffn_act_bits int. FFN activation bit-width. head_act_bits int. Head activation bit-width. query_weight_bits int. Query weight bit-width. key_weight_bits int. Key weight bit-width. value_weight_bits int. Value weight bit-width. output_weight_bits int. Output weight bit-width. ffn_weight_bits int. FFN weight bit-width. head_weight_bits int. Head weight bit-width. mixed_precision_apply bool. If true, apply mixed-precision according to the specified bit-widths. save_path str. (optional) Path to save the bit allocation. If empty, the allocation is not saved. load_path str. (optional) Path to load the bit allocation. If empty, allocation is not loaded. activation_16bits List[str]. (optional) Layer names to force 16-bit activations. weight_16bits List[str]. (optional) Layer names to force 16-bit weights.
Definition at line 352 of file quantization_config.py.
◆ default_config() [2/3]
| qubee.configs.quantization_config.BitConfig.default_config | ( | cls | ) |
Definition at line 415 of file quantization_config.py.
◆ from_kwargs() [2/3]
| qubee.configs.quantization_config.BitConfig.from_kwargs | ( | cls, | |
| ** | kwargs ) |
Definition at line 419 of file quantization_config.py.
◆ from_dict() [2/3]
| qubee.configs.quantization_config.BitConfig.from_dict | ( | cls, | |
| dict | config_dict ) |
Create BitConfig from dictionary (JSON structure)
Definition at line 435 of file quantization_config.py.
◆ to_dict() [2/3]
| qubee.configs.quantization_config.BitConfig.to_dict | ( | self | ) |
Definition at line 499 of file quantization_config.py.
◆ __init__() [3/3]
| qubee.configs.quantization_config.QuantizationConfig.__init__ | ( | self, | |
| bool | optional, | ||
| Optional[CalibrationConfig] | calibration = None, | ||
| Optional[BitConfig] | bit = None ) |
Initialize the QuantizationConfig.
- Parameters
-
calibration CalibrationConfig. Calibration configuration. bit BitConfig. Bit precision configuration.
Definition at line 556 of file quantization_config.py.
◆ default_config() [3/3]
| qubee.configs.quantization_config.QuantizationConfig.default_config | ( | cls | ) |
Definition at line 572 of file quantization_config.py.
◆ from_kwargs() [3/3]
| qubee.configs.quantization_config.QuantizationConfig.from_kwargs | ( | cls, | |
| ** | kwargs ) |
Definition at line 580 of file quantization_config.py.
◆ from_dict() [3/3]
| qubee.configs.quantization_config.QuantizationConfig.from_dict | ( | cls, | |
| dict | config_dict ) |
Definition at line 588 of file quantization_config.py.
◆ to_dict() [3/3]
| qubee.configs.quantization_config.QuantizationConfig.to_dict | ( | self | ) |
Definition at line 597 of file quantization_config.py.
◆ get_calibration_config()
| CalibrationConfig qubee.configs.quantization_config.get_calibration_config | ( | ** | kwargs | ) |
Create CalibrationConfig with partial parameters merged with defaults.
Definition at line 606 of file quantization_config.py.
◆ get_bit_config()
| BitConfig qubee.configs.quantization_config.get_bit_config | ( | ** | kwargs | ) |
Create BitConfig with partial parameters merged with defaults.
Definition at line 615 of file quantization_config.py.
◆ get_quantization_config()
| QuantizationConfig qubee.configs.quantization_config.get_quantization_config | ( | Optional[CalibrationConfig] | calibration = None, |
| Optional[BitConfig] | bit = None ) |
Create QuantizationConfig with partial parameters merged with defaults.
Definition at line 624 of file quantization_config.py.
Variable Documentation
◆ optional [1/3]
|
static |
Definition at line 22 of file quantization_config.py.
◆ method_list
|
static |
Definition at line 23 of file quantization_config.py.
◆ output_list
|
static |
Definition at line 24 of file quantization_config.py.
◆ mode_list
|
static |
Initial value:
= [
"percentile",
"max",
"maxPercentile",
"fastPercentile",
"histogramKL",
"histogramMSE",
]
Definition at line 25 of file quantization_config.py.
◆ DEFAULTS [1/2]
|
static |
Initial value:
= {
"quantization_method": 1,
"quantization_output": 0,
"quantization_mode": 2,
"percentile": 0.9999,
"topk_ratio": 0.01,
"max_each": 128,
"max_total": 65536,
"kernel_size": 9,
"stack_size": 32768,
"hist_num_cluster": 4,
"hist_search_percentile_min": 0.9999,
"hist_epsilon": 0.01,
"hist_adjust_coeff": 0.5,
"act_scale_min": 0.0005,
"weight_scale_min": 1e-6,
"min_clip_ratio": -1,
"layer_overrides": {"actScaleMin": {"0.0005": []}},
}
Definition at line 34 of file quantization_config.py.
◆ py_to_cpp_name [1/3]
|
static |
Initial value:
= {
"quantization_method": "method",
"quantization_output": "output",
"quantization_mode": "mode",
"percentile": "percentile",
"topk_ratio": "topKRatio",
"max_each": "maxEach",
"max_total": "maxTotal",
"kernel_size": "kernelSize",
"stack_size": "stackSize",
"hist_num_cluster": "numCluster",
"hist_search_percentile_min": "searchPercentileMin",
"hist_epsilon": "epsilon",
"hist_adjust_coeff": "adjustCoeff",
"act_scale_min": "actScaleMin",
"weight_scale_min": "weightScaleMin",
"min_clip_ratio": "minClipRatio",
}
Definition at line 54 of file quantization_config.py.
◆ min_max_check [1/3]
|
static |
Initial value:
= {
"quantization_method": [0, 3],
"quantization_output": [0, 2],
"quantization_mode": [0, 5],
"percentile": [0, 1],
"topk_ratio": [0, 1],
"max_each": [0, None],
"max_total": [0, None],
"kernel_size": [0, None],
"stack_size": [0, None],
"hist_num_cluster": [0, None],
"hist_search_percentile_min": [0, 1],
"hist_epsilon": [0, 1],
"hist_adjust_coeff": [0, None],
"act_scale_min": [0, 1],
"weight_scale_min": [0, 1],
"min_clip_ratio": [-1, 1],
}
Definition at line 73 of file quantization_config.py.
◆ quantization_method
| qubee.configs.quantization_config.CalibrationConfig.quantization_method = quantization_method |
Definition at line 150 of file quantization_config.py.
◆ quantization_output
| qubee.configs.quantization_config.CalibrationConfig.quantization_output = quantization_output |
Definition at line 151 of file quantization_config.py.
◆ quantization_mode
| qubee.configs.quantization_config.CalibrationConfig.quantization_mode = quantization_mode |
Definition at line 152 of file quantization_config.py.
◆ percentile
| qubee.configs.quantization_config.CalibrationConfig.percentile = percentile |
Definition at line 153 of file quantization_config.py.
◆ topk_ratio
| qubee.configs.quantization_config.CalibrationConfig.topk_ratio = topk_ratio |
Definition at line 154 of file quantization_config.py.
◆ max_each
| qubee.configs.quantization_config.CalibrationConfig.max_each = max_each |
Definition at line 155 of file quantization_config.py.
◆ max_total
| qubee.configs.quantization_config.CalibrationConfig.max_total = max_total |
Definition at line 156 of file quantization_config.py.
◆ kernel_size
| qubee.configs.quantization_config.CalibrationConfig.kernel_size = kernel_size |
Definition at line 157 of file quantization_config.py.
◆ stack_size
| qubee.configs.quantization_config.CalibrationConfig.stack_size = stack_size |
Definition at line 158 of file quantization_config.py.
◆ hist_num_cluster
| qubee.configs.quantization_config.CalibrationConfig.hist_num_cluster = hist_num_cluster |
Definition at line 159 of file quantization_config.py.
◆ hist_search_percentile_min
| qubee.configs.quantization_config.CalibrationConfig.hist_search_percentile_min = hist_search_percentile_min |
Definition at line 160 of file quantization_config.py.
◆ hist_epsilon
| qubee.configs.quantization_config.CalibrationConfig.hist_epsilon = hist_epsilon |
Definition at line 161 of file quantization_config.py.
◆ hist_adjust_coeff
| qubee.configs.quantization_config.CalibrationConfig.hist_adjust_coeff = hist_adjust_coeff |
Definition at line 162 of file quantization_config.py.
◆ act_scale_min
| qubee.configs.quantization_config.CalibrationConfig.act_scale_min = act_scale_min |
Definition at line 163 of file quantization_config.py.
◆ weight_scale_min
| qubee.configs.quantization_config.CalibrationConfig.weight_scale_min = weight_scale_min |
Definition at line 164 of file quantization_config.py.
◆ min_clip_ratio
| qubee.configs.quantization_config.CalibrationConfig.min_clip_ratio = min_clip_ratio |
Definition at line 165 of file quantization_config.py.
◆ layer_overrides
| qubee.configs.quantization_config.CalibrationConfig.layer_overrides = layer_overrides |
Definition at line 166 of file quantization_config.py.
◆ optional [2/3]
|
static |
Definition at line 296 of file quantization_config.py.
◆ DEFAULTS [2/2]
|
static |
Initial value:
= {
"query_act_bits": 8,
"key_act_bits": 8,
"value_act_bits": 8,
"output_act_bits": 16,
"ffn_act_bits": 16,
"head_act_bits": 8,
"query_weight_bits": 8,
"key_weight_bits": 8,
"value_weight_bits": 8,
"output_weight_bits": 8,
"ffn_weight_bits": 8,
"head_weight_bits": 8,
"mixed_precision_apply": False,
"save_path": "",
"load_path": "",
"activation_16bits": [],
"weight_16bits": [],
}
Definition at line 298 of file quantization_config.py.
◆ py_to_cpp_name [2/3]
|
static |
Initial value:
= {
"query_act_bits": "queryActBits",
"key_act_bits": "keyActBits",
"value_act_bits": "valueActBits",
"output_act_bits": "outputActBits",
"ffn_act_bits": "ffnActBits",
"head_act_bits": "headActBits",
"query_weight_bits": "queryWeightBits",
"key_weight_bits": "keyWeightBits",
"value_weight_bits": "valueWeightBits",
"output_weight_bits": "outputWeightBits",
"ffn_weight_bits": "ffnWeightBits",
"head_weight_bits": "headWeightBits",
"mixed_precision_apply": "mixedPrecisionApply",
"save_path": "savePath",
"load_path": "loadPath",
}
Definition at line 318 of file quantization_config.py.
◆ min_max_check [2/3]
|
static |
Initial value:
= {
"query_act_bits": [1, None],
"key_act_bits": [1, None],
"value_act_bits": [1, None],
"output_act_bits": [1, None],
"ffn_act_bits": [1, None],
"head_act_bits": [1, None],
"query_weight_bits": [1, None],
"key_weight_bits": [1, None],
"value_weight_bits": [1, None],
"output_weight_bits": [1, None],
"ffn_weight_bits": [1, None],
"head_weight_bits": [1, None],
}
Definition at line 337 of file quantization_config.py.
◆ query_act_bits
| qubee.configs.quantization_config.BitConfig.query_act_bits = query_act_bits |
Definition at line 395 of file quantization_config.py.
◆ key_act_bits
| qubee.configs.quantization_config.BitConfig.key_act_bits = key_act_bits |
Definition at line 396 of file quantization_config.py.
◆ value_act_bits
| qubee.configs.quantization_config.BitConfig.value_act_bits = value_act_bits |
Definition at line 397 of file quantization_config.py.
◆ output_act_bits
| qubee.configs.quantization_config.BitConfig.output_act_bits = output_act_bits |
Definition at line 398 of file quantization_config.py.
◆ ffn_act_bits
| qubee.configs.quantization_config.BitConfig.ffn_act_bits = ffn_act_bits |
Definition at line 399 of file quantization_config.py.
◆ head_act_bits
| qubee.configs.quantization_config.BitConfig.head_act_bits = head_act_bits |
Definition at line 400 of file quantization_config.py.
◆ query_weight_bits
| qubee.configs.quantization_config.BitConfig.query_weight_bits = query_weight_bits |
Definition at line 401 of file quantization_config.py.
◆ key_weight_bits
| qubee.configs.quantization_config.BitConfig.key_weight_bits = key_weight_bits |
Definition at line 402 of file quantization_config.py.
◆ value_weight_bits
| qubee.configs.quantization_config.BitConfig.value_weight_bits = value_weight_bits |
Definition at line 403 of file quantization_config.py.
◆ output_weight_bits
| qubee.configs.quantization_config.BitConfig.output_weight_bits = output_weight_bits |
Definition at line 404 of file quantization_config.py.
◆ ffn_weight_bits
| qubee.configs.quantization_config.BitConfig.ffn_weight_bits = ffn_weight_bits |
Definition at line 405 of file quantization_config.py.
◆ head_weight_bits
| qubee.configs.quantization_config.BitConfig.head_weight_bits = head_weight_bits |
Definition at line 406 of file quantization_config.py.
◆ mixed_precision_apply
| qubee.configs.quantization_config.BitConfig.mixed_precision_apply = mixed_precision_apply |
Definition at line 407 of file quantization_config.py.
◆ save_path
| qubee.configs.quantization_config.BitConfig.save_path = save_path |
Definition at line 408 of file quantization_config.py.
◆ load_path
| qubee.configs.quantization_config.BitConfig.load_path = load_path |
Definition at line 409 of file quantization_config.py.
◆ activation_16bits
| qubee.configs.quantization_config.BitConfig.activation_16bits = activation_16bits |
Definition at line 410 of file quantization_config.py.
◆ weight_16bits
| qubee.configs.quantization_config.BitConfig.weight_16bits = weight_16bits |
Definition at line 411 of file quantization_config.py.
◆ optional [3/3]
|
static |
Definition at line 547 of file quantization_config.py.
◆ py_to_cpp_name [3/3]
|
static |
Initial value:
= {
"calibration": "calibration",
"bit": "bit",
}
Definition at line 548 of file quantization_config.py.
◆ min_max_check [3/3]
|
static |
Definition at line 553 of file quantization_config.py.
◆ calibration
| qubee.configs.quantization_config.QuantizationConfig.calibration = calibration or CalibrationConfig.default_config() |
Definition at line 568 of file quantization_config.py.
◆ bit
| qubee.configs.quantization_config.QuantizationConfig.bit = bit or BitConfig.default_config() |
Definition at line 569 of file quantization_config.py.
Generated by
1.14.0