Quantization Configuration#
|
Mobilint SDK qb Compiler v1.0
MCS002-EN
|
Quantization configuration definitions used by Mobilint compiler for model quantization. More...
Classes | |
| class | qbcompiler.configs.quantization_config.CalibrationConfig |
| Configuration for calibration during quantization. More... | |
| class | qbcompiler.configs.quantization_config.BitConfig |
| Configuration for bit precision. More... | |
| class | qbcompiler.configs.quantization_config.QuantizationConfig |
| Unified quantization configuration. More... | |
Functions | |
| qbcompiler.configs.quantization_config.CalibrationConfig.__init__ (self, int quantization_method, int quantization_output, int quantization_mode, numbers.Number percentile, numbers.Number topk_ratio, int max_each, int max_total, numbers.Number hist_percentile, bool hist_use_gpu, int hist_num_bins, int hist_num_samples, int hist_buffer_size, numbers.Number hist_min_bin_width, numbers.Number hist_search_percentile_min, numbers.Number hist_search_percentile_max, int hist_num_search, int hist_search_type, numbers.Number act_scale_min, numbers.Number weight_scale_min, numbers.Number min_clip_ratio, Dict layer_overrides, bool optional=optional) | |
| Initialize the CalibrationConfig. | |
| qbcompiler.configs.quantization_config.CalibrationConfig.default_config (cls) | |
| qbcompiler.configs.quantization_config.CalibrationConfig.from_kwargs (cls, **kwargs) | |
| qbcompiler.configs.quantization_config.CalibrationConfig.from_dict (cls, dict config_dict) | |
| Create CalibrationConfig from dictionary (JSON structure) | |
| qbcompiler.configs.quantization_config.CalibrationConfig.to_dict (self) | |
| qbcompiler.configs.quantization_config.BitConfig.__init__ (self, int query_act_bits, int key_act_bits, int value_act_bits, int output_act_bits, int ffn_act_bits, int head_act_bits, int query_weight_bits, int key_weight_bits, int value_weight_bits, int output_weight_bits, int ffn_weight_bits, int head_weight_bits, bool mixed_precision_apply, str save_path, str load_path, List[str] activation_16bits, List[str] weight_16bits, bool optional=optional) | |
| Initialize the BitConfig. | |
| qbcompiler.configs.quantization_config.BitConfig.default_config (cls) | |
| qbcompiler.configs.quantization_config.BitConfig.from_kwargs (cls, **kwargs) | |
| qbcompiler.configs.quantization_config.BitConfig.from_dict (cls, dict config_dict) | |
| Create BitConfig from dictionary (JSON structure) | |
| qbcompiler.configs.quantization_config.BitConfig.to_dict (self) | |
| qbcompiler.configs.quantization_config.QuantizationConfig.__init__ (self, bool optional, Optional[CalibrationConfig] calibration=None, Optional[BitConfig] bit=None) | |
| Initialize the QuantizationConfig. | |
| qbcompiler.configs.quantization_config.QuantizationConfig.default_config (cls) | |
| qbcompiler.configs.quantization_config.QuantizationConfig.from_kwargs (cls, **kwargs) | |
| qbcompiler.configs.quantization_config.QuantizationConfig.from_dict (cls, dict config_dict) | |
| qbcompiler.configs.quantization_config.QuantizationConfig.to_dict (self) | |
| CalibrationConfig | qbcompiler.configs.quantization_config.get_calibration_config (**kwargs) |
| Create CalibrationConfig with partial parameters merged with defaults. | |
| BitConfig | qbcompiler.configs.quantization_config.get_bit_config (**kwargs) |
| Create BitConfig with partial parameters merged with defaults. | |
| QuantizationConfig | qbcompiler.configs.quantization_config.get_quantization_config (Optional[CalibrationConfig] calibration=None, Optional[BitConfig] bit=None) |
| Create QuantizationConfig with partial parameters merged with defaults. | |
Detailed Description
Quantization configuration definitions used by Mobilint compiler for model quantization.
Function Documentation
◆ __init__() [1/3]
| qbcompiler.configs.quantization_config.CalibrationConfig.__init__ | ( | self, | |
| int | quantization_method, | ||
| int | quantization_output, | ||
| int | quantization_mode, | ||
| numbers.Number | percentile, | ||
| numbers.Number | topk_ratio, | ||
| int | max_each, | ||
| int | max_total, | ||
| numbers.Number | hist_percentile, | ||
| bool | hist_use_gpu, | ||
| int | hist_num_bins, | ||
| int | hist_num_samples, | ||
| int | hist_buffer_size, | ||
| numbers.Number | hist_min_bin_width, | ||
| numbers.Number | hist_search_percentile_min, | ||
| numbers.Number | hist_search_percentile_max, | ||
| int | hist_num_search, | ||
| int | hist_search_type, | ||
| numbers.Number | act_scale_min, | ||
| numbers.Number | weight_scale_min, | ||
| numbers.Number | min_clip_ratio, | ||
| Dict | layer_overrides, | ||
| bool | optional = optional ) |
Initialize the CalibrationConfig.
- Parameters
-
quantization_method int. Calibration method index:
0: WChALayer - Weight per-channel, Activation per-layer.
1: WChAMulti - Weight per-channel, Activation multi-layer.
2: WChALayerZeropoint - Weight per-channel, Activation per-layer with zeropoint.
3: WChAMultiZeropoint - Weight per-channel, Activation multi-layer with zeropoint.
quantization_output int. Output quantization type index:
0: Layer - Per-layer quantization.
1: Ch - Per-channel quantization.
2: Sigmoid - Sigmoid-based quantization.
quantization_mode int. Quantization mode index:
0: max.
1: maxPercentile.
2: histogram.
percentile float. Percentile value for maxPercentile mode. topk_ratio float. Top-k ratio used in maxPercentile mode. max_each int. Maximum number of samples processed per iteration (maxPercentile). max_total int. Total maximum number of samples (maxPercentile). hist_percentile float. Percentile for histogram mode (when searchType is percentile). hist_use_gpu bool. Whether to use GPU for histogram computation. hist_num_bins int. Number of histogram bins. hist_num_samples int. Number of representative samples for metric computation. hist_buffer_size int. Buffer size for batch histogram computation. hist_min_bin_width float. Minimum bin width for histogram. hist_search_percentile_min float. Minimum search percentile for MSE/KL search. hist_search_percentile_max float. Maximum search percentile for MSE/KL search. hist_num_search int. Number of search iterations for MSE/KL search. hist_search_type int. Search type: 0=percentile, 1=mse, 2=kl. act_scale_min float. Minimum allowed activation scale (lower bound clamp). weight_scale_min float. Minimum allowed weight scale (lower bound clamp). min_clip_ratio float. Minimum clip ratio constraint applied during calibration. layer_overrides Dict. Layer-specific override settings. optional bool. Indicates whether this config is optional.
Definition at line 106 of file quantization_config.py.
◆ default_config() [1/3]
| qbcompiler.configs.quantization_config.CalibrationConfig.default_config | ( | cls | ) |
Definition at line 198 of file quantization_config.py.
◆ from_kwargs() [1/3]
| qbcompiler.configs.quantization_config.CalibrationConfig.from_kwargs | ( | cls, | |
| ** | kwargs ) |
Definition at line 202 of file quantization_config.py.
◆ from_dict() [1/3]
| qbcompiler.configs.quantization_config.CalibrationConfig.from_dict | ( | cls, | |
| dict | config_dict ) |
Create CalibrationConfig from dictionary (JSON structure)
Definition at line 218 of file quantization_config.py.
◆ to_dict() [1/3]
| qbcompiler.configs.quantization_config.CalibrationConfig.to_dict | ( | self | ) |
Definition at line 283 of file quantization_config.py.
◆ __init__() [2/3]
| qbcompiler.configs.quantization_config.BitConfig.__init__ | ( | self, | |
| int | query_act_bits, | ||
| int | key_act_bits, | ||
| int | value_act_bits, | ||
| int | output_act_bits, | ||
| int | ffn_act_bits, | ||
| int | head_act_bits, | ||
| int | query_weight_bits, | ||
| int | key_weight_bits, | ||
| int | value_weight_bits, | ||
| int | output_weight_bits, | ||
| int | ffn_weight_bits, | ||
| int | head_weight_bits, | ||
| bool | mixed_precision_apply, | ||
| str | save_path, | ||
| str | load_path, | ||
| List[str] | activation_16bits, | ||
| List[str] | weight_16bits, | ||
| bool | optional = optional ) |
Initialize the BitConfig.
- Parameters
-
query_act_bits int. Query activation bit-width. key_act_bits int. Key activation bit-width. value_act_bits int. Value activation bit-width. output_act_bits int. Output activation bit-width. ffn_act_bits int. FFN activation bit-width. head_act_bits int. Head activation bit-width. query_weight_bits int. Query weight bit-width. key_weight_bits int. Key weight bit-width. value_weight_bits int. Value weight bit-width. output_weight_bits int. Output weight bit-width. ffn_weight_bits int. FFN weight bit-width. head_weight_bits int. Head weight bit-width. mixed_precision_apply bool. If true, apply mixed-precision according to the specified bit-widths. save_path str. (optional) Path to save the bit allocation. If empty, the allocation is not saved. load_path str. (optional) Path to load the bit allocation. If empty, allocation is not loaded. activation_16bits List[str]. (optional) Layer names to force 16-bit activations. weight_16bits List[str]. (optional) Layer names to force 16-bit weights.
Definition at line 378 of file quantization_config.py.
◆ default_config() [2/3]
| qbcompiler.configs.quantization_config.BitConfig.default_config | ( | cls | ) |
Definition at line 441 of file quantization_config.py.
◆ from_kwargs() [2/3]
| qbcompiler.configs.quantization_config.BitConfig.from_kwargs | ( | cls, | |
| ** | kwargs ) |
Definition at line 445 of file quantization_config.py.
◆ from_dict() [2/3]
| qbcompiler.configs.quantization_config.BitConfig.from_dict | ( | cls, | |
| dict | config_dict ) |
Create BitConfig from dictionary (JSON structure)
Definition at line 461 of file quantization_config.py.
◆ to_dict() [2/3]
| qbcompiler.configs.quantization_config.BitConfig.to_dict | ( | self | ) |
Definition at line 525 of file quantization_config.py.
◆ __init__() [3/3]
| qbcompiler.configs.quantization_config.QuantizationConfig.__init__ | ( | self, | |
| bool | optional, | ||
| Optional[CalibrationConfig] | calibration = None, | ||
| Optional[BitConfig] | bit = None ) |
Initialize the QuantizationConfig.
- Parameters
-
calibration CalibrationConfig. Calibration configuration. bit BitConfig. Bit precision configuration.
Definition at line 582 of file quantization_config.py.
◆ default_config() [3/3]
| qbcompiler.configs.quantization_config.QuantizationConfig.default_config | ( | cls | ) |
Definition at line 598 of file quantization_config.py.
◆ from_kwargs() [3/3]
| qbcompiler.configs.quantization_config.QuantizationConfig.from_kwargs | ( | cls, | |
| ** | kwargs ) |
Definition at line 606 of file quantization_config.py.
◆ from_dict() [3/3]
| qbcompiler.configs.quantization_config.QuantizationConfig.from_dict | ( | cls, | |
| dict | config_dict ) |
Definition at line 614 of file quantization_config.py.
◆ to_dict() [3/3]
| qbcompiler.configs.quantization_config.QuantizationConfig.to_dict | ( | self | ) |
Definition at line 624 of file quantization_config.py.
◆ get_calibration_config()
| CalibrationConfig qbcompiler.configs.quantization_config.get_calibration_config | ( | ** | kwargs | ) |
Create CalibrationConfig with partial parameters merged with defaults.
Definition at line 633 of file quantization_config.py.
◆ get_bit_config()
| BitConfig qbcompiler.configs.quantization_config.get_bit_config | ( | ** | kwargs | ) |
Create BitConfig with partial parameters merged with defaults.
Definition at line 642 of file quantization_config.py.
◆ get_quantization_config()
| QuantizationConfig qbcompiler.configs.quantization_config.get_quantization_config | ( | Optional[CalibrationConfig] | calibration = None, |
| Optional[BitConfig] | bit = None ) |
Create QuantizationConfig with partial parameters merged with defaults.
Definition at line 651 of file quantization_config.py.
Variable Documentation
◆ optional [1/3]
|
static |
Definition at line 22 of file quantization_config.py.
◆ method_list
|
static |
Definition at line 23 of file quantization_config.py.
◆ output_list
|
static |
Definition at line 24 of file quantization_config.py.
◆ mode_list
|
static |
Definition at line 25 of file quantization_config.py.
◆ search_type_list
|
static |
Definition at line 26 of file quantization_config.py.
◆ DEFAULTS [1/2]
|
static |
Initial value:
= {
"quantization_method": 1,
"quantization_output": 0,
"quantization_mode": 1, # maxPercentile is now index 1
# maxPercentile config
"percentile": 0.9999,
"topk_ratio": 0.01,
"max_each": 128,
"max_total": 65536,
# histogram config
"hist_percentile": 0.9999,
"hist_use_gpu": True,
"hist_num_bins": 2048,
"hist_num_samples": 1024,
"hist_buffer_size": 1024,
"hist_min_bin_width": 0.0001,
"hist_search_percentile_min": 0.9999,
"hist_search_percentile_max": 0.999999,
"hist_num_search": 1024,
"hist_search_type": 0, # 0=percentile, 1=mse, 2=kl
# common
"act_scale_min": 0.0005,
"weight_scale_min": 1e-6,
"min_clip_ratio": -1,
"layer_overrides": {"actScaleMin": {"0.0005": []}},
}
Definition at line 28 of file quantization_config.py.
◆ py_to_cpp_name [1/3]
|
static |
Initial value:
= {
"quantization_method": "method",
"quantization_output": "output",
"quantization_mode": "mode",
# maxPercentile
"percentile": "percentile",
"topk_ratio": "topKRatio",
"max_each": "maxEach",
"max_total": "maxTotal",
# histogram
"hist_percentile": "percentile",
"hist_use_gpu": "useGPU",
"hist_num_bins": "numBins",
"hist_num_samples": "numSamples",
"hist_buffer_size": "bufferSize",
"hist_min_bin_width": "minBinWidth",
"hist_search_percentile_min": "searchPercentileMin",
"hist_search_percentile_max": "searchPercentileMax",
"hist_num_search": "numSearch",
"hist_search_type": "searchType",
# common
"act_scale_min": "actScaleMin",
"weight_scale_min": "weightScaleMin",
"min_clip_ratio": "minclipRatio",
}
Definition at line 55 of file quantization_config.py.
◆ min_max_check [1/3]
|
static |
Initial value:
= {
"quantization_method": [0, 3],
"quantization_output": [0, 2],
"quantization_mode": [0, 2], # 0=max, 1=maxPercentile, 2=histogram
# maxPercentile
"percentile": [0, 1],
"topk_ratio": [0, 1],
"max_each": [0, None],
"max_total": [0, None],
# histogram
"hist_percentile": [0, 1],
"hist_num_bins": [1, None],
"hist_num_samples": [1, None],
"hist_buffer_size": [1, None],
"hist_min_bin_width": [0, None],
"hist_search_percentile_min": [0, 1],
"hist_search_percentile_max": [0, 1],
"hist_num_search": [1, None],
"hist_search_type": [0, 2], # 0=percentile, 1=mse, 2=kl
# common
"act_scale_min": [0, 1],
"weight_scale_min": [0, 1],
"min_clip_ratio": [-1, 1],
}
Definition at line 81 of file quantization_config.py.
◆ quantization_method
| qbcompiler.configs.quantization_config.CalibrationConfig.quantization_method = quantization_method |
Definition at line 171 of file quantization_config.py.
◆ quantization_output
| qbcompiler.configs.quantization_config.CalibrationConfig.quantization_output = quantization_output |
Definition at line 172 of file quantization_config.py.
◆ quantization_mode
| qbcompiler.configs.quantization_config.CalibrationConfig.quantization_mode = quantization_mode |
Definition at line 173 of file quantization_config.py.
◆ percentile
| qbcompiler.configs.quantization_config.CalibrationConfig.percentile = percentile |
Definition at line 175 of file quantization_config.py.
◆ topk_ratio
| qbcompiler.configs.quantization_config.CalibrationConfig.topk_ratio = topk_ratio |
Definition at line 176 of file quantization_config.py.
◆ max_each
| qbcompiler.configs.quantization_config.CalibrationConfig.max_each = max_each |
Definition at line 177 of file quantization_config.py.
◆ max_total
| qbcompiler.configs.quantization_config.CalibrationConfig.max_total = max_total |
Definition at line 178 of file quantization_config.py.
◆ hist_percentile
| qbcompiler.configs.quantization_config.CalibrationConfig.hist_percentile = hist_percentile |
Definition at line 180 of file quantization_config.py.
◆ hist_use_gpu
| qbcompiler.configs.quantization_config.CalibrationConfig.hist_use_gpu = hist_use_gpu |
Definition at line 181 of file quantization_config.py.
◆ hist_num_bins
| qbcompiler.configs.quantization_config.CalibrationConfig.hist_num_bins = hist_num_bins |
Definition at line 182 of file quantization_config.py.
◆ hist_num_samples
| qbcompiler.configs.quantization_config.CalibrationConfig.hist_num_samples = hist_num_samples |
Definition at line 183 of file quantization_config.py.
◆ hist_buffer_size
| qbcompiler.configs.quantization_config.CalibrationConfig.hist_buffer_size = hist_buffer_size |
Definition at line 184 of file quantization_config.py.
◆ hist_min_bin_width
| qbcompiler.configs.quantization_config.CalibrationConfig.hist_min_bin_width = hist_min_bin_width |
Definition at line 185 of file quantization_config.py.
◆ hist_search_percentile_min
| qbcompiler.configs.quantization_config.CalibrationConfig.hist_search_percentile_min = hist_search_percentile_min |
Definition at line 186 of file quantization_config.py.
◆ hist_search_percentile_max
| qbcompiler.configs.quantization_config.CalibrationConfig.hist_search_percentile_max = hist_search_percentile_max |
Definition at line 187 of file quantization_config.py.
◆ hist_num_search
| qbcompiler.configs.quantization_config.CalibrationConfig.hist_num_search = hist_num_search |
Definition at line 188 of file quantization_config.py.
◆ hist_search_type
| qbcompiler.configs.quantization_config.CalibrationConfig.hist_search_type = hist_search_type |
Definition at line 189 of file quantization_config.py.
◆ act_scale_min
| qbcompiler.configs.quantization_config.CalibrationConfig.act_scale_min = act_scale_min |
Definition at line 191 of file quantization_config.py.
◆ weight_scale_min
| qbcompiler.configs.quantization_config.CalibrationConfig.weight_scale_min = weight_scale_min |
Definition at line 192 of file quantization_config.py.
◆ min_clip_ratio
| qbcompiler.configs.quantization_config.CalibrationConfig.min_clip_ratio = min_clip_ratio |
Definition at line 193 of file quantization_config.py.
◆ layer_overrides
| qbcompiler.configs.quantization_config.CalibrationConfig.layer_overrides = layer_overrides |
Definition at line 194 of file quantization_config.py.
◆ optional [2/3]
|
static |
Definition at line 322 of file quantization_config.py.
◆ DEFAULTS [2/2]
|
static |
Initial value:
= {
"query_act_bits": 8,
"key_act_bits": 8,
"value_act_bits": 8,
"output_act_bits": 16,
"ffn_act_bits": 16,
"head_act_bits": 8,
"query_weight_bits": 8,
"key_weight_bits": 8,
"value_weight_bits": 8,
"output_weight_bits": 8,
"ffn_weight_bits": 8,
"head_weight_bits": 8,
"mixed_precision_apply": False,
"save_path": "",
"load_path": "",
"activation_16bits": [],
"weight_16bits": [],
}
Definition at line 324 of file quantization_config.py.
◆ py_to_cpp_name [2/3]
|
static |
Initial value:
= {
"query_act_bits": "queryActBits",
"key_act_bits": "keyActBits",
"value_act_bits": "valueActBits",
"output_act_bits": "outputActBits",
"ffn_act_bits": "ffnActBits",
"head_act_bits": "headActBits",
"query_weight_bits": "queryWeightBits",
"key_weight_bits": "keyWeightBits",
"value_weight_bits": "valueWeightBits",
"output_weight_bits": "outputWeightBits",
"ffn_weight_bits": "ffnWeightBits",
"head_weight_bits": "headWeightBits",
"mixed_precision_apply": "mixedPrecisionApply",
"save_path": "savePath",
"load_path": "loadPath",
}
Definition at line 344 of file quantization_config.py.
◆ min_max_check [2/3]
|
static |
Initial value:
= {
"query_act_bits": [1, None],
"key_act_bits": [1, None],
"value_act_bits": [1, None],
"output_act_bits": [1, None],
"ffn_act_bits": [1, None],
"head_act_bits": [1, None],
"query_weight_bits": [1, None],
"key_weight_bits": [1, None],
"value_weight_bits": [1, None],
"output_weight_bits": [1, None],
"ffn_weight_bits": [1, None],
"head_weight_bits": [1, None],
}
Definition at line 363 of file quantization_config.py.
◆ query_act_bits
| qbcompiler.configs.quantization_config.BitConfig.query_act_bits = query_act_bits |
Definition at line 421 of file quantization_config.py.
◆ key_act_bits
| qbcompiler.configs.quantization_config.BitConfig.key_act_bits = key_act_bits |
Definition at line 422 of file quantization_config.py.
◆ value_act_bits
| qbcompiler.configs.quantization_config.BitConfig.value_act_bits = value_act_bits |
Definition at line 423 of file quantization_config.py.
◆ output_act_bits
| qbcompiler.configs.quantization_config.BitConfig.output_act_bits = output_act_bits |
Definition at line 424 of file quantization_config.py.
◆ ffn_act_bits
| qbcompiler.configs.quantization_config.BitConfig.ffn_act_bits = ffn_act_bits |
Definition at line 425 of file quantization_config.py.
◆ head_act_bits
| qbcompiler.configs.quantization_config.BitConfig.head_act_bits = head_act_bits |
Definition at line 426 of file quantization_config.py.
◆ query_weight_bits
| qbcompiler.configs.quantization_config.BitConfig.query_weight_bits = query_weight_bits |
Definition at line 427 of file quantization_config.py.
◆ key_weight_bits
| qbcompiler.configs.quantization_config.BitConfig.key_weight_bits = key_weight_bits |
Definition at line 428 of file quantization_config.py.
◆ value_weight_bits
| qbcompiler.configs.quantization_config.BitConfig.value_weight_bits = value_weight_bits |
Definition at line 429 of file quantization_config.py.
◆ output_weight_bits
| qbcompiler.configs.quantization_config.BitConfig.output_weight_bits = output_weight_bits |
Definition at line 430 of file quantization_config.py.
◆ ffn_weight_bits
| qbcompiler.configs.quantization_config.BitConfig.ffn_weight_bits = ffn_weight_bits |
Definition at line 431 of file quantization_config.py.
◆ head_weight_bits
| qbcompiler.configs.quantization_config.BitConfig.head_weight_bits = head_weight_bits |
Definition at line 432 of file quantization_config.py.
◆ mixed_precision_apply
| qbcompiler.configs.quantization_config.BitConfig.mixed_precision_apply = mixed_precision_apply |
Definition at line 433 of file quantization_config.py.
◆ save_path
| qbcompiler.configs.quantization_config.BitConfig.save_path = save_path |
Definition at line 434 of file quantization_config.py.
◆ load_path
| qbcompiler.configs.quantization_config.BitConfig.load_path = load_path |
Definition at line 435 of file quantization_config.py.
◆ activation_16bits
| qbcompiler.configs.quantization_config.BitConfig.activation_16bits = activation_16bits |
Definition at line 436 of file quantization_config.py.
◆ weight_16bits
| qbcompiler.configs.quantization_config.BitConfig.weight_16bits = weight_16bits |
Definition at line 437 of file quantization_config.py.
◆ optional [3/3]
|
static |
Definition at line 573 of file quantization_config.py.
◆ py_to_cpp_name [3/3]
|
static |
Initial value:
= {
"calibration": "calibration",
"bit": "bit",
}
Definition at line 574 of file quantization_config.py.
◆ min_max_check [3/3]
|
static |
Definition at line 579 of file quantization_config.py.
◆ calibration
| qbcompiler.configs.quantization_config.QuantizationConfig.calibration = calibration or CalibrationConfig.default_config() |
Definition at line 594 of file quantization_config.py.
◆ bit
| qbcompiler.configs.quantization_config.QuantizationConfig.bit = bit or BitConfig.default_config() |
Definition at line 595 of file quantization_config.py.
Generated by
1.14.0