qbcompiler.configs.llm_config.LlmConfig Class Reference#
|
Mobilint SDK qb Compiler v1.0
MCS002-EN
|
qbcompiler.configs.llm_config.LlmConfig Class Reference
Configuration for Large Language Model (LLM) compilation. More...
Inheritance diagram for qbcompiler.configs.llm_config.LlmConfig:
Public Member Functions | |
| __init__ (self, bool llm_config_apply, int max_data_length, int max_sequence_length, int max_cache_length, int max_core_data_length, int random_seq_length, bool use_full_seq_length, bool use_global_core, int batch_size, List[int] npu_core_ids, bool dynamic_rope, bool optional=optional) | |
| Initialize the LlmConfig: | |
| default_config (cls) | |
| from_kwargs (cls, **kwargs) | |
| from_dict (cls, dict config_dict) | |
| Create LlmConfig from dictionary (JSON structure) | |
| to_dict (self) | |
Public Attributes | |
| llm_config_apply = llm_config_apply | |
| max_data_length = max_data_length | |
| max_sequence_length = max_sequence_length | |
| max_cache_length = max_cache_length | |
| max_core_data_length = max_core_data_length | |
| random_seq_length = random_seq_length | |
| use_full_seq_length = use_full_seq_length | |
| use_global_core = use_global_core | |
| batch_size = batch_size | |
| npu_core_ids = npu_core_ids | |
| dynamic_rope = dynamic_rope | |
Static Public Attributes | |
| bool | optional = True |
| dict | DEFAULTS |
| dict | py_to_cpp_name |
| dict | min_max_check |
Detailed Description
Configuration for Large Language Model (LLM) compilation.
Definition at line 12 of file llm_config.py.
The documentation for this class was generated from the following file:
Generated by
1.14.0