qubee.configs.llm_config.LlmConfig Class Reference

qubee.configs.llm_config.LlmConfig Class Reference#

Mobilint SDK qb Compiler: qubee.configs.llm_config.LlmConfig Class Reference

Mobilint SDK qb Compiler v0.11.0.1

MCS002-EN

qubee
configs
llm_config
LlmConfig

Inheritance diagram for qubee.configs.llm_config.LlmConfig:

Public Member Functions
	__init__ (self, int max_data_len, int max_seq_len, int max_cache_len, int random_seq_len_calib, bool use_full_seq_len_calib, bool use_custom_mask_input, int start_dram_offset, bool use_global_core, int llm_batch_size, list[int] npu_core_ids, bool optional=True)
	default_config (cls)
	to_dict (self)

Public Attributes
	max_data_len = max_data_len
	max_seq_len = max_seq_len
	max_cache_len = max_cache_len
	random_seq_len_calib = random_seq_len_calib
	use_full_seq_len_calib = use_full_seq_len_calib
	use_custom_mask_input = use_custom_mask_input
	start_dram_offset = start_dram_offset
	use_global_core = use_global_core
	llm_batch_size = llm_batch_size
	npu_core_ids = npu_core_ids

Static Public Attributes
dict	py_to_cpp_name
dict	min_max_check

Detailed Description

Definition at line 21 of file llm_config.py.

Constructor & Destructor Documentation

◆ init()

qubee.configs.llm_config.LlmConfig.__init__	(		self,
		int	max_data_len,
		int	max_seq_len,
		int	max_cache_len,
		int	random_seq_len_calib,
		bool	use_full_seq_len_calib,
		bool	use_custom_mask_input,
		int	start_dram_offset,
		bool	use_global_core,
		int	llm_batch_size,
		list[int]	npu_core_ids,
		bool	optional = True )

Definition at line 48 of file llm_config.py.

Member Function Documentation

◆ default_config()

qubee.configs.llm_config.LlmConfig.default_config ( cls )

Definition at line 76 of file llm_config.py.

◆ to_dict()

qubee.configs.llm_config.LlmConfig.to_dict ( self )

Definition at line 102 of file llm_config.py.

Member Data Documentation

◆ py_to_cpp_name

dict qubee.configs.llm_config.LlmConfig.py_to_cpp_name

static

Initial value:

=  {
        "max_data_len": "maxDataLen",
        "max_seq_len": "maxSeqLen",
        "max_cache_len": "maxCacheLen",
        "random_seq_len_calib": "randomSeqLenCalib",
        "use_full_seq_len_calib": "useFullSeqLenCalib",
        "use_custom_mask_input": "useCustomMaskInput",
        "start_dram_offset": "startDramOffset",
        "use_global_core": "useGlobalCore",
        "llm_batch_size": "llmBatchSize",
        "npu_core_ids": "npuCoreIds",
    }

Definition at line 22 of file llm_config.py.

◆ min_max_check

dict qubee.configs.llm_config.LlmConfig.min_max_check

static

Initial value:

=  {
        "max_data_len": [0, None],
        "max_seq_len": [0, None],
        "max_cache_len": [0, None],
        "random_seq_len_calib": [0, None],
        "use_full_seq_len_calib": [0, 1],
        "use_custom_mask_input": [0, 1],
        "start_dram_offset": [0, None],
        "use_global_core": [0, 1],
        "llm_batch_size": [0, None],
    }

Definition at line 35 of file llm_config.py.

◆ max_data_len

qubee.configs.llm_config.LlmConfig.max_data_len = max_data_len

Definition at line 63 of file llm_config.py.

◆ max_seq_len

qubee.configs.llm_config.LlmConfig.max_seq_len = max_seq_len

Definition at line 64 of file llm_config.py.

◆ max_cache_len

qubee.configs.llm_config.LlmConfig.max_cache_len = max_cache_len

Definition at line 65 of file llm_config.py.

◆ random_seq_len_calib

qubee.configs.llm_config.LlmConfig.random_seq_len_calib = random_seq_len_calib

Definition at line 66 of file llm_config.py.

◆ use_full_seq_len_calib

qubee.configs.llm_config.LlmConfig.use_full_seq_len_calib = use_full_seq_len_calib

Definition at line 67 of file llm_config.py.

◆ use_custom_mask_input

qubee.configs.llm_config.LlmConfig.use_custom_mask_input = use_custom_mask_input

Definition at line 68 of file llm_config.py.

◆ start_dram_offset

qubee.configs.llm_config.LlmConfig.start_dram_offset = start_dram_offset

Definition at line 69 of file llm_config.py.

◆ use_global_core

qubee.configs.llm_config.LlmConfig.use_global_core = use_global_core

Definition at line 70 of file llm_config.py.

◆ llm_batch_size

qubee.configs.llm_config.LlmConfig.llm_batch_size = llm_batch_size

Definition at line 71 of file llm_config.py.

◆ npu_core_ids

qubee.configs.llm_config.LlmConfig.npu_core_ids = npu_core_ids

Definition at line 72 of file llm_config.py.

The documentation for this class was generated from the following file:

llm_config.py