qbcompiler.configs.models.LlmConfig Class Reference

Mobilint SDK qb Compiler v1.2 MCS002-KR

• qbcompiler
• configs
• models
• LlmConfig

Classes | Public Member Functions | Static Public Attributes | List of all members

qbcompiler.configs.models.LlmConfig Class Reference

Configuration for Large Language Model (LLM) compilation. More...

Inheritance diagram for qbcompiler.configs.models.LlmConfig:

Classes
class	Attributes

Public Member Functions
"LlmConfig"	with_updates (self, **kwargs)
	Return a copy with updated fields.

Static Public Attributes
	model_config
bool	apply
int	npu_parallel_degree
	Attributes
	default_factory
	alias

Detailed Description

Configuration for Large Language Model (LLM) compilation.

Defines LLM-specific settings including sequence lengths, cache configurations, and runtime parameters for efficient LLM inference.

Parameters

apply	bool. If True, apply LLM-specific configurations
npu_parallel_degree	int. Number of NPU partitions for FFN tensor parallelism (1 = disabled). Applied before OptimizeFFN if both active.
attributes	Attributes. LLM attributes configuration

Definition at line 950 of file models.py.

Member Function Documentation

◆ with_updates()

"LlmConfig" qbcompiler.configs.models.LlmConfig.with_updates	(		self,
		**	kwargs )

Return a copy with updated fields.

Definition at line 1087 of file models.py.

Member Data Documentation

◆ model_config

qbcompiler.configs.models.LlmConfig.model_config

static

Initial value:

= ConfigDict(

populate_by_name=True,

extra="forbid",

)

Definition at line 962 of file models.py.

◆ apply

bool qbcompiler.configs.models.LlmConfig.apply

static

Initial value:

= Field(

default=False, description="If True, apply LLM-specific configurations"

)

Definition at line 967 of file models.py.

◆ npu_parallel_degree

int qbcompiler.configs.models.LlmConfig.npu_parallel_degree

static

Initial value:

= Field(

default=1,

alias="npuParallelDegree",

description="Number of NPU partitions for FFN tensor parallelism (1 = disabled). Applied before OptimizeFFN if both active.",

)

Definition at line 970 of file models.py.

◆ Attributes

qbcompiler.configs.models.LlmConfig.Attributes

static

Definition at line 1085 of file models.py.

◆ default_factory

qbcompiler.configs.models.LlmConfig.default_factory

static

Definition at line 1085 of file models.py.

◆ alias

qbcompiler.configs.models.LlmConfig.alias

static

Definition at line 1085 of file models.py.

---

The documentation for this class was generated from the following file:

• models.py

---

Generated by  1.14.0