CacheInfo¶
Struct representing KV-cache information.
Attributes¶
| Name | Description |
|---|---|
cache_type |
|
name |
|
layer_hash |
|
size |
|
num_batches |
cache_type¶
name¶
layer_hash¶
size¶
num_batches¶
Methods¶
| Name | Description |
|---|---|
__init__ |
|
from_cpp |
__init__¶
def __init__(
self,
cache_type: CacheType = CacheType.Error,
name: str = "",
layer_hash: str = "",
size: int = 0,
num_batches: int = 0,
)