CacheInfo¶
Struct representing KV-cache information.
Attributes¶
| Member | Description |
|---|---|
cache_type |
|
name |
|
layer_hash |
|
size |
|
num_batches |
Struct representing KV-cache information.
| Member | Description |
|---|---|
cache_type |
|
name |
|
layer_hash |
|
size |
|
num_batches |