# ModelConfig

```python
class ModelConfig
```

Configures a core mode and core allocation of a model for NPU inference. The [`ModelConfig`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/ModelConfig/index.md>) class provides methods for setting a core mode and allocating cores for NPU inference. Supported core modes are single-core, multi-core, global4-core, and global8-core. Users can also specify which cores to allocate for the model. Additionally, the configuration offers an option to enforce the use of a specific NPU bundle.

> [!NOTE]
>
> Deprecated functions are included for backward compatibility, but it is recommended to use the newer core mode configuration methods.

## Methods

| Name | Description |
| --- | --- |
| [`__init__`](<#init>) | Default constructor. This default-constructed object is initially set to auto-core mode. |
| [`set_auto_core_mode`](<#set-auto-core-mode>) | Sets the model to detect [CoreMode](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreMode/index.md>) automatically. |
| [`set_single_core_mode`](<#set-single-core-mode>) | Sets the model to use single-core mode for inference with a specified number of local cores. |
| [`set_global_core_mode`](<#set-global-core-mode>) |  |
| [`set_global4_core_mode`](<#set-global4-core-mode>) | Sets the model to use global4-core mode for inference with a specified set of NPU clusters. |
| [`set_global8_core_mode`](<#set-global8-core-mode>) | Sets the model to use global8-core mode for inference. |
| [`get_core_mode`](<#get-core-mode>) | Gets the core mode to be applied to the model. |
| [`set_multi_core_mode`](<#set-multi-core-mode>) | Sets the model to use multi-core mode for batch inference. |
| [`get_core_allocation_policy`](<#get-core-allocation-policy>) | Gets the core allocation policy to be applied to the model. |
| [`get_num_cores`](<#get-num-cores>) | Gets the number of cores to be allocated for the model. |
| [`force_single_npu_bundle`](<#force-single-npu-bundle>) | Forces the use of a specific NPU bundle. |
| [`get_forced_npu_bundle_index`](<#get-forced-npu-bundle-index>) | Retrieves the index of the forced NPU bundle. |
| [`set_async_pipeline_enabled`](<#set-async-pipeline-enabled>) | Enables or disables the asynchronous pipeline required for asynchronous inference. |
| [`get_async_pipeline_enabled`](<#get-async-pipeline-enabled>) | Returns whether the asynchronous pipeline is enabled in this configuration. |
| [`set_activation_slots`](<#set-activation-slots>) | Sets activation buffer slots for multi-activation supported model. |
| [`get_activation_slots`](<#get-activation-slots>) | Returns activation buffer slot count. |
| [`get_core_ids`](<#get-core-ids>) | Returns the list of NPU CoreIds to be used for model inference. |
| [`get_clusters`](<#get-clusters>) | Returns the list of clusters to be used for model inference. |
| [`__repr__`](<#repr>) |  |

### \_\_init\_\_

```python
def __init__(self, num_cores: Optional[int] = None)
```

Default constructor. This default-constructed object is initially set to auto-core mode.

### set\_auto\_core\_mode

```python
def set_auto_core_mode(self) -> bool
```

Sets the model to detect [CoreMode](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreMode/index.md>) automatically.

In auto-core mode, the model automatically detects a supported [CoreMode](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreMode/index.md>) while using all available NPU cores.

> [!NOTE]
>
> If the model has more than one [CoreMode](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreMode/index.md>), [`CoreMode.Auto`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreMode/#auto>) is not supported.

> [!NOTE]
>
> The activation buffer slot count is reset to -1 when `set_auto_core_mode` is called. A value of -1 means "not set": when the model is created, the slot count is decided from the detected [CoreMode](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreMode/index.md>) of each bundle, following the same per-CoreMode defaults as `set_activation_slots` describes. Call `set_activation_slots` after `set_auto_core_mode` to override it.

**Returns**

True if the mode was successfully set, False otherwise.

### set\_single\_core\_mode

```python
def set_single_core_mode(self, num_cores: int) -> bool
```

```python
def set_single_core_mode(self, core_ids: List[CoreId]) -> bool
```

Sets the model to use single-core mode for inference with a specified number of local cores.

In single-core mode, each local core executes model inference independently. The number of cores used is specified by the `num_cores` parameter, and the core allocation policy is set to [`CoreAllocationPolicy.Auto`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreAllocationPolicy/#auto>), meaning the model will be automatically allocated to available local cores when the model is launched to the NPU, specifically when the [`Model.launch()`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/Model/#launch>) function is called. Or The user can specify a list of CoreIds to determine which cores to use for inference.

**Parameters**

- `num_cores` *\[in\]* — The number of local cores to use for inference.
- `core_ids` *\[in\]* — A list of CoreIds to be used for model inference.

**Returns**

True if the mode was successfully set, False otherwise.

> [!NOTE]
>
> Supported call forms (provide exactly one of `num_cores` / `core_ids`):
>
> - `set_single_core_mode(num_cores)`: auto-allocate `num_cores` local cores. (deprecated in latest models)
> - `set_single_core_mode(core_ids)`: use exactly the cores listed in `core_ids`.

### set\_global\_core\_mode

```python
def set_global_core_mode(self, clusters: List[Cluster]) -> bool
```

> [!WARNING]
>
> **Deprecated**

### set\_global4\_core\_mode

```python
def set_global4_core_mode(
    self, clusters: List[Cluster] = [Cluster.Cluster0, Cluster.Cluster1]
) -> bool
```

Sets the model to use global4-core mode for inference with a specified set of NPU clusters.

For Aries NPU, there are two clusters, each consisting of four local cores. In global4-core mode, four local cores within the same cluster work together to execute the model inference.

**Parameters**

- `clusters` *\[in\]* — A list of clusters to be used for model inference.

**Returns**

True if the mode was successfully set, False otherwise.

### set\_global8\_core\_mode

```python
def set_global8_core_mode(self) -> bool
```

Sets the model to use global8-core mode for inference.

For Aries NPU, there are two clusters, each consisting of four local cores. In global8-core mode, all eight local cores across the two clusters work together to execute the model inference.

**Returns**

True if the mode was successfully set, False otherwise.

### get\_core\_mode

```python
def get_core_mode(self) -> CoreMode
```

Gets the core mode to be applied to the model.

This reflects the core mode that will be used when the model is created.

**Returns**

The [`CoreMode`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreMode/index.md>) to be applied to the model.

### set\_multi\_core\_mode

```python
def set_multi_core_mode(
    self, clusters: List[Cluster] = [Cluster.Cluster0, Cluster.Cluster1]
) -> bool
```

Sets the model to use multi-core mode for batch inference.

In multi-core mode, on Aries NPU, the four local cores within a cluster work together to process batch inference tasks efficiently. This mode is optimized for batch processing.

**Parameters**

- `clusters` *\[in\]* — A list of clusters to be used for multi-core batch inference.

**Returns**

True if the mode was successfully set, False otherwise.

### get\_core\_allocation\_policy

```python
def get_core_allocation_policy(self) -> CoreAllocationPolicy
```

Gets the core allocation policy to be applied to the model.

This reflects the core allocation policy that will be used when the model is created.

**Returns**

The [`CoreAllocationPolicy`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreAllocationPolicy/index.md>) to be applied to the model.

### get\_num\_cores

```python
def get_num_cores(self) -> int
```

Gets the number of cores to be allocated for the model.

This represents the number of cores that will be allocated for inference when the model is launched to the NPU.

**Returns**

The number of cores to be allocated for the model.

### force\_single\_npu\_bundle

```python
def force_single_npu_bundle(self, npu_bundle_index: int) -> bool
```

Forces the use of a specific NPU bundle.

This function forces the selection of a specific NPU bundle. If a non-negative index is provided, the corresponding NPU bundle is selected and runs without CPU offloading. If -1 is provided, all NPU bundles are used with CPU offloading enabled.

**Parameters**

- `npu_bundle_index` *\[in\]* — The index of the NPU bundle to force. A non-negative integer selects a specific NPU bundle (runs without CPU offloading), or -1 to enable all NPU bundles with CPU offloading.

**Returns**

True if the index is valid and the NPU bundle is successfully set, False if the index is invalid (less than -1).

### get\_forced\_npu\_bundle\_index

```python
def get_forced_npu_bundle_index(self) -> bool
```

Retrieves the index of the forced NPU bundle.

This function returns the index of the NPU bundle that has been forced using the `force_single_npu_bundle` function. If no NPU bundle is forced, the returned value will be -1.

**Returns**

The index of the forced NPU bundle, or -1 if no bundle is forced.

### set\_async\_pipeline\_enabled

```python
def set_async_pipeline_enabled(self, enable: bool) -> None
```

Enables or disables the asynchronous pipeline required for asynchronous inference.

Call this function with `enable` set to `True` if you intend to use [`Model.infer_async()`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/Model/#infer-async>), as the asynchronous pipeline is necessary for their operation.

If you are only using synchronous inference, such as [`Model.infer()`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/Model/#infer>) or [`Model.infer_to_float()`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/Model/#infer-to-float>), it is recommended to keep the asynchronous pipeline disabled to avoid unnecessary overhead.

**Parameters**

- `enable` *\[in\]* — Set to `True` to enable the asynchronous pipeline; set to `False` to disable it.

### get\_async\_pipeline\_enabled

```python
def get_async_pipeline_enabled(self) -> bool
```

Returns whether the asynchronous pipeline is enabled in this configuration.

**Returns**

`True` if the asynchronous pipeline is enabled; `False` otherwise.

### set\_activation\_slots

```python
def set_activation_slots(self, num: int) -> None
```

Sets activation buffer slots for multi-activation supported model.

all this function if you want to set the number of activation buffer slots manually.

If you do not call this function, the default number of activation buffer slots is set differently depending on the [CoreMode](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreMode/index.md>).

- [`CoreMode.Single`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreMode/#single>) : 2 \* (the number of target core ids)
- [`CoreMode.Multi`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreMode/#multi>) : 2 \* (the number of target clusters)
- [`CoreMode.Global4`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreMode/#global4>) : 2 \* (the number of target clusters)
- [`CoreMode.Global8`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreMode/#global8>) : 2

> [!NOTE]
>
> This function has no effect on MXQ file in version earlier than MXQv7.

> [!NOTE]
>
> Currently, LLM model's activation slot is fixed to 1 and ignoring `count`.

**Parameters**

- `count` *\[in\]* — Multi activation counts. Must be \>= 1.

### get\_activation\_slots

```python
def get_activation_slots(self) -> int
```

Returns activation buffer slot count.

> [!NOTE]
>
> This function has no meaning on MXQ file in version earlier than MXQv7.

**Returns**

Activation buffer slot count.

### get\_core\_ids

```python
def get_core_ids(self) -> List[CoreId]
```

Returns the list of NPU CoreIds to be used for model inference.

The returned list reflects only what was set on this [ModelConfig](<https://docs.mobilint.com/runtime/en/v1.5/api/python/ModelConfig/index.md>). What it contains depends on the configured [CoreMode](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreMode/index.md>):

- Single mode via `set_single_core_mode(core_ids)`: the specified core\_ids.
- Single mode via `set_single_core_mode(num_cores)`: empty. The actual cores are resolved from the target NPU when the model is created.
- Multi, Global4, Global8, and Auto modes: empty. Use [`get_clusters()`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/ModelConfig/#get-clusters>) for the cluster-based modes.

**Returns**

A list of NPU CoreIds.

### get\_clusters

```python
def get_clusters(self) -> List[Cluster]
```

Returns the list of clusters to be used for model inference.

The returned list reflects only what was set on this [ModelConfig](<https://docs.mobilint.com/runtime/en/v1.5/api/python/ModelConfig/index.md>). What it contains depends on the configured [CoreMode](<https://docs.mobilint.com/runtime/en/v1.5/api/python/CoreMode/index.md>):

- Multi mode via [`set_multi_core_mode()`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/ModelConfig/#set-multi-core-mode>) and Global4 mode via [`set_global4_core_mode()`](<https://docs.mobilint.com/runtime/en/v1.5/api/python/ModelConfig/#set-global4-core-mode>): the specified clusters (defaults to all clusters).
- Single, Global8, and Auto modes: empty.

**Returns**

A list of NPU Clusters.

### \_\_repr\_\_

```python
def __repr__(self)
```
