ModelConfig Configuration

Note

ModelConfig belongs to Mobilint’s runtime library — the C++ and Python SDK that loads a compiled model and runs inference on an ARIES device. Installing and using the runtime is covered in the Runtime Manual. This page focuses on one runtime object, ModelConfig, and how it assigns NPU cores and a Core Mode to each model.

An ARIES device has eight local cores, arranged as two clusters of four (each cluster also has a global core that coordinates the multi-core modes). When you load a model, a ModelConfig tells the runtime how many — and which — cores the model may use and which core mode to run in. Because each model can be pinned to its own cores, a single ARIES device can be partitioned across several models running at the same time.

The workflow is always the same: create a ModelConfig, call one method to choose the mode and cores, then pass it in when you create the model.

Choosing cores and a core mode

Each core mode maps to one ModelConfig method. The mode itself — Single, Multi, Global4, Global8 — is explained in Core Mode; the method just applies it to a model.

Goal	C++ method / Python method	What the model uses
Single-core, specific cores	setSingleCoreMode({...}) / set_single_core_mode(core_ids=[...])	exactly the local cores you list
Single-core, all cores	setSingleCoreMode() / set_single_core_mode(num_cores=8)	all 8 local cores
Multi (batch)	setMultiCoreMode({clusters}) / set_multi_core_mode([clusters])	the 4 local cores of each listed cluster batch-process multiple inputs
Global4	setGlobal4CoreMode({clusters}) / set_global4_core_mode([clusters])	the 4 local cores of each listed cluster, collaborating on one input
Global8	setGlobal8CoreMode() / set_global8_core_mode()	all 8 local cores across both clusters, collaborating on one input

Example

A few representative configurations. Each creates a ModelConfig, selects a mode, and builds the model with it (the launch call is omitted for brevity).

#include "qbruntime/qbruntime.h"
using namespace mobilint;

const char* MXQ_PATH = "model_name.mxq";

StatusCode sc;
auto acc = Accelerator::create(sc);

// Single-core mode on two specific local cores
ModelConfig mc_single;
mc_single.setSingleCoreMode({{Cluster::Cluster0, Core::Core0},
                             {Cluster::Cluster0, Core::Core1}});
auto model_single = Model::create(MXQ_PATH, mc_single, sc);

// Multi-core mode on Cluster0
ModelConfig mc_multi;
mc_multi.setMultiCoreMode({Cluster::Cluster0});
auto model_multi = Model::create(MXQ_PATH, mc_multi, sc);

// Global4 mode on Cluster1
ModelConfig mc_g4;
mc_g4.setGlobal4CoreMode({Cluster::Cluster1});
auto model_g4 = Model::create(MXQ_PATH, mc_g4, sc);

// Global8 mode
ModelConfig mc_g8;
mc_g8.setGlobal8CoreMode();
auto model_g8 = Model::create(MXQ_PATH, mc_g8, sc);

from qbruntime import Accelerator, Model, ModelConfig, CoreId, Cluster, Core

acc = Accelerator()
MXQ_PATH = "model_name.mxq"

# Single-core mode on two specific local cores
mc_single = ModelConfig()
mc_single.set_single_core_mode(core_ids=[
    CoreId(Cluster.Cluster0, Core.Core0),
    CoreId(Cluster.Cluster0, Core.Core1),
])
model_single = Model(MXQ_PATH, mc_single)

# Multi-core mode on Cluster0
mc_multi = ModelConfig()
mc_multi.set_multi_core_mode([Cluster.Cluster0])
model_multi = Model(MXQ_PATH, mc_multi)

# Global4 mode on Cluster1
mc_g4 = ModelConfig()
mc_g4.set_global4_core_mode([Cluster.Cluster1])
model_g4 = Model(MXQ_PATH, mc_g4)

# Global8 mode
mc_g8 = ModelConfig()
mc_g8.set_global8_core_mode()
model_g8 = Model(MXQ_PATH, mc_g8)