Runtime Library Installation

Runtime Library Installation#

This section explains how to install the Mobilint NPU qb Runtime, or maccel (pronounced em-accel) which is required to run inference on Mobilint NPUs using precompiled or custom models.

Attention

  • Please complete driver installation before proceeding with runtime setup.

  • Before installing the runtime library, please check compatible driver and firmware versions from here.

Table of Contents#

Linux/Ubuntu#

Option 1: Installing via APT on Linux/Ubuntu#

This method installs the runtime system-wide using Debian based OS’s package manager apt.

Installation Process#

  1. Add the Mobilint APT repository to your system before proceeding with the installation.

    Tip

    If you’ve already added the Mobilint APT repository during driver installation, you can skip this step.

    # Add Mobilint's official GPG key:
sudo apt update
sudo apt install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://dl.mobilint.com/apt/gpg.pub -o /etc/apt/keyrings/mblt.asc
sudo chmod a+r /etc/apt/keyrings/mblt.asc

# Add the repository to apt sources:
printf "%s\n" \
    "deb [signed-by=/etc/apt/keyrings/mblt.asc] https://dl.mobilint.com/apt \
    stable multiverse" \
    "deb [signed-by=/etc/apt/keyrings/mblt.asc] https://dl.mobilint.com/apt \
    $(. /etc/os-release && echo "${UBUNTU_CODENAME:-$VERSION_CODENAME}") multiverse" | \
    sudo tee /etc/apt/sources.list.d/mobilint.list > /dev/null

# Update available packages
sudo apt update

  2. Install via apt package manager.

    sudo apt install mobilint-npu-runtime

  3. Verify installation.

    dpkg -L mobilint-npu-runtime

  4. Install python runtime library package. Replace {version} and {python_version} appropriately.

    pip install /opt/maccel/maccel-{version}-cp{python_version}*.whl

  5. Verify python runtime library installation.

    python -c "import maccel"

Troubleshooting Anaconda Error#

If you are using anaconda virtual environment, you can get error message like below.

../lib/libstdc++.so.6: version `GLIBCXX_3.4.32' not found

This issue is caused by the version confliction of libstdc++ bundled with Anaconda, and the appropriate solution may vary depending on the Anaconda version. We recommend searching online for the most suitable fix.

Additionally, to prevent such conflicts, it is recommended to use Python’s built-in virtual environment (venv), which utilizes the system’s libstdc++ library.

Option 2: Manual Installation from Download Center#

Requirements#

  • Download the appropriate runtime library from Mobilint’s official Download Center.

Installation Process#

  1. Unzip the downloaded runtime library file. {RUNTIME_VERSION} corresponding to your runtime library version.

    tar -xvzf qb_runtime_aries2-v4_v{RUNTIME_VERSION}.tar.gz

    You should see a directory structure like this:

    qb_runtime_aries2-v4_v{RUNTIME_VERSION}
├── Makefile
├── maccel
│   ├── maccel
│   │ ├── include    # include path
│   │ │   └── maccel
│   │ ├── lib        # library path
│   │ └── python
│   └── resnet50
├── mbltweb
└── mobilint-cli

There are two options from here: System-wide installation, or using without installing.

Option 2-1: System-Wide Installation#

Install the runtime library system-wide using the following commands.

cd qb_runtime_aries2-v4_v{RUNTIME_VERSION}
sudo make install

Option 2-2: Use Without Installation#

Specify the paths to qb_runtime_aries2-v4_v{RUNTIME_VERSION}/maccel/maccel/include and qb_runtime_aries2-v4_v{RUNTIME_VERSION}/maccel/maccel/lib during compilation as shown below:

g++ -o {output_binary} {source_code} -I{path_to_include} \
    -L{path_to_library} -lmaccel

Then export environment variable LD_LIBRARY_PATH to let linker know runtime library(*.so) path.

export LD_LIBRARY_PATH={path_to_library}

Option 3: Installation via pip#

Note

Installing the Python library package via pip on Ubuntu is supported starting from runtime library version 0.29.0.

The Python library can be installed using pip. Run the following command to install the Python library package:

pip install maccel

Installing on Windows#

Requirements#

  • Download the appropriate runtime library from Mobilint’s official Download Center.

Installation Process#

On Windows, only non-installation method is supported. Please refer to section Comiling C++ code - Windows.

Contents