Overview

Overview#

REGULUS is Mobilint’s ultra-low-power AI System-on-Chip. It combines a Mobilint NPU with an Arm Cortex-A53 application processor and the video and signal-processing blocks an edge vision device needs, so camera input, preprocessing, inference, and postprocessing can all run on one chip.

This manual is for developers who write applications for REGULUS-based devices. It currently covers the REGULUS Vision DSP: what it offloads from the CPU, and the C API that application code uses to reach it.

On-Chip Processing Blocks#

Besides the NPU and the CPU, REGULUS has the following blocks for image, video, and signal processing.

Block

Function

MIPI CSI-2 RX

Receives video from a MIPI camera.

ISP

Converts the raw sensor signal into a color image (BGR, RGB, YUV formats).

CODEC

JPEG encoder and decoder.

CONVERT

Color-space and size conversion.

HEVC

H.264 and H.265 encoder and decoder.

DMA350

DMA engine for 2D copy, mirror, rotation, and crop.

Vision DSP

Programmable DSP that runs the pre- and postprocessing steps around NPU inference.

The ISP and the fixed-function accelerators are reached through Linux V4L2 and GStreamer. The Vision DSP is reached through the SP500 DSP API that this manual documents.

Where the Vision DSP Fits#

An AI application on REGULUS moves its data through the following stages, whatever the model:

Input -> Preprocessing (CPU or DSP) -> Inference (NPU) -> Postprocessing (CPU or DSP) -> Output

  • Input is the data the model runs on, such as a camera frame that has passed through the ISP, or an audio clip.

  • Preprocessing turns the input into the tensor the model takes, for example by resizing and padding a frame (letterbox) or by converting audio into a log-mel spectrogram.

  • Inference runs the compiled model on the NPU.

  • Postprocessing turns the model outputs into results the application can use, for example by decoding YOLO head outputs into boxes.

Preprocessing and postprocessing can run on either the CPU or the Vision DSP. Running them on the DSP frees the CPU for the rest of the application. The DSP also shares memory with the NPU, so a tensor the DSP has prepared can be handed to the NPU without a copy.

See DSP Overview for the operations the DSP provides and how the API is organized, and DSP Programming Guide to build and run a first application.

Manual Versions#

The version of this manual follows the REGULUS package release, using its major and minor numbers only. Manual v4.1 covers every v4.1.x release of the REGULUS package; a patch release does not change the documented interface. Each version documents the SP500 DSP API that its REGULUS release ships: REGULUS v4.1 ships SP500 DSP API 1.0.0. Use the version selector to open the manual that matches the REGULUS package release installed on your device.