DSP Overview

DSP Overview#

The SP500 DSP API lets an application on the REGULUS Cortex-A53 host run operations on the Vision DSP. The API consists of one shared library, libsp500.so, and four C headers.

Supported Operations#

Operation	Header	Description
Letterbox	`sp500/letterbox.h`	Aspect-preserving resize with constant padding. Prepares a camera frame as a fixed-size model input and returns the scale and padding needed to map detections back to the source frame.
Log-mel spectrogram	`sp500/log_mel.h`	Whisper-style log-mel spectrogram from float32 PCM, for speech models such as Whisper and Qwen3-ASR. A CPU reference implementation is included.
YOLO decode	`sp500/yolo.h`	Turns the raw output tensors of a YOLO network into a list of detections: box decoding, confidence filtering, and duplicate removal, in one DSP round trip. Supports detection, segmentation, and pose.

sp500/dsp.h is the foundation header. It defines the status codes, attaching to the DSP, shared memory, and buffers. Each module header includes it, so include only the module headers you need.

Architecture#

 Application (Cortex-A53)
        |  sp500_* calls
        v
 libsp500.so  ----------------------->  Vision DSP firmware
        |        request / completion          |
        v                                      v
 +----------------------- shared-memory window ------------------------+
 |  input and output buffers, read and written in place by the DSP     |
 |  and, through their physical address, by the NPU                     |
 +---------------------------------------------------------------------+

The application calls the API on the host. Each call sends one request to the DSP firmware and waits for its completion.
Tensors are passed as buffers. A buffer allocated in the shared-memory window is read and written by the DSP in place. A buffer in ordinary application memory is staged through shared memory, which costs one copy in each direction.
The window is physically contiguous. The physical address of a shared-memory buffer can be handed to the NPU runtime, so the DSP and the NPU work on the same memory with no copy in between.

Design of the API#

Plain C, Stable ABI#

The headers are plain C and include only <stdint.h> and <stddef.h>. They work from C, from any C++ compiler and standard library, and from any language with a C FFI. Every name they define starts with sp500_ or SP500_.

The functions form the ABI of the shared library, and follow these rules:

Every function returns sp500_status (an int32_t) or a fixed-width scalar. Enumerations name constants only; values cross the boundary as int32_t or uint32_t.
No struct is passed or returned by value across the boundary; only pointers are.
Every struct ends with a reserved array that must be zero. A later version may give those words a meaning whose zero value is today’s behavior.
No function throws an exception.

Parameter Structs and Defaults#

Each operation takes a parameter struct, and each parameter struct has an *_params_init() function that writes its defaults. The defaults are not all zero, so always call the init function before setting fields. An all-zero struct is rejected because the required fields are invalid at zero, but a struct that was zeroed and then given only its required fields runs with every other option at zero instead of at its default.

Buffers#

Every tensor is an sp500_buffer. The buffer records where its memory lives, so the same function accepts both kinds:

Constructor	Memory	Cost
`sp500_buffer_alloc()`	Shared-memory window	No copy. Release it with `sp500_buffer_release()`.
`sp500_buffer_wrap()`	Memory the application already owns	One copy in each direction per call. Nothing to release.
`sp500_buffer_none()`	None	Marks an optional tensor as unused.

Use shared-memory buffers for anything large, and for anything the NPU also reads or writes.

Threading#

Every function may be called concurrently from several threads and several processes. Each DSP call takes one lock shared by all threads and processes for the length of its round trip, so concurrent calls are safe but run one at a time. To make a sequence of calls atomic as a whole, wrap it in sp500_dsp_lock() and sp500_dsp_unlock().

Status Codes#

Every call returns an sp500_status. SP500_OK is 0 and every error is negative, so both rc != SP500_OK and rc < 0 are correct tests. sp500_status_string() converts a code to text. Output values are meaningful only after SP500_OK.

Code	Value	Meaning
`SP500_OK`	0	Completed; output buffers are valid.
`SP500_ERR_INVALID_ARG`	-1	A parameter is missing, out of range, or inconsistent with another.
`SP500_ERR_UNSUPPORTED`	-2	The module cannot run that combination.
`SP500_ERR_BUFFER_TOO_SMALL`	-3	A supplied buffer cannot hold the result.
`SP500_ERR_NOT_READY`	-4	`sp500_dsp_open()` has not succeeded.
`SP500_ERR_NO_MEMORY`	-5	Host or shared-memory allocation failed.
`SP500_ERR_TIMEOUT`	-6	The DSP did not answer within the deadline.
`SP500_ERR_DSP`	-7	The DSP reported a failure.
`SP500_ERR_INTERNAL`	-8	Unexpected internal condition.

Treat an unrecognized negative value as a generic failure; a later version may add codes. Two results are successes rather than errors: a frame with nothing in it returns SP500_OK with a detection count of 0, and a log-mel request with fewer samples than one hop returns SP500_OK with 0 frames.

Versioning#

The library version is the API version. sp500_version() returns the version of the loaded library, encoded like SP500_VERSION_INT (major * 10000 + minor * 100 + patch), and sp500_version_string() returns it as text.

A major release changes the library SONAME and requires rebuilding the application.
Minor and patch releases are drop-in replacements.

The library is installed under three names:

libsp500.so       -> libsp500.so.1        (link time: what -lsp500 finds)
libsp500.so.1     -> libsp500.so.1.0.0    (run time: the SONAME)
libsp500.so.1.0.0                         (the library itself)

An application binary records libsp500.so.1. See DSP Programming Guide for how to copy the library to a device.