콘텐츠로 이동

log_mel.h

Whisper-style log-mel spectrogram on the SP500 DSP.

Turns float32 PCM into the log-mel spectrogram that Whisper and Qwen3-ASR encoders consume. The maths follows the Hugging Face WhisperFeatureExtractor exactly: reflect-padded frames of n_fft samples every hop_length, a periodic Hann window, the power spectrum, a slaney mel filter bank, log10(max(1e-10, mel)), then optionally the Whisper normalisation (clamp at max - 8, then (x + 4) / 4).

Output layout

The result is frame-major: features[t * n_mels + m] for t < n_frames, with n_frames = n_samples / hop_length - the trailing centred frame is dropped, as Hugging Face does. This is the [n_frames, n_mels] layout the MXQ Whisper models take. Hugging Face's own input_features is the transpose.

Example

sp500_dsp_open(0);

sp500_log_mel_params p;
sp500_log_mel_params_init(&p);
p.n_samples = 16000 * 30;          // 30 s at 16 kHz

const int32_t frames = sp500_log_mel_num_frames(p.n_samples, p.hop_length);

sp500_buffer audio = sp500_buffer_wrap(pcm, p.n_samples * sizeof(float));
sp500_buffer feats = sp500_buffer_alloc(frames * p.n_mels * sizeof(float),
                                        SP500_ALIGN_DSP);

sp500_log_mel_result r;
if(sp500_log_mel(&audio, &p, &feats, &r) == SP500_OK)
{
    // feats.data holds r.n_frames * p.n_mels floats, frame-major.
}
sp500_buffer_release(&feats);

Note

A 30 s clip is a few thousand frames of DSP work. If a call returns SP500_ERR_TIMEOUT, raise the deadline with sp500_set_dsp_timeout_ms() rather than retrying - see the warning on that function.

Data structures

Name Description
sp500_log_mel_params Front-end parameters.
sp500_log_mel_result Values reported by a log-mel call.

Macros

SP500_LOG_MEL_MAX_MELS

#define SP500_LOG_MEL_MAX_MELS 256

Largest supported mel bin count.

SP500_LOG_MEL_MAX_FFT

#define SP500_LOG_MEL_MAX_FFT 2048

Largest supported FFT size.

Functions

sp500_log_mel_params_init

void sp500_log_mel_params_init(sp500_log_mel_params *params)

Fill params with the Whisper defaults documented above.

sp500_log_mel_num_frames

int32_t sp500_log_mel_num_frames(int32_t n_samples, int32_t hop_length)

Frames produced for n_samples: n_samples / hop_length, or 0 for non-positive arguments. Use it to size the feature buffer.

sp500_log_mel_num_freq

int32_t sp500_log_mel_num_freq(int32_t n_fft)

Frequency bins for n_fft: n_fft / 2 + 1, or 0 if n_fft is not supported. Use it to size a custom mel_filters bank.

sp500_log_mel

sp500_status sp500_log_mel(const sp500_buffer *audio, const sp500_log_mel_params *params, sp500_buffer *features, sp500_log_mel_result *result)

Compute the log-mel spectrogram of float32 PCM on the DSP.

Parameters

  • audio — Input samples; at least n_samples * sizeof(float) bytes. 16 kHz mono for Whisper.
  • params — Front-end parameters; see sp500_log_mel_params.
  • features — Receives n_frames * n_mels float32, frame-major; at least n_frames * n_mels * sizeof(float) bytes.
  • result — Receives the frame count and peak. May be NULL.

Returns

SP500_OK on success. SP500_ERR_UNSUPPORTED for an n_fft the DSP cannot transform, or for a mel bank it cannot run - one whose weights are all zero because the frequency range covers no FFT bin, or one too dense to describe. SP500_ERR_INVALID_ARG for any other bad parameter, SP500_ERR_BUFFER_TOO_SMALL for an undersized buffer, SP500_ERR_NO_MEMORY if shared memory is exhausted, SP500_ERR_NOT_READY, SP500_ERR_TIMEOUT or SP500_ERR_DSP.

Note

n_samples < hop_length yields zero frames. That is reported as SP500_OK with result->n_frames == 0 and nothing written.

sp500_log_mel_cpu

sp500_status sp500_log_mel_cpu(const float *audio, const sp500_log_mel_params *params, float *features, int32_t features_count, sp500_log_mel_result *result)

Host CPU reference for sp500_log_mel(): same tables, same per-frame maths, no DSP involved.

Useful during bring-up to check the DSP result, and as a fallback when the DSP is busy. It does not require sp500_dsp_open().

Parameters

  • audio — params->n_samples float32 samples.
  • params — The same parameters the DSP call would take.
  • features — Receives n_frames * n_mels float32, frame-major.
  • features_count — Number of floats features can hold.
  • result — Receives the frame count and peak. May be NULL; host_seconds is the CPU time of this call.

Returns

SP500_OK, SP500_ERR_INVALID_ARG, SP500_ERR_UNSUPPORTED (the same n_fft and mel-bank cases as the DSP call), SP500_ERR_BUFFER_TOO_SMALL or SP500_ERR_NO_MEMORY. It returns the same status as sp500_log_mel() for the same parameters.