mirror/whisper.cpp
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Remove unnecessary memory reallocation in fft (#2080)

Browse Source 
fft_out needs to be twice the frame_size, not the frame_step.  It is resized in fft() anyway, but this change prevents an unnecessary reallocation.

n_fft must match the mel filter size, so it is best not to calculate it from the framesize.

We only need to get the magnitudes for half the spectrum since the other half is a mirror and not used in the mel filter loop later.
This commit is contained in:
goldwaving 2024-04-28 15:06:12 -02:30 committed by GitHub
parent 858452d58d
commit 22b6598cc9
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 6 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

10
whisper.cpp
View File

@ -2900,11 +2900,13 @@ static void log_mel_spectrogram_worker_thread(int ith, const std::vector<float>
int n_samples, int frame_size, int frame_step, int n_threads,
const whisper_filters & filters, whisper_mel & mel) {
std::vector<float> fft_in(frame_size, 0.0);
std::vector<float> fft_out(2 * frame_step);
// make sure n_fft == 1 + (WHISPER_N_FFT / 2), bin_0 to bin_nyquist
int n_fft = 1 + (frame_size / 2);
std::vector<float> fft_out(2 * frame_size);
int n_fft = filters.n_fft;
int i = ith;
// make sure n_fft == 1 + (WHISPER_N_FFT / 2), bin_0 to bin_nyquist
assert( n_fft == 1 + (frame_size / 2) );
// calculate FFT only when fft_in are not all zero
for (; i < std::min(n_samples / frame_step + 1, mel.n_len); i += n_threads) {
const int offset = i * frame_step;
@ -2923,7 +2925,7 @@ static void log_mel_spectrogram_worker_thread(int ith, const std::vector<float>
// Calculate modulus^2 of complex numbers
// Use pow(fft_out[2 * j + 0], 2) + pow(fft_out[2 * j + 1], 2) causes inference quality problem? Interesting.
for (int j = 0; j < frame_size; j++) {
for (int j = 0; j < n_fft; j++) {
fft_out[j] = (fft_out[2 * j + 0] * fft_out[2 * j + 0] + fft_out[2 * j + 1] * fft_out[2 * j + 1]);
}