A Systematic Evaluation of Adversarial Attacks against Speech Emotion Recognition Models

The model and its baseline accuracies

Nicolas Facchinetti, Federico Simonetta, Stavros Ntalampiras

Published in Intelligent Computing 2024

Written with GPT-5.4-mini

Tiny changes in the spectrogram can make the model hear the wrong emotion.

The paper checks how easy it is to fool one speech-emotion classifier. The model is a CNN-LSTM, and it is tested on three datasets: EmoDB (German), EMOVO (Italian), and RAVDESS (English). On clean audio, it scores 0.909, 0.872, and 0.911. Those numbers mean the share of test clips the model labels correctly before anyone attacks it.

Then the authors try seven attacks: FGSM, BIM, DeepFool, JSMA, Carlini & Wagner, PixelAttack, and BoundaryAttack. JSMA is the harshest: accuracy falls to about 0.01-0.02, so the model is right only 1%-2% of the time. PixelAttack is also effective, even though it changes only a few pixels in the spectrogram. BoundaryAttack matters for a different reason: it still works even without direct access to gradients.

The nice part is that the paper does not just say “the model failed”. It also checks how much each attack changes the input. That matters, because the most damaging attack is not always the one that makes the biggest visual mess. In other words, a small edit can be enough.

The language comparison is almost flat: after attack, mean white-box accuracy is 0.063 for EmoDB, 0.067 for RAVDESS, and 0.068 for EMOVO. So German, English, and Italian are similarly fragile here. Gender differences exist, but they are smaller. The bottom line is simple: this kind of SER system can look fine on normal audio and still be very easy to trick.