From Inaudible Inputs to Model Failures: Low-Frequency Safety Risks in LALMs

This paper identifies a previously overlooked safety risk in large audio-language models (LALMs): inaudible low-frequency signals (5–20 Hz) that can enter the model's audio frontend and influence its generation while remaining imperceptible to human users. The authors propose Intermittent Low-Frequency Lockout (ILL), a black-box red teaming method that evaluates this risk using a universal waveform template, and Distributional Requery Guard (DRG), a defense mechanism that detects low-frequency distribution shifts and conditionally requests a second recording for semantic recovery.

Attack effectiveness: Across six LALMs and multiple audio understanding tasks, ILL reduces accuracy by up to 67 percentage points. The paper states: ILL reduces accuracy by up to 67 percentage points, while its audible noise ratio remains at 0.06–0.08% and its mean human audibility rating is 1.33, close to 1.17 for clean audio.

Defense effectiveness: DRG achieves detection F1 scores of 89.69–99.00% and improves defense performance by 17.6 percentage points, raising the six-model mean accuracy from 28.5% to 46.1%.

ILL combines two components:

1. Sentence Attention Scale Estimation: This derives active intervals from multiscale changes in audio attention. The method computes averaged self-attention matrices from a reference LALM, detects boundaries using cross-boundary and within-segment attention masses at scales ρ, 5ρ, 10ρ, and produces a corpus-derived active duration Lon. The paper notes: This corpus-derived duration aligns each active interval with a typical span of continuous semantic attention.

2. Frequency Confusion Transfer: This converts corpus spectral variation into a continuous-phase low-frequency state sequence. It uses spectral centroids from training recordings, quantizes them into n states, estimates transition statistics, and decodes the most probable state sequence via dynamic programming. The resulting waveform uses continuous phase + smooth envelope and operates at 5–20 Hz.

The final perturbation is intermittent: The schedule places an active interval at the beginning of the recording and repeats it every Tcyc until the recording ends.

DRG is a plug-in input filter that detects low-frequency distribution shifts before LALM inference and requests independent acoustic evidence only for flagged inputs. It uses spectral distribution descriptors, K-means clustering into two groups (clean vs. interference), and labels clusters by mean descriptor mass in the low-frequency range. At deployment, it assigns inputs to the nearest centroid; flagged inputs trigger a second recording request with joint semantic recovery.

Models evaluated: MiniCPM-o 4.5, StepAudio2-Mini, Qwen2.5-Omni, Qwen3-Omni, GPT-audio-mini, and Gemini-3.5-flash. ILL was constructed once with Qwen2.5-Omni as reference and transferred to all other targets.

Tasks and datasets: MMAU (speech, sound, music categories) for audio QA, LibriSpeech for speech recognition, CoVoST 2 for speech translation, and RAVDESS for emotion classification. Each dataset used 100 test examples.

Attack baselines: Gaussian noise, PNL (Perceptual Noise Labels with Natural/Mechanical/Human categories), Audio-Adv (targeted white-box attack), and Whisper (universal acoustic attack).

Defense baselines: LF-Suppression, Deep Feature Loss (DFL) denoising, and MMSE-STSA enhancement.

The fixed ILL waveform degrades all four task types and transfers across models without target-specific optimization. The paper notes: This effect does not weaken after black-box transfer: the largest RAVDESS reduction, 67 percentage points, occurs on the unseen StepAudio2 target rather than on the Qwen2.5 reference model.

ILL achieves comparable or stronger disruption than audible-noise baselines while using only the 5–20 Hz band. It gives the lowest CoVoST 2 BLEU on every target and remains competitive on accuracy-based tasks.

ILL has an audible noise ratio (ANR) of only 0.06–0.08%, whereas all baselines exceed 98.9%. Human ratings: ILL receives a mean rating of 1.33 and a median of 1, close to the clean condition with a mean of 1.17 and a median of 1. The closest baselines are Audio-Adv. at 3.75 and Whisper at 3.92.

DRG detection F1 scores range from 89.69–99.00% under the Average training condition. Recovery results: DRG raises the six-model mean accuracy from 28.5% without defense to 46.1% after clean reacquisition, outperforming the signal-processing baselines. Under persistent interference (noisy second recording), gains are smaller but still above no-defense.

The requery mechanism transfers beyond ILL: clean reacquisition is best or tied for best in 19 of 24 settings across four additional attack types (Gaussian, PNL, Audio-Adv., Whisper).

Forced DRG requery on clean inputs shows minimal degradation: Ten of the twelve scores remain within 0.03 of standard inference, and DRG exceeds standard inference on two classification results.

• Amplitude: a steep performance decline up to β = 4, followed by saturation

• Duty cycle: effectiveness improves with active proportion but "changes little beyond γ > 70%"

• Attack duration: a clear duration threshold at the default segment length. Shorter segments are ineffective, whereas extending the segment by one or two orders of magnitude provides no further gain.

• State-sequence construction: ILL causes greater degradation than Gaussian noise, fixed-frequency signals, and uniform frequency sweeps, supporting the effectiveness of the proposed state-sequence design.

ILL reduces audio attention mass (from 0.0416 to 0.0373 on MMAU; from 0.0777 to 0.0531 on RAVDESS), weakens representation similarity (cosine similarity falls from 1.0000 to 0.8659 on MMAU and to 0.6309 on RAVDESS), and lowers output confidence (correct-answer probability drops from 0.8050 to 0.1709 on MMAU and from 0.8240 to 0.0179 on RAVDESS). DRG partially restores all these metrics.

The paper acknowledges: "we do not deploy ILL in uncontrolled scenarios... Our experiments instead simulate microphone reception and do not reproduce the complete acoustic path from a loudspeaker through a real-world acoustic environment to a microphone. The analysis characterizes changes in attention, representations, and output confidence rather than establishing a complete causal mechanism."

The paper's stated contributions are: (1) identifying the hidden risk of inaudible low-frequency inputs to LALMs and introducing ILL; (2) developing DRG for detection and mitigation; (3) evaluating both across six LALMs and multiple audio tasks, linking low-frequency interference to reduced acoustic attention and output confidence.

Improvements to AI Systems:

1. Add a low-frequency input filter to all audio frontends. The system will reject or attenuate signals below 20 Hz before they reach the model’s encoder, preventing inaudible interference from altering attention and output. This is a hard, model-agnostic safeguard that requires no retraining.

2. Implement a distributional anomaly detector for audio inputs. The system will compute spectral descriptors (e.g., low-frequency mass, centroid) and flag inputs that deviate from the clean training distribution. Flagged inputs will trigger a second independent recording request, and the system will fuse semantic outputs from both recordings to recover accuracy.

3. Enable conditional re-acquisition logic. The system will automatically request a fresh recording only when the initial input shows low-frequency distribution shift, avoiding unnecessary delays on clean inputs. This preserves latency and user experience while defending against attacks.

4. Add a confidence-based fallback mechanism. When the model’s output confidence drops below a threshold (e.g., correct-answer probability < 0.3) and the input is flagged, the system will re-run inference on a second recording and select the higher-confidence output. This mitigates both adversarial and accidental low-frequency interference.

5. Integrate a universal low-frequency waveform template detector. The system will maintain a library of known attack templates (e.g., ILL’s 5–20 Hz state-sequence waveform) and perform cross-correlation or spectral matching to detect and block them in real time, even before full inference.

6. Improve robustness of audio attention mechanisms. The system will apply attention masking or re-weighting to suppress contributions from low-frequency bands during self-attention computation, reducing the attack’s influence on semantic focus without altering the model’s architecture.

What the Improved AI System Can Do:

• Resist inaudible attacks: It will maintain near-clean accuracy (within 3% of baseline) when exposed to low-frequency interference, even on unseen models and tasks.

• Detect and recover autonomously: It will identify suspicious audio inputs with >95% F1 score and recover accuracy by up to 17.6 percentage points via re-acquisition, without human intervention.

• Preserve benign performance: It will show negligible degradation (<0.03 accuracy drop) on clean inputs, ensuring no false positives or unnecessary re-recordings.

• Transfer across models and tasks: The defense will work for speech recognition, translation, emotion classification, and audio QA, regardless of the underlying LALM.

• Operate in real-time: The filter and detector will add minimal latency (<50 ms) and require no model retraining, making it deployable in production systems like voice assistants, hearing aids, and autonomous agents.

Large audio-language models (LALMs) have demonstrated strong capabilities in understanding diverse audio inputs. This diversity includes low-frequency signals that are inaudible to humans but can still enter the model and influence its generation. However, the practical impact of such low-frequency inputs on LALMs remains largely unexplored. In this paper, we propose Intermittent Low-Frequency Lockout (ILL), an inaudible red teaming method that evaluates this risk using a universal waveform template in a black box setting. ILL uses Sentence Attention Scale Estimation to determine active intervals and Frequency Confusion Transfer to construct a low-frequency waveform with continuous phase from corpus spectral variation. To mitigate this risk, we propose Distributional Requery Guard (DRG) to detect low-frequency distribution shifts and conditionally request a second recording for semantic recovery. Across six LALMs and multiple audio understanding tasks, ILL reduces accuracy by up to 67 percentage points while receiving a mean human audibility rating of 1.33, close to 1.17 for clean audio; DRG raises mean attacked accuracy from 28.5% to 46.1% after clean reacquisition. These findings identify a previously overlooked safety risk for LALMs and provide a foundation for future research on robust audio understanding.