Efficient and Robust Video Defense Framework against 3D-field Personalized Talking Face
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.
Tom: We're looking at the paper: "Efficient and Robust Video Defense Framework against 3D-field Personalized Talking Face".
Jane: Here's the gist of the paper. State-of-the-art 3D-field video-referenced Talking Face Generation (TFG) methods synthesize high-fidelity personalized talking-face videos in real time by modeling 3D geometry and appearance from reference portrait video.
Tom: This capability raises significant privacy concerns regarding malicious misuse of personal portraits.
Jane: To address this, we propose a novel and efficient video defense framework against 3D-field TFG methods, which protects portrait video by perturbing the 3D information acquisition process while maintain high-fidelity video quality.
Tom: Recent advances in 3D-field video-referenced Talking Face Generation (TFG) methods leverage 3D neural fields to capture and store a individual’s comprehensive geometric and appearance information from a short reference video during training.
Paper summary: Tom: The core idea here is moving the defense from just fixing individual pixels to perturbing the way the three dimensional data itself is captured. Jane They introduce a few key parts for this, starting with something called a similarity guided parameter sharing mechanism which helps with computational efficiency. Tom And then they use a multi scale dual domain attention module that tries to optimize both the spatial and frequency perturbations at the same time. Jane What's interesting is that they use frequency domain perturbation first instead of just spatial ones because those tend to be more robust against changes in scale and color, which helps with purification models. Tom Then they have this multiscale optimization strategy that refines those perturbations under different resolution conditions, like when the video gets resized or compressed, to make it even stronger against scaling. Jane And finally, they add a spatial domain attention mechanism to selectively suppress those pixel level perturbations that are redundant.
Lu: From my side, what I find interesting is how they are tackling the trade off between efficiency and robustness simultaneously by combining these different perturbation strategies. Meng But from a deployment standpoint, I have to ask about the cost. How much does this framework add to the inference time when we're actually trying to run it on a live service? Tom That's what I want to know, Meng. The abstract says they achieve forty-seven times acceleration over the fastest baseline while maintaining high fidelity, which sounds pretty good for speed <ref:2512.21019#pg2>. Jane It also mentions that their method shows strong robustness against purification models like JPEG, DiffPure, and FreqPure. Lu And they even did a cross model evaluation where the synchronization values of the generated videos show that VDF keeps its adversarial effects even on models that weren't used during their optimization process.
Meng: Cross model evaluation is important because it suggests this isn't just a defense tailored perfectly to one specific generation model. Tom And they claim it achieves sixty times acceleration over baselines in terms of computational cost, which means lower running expenses for the service. Jane The results show that when you test SyncTalk based on these purified images, VDF gets the best results across all metrics compared to other state-of-the-art methods.
Lalam: My analysis confirms what I read: VDF achieves a FID score of one hundred seventy-four point eight eight, which means the generated videos deviate more strongly from the original identity than other methods. Lu That deviation is exactly what we want in a defense framework—it means the protection mechanism is actively disrupting the generation process in a way that's measurable.
Meng: So, for someone who only listens to this show, it means that if you are building a system to generate personalized talking faces from reference portraits, this VDF approach offers much better speed and resilience against common compression or scaling attacks than just applying simple 2D defenses frame by frame <ref:2512.21019#pg1>.
Conclusion: Tom: So wrapping up the paper "Efficient and Robust Video Defense Framework against three dee-field Personalized Talking Face," the main point is that they solved the problem of needing a defense that works well for three dimensional field models without destroying video quality <ref:2512.21019#pg1,Efficient and Robust Video Defense Framework against 3D-field Personalized Talking Face>. Jane They did this by focusing on perturbing the three dimensional information acquisition process itself, using frequency domain perturbations combined with adaptive spatial attention and parameter sharing. Tom The implication is that we can now protect these types of personalized video synthesis systems more effectively in real time than before, without incurring the massive computational penalties of older image based defenses. Jane It shifts the focus from just patching individual frames to defending the core three dee representation being learned by the model <ref:2512.21019#pg1>.
Lu: What I think is that their success with cross model evaluation is really telling because it shows a level of generality in their defense mechanism, suggesting it could be more broadly applicable across different TFG architectures. Meng From an engineering perspective, if we can achieve this kind of acceleration and robustness against scaling operations like JPEG or Resize, that’s what makes a defense practical for production deployment. Tom The paper flags that the main limitation they mentioned is that they focused specifically on defending against three dee-field TFG methods, so we don't automatically know how this framework performs against other types of generation models <ref:2512.21019#pg1,against 3D-field TFG methods>.
Jane: So the takeaway for listeners is that privacy in personalized video generation isn't just about applying a filter to an image; it's about designing a defense that understands the underlying three dee geometry and appearance modeling process <ref:2512.21019#pg1>. Tom That’s what this work does by looking at how to perturb that acquisition process efficiently.
Lu: The authors suggest future work could be developing one unified video defense framework that could handle both image referenced and video referenced TFG models at the same time, which would be a big step forward for comprehensive privacy protection.
Meng: And for us in shipping models, the promise here is a way to keep things fast while still getting strong results against common attacks on generated content.
Rui-qing Sun, §Xingshan Yao, Tian Lan, Jia-Ling Shi, Chen-Hao Cui, Hui-Yang Zhao, §Zhijing Wu, Chen Yang, §Xian-Ling Mao
Beijing Institute of Technology
cs.CV
Submitted: 2025-12-24
Updated: 2026-10-31
Project page: https://zkaochi.github.io/VDF
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
Importance score: 89/100
The gist: State-of-the-art 3D-field video-referenced Talking Face Generation (TFG) methods synthesize high-fidelity personalized talking-face videos in real time by modeling 3D geometry and appearance from…
Key concepts
- 3D-field TFG Methods
- These are advanced AI models that use 3D neural fields to capture a person's complete geometric and appearance data from a short reference video. They generate high-fidelity, personalized talking face videos in real time, allowing for precise control over head poses and facial movements based on the input portrait.
- Frequency Domain Perturbation
- Instead of changing pixels directly in the image (spatial domain), this technique introduces subtle changes to the frequency components of the video. This approach is chosen because it offers better resistance against variations in scale, such as resizing or color compression, making the resulting defense more robust.
- Multi-scale Dual-Domain Attention Module
- This module works by applying attention mechanisms across different scales and domains (spatial and frequency). It helps the system jointly optimize perturbations to ensure that the introduced changes are effective while suppressing unnecessary pixel-level noise, leading to a refined defense against scaling attacks.
Terminology
Summary
State-of-the-art 3D-field video-referenced Talking Face Generation (TFG) methods synthesize high-fidelity personalized talking-face videos in real time by modeling 3D geometry and appearance from reference portrait video. p. 0 Efficient and Robust Video Defense Framework against 3D-field Personalized
Talking Face
Rui-qing Sun1,† Xingshan Yao1,† Tian Lan1
Jia-Ling Shi1 Chen-Hao Cui1 Hui-Yang Zhao1
Zhijing Wu1 Chen Yang1,∗ Xian-Ling Mao1,∗
1Beijing Institute of Technology
Abstract
State-of-the-art 3D-field video-referenced Talking Face
Generation (TFG) methods synthesize high-fidelity personalized talking-face videos in real time by modeling 3D
geometry and appearance from reference portrait video. This capability raises significant privacy concerns regarding malicious misuse of personal portraits. p. 0 Efficient and Robust Video Defense Framework against 3D-field Personalized
Talking Face
Rui-qing Sun1,† Xingshan Yao1,† Tian Lan1
Jia-Ling Shi1 Chen-Hao Cui1 Hui-Yang Zhao1
Zhijing Wu1 Chen Yang1,∗ Xian-Ling Mao1,∗
1Beijing Institute of Technology
Abstract
State-of-the-art 3D-field video-referenced Talking Face
Generation (TFG) methods synthesize high-fidelity personalized talking-face videos in real time by modeling 3D
geometry and appearance from reference portrait video. This capability raises significant privacy concerns regarding malicious misuse of personal portraits. While image-based defenses could apply per-frame 2D perturbations, they incur prohibitive computational costs, severe video quality
degradation, failing to disrupt 3D information for video
protection. To address this, we propose a novel and efficient video defense framework against 3D-field TFG methods, which protects portrait video by perturbing the 3D information acquisition process while maintain high-fidelity
video quality.Where the paper says it
Where the paper says it
Where the paper says it
Motivation for the Defense Framework
Recent advances in 3D-field video-referenced Talking Face Generation (TFG) methods leverage 3D neural fields to capture and store a individual’s comprehensive geometric and appearance information from a short reference video during training. These methods synthesize high-fidelity personalized videos at over 30 frames per second (FPS) while enabling precise control of head poses and facial dynamics through additional motion parameters. p. 0 ∗Corresponding author. image-referenced TFG approaches [16, 27], these 3D field–
based methods synthesize high-fidelity personalized videos
at over 30 frames per second (FPS) while enabling precise control of head poses and facial dynamics through additional motion parameters. Consequently, significant privacy concerns have emerged regarding malicious exploitation of such systems—particularly when driven by toxic
text prompts or harmful audio inputs to synthesize nonconsensual portrait videos. However, to the best of our knowledge, defenses specifically designed for 3D-field video-referenced TFG methods remain largely unexplored. However, to the best of our knowledge, defenses specifically designed for 3D-field video-referenced TFG methods remain largely unexplored. Intuitively, a simple solution is to decompose an reference portrait video into individual frames and apply image-level defenses frame-byframe [8, 20, 21, 33, 42] developed for general imagebased TFG models. Intuitively, a simple solution is to decompose an reference portrait video into individual frames and apply image-level defenses frame-byframe [8, 20, 21, 33, 42] developed for general imagebased TFG models. Unfortunately, this defense approach
suffers from four critical limitations against 3D-field TFG
methods: (1) Efficiency: Frame-by-frame defense leads to
extremely low efficiency, which makes it impractical for
real-world use; (2) Robustness: Video transmission operations such as resizing and color compression [34] can easily destroy spatial-domain perturbations injected by existing
image-based defense methods, and purification methods designed to remove harmful noise in images can also eliminate
these perturbations [23, 26, 48]; (3) Effectiveness: They
cannot apply effective perturbation on frequency video information, leading to worse protection performance, as illustrated in Fig 1; and (4) Visual Quality: Defended videos
exhibit poor visual quality due to the numerous perturbation
on spatial-domain.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Defense Strategy Components
Our proposed framework overcomes the weaknesses of image-based defense framework by introducing several key designs. Specifically, our method introduces: (1) a similarity-guided parameter sharing mechanism for computational efficiency, and (2) a multi-scale dual-domain attention module to jointly optimize spatial-frequency perturbations. p. 0 To address this, we propose a novel and efficient video defense framework against 3D-field TFG methods, which protects portrait video by perturbing the 3D information acquisition process while maintain high-fidelity
video quality. Specifically, our method introduces: (1) a
similarity-guided parameter sharing mechanism for computational efficiency, and (2) a multi-scale dual-domain attention module to jointly optimize spatial-frequency perturbations. To efficiently protect portrait
privacy from such personalized TFG models, we propose a video-level defense framework (VDF) that achieves both high efficiency and
strong robustness. fense Effectiveness and Robustness: We first introduce
effective perturbation using frequency domain perturbation
rather than the spatial domain perturbations, which also exhibit better robustness against scale variations and color
[11, 46] and purification models. fense Effectiveness and Robustness: We first introduce
effective perturbation using frequency domain perturbation
rather than the spatial domain perturbations, which also exhibit better robustness against scale variations and color
[11, 46] and purification models. Then, we design a multiscale optimization strategy that refines perturbations under
varying resolution conditions to further strengthen robustness against scaling such as resizing and color compression; and (3) Visual Quality: An additional spatial-domain
attention mechanism is designed to adaptively suppresses
redundant pixel-level perturbations, As the global nature of
frequency-domain perturbations inevitably introduces some
noise redundancy. fense Effectiveness and Robustness: We first introduce
effective perturbation using frequency domain perturbation
rather than the spatial domain perturbations, which also exhibit better robustness against scale variations and color
[11, 46] and purification models. Then, we design a multiscale optimization strategy that refines perturbations under
varying resolution conditions to further strengthen robustness against scaling such as resizing and color compression; and (3) Visual Quality: An additional spatial-domain
attention mechanism is designed to adaptively suppresses
redundant pixel-level perturbations, As the global nature of
frequency-domain perturbations inevitably introduces some
noise redundancy.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Experimental Validation and Performance
Extensive experiments demonstrate that our proposed framework exhibits strong defense capability and achieves a 47× acceleration over the fastest baseline while maintaining high fidelity. Our proposed method exhibit powerful robustness to scaling operations and state-of-the-art purification models like JPEG [34], DiffPure [23] and FreqPure [26]. p. 1 Following prior work, we conduct extensive experiments
to evaluate our method on a corresponding personalized
TFG dataset [28, 44]. Experimental results demonstrate
several important findings: (1) Compared with conventional
image-based generalized TFG defenses or spatial-domain
method, our proposed framework effectively resists attacks
from 3D-field based personalized TFG models while maintaining high visual quality (Section and Section ); (2)
Our proposed method exhibit powerful robustness to scaling operations and state-of-the-art purification models like
JPEG [34], DiffPure [23] and FreqPure [26] (Section 4. Experimental results demonstrate
several important findings: (1) Compared with conventional
image-based generalized TFG defenses or spatial-domain
method, our proposed framework effectively resists attacks
from 3D-field based personalized TFG models while maintaining high visual quality (Section and Section ); (2)
Our proposed method exhibit powerful robustness to scaling operations and state-of-the-art purification models like
JPEG [34], DiffPure [23] and FreqPure [26] (Section 4. 4);
and (3) Computational cost is much lower than baselines,
achieving 60x acceleration. Using 11 videos of 45
seconds each as reference videos, we applied state-of-theart protection methods and computed metrics between the
protected and original reference videos. As shown in Table 1, our method achieves the best results across all metrics, demonstrating its superior ability to
retain the visual information of the original video. Notably, although VDF adds
smaller perturbations, the generated identities still deviate
significantly from the protected ones, reflecting a trade-off
between visual quality and robustness to purification. Noise Domain Spatial Mask VGG Init Noise Protected Image
SSIM PSNR FID Time Cost
Spatial – ✓ ✓ 0. Adversarial Examples for Image-based Talking
Face Generation
Previous defense approaches have primarily focused on
image-based TFG models. These methods protect identity
privacy by introducing carefully designed perturbations to
the portrait images, misleading the latent diffusion model
to learn incorrect representations, thereby degrading the visual quality of the generated talking-face videos.Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Where the paper says it
Robustness and Transferability Analysis
We evaluate four purification methods—JPEG [34], Resize [41], DiffPure [23], and FreqPure [26]—by training and testing SyncTalk on the purified videos. VDF achieves the best performance under JPEG and Resize purification, benefiting from its frequency-domain perturbations and multi-scale robustness that are difficult to remove. p. 7 Anti-Purification Experiments
We evaluate four purification methods—JPEG [34], Resize
[41], DiffPure [23], and FreqPure [26]—by training and
testing SyncTalk on the purified videos. As shown in Table3, VDF achieves the best performance under JPEG and
Resize purification, benefiting from its frequency-domain
perturbations and multi-scale robustness that are difficult to
remove. 4);
and (3) Computational cost is much lower than baselines,
achieving 60x acceleration. In summary, our main contributions are four-fold:
• We identify four essential properties that a practical portrait video defense method against 3D-field TFG methods
should possess: high computational efficiency, robustness
to video scaling and purification, defense effectiveness
and high-fidelity visual quality.Where the paper says it
Where the paper says it
Conclusion and Future Work
We propose a dual-domain attention–based video defense framework that protects portrait privacy from 3D-field personalized TFG models. By combining frequency-domain perturbations with adaptive spatial attention and parameter sharing mechanism, our method achieves high visual quality, robustness to scaling and purification, and superior efficiency in real-world privacy defense. p. 7 Conclusion
We propose a dual-domain attention–based video defense
framework that protects portrait privacy from 3D-field personalized TFG models. By combining frequency-domain
perturbations with adaptive spatial attention and parameter
sharing mechanism, our method achieves high visual
quality, robustness to scaling and purification, and superior
efficiency in real-world privacy defense. By combining frequency-domain
perturbations with adaptive spatial attention and parameter
sharing mechanism, our method achieves high visual
quality, robustness to scaling and purification, and superior
efficiency in real-world privacy defense. In future work, we
aim to develop a unified video defense framework capable
of simultaneously countering privacy threats posed byWhere the paper says it
Where the paper says it
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Privacy Protection against 3D-field TFG models: The improved system can protect portrait videos by perturbing 3D information acquisition while maintaining high fidelity, as demonstrated by achieving a
47x acceleration over the fastest baseline while maintaining high fidelity.
-
Efficiency Enhancement: The system can process long reference videos efficiently by using a
similarity-guided parameter sharing mechanism,
which reduces computational costs and achieves60x acceleration
compared to baselines. -
Robustness to Video Degradation: The improved AI system can maintain defense effectiveness against video transmission issues like resizing and color compression because it introduces perturbations in the frequency domain, which are
more resilient to scale variations and color
than spatial domain perturbations. -
Purification Resilience: The system exhibits strong robustness against state-of-the-art purification attacks like JPEG, DiffPure, and FreqPure, as shown by achieving the
best performance under JPEG and Resize purification.
-
High Visual Quality Preservation: The system can maintain high fidelity during defense by employing a
multi-scale dual-domain attention module to jointly optimize spatial-frequency perturbations,
which includes anadditional spatial-domain attention mechanism designed to adaptively suppresses redundant pixel-level perturbations.
-
Cross-Model Generalization: The framework can be applied effectively across different 3D architectures, as evidenced by retaining strong synchronization values even when optimized on one model (NeRF) and tested on another (3DGS).
-
Disruption of Identity Extraction: The system severely disrupts audio-driven identity extraction, achieving a
lowest Sync score (0.1700),
which prevents the generation of high-quality forged videos by models like Synctalk.
Abstract
Talking Face Generation (TFG) has made remarkable progress in synthesizing realistic speaker videos, evolving into two paradigms based on identity incorporation: image-based 2D TFG and video-based 3D TFG. Despite enabling efficient content creation, TFG also raises risks of unauthorized portrait forgery and privacy abuse, motivating proactive defenses. Existing defenses mainly target image-based 2D TFG, while, to our knowledge, no proactive defense has been developed for video-based 3D TFG. A straightforward solution is to apply existing image-based defenses independently to each video frame. However, experiments show that this adaptation is ineffective and computationally expensive: video-based 3D TFG lacks the image-reference modules and reverse diffusion processes exploited by existing defenses, and per-frame optimization incurs substantial overhead. To address this gap, we analyze the vulnerabilities of video-based 3D TFG and propose ErodedFace, an efficient proactive defense framework targeting reference-video preprocessing. ErodedFace uses a curriculum-inspired two-stage strategy: Stage I establishes scale-robust disruption through similarity-guided parameter inheritance and adaptive multi-scale optimization; Stage II improves perceptual fidelity and transformation robustness without weakening protection. Extensive experiments show that ErodedFace effectively protects portrait videos against unauthorized video-based 3D TFG, achieves a 10.0x speedup in defense generation over existing image-based 2D TFG defenses, and remains robust to common transformations and recent purification methods.
Sources
- Your Classifier is Secretly an Energy Based Model and You Should Treat it Like One
- Stochastic Security: Adversarial Defense Using Long-Run Dynamics of Energy-Based Models
- Let Them Talk: Audio-Driven Multi-Person Conversational Video Generation
- Mist: Towards Improved Adversarial Examples for Diffusion Models
- Towards Deep Learning Models Resistant to Adversarial Attacks
- Diffusion-based Adversarial Purification from the Perspective of the Frequency Domain
- Open-Sora 2.0: Training a Commercial-Level Video Generation Model in $200k
- SyncTalk++: High-Fidelity and Efficient Synchronized Talking Heads Synthesis Using Gaussian Splatting
- Raising the Cost of Malicious AI-Powered Image Editing
- JPEG Compressed Images Can Bypass Protections Against AI Editing
- Real-time Neural Radiance Talking Portrait Synthesis via Audio-spatial Decomposition
- Mitigating Adversarial Effects Through Randomization
- DFA-NeRF: Personalized Talking Head Generation via Disentangled Face Attributes Neural Rendering
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs