Efficient and Robust Video Defense Framework against 3D-field Personalized Talking Face

arXiv:2512.21019 · cs.CV · Submitted 2025-12-24 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Efficient and Robust Video Defense Framework against 3D-field Personalized Talking Face".

Jane: Here's the gist of the paper. State-of-the-art 3D-field video-referenced Talking Face Generation (TFG) methods synthesize high-fidelity personalized talking-face videos in real time by modeling 3D geometry and appearance from reference portrait video.

Tom: This capability raises significant privacy concerns regarding malicious misuse of personal portraits.

Jane: To address this, we propose a novel and efficient video defense framework against 3D-field TFG methods, which protects portrait video by perturbing the 3D information acquisition process while maintain high-fidelity video quality.

Tom: Recent advances in 3D-field video-referenced Talking Face Generation (TFG) methods leverage 3D neural fields to capture and store a individual’s comprehensive geometric and appearance information from a short reference video during training.

Paper summary: Tom: The core idea here is moving the defense from just fixing individual pixels to perturbing the way the three dimensional data itself is captured. Jane They introduce a few key parts for this, starting with something called a similarity guided parameter sharing mechanism which helps with computational efficiency. Tom And then they use a multi scale dual domain attention module that tries to optimize both the spatial and frequency perturbations at the same time. Jane What's interesting is that they use frequency domain perturbation first instead of just spatial ones because those tend to be more robust against changes in scale and color, which helps with purification models. Tom Then they have this multiscale optimization strategy that refines those perturbations under different resolution conditions, like when the video gets resized or compressed, to make it even stronger against scaling. Jane And finally, they add a spatial domain attention mechanism to selectively suppress those pixel level perturbations that are redundant.

Lu: From my side, what I find interesting is how they are tackling the trade off between efficiency and robustness simultaneously by combining these different perturbation strategies. Meng But from a deployment standpoint, I have to ask about the cost. How much does this framework add to the inference time when we're actually trying to run it on a live service? Tom That's what I want to know, Meng. The abstract says they achieve forty-seven times acceleration over the fastest baseline while maintaining high fidelity, which sounds pretty good for speed <ref:2512.21019#pg2>. Jane It also mentions that their method shows strong robustness against purification models like JPEG, DiffPure, and FreqPure. Lu And they even did a cross model evaluation where the synchronization values of the generated videos show that VDF keeps its adversarial effects even on models that weren't used during their optimization process.

Meng: Cross model evaluation is important because it suggests this isn't just a defense tailored perfectly to one specific generation model. Tom And they claim it achieves sixty times acceleration over baselines in terms of computational cost, which means lower running expenses for the service. Jane The results show that when you test SyncTalk based on these purified images, VDF gets the best results across all metrics compared to other state-of-the-art methods.

Lalam: My analysis confirms what I read: VDF achieves a FID score of one hundred seventy-four point eight eight, which means the generated videos deviate more strongly from the original identity than other methods. Lu That deviation is exactly what we want in a defense framework—it means the protection mechanism is actively disrupting the generation process in a way that's measurable.

Meng: So, for someone who only listens to this show, it means that if you are building a system to generate personalized talking faces from reference portraits, this VDF approach offers much better speed and resilience against common compression or scaling attacks than just applying simple 2D defenses frame by frame <ref:2512.21019#pg1>.

Conclusion: Tom: So wrapping up the paper "Efficient and Robust Video Defense Framework against three dee-field Personalized Talking Face," the main point is that they solved the problem of needing a defense that works well for three dimensional field models without destroying video quality <ref:2512.21019#pg1,Efficient and Robust Video Defense Framework against 3D-field Personalized Talking Face>. Jane They did this by focusing on perturbing the three dimensional information acquisition process itself, using frequency domain perturbations combined with adaptive spatial attention and parameter sharing. Tom The implication is that we can now protect these types of personalized video synthesis systems more effectively in real time than before, without incurring the massive computational penalties of older image based defenses. Jane It shifts the focus from just patching individual frames to defending the core three dee representation being learned by the model <ref:2512.21019#pg1>.

Lu: What I think is that their success with cross model evaluation is really telling because it shows a level of generality in their defense mechanism, suggesting it could be more broadly applicable across different TFG architectures. Meng From an engineering perspective, if we can achieve this kind of acceleration and robustness against scaling operations like JPEG or Resize, that’s what makes a defense practical for production deployment. Tom The paper flags that the main limitation they mentioned is that they focused specifically on defending against three dee-field TFG methods, so we don't automatically know how this framework performs against other types of generation models <ref:2512.21019#pg1,against 3D-field TFG methods>.

Jane: So the takeaway for listeners is that privacy in personalized video generation isn't just about applying a filter to an image; it's about designing a defense that understands the underlying three dee geometry and appearance modeling process <ref:2512.21019#pg1>. Tom That’s what this work does by looking at how to perturb that acquisition process efficiently.

Lu: The authors suggest future work could be developing one unified video defense framework that could handle both image referenced and video referenced TFG models at the same time, which would be a big step forward for comprehensive privacy protection.

Meng: And for us in shipping models, the promise here is a way to keep things fast while still getting strong results against common attacks on generated content.

Rui-qing Sun, §Xingshan Yao, Tian Lan, Jia-Ling Shi, Chen-Hao Cui, Hui-Yang Zhao, §Zhijing Wu, Chen Yang, §Xian-Ling Mao

Beijing Institute of Technology

cs.CV

Submitted: 2025-12-24

Updated: 2026-10-31

Project page: https://zkaochi.github.io/VDF

License: http://creativecommons.org/licenses/by-nc-nd/4.0/

Importance score: 89/100

The gist: State-of-the-art 3D-field video-referenced Talking Face Generation (TFG) methods synthesize high-fidelity personalized talking-face videos in real time by modeling 3D geometry and appearance from…

Key concepts

3D-field TFG Methods
These are advanced AI models that use 3D neural fields to capture a person's complete geometric and appearance data from a short reference video. They generate high-fidelity, personalized talking face videos in real time, allowing for precise control over head poses and facial movements based on the input portrait.
Frequency Domain Perturbation
Instead of changing pixels directly in the image (spatial domain), this technique introduces subtle changes to the frequency components of the video. This approach is chosen because it offers better resistance against variations in scale, such as resizing or color compression, making the resulting defense more robust.
Multi-scale Dual-Domain Attention Module
This module works by applying attention mechanisms across different scales and domains (spatial and frequency). It helps the system jointly optimize perturbations to ensure that the introduced changes are effective while suppressing unnecessary pixel-level noise, leading to a refined defense against scaling attacks.

Terminology

Summary

State-of-the-art 3D-field video-referenced Talking Face Generation (TFG) methods synthesize high-fidelity personalized talking-face videos in real time by modeling 3D geometry and appearance from reference portrait video. p. 0

Where the paper says it

Efficient and Robust Video Defense Framework against 3D-field Personalized Talking Face Rui-qing Sun1,† Xingshan Yao1,† Tian Lan1 Jia-Ling Shi1 Chen-Hao Cui1 Hui-Yang Zhao1 Zhijing Wu1 Chen Yang1,∗ Xian-Ling Mao1,∗ 1Beijing Institute of Technology Abstract State-of-the-art 3D-field video-referenced Talking Face Generation (TFG) methods synthesize high-fidelity personalized talking-face videos in real time by modeling 3D geometry and appearance from reference portrait video. This capability raises significant privacy concerns regarding malicious misuse of personal portraits. p. 0

Where the paper says it

Efficient and Robust Video Defense Framework against 3D-field Personalized Talking Face Rui-qing Sun1,† Xingshan Yao1,† Tian Lan1 Jia-Ling Shi1 Chen-Hao Cui1 Hui-Yang Zhao1 Zhijing Wu1 Chen Yang1,∗ Xian-Ling Mao1,∗ 1Beijing Institute of Technology Abstract State-of-the-art 3D-field video-referenced Talking Face Generation (TFG) methods synthesize high-fidelity personalized talking-face videos in real time by modeling 3D geometry and appearance from reference portrait video. This capability raises significant privacy concerns regarding malicious misuse of personal portraits.

This capability raises significant privacy concerns regarding malicious misuse of personal portraits. To address this, we propose a novel and efficient video defense framework against 3D-field TFG methods, which protects portrait video by perturbing the 3D information acquisition process while maintain high-fidelity video quality. p. 0
Where the paper says it

While image-based defenses could apply per-frame 2D perturbations, they incur prohibitive computational costs, severe video quality degradation, failing to disrupt 3D information for video protection. To address this, we propose a novel and efficient video defense framework against 3D-field TFG methods, which protects portrait video by perturbing the 3D information acquisition process while maintain high-fidelity video quality.

Motivation for the Defense Framework

Recent advances in 3D-field video-referenced Talking Face Generation (TFG) methods leverage 3D neural fields to capture and store a individual’s comprehensive geometric and appearance information from a short reference video during training. These methods synthesize high-fidelity personalized videos at over 30 frames per second (FPS) while enabling precise control of head poses and facial dynamics through additional motion parameters. p. 0

Where the paper says it

∗Corresponding author. image-referenced TFG approaches [16, 27], these 3D field– based methods synthesize high-fidelity personalized videos at over 30 frames per second (FPS) while enabling precise control of head poses and facial dynamics through additional motion parameters.

However, to the best of our knowledge, defenses specifically designed for 3D-field video-referenced TFG methods remain largely unexplored. p. 0
Where the paper says it

Consequently, significant privacy concerns have emerged regarding malicious exploitation of such systems—particularly when driven by toxic text prompts or harmful audio inputs to synthesize nonconsensual portrait videos. However, to the best of our knowledge, defenses specifically designed for 3D-field video-referenced TFG methods remain largely unexplored.

A simple solution is to decompose an reference portrait video into individual frames and apply image-level defenses frame-byframe. p. 0
Where the paper says it

However, to the best of our knowledge, defenses specifically designed for 3D-field video-referenced TFG methods remain largely unexplored. Intuitively, a simple solution is to decompose an reference portrait video into individual frames and apply image-level defenses frame-byframe [8, 20, 21, 33, 42] developed for general imagebased TFG models.

This frame-by-frame defense approach suffers from four critical limitations against 3D-field TFG methods: (1) Efficiency, (2) Robustness, (3) Effectiveness, and (4) Visual Quality. p. 0
Where the paper says it

Intuitively, a simple solution is to decompose an reference portrait video into individual frames and apply image-level defenses frame-byframe [8, 20, 21, 33, 42] developed for general imagebased TFG models. Unfortunately, this defense approach suffers from four critical limitations against 3D-field TFG methods: (1) Efficiency: Frame-by-frame defense leads to extremely low efficiency, which makes it impractical for real-world use; (2) Robustness: Video transmission operations such as resizing and color compression [34] can easily destroy spatial-domain perturbations injected by existing image-based defense methods, and purification methods designed to remove harmful noise in images can also eliminate these perturbations [23, 26, 48]; (3) Effectiveness: They cannot apply effective perturbation on frequency video information, leading to worse protection performance, as illustrated in Fig 1; and (4) Visual Quality: Defended videos exhibit poor visual quality due to the numerous perturbation on spatial-domain.

Defense Strategy Components

Our proposed framework overcomes the weaknesses of image-based defense framework by introducing several key designs. Specifically, our method introduces: (1) a similarity-guided parameter sharing mechanism for computational efficiency, and (2) a multi-scale dual-domain attention module to jointly optimize spatial-frequency perturbations. p. 0

Where the paper says it

To address this, we propose a novel and efficient video defense framework against 3D-field TFG methods, which protects portrait video by perturbing the 3D information acquisition process while maintain high-fidelity video quality. Specifically, our method introduces: (1) a similarity-guided parameter sharing mechanism for computational efficiency, and (2) a multi-scale dual-domain attention module to jointly optimize spatial-frequency perturbations.

We first introduce effective perturbation using frequency domain perturbation rather than the spatial domain perturbations, which also exhibit better robustness against scale variations and color [11, 46]. p. 1
Where the paper says it

To efficiently protect portrait privacy from such personalized TFG models, we propose a video-level defense framework (VDF) that achieves both high efficiency and strong robustness. fense Effectiveness and Robustness: We first introduce effective perturbation using frequency domain perturbation rather than the spatial domain perturbations, which also exhibit better robustness against scale variations and color [11, 46] and purification models.

Then, we design a multiscale optimization strategy that refines perturbations under varying resolution conditions to further strengthen robustness against scaling such as resizing and color compression. p. 1
Where the paper says it

fense Effectiveness and Robustness: We first introduce effective perturbation using frequency domain perturbation rather than the spatial domain perturbations, which also exhibit better robustness against scale variations and color [11, 46] and purification models. Then, we design a multiscale optimization strategy that refines perturbations under varying resolution conditions to further strengthen robustness against scaling such as resizing and color compression; and (3) Visual Quality: An additional spatial-domain attention mechanism is designed to adaptively suppresses redundant pixel-level perturbations, As the global nature of frequency-domain perturbations inevitably introduces some noise redundancy.

An additional spatial-domain attention mechanism is designed to adaptively suppresses redundant pixel-level perturbations. p. 1
Where the paper says it

fense Effectiveness and Robustness: We first introduce effective perturbation using frequency domain perturbation rather than the spatial domain perturbations, which also exhibit better robustness against scale variations and color [11, 46] and purification models. Then, we design a multiscale optimization strategy that refines perturbations under varying resolution conditions to further strengthen robustness against scaling such as resizing and color compression; and (3) Visual Quality: An additional spatial-domain attention mechanism is designed to adaptively suppresses redundant pixel-level perturbations, As the global nature of frequency-domain perturbations inevitably introduces some noise redundancy.

Experimental Validation and Performance

Extensive experiments demonstrate that our proposed framework exhibits strong defense capability and achieves a 47× acceleration over the fastest baseline while maintaining high fidelity. Our proposed method exhibit powerful robustness to scaling operations and state-of-the-art purification models like JPEG [34], DiffPure [23] and FreqPure [26]. p. 1

Where the paper says it

Following prior work, we conduct extensive experiments to evaluate our method on a corresponding personalized TFG dataset [28, 44]. Experimental results demonstrate several important findings: (1) Compared with conventional image-based generalized TFG defenses or spatial-domain method, our proposed framework effectively resists attacks from 3D-field based personalized TFG models while maintaining high visual quality (Section and Section ); (2) Our proposed method exhibit powerful robustness to scaling operations and state-of-the-art purification models like JPEG [34], DiffPure [23] and FreqPure [26] (Section 4.

Computational cost is much lower than baselines, achieving 60x acceleration. p. 1
Where the paper says it

Experimental results demonstrate several important findings: (1) Compared with conventional image-based generalized TFG defenses or spatial-domain method, our proposed framework effectively resists attacks from 3D-field based personalized TFG models while maintaining high visual quality (Section and Section ); (2) Our proposed method exhibit powerful robustness to scaling operations and state-of-the-art purification models like JPEG [34], DiffPure [23] and FreqPure [26] (Section 4. 4); and (3) Computational cost is much lower than baselines, achieving 60x acceleration.

In comparison with state-of-the-art methods on generated videos by Synctalk based on pured images, our method achieves the best results across all metrics. p. 6
Where the paper says it

Using 11 videos of 45 seconds each as reference videos, we applied state-of-theart protection methods and computed metrics between the protected and original reference videos. As shown in Table 1, our method achieves the best results across all metrics, demonstrating its superior ability to retain the visual information of the original video.

Furthermore, VDF exhibits substantially higher FID score (174. p. 7
Where the paper says it

Notably, although VDF adds smaller perturbations, the generated identities still deviate significantly from the protected ones, reflecting a trade-off between visual quality and robustness to purification. Noise Domain Spatial Mask VGG Init Noise Protected Image SSIM PSNR FID Time Cost Spatial – ✓ ✓ 0.

88), meaning the generated videos deviate more strongly from the original identity. p. 3
Where the paper says it

Adversarial Examples for Image-based Talking Face Generation Previous defense approaches have primarily focused on image-based TFG models. These methods protect identity privacy by introducing carefully designed perturbations to the portrait images, misleading the latent diffusion model to learn incorrect representations, thereby degrading the visual quality of the generated talking-face videos.

Robustness and Transferability Analysis

We evaluate four purification methods—JPEG [34], Resize [41], DiffPure [23], and FreqPure [26]—by training and testing SyncTalk on the purified videos. VDF achieves the best performance under JPEG and Resize purification, benefiting from its frequency-domain perturbations and multi-scale robustness that are difficult to remove. p. 7

Where the paper says it

Anti-Purification Experiments We evaluate four purification methods—JPEG [34], Resize [41], DiffPure [23], and FreqPure [26]—by training and testing SyncTalk on the purified videos. As shown in Table3, VDF achieves the best performance under JPEG and Resize purification, benefiting from its frequency-domain perturbations and multi-scale robustness that are difficult to remove.

We conduct a cross-model evaluation, and the synchronization values of the generated videos indicate that VDF retains strong adversarial effects even on models that were not used during optimization. This cross-model robustness suggests promising potential for broader applicability and further validates the effectiveness of our method. p. 1
Where the paper says it

4); and (3) Computational cost is much lower than baselines, achieving 60x acceleration. In summary, our main contributions are four-fold: • We identify four essential properties that a practical portrait video defense method against 3D-field TFG methods should possess: high computational efficiency, robustness to video scaling and purification, defense effectiveness and high-fidelity visual quality.

Conclusion and Future Work

We propose a dual-domain attention–based video defense framework that protects portrait privacy from 3D-field personalized TFG models. By combining frequency-domain perturbations with adaptive spatial attention and parameter sharing mechanism, our method achieves high visual quality, robustness to scaling and purification, and superior efficiency in real-world privacy defense. p. 7

Where the paper says it

Conclusion We propose a dual-domain attention–based video defense framework that protects portrait privacy from 3D-field personalized TFG models. By combining frequency-domain perturbations with adaptive spatial attention and parameter sharing mechanism, our method achieves high visual quality, robustness to scaling and purification, and superior efficiency in real-world privacy defense.

In future work, we aim to develop a unified video defense framework capable of simultaneously countering privacy threats posed by both image-referenced and video-referenced TFG models. p. 7
Where the paper says it

By combining frequency-domain perturbations with adaptive spatial attention and parameter sharing mechanism, our method achieves high visual quality, robustness to scaling and purification, and superior efficiency in real-world privacy defense. In future work, we aim to develop a unified video defense framework capable of simultaneously countering privacy threats posed by

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Privacy Protection against 3D-field TFG models: The improved system can protect portrait videos by perturbing 3D information acquisition while maintaining high fidelity, as demonstrated by achieving a 47x acceleration over the fastest baseline while maintaining high fidelity.

  2. Efficiency Enhancement: The system can process long reference videos efficiently by using a similarity-guided parameter sharing mechanism, which reduces computational costs and achieves 60x acceleration compared to baselines.

  3. Robustness to Video Degradation: The improved AI system can maintain defense effectiveness against video transmission issues like resizing and color compression because it introduces perturbations in the frequency domain, which are more resilient to scale variations and color than spatial domain perturbations.

  4. Purification Resilience: The system exhibits strong robustness against state-of-the-art purification attacks like JPEG, DiffPure, and FreqPure, as shown by achieving the best performance under JPEG and Resize purification.

  5. High Visual Quality Preservation: The system can maintain high fidelity during defense by employing a multi-scale dual-domain attention module to jointly optimize spatial-frequency perturbations, which includes an additional spatial-domain attention mechanism designed to adaptively suppresses redundant pixel-level perturbations.

  6. Cross-Model Generalization: The framework can be applied effectively across different 3D architectures, as evidenced by retaining strong synchronization values even when optimized on one model (NeRF) and tested on another (3DGS).

  7. Disruption of Identity Extraction: The system severely disrupts audio-driven identity extraction, achieving a lowest Sync score (0.1700), which prevents the generation of high-quality forged videos by models like Synctalk.

Abstract

Talking Face Generation (TFG) has made remarkable progress in synthesizing realistic speaker videos, evolving into two paradigms based on identity incorporation: image-based 2D TFG and video-based 3D TFG. Despite enabling efficient content creation, TFG also raises risks of unauthorized portrait forgery and privacy abuse, motivating proactive defenses. Existing defenses mainly target image-based 2D TFG, while, to our knowledge, no proactive defense has been developed for video-based 3D TFG. A straightforward solution is to apply existing image-based defenses independently to each video frame. However, experiments show that this adaptation is ineffective and computationally expensive: video-based 3D TFG lacks the image-reference modules and reverse diffusion processes exploited by existing defenses, and per-frame optimization incurs substantial overhead. To address this gap, we analyze the vulnerabilities of video-based 3D TFG and propose ErodedFace, an efficient proactive defense framework targeting reference-video preprocessing. ErodedFace uses a curriculum-inspired two-stage strategy: Stage I establishes scale-robust disruption through similarity-guided parameter inheritance and adaptive multi-scale optimization; Stage II improves perceptual fidelity and transformation robustness without weakening protection. Extensive experiments show that ErodedFace effectively protects portrait videos against unauthorized video-based 3D TFG, achieves a 10.0x speedup in defense generation over existing image-based 2D TFG defenses, and remains robust to common transformations and recent purification methods.

Sources

Related papers