Peak-End-Net: A Peak-End Rule Inspired Framework for Generalizable Video Aesthetic Assessment

arXiv:2607.13941 · cs.CV · Submitted 2026-07-15 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "Peak-End-Net: A Peak-End Rule Inspired Framework for Generalizable Video Aesthetic Assessment".

Jane: Here's the gist of the paper. Video aesthetic assessment (VAA) aims to predict how aesthetically pleasing a video is, yet remains far less explored than other visual assessment tasks.

Tom: In this paper, we revisit VAA from a psychological perspective and propose Peak-End-Net, a lightweight and interpretable framework inspired by the peak-end rule.

Jane: Psychological Foundation for Temporal Aggregation The authors draw inspiration from the psychologically grounded peak-end rule

13: , which suggests that people tend to judge a temporal experience mainly according to its salient moments and the ending.

Tom: This principle is especially plausible for video aesthetics, where a few striking moments and the final impression may disproportionately shape the overall score.

Title and authors: Tom: So Jane, this paper, "Peak-End-Net: A Peak-End Rule Inspired Framework for Generalizable Video Aesthetic Assessment," it really dives into why assessing video aesthetics is so hard because it's so subjective.

Jane: Yeah, that's the thing. People judge a video not just by looking at every single frame equally, but how they summarize or remember the whole experience, which is what this paper calls the peak-end rule thirteen <ref:2607.13941#pg1>.

Lu: Exactly. It suggests that people tend to judge a temporal experience mainly according to its most intense moments and the ending segment rather than uniformly averaging every moment >. That’s a psychological principle being applied here because current methods often just treat video aesthetics like any other generic regression problem, which limits how well they generalize >.

Meng: So what's the actual mechanism? Is this some complicated thing we have to train on millions of videos just to capture the 'peak' and 'end' >? Because if it requires massive manual annotation, I’m not deploying this on production traffic.

Tom: The paper says they use frame-wise aesthetic priors from a pre-trained Image Aesthetic Assessment head, which is trained on AVA data, to get those initial scores for every single frame without needing human frame-level annotations >. That transfers image knowledge to video assessment without that manual work >.

Jane: But it doesn't just look at the peak and the end; they introduce something called an aesthetic rhythm pattern encoding which models the temporal variation structure, capturing trends and fluctuations across the whole video >. This complements just having a peak and an end by preserving how the aesthetic experience unfolds over time >.

Lu: They combine these cues into an attention logit, which is like a weighted sum of the base score plus terms for peak, valley, and end responses, then they compute the final video representation as a weighted sum over all time steps >. This weighting function is w equals exponent of r t divided by the sum over j equals one to T exponent of r j >.

Meng: That sounds computationally heavy if you have a very long video sequence, because you’re doing that summation over every single time step in the final representation >. How does it handle scalability when we're dealing with long-form content?

Title and authors: Tom: The architecture is designed to be lightweight because it builds on a frozen vision transformer backbone, and the total number of trainable parameters they have is only one point zero five million, which is less than one percent of models like VADB-Net >. That keeps the inference cost down significantly >.

Jane: And for making sure it works well when we test it on different datasets, they use a dynamic gated fusion mechanism to combine the VAA prediction with an image-aesthetic term from their IAA head >. This lets them adaptively control how much influence each prediction has on the final overall score >.

Lu: The loss function for this part is formalized as LGF equals expectation of V and y in DVIDE-3K, which is the squared difference between the fused overall score and the ground truth VAA Overall Score, plus a binary cross-entropy term for tuning that gating weight lambda > <ref:2607.13941#pg1>. This shows they are explicitly training it to be robust across different data distributions >.

Meng: So, if I were deploying this, would I worry about it breaking down when the visual quality in the video changes drastically from one scene to the next >? That's a real deployment concern for me >.

Tom: The dynamic gated fusion is specifically designed to improve generalization under cross-dataset distribution shift, and experiments show it achieves larger gains in both SRCC and PLCC on DIVIDE-3K compared to just using one component alone > <ref:2607.13941#pg1>.

Jane: It seems like the core idea here is that explicitly modeling the temporal dynamics through peak-end rule aggregation helps provide a more reliable and consistent multi-dimensional aesthetic prediction across different types of video content, especially when visual quality fluctuates rapidly >.

Lu: That’s because you're not just looking at static frames; you’re capturing the rhythm of how the aesthetic experience unfolds, which I think is what makes it better than just a simple average score >.

Meng: What about the scalability if we want to adapt this to assess a different kind of aesthetic dimension, like motion quality instead of just visual appeal >? It seems tied to those specific heads they use >.

Title and authors: Tom: The paper suggests that the VAA head is designed such that it can be adapted by adjusting those prediction heads, which means you could potentially tune it for other aesthetic dimensions without starting from scratch on a new dataset >. That’s a scalable feature >.

Jane: So, to wrap up this part, Peak-End-Net achieves state-of-the-art performance in both the in-domain evaluation on VADB and the zero shot transfer testing on DIVIDE-3K, showing that these psychological principles actually work well when grounded in deep learning > <ref:2607.13941#pg1>.

Lu: It validates the effectiveness of explicitly textitasizing salient moments and the ending segment of a video, which is a really strong point for understanding human perception through this model >.

Meng: For deployment, it's efficient because it only has one point zero five million parameters, so I think that makes it viable for large-scale video aesthetic filtering or even as a reward model for video editing applications >.

Tom: It’s definitely not a massive leap in raw capability compared to some of the other models on the table, but its efficiency and interpretability make it very attractive for real systems where you need something fast and explainable >.

Jane: So, we’re seeing a framework that leverages psychological principles to build a video aesthetic assessment system that’s both accurate and adaptable across different data settings >.

Lu: It really highlights the importance of grounding models in established cognitive theories when dealing with subjective judgments like aesthetics >.

Meng: I just hope the dynamic fusion mechanism holds up under heavy, real-world load, because that’s where most of these novel methods struggle to stay stable >.

Tom: Well, it seems to be a solid direction for exploring how we can build more interpretable models for complex perceptual tasks like video aesthetics >.

Jane: That brings us to the conclusion of this talk on Peak-End-Net, which presents this psychologically grounded framework inspired by the peak-end rule that uses frame-wise aesthetic priors and dynamic gated fusion to achieve state of the art performance on both VADB and DIVIDE 3K > <ref:2607.13941#pg1,framework inspired by the peak-end rule>.

The paper's summary: Tom: So, we just talked about how they use that peak-end rule idea to judge video aesthetics by looking at the most important moments and the end of the video >

Jane: Right, and what this paper really shows is that they made it work in a way that’s actually useful for real videos >

Lu: Basically, they take those image aesthetic scores—the ones trained on something called AVA data—and use them to predict frame-by-frame scores for the video without needing anyone to manually label every single frame >

Meng: But wait, if you're just looking at individual frames, doesn't that ignore the whole flow of the video?

Tom: That’s where they introduce something called an aesthetic rhythm pattern encoding which models how those frame scores change over time >

Jane: It’s like they’re tracking the ups and downs, the trends and fluctuations in how aesthetically pleasing a part of the video is, which adds another layer to just looking at peaks and ends >

Lu: They combine all these cues—the peak-end stuff plus that rhythm encoding—into an attention logit, which is essentially a score for every time step in the video >

Tom: And they compute the final video representation as a weighted sum where each time step has a weight based on how much attention it gets from those cues >

Jane: That weighting function, w equals exponent of r t divided by the sum over j equals one to T exponent of r j, means later moments or more important moments get more influence on the final score >

Lu: It’s also important because they use a dynamic gated fusion mechanism to combine that video score with an image aesthetic term from their IAA head >

Meng: I remember you mentioning that fusion mechanism before, but what does it actually do for robustness across different datasets?

Tom: It adaptively integrates the image-aesthetic term into the final prediction, controlling how much influence each part has on the overall score using a parameter called lambda >

Jane: The authors are showing that this fusion is crucial for cross-dataset transferability, and they see big gains in metrics like SRCC and PLCC when they use it compared to just one of those components alone >

Lu: They formalized the training for this stage with a loss function called LGF, which involves the squared difference between the fused overall score and the true video aesthetic score in that DIVIDE-3K dataset > <ref:2607.13941#pg1>

Tom: The results on VADB, their in-domain test set, show that Peak-End-Net ranks first across all four metrics compared to other methods they tested >

Jane: And then they move to DIVIDE-3K, the cross-dataset test, and Peak-End-Net Stage two achieves the best performance on every single metric there > <ref:2607.13941#pg1>

Lu: The main point is that this peak-end rule approach gives us an explicit temporal bias that focuses on those salient moments and the ending segment of a video >

Tom: So what does this mean for the real world, beyond just getting a high score on a test set?

Jane: It means we’re moving toward models that capture the actual way humans perceive time in videos, which is more consistent than just looking at static features >

Lu: Plus, because it's built on a frozen vision transformer and only has about one million parameters, it’s super efficient for deployment >

Meng: Efficiency is key. If this runs on our production hardware for filtering millions of videos an hour without blowing the budget, that’s a win >

Tom: Exactly. And they also noted that the VAA head can be adapted to handle different aesthetic dimensions by just tweaking those prediction heads, which makes it more scalable than some of the big language model approaches we see now >

Jane: It seems like they’re really pushing for models that are not just accurate but also flexible enough to change what they're measuring, which is a pretty cool direction for perceptual understanding >

Lu: It gives us a solid foundation to explore how psychological principles can actually guide the training of these visual assessment models >

The paper's improvements: Tom: We just talked about how they used peak-end aggregation and rhythm encoding to model the temporal structure of aesthetic cues in videos >

Jane: Right, and now we’re talking about how they try to make this framework even better for different kinds of use cases >

Lu: The paper suggests a dynamic gated fusion mechanism to improve generalization when testing on different datasets, especially when there's a shift in the data distribution >

Meng: So you’re saying it can handle videos from one source and still give a decent score on another source without retraining everything from scratch >

Tom: That’s right. This module adaptively integrates an image-aesthetic term into the final overall score, controlled by a parameter called lambda >

Jane: It lets them decide how much weight to put on their video assessment versus that transferable image knowledge, which is smart for robustness >

Lu: They formalized this training stage with a loss function LGF, which measures the squared difference between the fused overall score and the true video aesthetic score in a new dataset >

Meng: How does that help with deployment? If we’re shipping this to users, what breaks at scale when you start switching between sources?

Tom: The training objective for this part is designed to keep that fusion stable, and experiments on DIVIDE-3K show it gives the biggest jumps in metrics like SRCC and PLCC > <ref:2607.13941#pg1>

Jane: It suggests that by combining the specific temporal cues with a dynamic gate, you get a much more reliable overall assessment across varied content >

Lu: They also pointed out that this framework is lightweight because it’s built on a frozen vision transformer and only has about one million parameters total >

Meng: one million parameters is manageable, I guess. But what about the flexibility to change what we're assessing?

Tom: The authors say you can adapt the model to different aesthetic dimensions simply by adjusting those prediction heads in the VAA head, not by starting a whole new model from zero >

Jane: That’s a big deal because it means this isn't just locked into judging visual appeal; you could tune it for motion quality or something else without retraining the whole system >

Lu: It gives us a way to explore psychologically grounded modeling principles that can be applied to any aesthetic dimension by adjusting the prediction heads >

Meng: If we could do that, it would make integrating this into our existing content pipeline much easier, because we wouldn't need a new model for every new aesthetic dimension >

Tom: It means you could use this for things like large-scale video filtering or even as a reward model for AI video generation and editing applications >

Jane: So the implication here is that this peak-end rule approach isn't just about getting a score; it’s about building a flexible tool that understands the actual experience of watching a video >

Lu: It really highlights how grounding models in cognitive theories can lead to systems that are both accurate and adaptable across different data settings >

Conclusion: Tom: So to wrap up, we’ve seen how Peak-End-Net uses psychological principles like the peak-end rule to build a video aesthetic assessment framework that works well on both real data and new ones >

Jane: Exactly, it’s about using those specific moments and the ending of a video to judge its overall quality >

Lu: The main thing is that they prove you can transfer image aesthetic knowledge to video assessment without needing any manual frame-level annotation >

Meng: And it’s efficient. If we’re thinking about deploying this, we’re talking about a model with only one point zero five million parameters, which is way lighter than some of the bigger models we see right now >

Tom: It keeps the inference cost low for us and makes it practical for things like large-scale video filtering >

Jane: It’s a solid method that captures the actual human perception of video flow, which is something I think matters for how we understand media consumption >

Lu: They also showed that this approach is very adaptable because you can change what you're assessing by adjusting the prediction heads in the VAA head without starting over >

Meng: That adaptability is important. It means we don’t have to build a whole new model every time we want to measure something different, which saves us time and resources >

Tom: I think that ability to tune it for different aesthetic dimensions is what makes this framework interesting from an engineering standpoint >

Jane: It really shows how grounding a model in human psychology can lead to systems that are both accurate and flexible across different uses >

Lu: Overall, Peak-End-Net presents a framework inspired by the peak-end rule that uses frame-wise aesthetic priors and dynamic gated fusion to achieve state of the art performance on both VADB and DIVIDE 3K > <ref:2607.13941#pg1,framework inspired by the peak-end rule>

Meng: So it’s a solid result for a framework that actually has some real engineering constraints built in >

Tom: It validates the idea that explicit temporal modeling, like peak-end aggregation, is useful when you want to capture how an experience unfolds over time >

Jane: We’ve seen how this can be used for filtering or as a reward model for video generation, which points toward really practical applications in content creation >

Lu: And I think the biggest thing here is that it gives us a direction to explore how these psychological principles can guide the training of these visual assessment models >

Alibaba Group · Beijing University of Posts and Telecommunications

cs.CV

Submitted: 2026-07-15

Updated: 2026-11-16

Comments: Accepted to ACM MM 2026, Code: https://github.com/AMAP-ML/Peak-End-Net

Code: https://github.com/AMAP-ML/Peak-End-Net

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 100/100

The gist: Video aesthetic assessment (VAA) aims to predict how aesthetically pleasing a video is, yet remains far less explored than other visual assessment tasks.

Key concepts

Peak-End Rule
This psychological principle suggests that people judge an experience based mainly on its most striking moments (peaks) and the ending. The paper applies this to video aesthetics, assuming a few key frames and the final impression disproportionately influence the overall aesthetic score of the entire video.
Frame Aesthetic Priors
The method uses an Image Aesthetic Assessment (IAA) head, pretrained on AVA data, to predict aesthetic scores for every frame. This allows image-level aesthetic knowledge to be transferred to video assessment without needing manual annotations for each individual frame.
Aesthetic Rhythm Pattern Encoding
This component models the temporal structure of aesthetic cues in a video by capturing trends, fluctuations, and progression over time. It complements peak-end aggregation by ensuring the model preserves the natural rhythm of how an aesthetic experience unfolds throughout the video.

Terminology

Summary

Video aesthetic assessment (VAA) aims to predict how aesthetically pleasing a video is, yet remains far less explored than other visual assessment tasks. p. 0 In this paper, we revisit VAA from a psychological perspective and propose Peak-End-Net, a lightweight and interpretable framework inspired by the peak-end rule. p. 0

Psychological Foundation for Temporal Aggregation

The authors draw inspiration from the psychologically grounded peak-end rule [13], which suggests that people tend to judge a temporal experience mainly according to its salient moments and the ending. This principle is especially plausible for video aesthetics, where a few striking moments and the final impression may disproportionately shape the overall score. p. 1 To make this idea feasible, they introduce frame-wise aesthetic priors by employing an IAA head pretrained on AVA [30] to infer frame-wise aesthetic scores. p. 1 This allows image-level aesthetic knowledge to be transferred to video-level assessment without requiring manual frame-level annotation. p. 1

Modeling Temporal Progression and Salience

Beyond isolated peaks and ends, the authors introduce an aesthetic rhythm pattern encoding which models the temporal variation structure of framewise aesthetic cues, including trends, fluctuations, and overall progression. This component complements peak-end aggregation by preserving the rhythm with which the aesthetic experience unfolds throughout the video. p. 1 The attention weight computation combines these cues into an attention logit r t = sigma(r base) + alpha · r peak t + beta · r valley t + gamma · r end t. p. 3 The final video-level representation is computed as a weighted sum fvideo = sum over t=1 to T of w t f t in R d, where w t = exp(r t) / sum over j=1 to T exp(r j). p. 3

Robustness Through Dynamic Fusion

To improve generalization under cross-dataset distribution shift, the authors refine the predicted overall score via a dynamic gated fusion mechanism. This module adaptively integrates an image-aesthetic term into the final overall-score prediction by computing yˆ fusion overall = lambda yˆ overall + (1 − lambda) yˆIAA, where lambda controls the relative contribution of the VAA prediction and the global IAA score. p. 4 The training objective for this stage is formalized as LGF = E(V,y) in DVIDE-3K (yˆ fusion overall − y i) squared + BCE(lambda, lambda∗). p. 4

Efficiency and Performance Validation

The framework is lightweight by design because it builds on a frozen ViT backbone [9, 33] and incorporates only a small number of learnable modules. The total number of trainable parameters in their method is only 1. p. 005M, accounting for less than 1% of the total parameter count compared to VADB-Net and Q-Align. p. 7 Extensive experiments on two existing VAA benchmarks demonstrate that their approach achieves state-of-the-art performance on both in-domain evaluation on VADB and crossdomain testing on DIVIDE-3K. p. 0

Key Results and Contributions

In the in-domain evaluation on VADB, Peak-End-Net achieves the best overall performance ranking first on the overall score under all four metrics. Furthermore, cross-dataset testing on DIVIDE-3K shows that Peak-End-Net (Stage 2) achieves the best performance on all four metrics, with 0. p. 54905 on RMSE, 0. p. 55427 on SRCC, 0. p. 55773 on PLCC, and 0. p. 53745 on KRCC. p. 5 The paper concludes that the peak-end rule-based aggregation introduces an explicit temporal inductive bias that emphasizes aesthetically salient moments and the ending segment of a video. p. 5 The key contribution lies in the underlying idea. p. 7

Conclusion

In this paper, Peak-End-Net presents a psychologically grounded framework for video aesthetic assessment inspired by the peak-end rule. By leveraging frame-wise aesthetic priors from a pretrained IAA head, their method transfers image-level aesthetic knowledge to video-level modeling without requiring frame-level video annotations. p. 7 Complemented by an aesthetic rhythm encoder that captures temporal progression and a dynamic gated fusion mechanism that adaptively integrates image-aesthetic priors for robust overall-score prediction, Peak-End-Net achieves state-of-the-art performance on both settings: in-domain evaluation on VADB and zero-shot transfer to DIVIDE-3K. p. 8 Extensive ablation studies further validate the contribution of each component and confirm the robustness of the proposed design. p. 8 The authors hope this work encourages future research to explore psychologically grounded modeling principles for perceptual video understanding. p. 8

References

[1] Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat et al. 2023. Gpt-4 technical report. p. 8 p. 8 arXiv preprint arXiv:2303. p. 8 p. 808774 (2023). p. 8 p. 8

[11] Kaiming He, Haoqi Fan, Yuxin Wu, Saining Xie, and Ross Girshick. p. 8 2020. Momentum contrast for unsupervised visual representation learning. p. 8 p. 8 In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. p. 8 p. 8 9729–9738 (2020). p. 8 p. 8

[13] Daniel Kahneman, Barbara L Fredrickson, Charles A Schreiber, and Donald A Redelmeier. p. 8 p. 8 1993. When more pain is preferred to less: Adding a better end. p. 8 p. 8 Psychological science 4, 6 (1993), 401–405

[30] Naila Murray, Luca Marchesotti, and Florent Perronnin. 2012. AVA: A large-scale database for aesthetic visual analysis. p. 8 p. 8 In 2012 IEEE conference on computer vision and pattern recognition. p. 8 p. 8 IEEE, 2408–2415

[33] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell et al. 2021. Learning transferable visual models from natural language supervision. p. 8 p. 8 In International conference on machine learning. p. 8 p. 8 p. 8 p. 8 PMLR (2021). p. 8 p. 8

[38] Michael Tschannen, Alexey Gritsenko, Xiao Wang, Muhammad Ferjad Naeem et al. p. 8 2025. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. p. 8 p. 8 arXiv preprint arXiv:2502. p. 8 p. 814786 (2025). p. 8 p. 8

[46] Haoning Wu, Zicheng Zhang, Weixia Zhang, Chaofeng Chen et al. p. 9 2023. Q-align: Teaching lmms for visual scoring via discrete text-defined levels. p. 9 p. 9 arXiv preprint arXiv:2312. p. 9 p. 917090 (2023). p. 9 p. 9

[5] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. p. 8 p. 8 2020. A simple framework for contrastive learning of visual representations. p. 8 p. 8 In International conference on machine learning. PmLR, 1597–1607

[8] Yubin Deng, Chen Change Loy, and Xiaoou Tang. 2017. Image aesthetic assessment: An experimental survey. p. 8 p. 8 IEEE Signal Processing Magazine 34, 4 (2017), 80–106

[32] Qianqian Qiao, DanDan Zheng, Yihang Bo et al. 2025. VADB: A Large-Scale Video Aesthetic Database with Professional and Multi-Dimensional Annotations. p. 8 p. 8 arXiv preprint arXiv:2510. p. 8 p. 825238 (2025). p. 8 p. 8

[49] Fengchuang Xing, Mingjie Li, Yuan-Gen Wang et al. p. 9 p. 9 2024. Clipvqa: Video quality assessment via clip. p. 9 p. 9 IEEE Transactions on Broadcasting 71, 1 (2024), 291–306

[53] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sigmoid loss for language image pre-training. p. 9 p. 9 In Proceedings of the IEEE/CVF international conference on computer vision. p. 8 p. 8 11975–11986 (2023). p. 9 p. 9

[40] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit et al. p. 8 2017. Attention is all you need. p. 8 p. 8 Advances in neural information processing systems 30 (2017). p. 8 p. 8

[9] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov et al. p. 8 p. 8 2020. An image is worth 16x16 words: Transformers for image recognition at scale. p. 8 p. 8 arXiv preprint arXiv:2010. p. 8 p. 811929 (2020). p. 8 p. 8

[34] Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. p. 8 p. 8 2012. Ucf101: A dataset of 101 human actions classes from videos in the wild. p. 8 p. 8 arXiv preprint arXiv:1212. p. 8 p. 80402 (2012). p. 8 p. 8

[36] Yunlong Tang, Jing Bi, Siting Xu, Luchuan Song et al. p. 8 2025. Video understanding with large language models: A survey. p. 8 p. 8 IEEE Transactions on Circuits and Systems for Video Technology (2025). p. 8 p. 8

[37] Bart Thomee, David A Shamma et al. p. 8 p. 8 2016. Yfcc100m: The new data in multimedia research. p. 8 p. 8 Commun. ACM 59, 2 (2016), 64–73

[48] Wulin Xie, Rui Dai, Ruidong Ding et al. 2026. Q-Hawkeye: Reliable Visual Policy Optimization for Image Quality Assessment. p. 9 p. 9 arXiv preprint arXiv:2601. p. 9 p. 922920 (2026). p. 9 p. 9

A Detailed Architectural Overview

The framework first extracts frame-level visual features using a frozen vision transformer (ViT) [9, 33]. These features are then fed into the Frame Aesthetic Perceiver (FAP), which uses an IAA head ΦIAA to predict a frame-wise aesthetic score s t = 1/K - 1 sum over k=1 to K k · p t,k. p. 4 The peak-end rule is applied by defining peak and valley responses using temporal differences ΔL+, ΔR+, ΔL-, and ΔR- for the sequence of sampled frame-level aesthetic scores s t. p. 3 The final video representation fvideo is computed as a weighted sum fvideo = sum over t=1 to T of w t f t in R d, where w t = exp(r t) / sum over j=1 to T exp(r j). p. 3

Ablation and Ablation Studies

Ablation studies confirm the effectiveness of the components; for instance, combining Peak-End + ΦIAA + ΦRE achieves the best performance among all variants on VADB. The sensitivity analysis shows that using only one temporal cue yields limited performance, while combining any two components generally leads to better results. p. 7 The dynamic gated fusion module is crucial for cross-dataset transferability, as it achieves the largest gains in SRCC and PLCC on DIVIDE-3K. p. 7

Discussion and Future Directions

The authors discuss that compared to LLM-based approaches, their method is more efficient and can be naturally scaled to arbitrary aesthetic dimensions by adjusting the prediction heads. The key contribution lies in the underlying idea of psychologically grounded temporal aggregation. p. 7 The model can also be used for large-scale video aesthetic filtering and as a reward model for RL-based video generation or editing. p. 7 The final prediction is refined via gated fusion to produce a more robust overall assessment. p. 4

Peak-End-Net achieves state-of-the-art performance on both settings: in-domain evaluation on VADB and zero-shot transfer to DIVIDE-3K. The proposed method validates the effectiveness of explicitly textitasizing salient moments and the ending segment of a video. p. 6 The zero-shot results demonstrate that imageaesthetic priors are valuable not only for frame-level guidance but also for improving the robustness of overall aesthetic assessment in cross-dataset evaluation. p. 6

[1] Josh Achiam, Steven Adler, Sandhini Agarwal et al. 2023. Gpt-4 technical report. arXiv preprint arXiv:2303.08774 (2023).

[11] Kaiming He, Haoqi Fan, Yuxin Wu et al. p. 8 2020. Momentum contrast for unsupervised visual representation learning. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 9729–9738 (2020).

[13] Daniel Kahneman, Barbara L Fredrickson, Charles A Schreiber, and Donald A Redelmeier. 1993. When more pain is preferred to less: Adding a better end. Psychological science 4, 6 (1993), 401–405

[30] Naila Murray, Luca Marchesotti, and Florent Perronnin. 2012. AVA: A large-scale database for aesthetic visual analysis. In 2012 IEEE conference on computer vision and pattern recognition. IEEE, 2408–2415

[33] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh et al. 2021. Learning transferable visual models from natural language supervision. In International conference on machine learning. PMLR (2021).

[38] Michael Tschannen, Alexey Gritsenko, Xiao Wang et al. p. 8 2025. Siglip 2: Multilingual vision-language encoders with improved semantic understanding, localization, and dense features. arXiv preprint arXiv:2502.14786 (2025).

[46] Haoning Wu, Zicheng Zhang, Weixia Zhang et al. p. 9 2023. Q-align: Teaching lmms for visual scoring via discrete text-defined levels. arXiv preprint arXiv:2312.17090 (2023).

[5] Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. A simple framework for contrastive learning of visual representations. In International conference on machine learning. PmLR, 1597–1607

[8] Yubin Deng, Chen Change Loy, and Xiaoou Tang. 2017. Image aesthetic assessment: An experimental survey. IEEE Signal Processing Magazine 34, 4 (2017), 80–106

[32] Qianqian Qiao, DanDan Zheng, Yihang Bo et al. 2025. VADB: A Large-Scale Video Aesthetic Database with Professional and Multi-Dimensional Annotations. arXiv preprint arXiv:2510.25238 (2025).

[49] Fengchuang Xing, Mingjie Li, Yuan-Gen Wang et al. 2024. Clipvqa: Video quality assessment via clip. IEEE Transactions on Broadcasting 71, 1 (2024), 291–306

[53] Xiaohua Zhai, Basil Mustafa, Alexander Kolesnikov, and Lucas Beyer. 2023. Sigmoid loss for language image pre-training. In Proceedings of the IEEE/CVF international conference on computer vision. 11975–11986 (2023).

[40] Ashish Vaswani, Noam Shazeer, Niki Parmar et al. p. 8 2017. Attention is all you need. Advances in neural information processing systems 30 (2017).

[9] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov et al. 2020. An image is worth 16x16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929 (2020).

[34] Khurram Soomro, Amir Roshan Zamir, and Mubarak Shah. 2012. Ucf101: A dataset of 101 human actions classes from videos in the wild. arXiv preprint arXiv:1212.0402 (2012).

[36] Yunlong Tang, Jing Bi, Siting Xu et al. p. 8 2025. Video understanding with large language models: A survey. IEEE Transactions on Circuits and Systems for Video Technology (2025).

[37] Bart Thomee, David A Shamma et al. 2016. Yfcc100m: The new data in multimedia research. Commun. ACM 59, 2 (2016), 64–73

[48] Wulin Xie, Rui Dai, Ruidong Ding et al. 2026. Q-Hawkeye: Reliable Visual Policy Optimization for Image Quality Assessment. arXiv preprint arXiv:2601.22920 (2026).

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Peak-End-Net can be used for large-scale video aesthetic filtering and as a reward model for RL-based video generation or editing because it is lightweight and parameter efficient, requiring only a small number of trainable parameters compared to models like VADB-Net and Q-Align.

  2. The system can achieve robust overall score prediction in cross-dataset evaluation by adaptively integrating the holistic judgment from the VAA branch with the transferable insight from the IAA branch through a dynamic gated fusion mechanism, as described by equation 12.

  3. The model can provide more reliable and consistent multi-dimensional aesthetic predictions across various video content, especially when visual quality changes significantly across time, by explicitly modeling temporal aesthetic dynamics through peak-end rule aggregation and rhythm pattern encoding.

  4. The framework can be adapted to arbitrary aesthetic dimensions by adjusting the prediction heads in the VAA head, offering a more scalable solution compared to LLM-based approaches which are less suitable for multi-dimensional scoring due to higher inference costs.

Sources

Related papers