PixelSpline: Continuous Space-Time Video Super-Resolution via Endpoint-Anchored Shared Pixel Trajectories

arXiv:2611.06870 · cs.CV · Submitted 2026-11-04 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: We're looking at the paper: "PixelSpline: Continuous Space-Time Video Super-Resolution via Endpoint-Anchored Shared Pixel Trajectories".

Jane: Here's the gist of the paper. Continuous space-time video super-resolution (STVSR) methods are introduced that combine long-context encoding, continuous HR feature lifting, and an endpoint-anchored PixelSpline to reconstruct videos with both higher spatial resolution and higher frame rate This framework explicitly models each pixel’s motion as a single continuous trajectory shared by both endpoints Pruning redundant reasoning tokens Existing continuous space-time video super-resolution methods often infer motion from only two endpoint frames and model the two motion directions independently, making them vulnerable to ambiguous correspondence and inconsistent trajectories Long-range observations provide more reliable endpoint representations, while continuous upsampling constructs detailed appearance features directly on the requested HR grid PixelSpline explicitly models each pixel’s motion as a single continuous trajectory shared by both endpoints Using this trajectory, our method warps endpoint colors into a coarse intermediate frame and aligns the HR features for residual reconstruction How the training signal is built The framework is trained using four main objectives First, the final reconstruction is supervised by a Charbonnier loss where the loss is defined as L img = rho (widehat I t H - I t H) Second, when several target times are sampled from the same interval, we match their first-order temporal changes using L temp = rho ((widehat I(tj+1) H - widehat I tj H) -(I(tj+1) H - I tj H)) Third, we directly supervise the motions predicted by PixelSpline using pseudo-ground-truth optical flows generated by a frozen SEA-RAFT model, defined as L flow = (1/2) sum i in 0,1 rho ((M(i->t) - M(i->t) star) / (W s,H s)) Finally, the coarse warped frame is encouraged to match a blurry target obtained by downsampling and then upsampling the HR ground truth via L warp = rho (widetilde I t H - I t blur) Where the gains come from The long-context encoder aggregates information across multiple LR observations to produce target-pair-aware endpoint representations, improving correspondence estimation in ambiguous regions This encoder uses a multi-frame sequence encoding where features are propagated through a bidirectional deformable ConvLSTM The target-pair dual-scope attention module refines the query triplet using local, global, and pooled spatial scopes to produce context-enhanced anchor features The continuous feature upsampling module follows continuous upsampling filters (CUF) and the hyper-upsampling design used in STAVSR

22: to map LR anchor features onto an HR grid Pruning redundant reasoning tokens The core innovation is the endpoint-anchored PixelSpline which constructs one explicit trajectory for every matched point from the first anchor to the second This shared curve ensures that corresponding pixels transported from the two anchors must meet at the same location in every intermediate frame, ensuring consistent motion trajectories between the two anchors The shared trajectory is used for two forward-warping operations: first, bicubically upsample anchor RGB frames and forward-warp them with M0→t and M1→t to yield a coarse frame IeH t Second, apply the same two motions to the lifted features F H 0 and F H 1 to produce an aligned HR feature F H t via SoftSplatting Where the gains come from The proposed method demonstrates a favorable quality–efficiency trade-off, running 2.24× faster than the V3 baseline with only a 0.90 dB lower PSNR on Adobe240 Average Qualitative comparisons show that our method better preserves fine textures and high-frequency structures across both GoPro and REDS datasets Furthermore, the method achieves the lowest tOF on REDS, reducing the error from 8.923 for V3 to 8.171 The decoupled reconstruction settings also show competitive results when spatial and temporal reconstruction are evaluated separately Pruning redundant reasoning tokens The main contributions of the work are summarized as follows: • We introduce long-context endpoint encoding that uses observations beyond the queried pair to obtain more stable and globally informed endpoint representations • We propose an endpoint-anchored shared PixelSpline that explicitly models per-pixel motion as one continuous trajectory evaluated from both endpoints • We develop a CUF-based continuous feature lifting module that adapts continuous upsampling filters to map context-enhanced LR endpoint features onto arbitrary HR grids before trajectory-guided temporal transport Where the gains come from The model's performance is verified across full C-STVSR task, spatial-only reconstruction, and temporal-only reconstruction settings The ablation study confirms that both endpoint-consistent control sharing and long-range temporal context contribute significantly to reconstruction quality The final results show that our method approaches the reconstruction quality of V3 while requiring less than half of its end-to-end inference time Pruning redundant reasoning tokens We presented a continuous space-time video superresolution framework that combines long-context encoding, continuous HR feature upsampling, and an endpoint-anchored PixelSpline to reconstruct videos with both higher spatial resolution and higher frame rate By constructing a single correspondence-consistent trajectory shared by both endpoints, our method aligns pixel and feature transport while exploiting observations beyond the queried frame pair Experiments across joint, spatial-only, and temporal-only super-resolution settings demonstrate competitive reconstruction quality, improved temporal consistency, and a favorable quality–efficiency trade-off Despite these results, the reconstruction performance in terms of PSNR and SSIM can be further improved compared to the quality-leading method in several settings Moreover, although our method offers practical inference efficiency, its training process remains computationally demanding and time-consuming Improving reconstruction accuracy while developing more efficient training strategies constitutes a promising direction for future work Where the gains come from The paper was supported in part by the U.S.

Tom: National Science Foundation (NSF) under Awards IIS-2232298, IIS2442792, and OIA-2242812 The reconstruction quality is evaluated using PSNR and SSIM metrics across various datasets including Adobe 133 high-frame-rate videos captured at 100 fps with a spatial resolution of 1280 × 720 The results on REDS show the ability to faithfully recover fine stripe patterns on the tiger and preserve the folded structure of the pedestrian’s shirt collar with smooth, well-defined boundaries The temporal optical-flow consistency (tOF) measures how close the reconstructed motion is to the reference, indicating less effect from temporal artifacts such as flicker or irregular motion The end-to-end latency for our method is 406.41 ms per output frame, corresponding to a 2.24× speedup compared to V3 The paper concludes that the continuous formulation supports both boundary cases without architectural changes, showing competitiveness when spatial and temporal reconstruction are evaluated separately The work is supported in part by the U.S.

Jane: National Science Foundation (NSF) under Awards IIS-2232298, IIS2442792, and OIA-2242812 Pruning redundant reasoning tokens The references provided are:

1: Wenbo Bao, Wei-Sheng Lai, Chao Ma, Xiaoyun Zhang, Zhiyong Gao, and Ming-Hsuan Yang.

Tom: Depth-aware video frame interpolation.

Paper summary: Tom: So looking at PixelSpline: Continuous Space-Time Video Super-Resolution via Endpoint-Anchored Shared Pixel Trajectories, the title itself is pretty descriptive about what they've done, right?

Jane: It’s about combining continuous space time video super resolution with a specific mechanism for anchoring those points >

Lu: The authors are essentially showing how to build a method that uses long context and continuous feature lifting alongside this shared trajectory idea >

Meng: So what does this mean for the people who actually use these models? Is it something practical for deploying on a server?

Lalam: It means you can get high spatial resolution and higher frame rate while maintaining better temporal consistency, which is usually a pain in video stuff >

Tom: I think the main thing to grasp is that they solved the problem of inconsistent motion trajectories between endpoints by forcing them onto one continuous curve >

Jane: That shared curve ensures that everything transported between those two points stays geometrically consistent across time, which reduces ghosting and jumps >

Lu: The authors are bringing together ideas from different areas, like three dee Fourier fields and pairwise trajectory methods, into one cohesive structure > <ref:2611.06870#pg1>

Meng: It’s a complex training process though. They mentioned that while the quality is good, the training itself remains computationally demanding and time consuming >

Tom: That's fair. I think it's a trade off: you get better results with more effort in training, but the inference speed gain is still quite substantial compared to older models like V3 >

Jane: So ultimately, PixelSpline shows that by being explicit about the geometry of motion using that shared curve, you can achieve better spatial and temporal reconstruction without needing completely different architectures >

Conclusion: Tom: So we've been diving into PixelSpline, this paper that’s all about super resolution for space-time video using these continuous trajectories to link endpoints together across time and space, right?

Jane: It sounds a bit complicated with all those names but the core idea is simple: they found a way to make sure every single pixel moves consistently between two frames, no matter how far apart they are in time or space.

Lu: Exactly. They’re using this endpoint-anchored shared trajectory to warp colors and align features, which means they get both higher picture quality and a faster frame rate simultaneously.

Meng: From my side at the startup, the key I want to know is how much extra training time this continuous modeling adds compared to just using standard interpolation methods.

Tom: Well, the authors show that while the training itself is more complex because they're modeling these continuous paths explicitly, the resulting model is running about two point two four times faster than a baseline called V3, which is a solid speed gain for deployment.

Jane: That speedup comes from this "long-context encoding" and how they build those continuous upsampling filters to map features onto the final high resolution grid.

Lu: The main contribution, the big idea here, is that by using this single shared curve for every matched point, you get a much more reliable motion prediction than just looking at two frames independently.

Tom: It’s about moving beyond independent motion modeling to a globally consistent trajectory across the whole sequence.

Meng: But if they are relying on long-range observations and these complex representations, does that mean this method breaks down if you have really weird, unpredictable motion in the video?

Jane: The paper tests it across different settings—joint space and temporal reconstruction—and shows it still keeps a decent quality, even when things get messy.

Lu: They showed that both using the endpoint-consistent control sharing and having that long-range temporal context really help boost the final reconstruction quality, even if they aren't perfect.

Tom: The authors admit that while this method is fast and good, achieving the absolute highest PSNR or SSIM scores in every single test setting still requires more tuning than what they achieved.

Jane: So it’s a very strong contender for practical applications where you need a balance between quality, speed, and temporal smoothness.

Lu: It really pushes the idea that explicitly modeling the continuous space-time path is a better way to handle video dynamics than relying on simpler frame-to-frame assumptions.

Tom: We’ve seen how they manage the training signal with four different objectives, from matching final images to supervising motions with optical flow, which shows a really thorough approach.

Jane: And that's what we're looking at now—how this trajectory idea shifts the way we think about reconstructing video motion in high-resolution environments.

Wanqi Yuan, Nianyi Li

Clemson University

cs.CV

Submitted: 2026-11-04

Updated: 2026-11-04

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 78/100

The gist: Continuous space-time video super-resolution (STVSR) methods are introduced that combine long-context encoding, continuous HR feature lifting…

Key concepts

Long-Context Encoding
This technique uses observations beyond the two queried frames to create more stable and globally informed representations of the video. It employs a multi-frame sequence encoding where features are propagated through a bidirectional deformable ConvLSTM, helping to better estimate correspondences in ambiguous areas.
Endpoint-Anchored PixelSpline
This is the core innovation that explicitly models each pixel's motion as one continuous trajectory connecting the two video endpoints. This shared curve ensures that corresponding pixels from both anchors meet at the same location in every intermediate frame, guaranteeing consistent motion trajectories.
Continuous Feature Lifting
This module uses continuous upsampling filters (CUF) and hyper-upsampling designs to map low-resolution anchor features onto a high-resolution grid. It adapts these filters to effectively lift the LR features into the HR space before the trajectory-guided temporal transport occurs.

Terminology

Summary

Continuous space-time video super-resolution (STVSR) methods are introduced that combine long-context encoding, continuous HR feature lifting, and an endpoint-anchored PixelSpline to reconstruct videos with both higher spatial resolution and higher frame rate This framework explicitly models each pixel’s motion as a single continuous trajectory shared by both endpoints

Pruning redundant reasoning tokens

Existing continuous space-time video super-resolution methods often infer motion from only two endpoint frames and model the two motion directions independently, making them vulnerable to ambiguous correspondence and inconsistent trajectories Long-range observations provide more reliable endpoint representations, while continuous upsampling constructs detailed appearance features directly on the requested HR grid PixelSpline explicitly models each pixel’s motion as a single continuous trajectory shared by both endpoints Using this trajectory, our method warps endpoint colors into a coarse intermediate frame and aligns the HR features for residual reconstruction

How the training signal is built

The framework is trained using four main objectives First, the final reconstruction is supervised by a Charbonnier loss where the loss is defined as L img = rho (widehat I t H - I t H) Second, when several target times are sampled from the same interval, we match their first-order temporal changes using L temp = rho ((widehat I(tj+1) H - widehat I tj H) -(I(tj+1) H - I tj H)) Third, we directly supervise the motions predicted by PixelSpline using pseudo-ground-truth optical flows generated by a frozen SEA-RAFT model, defined as L flow = (1/2) sum i in 0,1 rho ((M(i->t) - M(i->t) star) / (W s,H s)) Finally, the coarse warped frame is encouraged to match a blurry target obtained by downsampling and then upsampling the HR ground truth via L warp = rho (widetilde I t H - I t blur)

Where the gains come from

The long-context encoder aggregates information across multiple LR observations to produce target-pair-aware endpoint representations, improving correspondence estimation in ambiguous regions This encoder uses a multi-frame sequence encoding where features are propagated through a bidirectional deformable ConvLSTM The target-pair dual-scope attention module refines the query triplet using local, global, and pooled spatial scopes to produce context-enhanced anchor features The continuous feature upsampling module follows continuous upsampling filters (CUF) and the hyper-upsampling design used in STAVSR [22] to map LR anchor features onto an HR grid

The core innovation is the endpoint-anchored PixelSpline which constructs one explicit trajectory for every matched point from the first anchor to the second This shared curve ensures that corresponding pixels transported from the two anchors must meet at the same location in every intermediate frame, ensuring consistent motion trajectories between the two anchors The shared trajectory is used for two forward-warping operations: first, bicubically upsample anchor RGB frames and forward-warp them with M0→t and M1→t to yield a coarse frame IeH t Second, apply the same two motions to the lifted features F H 0 and F H 1 to produce an aligned HR feature F H t via SoftSplatting

The proposed method demonstrates a favorable quality–efficiency trade-off, running 2.24× faster than the V3 baseline with only a 0. p. 190 dB lower PSNR on Adobe240 Average Qualitative comparisons show that our method better preserves fine textures and high-frequency structures across both GoPro and REDS datasets Furthermore, the method achieves the lowest tOF on REDS, reducing the error from 8. p. 5923 for V3 to 8. p. 7171 The decoupled reconstruction settings also show competitive results when spatial and temporal reconstruction are evaluated separately

The main contributions of the work are summarized as follows:

• We introduce long-context endpoint encoding that uses observations beyond the queried pair to obtain more stable and globally informed endpoint representations

• We propose an endpoint-anchored shared PixelSpline that explicitly models per-pixel motion as one continuous trajectory evaluated from both endpoints

• We develop a CUF-based continuous feature lifting module that adapts continuous upsampling filters to map context-enhanced LR endpoint features onto arbitrary HR grids before trajectory-guided temporal transport

The model's performance is verified across full C-STVSR task, spatial-only reconstruction, and temporal-only reconstruction settings The ablation study confirms that both endpoint-consistent control sharing and long-range temporal context contribute significantly to reconstruction quality The final results show that our method approaches the reconstruction quality of V3 while requiring less than half of its end-to-end inference time

We presented a continuous space-time video superresolution framework that combines long-context encoding, continuous HR feature upsampling, and an endpoint-anchored PixelSpline to reconstruct videos with both higher spatial resolution and higher frame rate By constructing a single correspondence-consistent trajectory shared by both endpoints, our method aligns pixel and feature transport while exploiting observations beyond the queried frame pair Experiments across joint, spatial-only, and temporal-only super-resolution settings demonstrate competitive reconstruction quality, improved temporal consistency, and a favorable quality–efficiency trade-off Despite these results, the reconstruction performance in terms of PSNR and SSIM can be further improved compared to the quality-leading method in several settings Moreover, although our method offers practical inference efficiency, its training process remains computationally demanding and time-consuming Improving reconstruction accuracy while developing more efficient training strategies constitutes a promising direction for future work

The paper was supported in part by the U.S. National Science Foundation (NSF) under Awards IIS-2232298, IIS2442792, and OIA-2242812 The reconstruction quality is evaluated using PSNR and SSIM metrics across various datasets including Adobe 133 high-frame-rate videos captured at 100 fps with a spatial resolution of 1280 × 720 The results on REDS show the ability to faithfully recover fine stripe patterns on the tiger and preserve the folded structure of the pedestrian’s shirt collar with smooth, well-defined boundaries The temporal optical-flow consistency (tOF) measures how close the reconstructed motion is to the reference, indicating less effect from temporal artifacts such as flicker or irregular motion The end-to-end latency for our method is 406. p. 641 ms per output frame, corresponding to a 2. p. 524× speedup compared to V3 The paper concludes that the continuous formulation supports both boundary cases without architectural changes, showing competitiveness when spatial and temporal reconstruction are evaluated separately The work is supported in part by the U. p. 7S. National Science Foundation (NSF) under Awards IIS-2232298, IIS2442792, and OIA-2242812

The references provided are: [1] Wenbo Bao, Wei-Sheng Lai, Chao Ma, Xiaoyun Zhang, Zhiyong Gao, and Ming-Hsuan Yang. Depth-aware video frame interpolation. p. 8 In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3698–3707. p. 8 IEEE, 2019 [2] Alexander Becker, Rodrigo Caye Daudt, Dominik Narnhofer, Torben Peters, Nando Metzger, Jan Dirk Wegner, and Konrad Schindler. p. 8 Thera: Aliasing-free arbitrary-scale super-resolution with neural heat fields. p. 8 arXiv preprint arXiv:2311. p. 817643, 2023 [3] Alexander Becker, Julius Erbach, Dominik Narnhofer, and Konrad Schindler. p. 8 Continuous space-time video super-resolution with 3d fourier fields. p. 8 arXiv preprint arXiv:2509. p. 826325, 2025 [4] Yinbo Chen, Sifei Liu, and Xiaolong Wang. p. 8 Learning continuous image representation with local implicit image function. p. 8 In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 8628–8638, 2021 [5] Yi-Hsin Chen, Si-Cun Chen, Yen-Yu Lin, and Wen-Hsiao Peng. p. 8 Motif: Learning motion trajectories with local implicit neural functions for continuous space-time video superresolution. p. 8 In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 23074–23084. p. 8 IEEE, 2023 [6] Zeyuan Chen, Yinbo Chen, Jingwen Liu, Xingqian Xu, Vidit Goel, Zhangyang Wang, Humphrey Shi, and Xiaolong Wang. p. 8 Videoinr: Learning video implicit neural representation for continuous space-time super-resolution. p. 8 In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2037–2047. p. 8 IEEE, 2022 [7] Hee Min Choi, Hyoa Kang, Suji Kim, Dokwan Oh, and Nam Ik Cho. p. 8 Time without time: Pseudo-temporal representation for space-time super-resolution. p. 8 In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 6812–6822, 2026 [8] Zhicheng Geng, Luming Liang, Tianyu Ding, and Ilya Zharkov. p. 8 Rstt: Real-time spatial temporal transformer for space-time video super-resolution. p. 8 In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 17420–17430. p. 8 IEEE, 2022 [9] Muhammad Haris, Greg Shakhnarovich, and Norimichi Ukita. p. 8 Space-time-aware multi-resolution video enhancement. p. 8 In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 2856–2865. p. 8 IEEE, 2020 [10] Mengshun Hu, Kui Jiang, Zhixiang Nie, and Zheng Wang. p. 8 You only align once: Bidirectional interaction for spatialtemporal video super-resolution. p. 8 In Proceedings of the 30th ACM International Conference on Multimedia, pages 847–855, 2022 [11] Mengshun Hu, Kui Jiang, Zhixiang Nie, Jiahuan Zhou, and Zheng Wang. p. 8 Store and fetch immediately: Everything is all you need for space-time video super-resolution. p. 8 In Proceedings of the AAAI Conference on Artificial Intelligence, pages 863–871, 2023 [12] Xuecai Hu, Haoyuan Mu, Xiangyu Zhang, Zilei Wang, Tieniu Tan, and Jian Sun. p. 8 Meta-sr: A magnification-arbitrary network for super-resolution. p. 8 In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1575–1584. p. 8 IEEE, 2019 [13] Huaizu Jiang, Deqing Sun, Varun Jampani, Ming-Hsuan Yang, Erik Learned-Miller, and Jan Kautz. p. 8 Super slomo: High quality estimation of multiple intermediate frames for video interpolation. p. 8 In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 9000–9008, 2018 [14] Eunjin Kim, Hyeonjin Kim, Kyong Hwan Jin, and Jaejun Yoo. p. 8 Bf-stvsr: B-splines and fourier—best friends for high fidelity spatial-temporal video super-resolution. p. 8 In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 28009–28018 [15] Tae Hyun Kim, Seungjun Nah, and Kyoung Mu Lee. p. 8 Deep multi-scale convolutional neural network for dynamic scene deblurring. p. 8 In Conference on Computer Vision and Pattern Recognition, pages 1–21. p. 8 IEEE, 2017 [16] Jaewon Lee and Kyong Hwan Jin. p. 8 Local texture estimator for implicit representation function. p. 8 In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 1919–1928. p. 8 IEEE, 2022 [17] Zekun Li, Hongying Liu, Fanhua Shang, Yuanyuan Liu, Liang Wan, and Wei Feng. p. 8 Savsr: Arbitrary-scale video super-resolution via a learned scale-adaptive network. p. 8 In Proceedings of the AAAI Conference on Artificial Intelligence, pages 3288–3296 [18] Ce Liu and Deqing Sun. p. 8 A bayesian approach to adaptive video super resolution. p. 8 In CVPR 2011, pages 209–216. p. 8 IEEE, 2011 [19] Chunxu Liu, Guozhen Zhang, Rui Zhao, and Limin Wang. p. 8 Sparse global matching for video frame interpolation with large motion. p. 8 In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 19125–19134 [20] Seungjun Nah, Sungyong Baik, Seokil Hong, Gyeongsik Moon, Sanghyun Son, Radu Timofte, and Kyoung Mu Lee. p. 8 Ntire 2019 challenge on video deblurring and superresolution: Dataset and study. p. 8 In CVPR Workshops, 2019 [23] Vincent Sitzmann, Julien Martel, Alexander Bergman, David Lindell, and Gordon Wetzstein. p. 8 Implicit neural representations with periodic activation functions. p. 8 Advances in neural information processing systems, 33:7462–7473, 2020 [24] Shuochen Su, Mauricio Delbracio, Jue Wang, Guillermo Sapiro, Wolfgang Heidrich, and Oliver Wang. p. 9 Deep video deblurring for hand-held cameras. p. 9 In 2017 IEEE Conference on computer vision and pattern recognition (CVPR), pages 237–246. p. 9 IEEE, 2017 [25] Xintao Wang, Kelvin CK Chan, Ke Yu, Chao Dong, and Chen Change Loy. p. 9 Edvr: Video restoration with enhanced deformable convolutional networks. p. 9 In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1954–1963 [26] Yihan Wang, Lahav Lipson, and Jia Deng. p. 9 Sea-raft: Simple, efficient, accurate raft for optical flow. p. 9 In European Conference on Computer Vision, pages 36–54. p. 8 Springer, 2024 [27] Xiaoyu Xiang, Yapeng Tian, Yulun Zhang, Yun Fu, Jan P Allebach, and Chenliang Xu. p. 9 Zooming slow-mo: Fast and accurate one-stage space-time video super-resolution. p. 9 In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3367–3376 [28] Gang Xu, Jun Xu, Zhen Li, Liang Wang, Xing Sun, and Ming-Ming Cheng. p. 9 Temporal modulation network for controllable space-time video super-resolution. p. 9 arXiv preprint arXiv:2104. p. 910642, 2021 [29] Haofei Xu, Jing Zhang, Jianfei Cai, Hamid Rezatofighi, and Dacheng Tao. p. 9 Gmflow: Learning optical flow via global matching. p. 9 In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 8111–8120 [30] Yulun Zhang, Kunpeng Li, Kai Li, Lichen Wang, Bineng Zhong, and Yun Fu. p. 9 Image super-resolution using very deep residual channel attention networks. p. 9 In European conference on computer vision, pages 294–310. p. 9 Springer, 2018 [31] PixelSpline: Continuous Space-Time Video Super-Resolution via Endpoint-Anchored Shared Pixel Trajectories Wanqi Yuan Nianyi Li Clemson University Clemson, South Carolina, United States wanqiy@clemson. p. 0edu nianyil@clemson. p. 0 p. 0edu https://wanqiyuan. p. 0github.io/PixelSpline/ [32] Table 1 Quantitative comparison of C-STVSR reconstruction quality under S×4 spatial and T×8 temporal upsampling, reported as PSNR ↑ / SSIM ↑. p. 5 Best and second-best results are highlighted in bold and underlined, GoPro Center Adobe Center REDS Center REDS Average Adobe Average REDS Average [3] [31] Table 2 End-to-end latency and throughput under the timing protocol described in the caption. p. 5 As shown in Fig. p. 1 1, our method requires 406. p. 541 ms per output frame, compared to 912. p. 521 ms for V3 [3] [31] Table 3 Temporal consistency on REDS under S×4 spatial and T×8 temporal upsampling, measured by tOF ↓. p. 6 Method TMNet VideoINR MoTIF BF-STVSR V3 Ours tOF ↓ 9. p. 6944 9.495 9.391 9.237 8.923 8.171 [3] [31] Figure 4 Qualitative comparison of temporal consistency of C-STVSR methods under S×4 spatial and T×8 temporal upsampling

--- Page 1 ---

PixelSpline: Continuous Space-Time Video Super-Resolution via Endpoint-Anchored Shared Pixel Trajectories Wanqi Yuan Nianyi Li Clemson University Clemson, South Carolina, United States wanqiy@clemson.edu nianyil@clemson.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Long-context endpoint encoding allows for more stable and globally informed endpoint representations by using observations beyond a queried pair, which improves correspondence estimation in ambiguous regions.

  2. An endpoint-anchored shared PixelSpline explicitly models each pixel’s motion as a single continuous trajectory shared by both endpoints, ensuring that corresponding pixels transported from the two anchors must meet at the same location in every intermediate frame, ensuring consistent motion trajectories between the two anchors.

  3. A CUF-based continuous feature lifting module maps LR anchor features to HR grids using filters dependent only on query geometry and scale, allowing these same filters can be shared by both anchors when they are lifted to the same target grid.

  4. The training objective includes a flow loss that directly supervise[s] the motions predicted by PixelSpline using pseudo-ground-truth optical flows generated by a frozen SEA-RAFT model, ensuring that the explicit pixel-transport path is consistent with estimated motion.

Sources

Related papers