A Multimodal Sequence-to-Sequence Model for Cross-Subject Prediction of Brain Responses to Naturalistic Stimuli

Name: A Multimodal Sequence-to-Sequence Model for Cross-Subject Prediction of Brain Responses to Naturalistic Stimuli
Uploaded: 2026-10-10T15:24:36.477256+00:00
Duration: 347 s
Description: This research proposes a sequence-to-sequence Transformer to predict whole-brain fMRI responses to naturalistic movies by autoregressively processing visual, auditory, and language inputs. It uses specialized models like VideoMAE and HuBERT for feature extraction and integrates narrative summaries to capture complex temporal dependencies in brain activity.

This research proposes a sequence-to-sequence Transformer to predict whole-brain fMRI responses to naturalistic movies by autoregressively processing visual, auditory, and language inputs. It uses specialized models like VideoMAE and HuBERT for feature extraction and integrates narrative summaries to capture complex temporal dependencies in brain activity.

Open the full episode