MindEye2: Shared-Subject Models Enable fMRI-To-Image With 1 Hour of Data
cs.CV, cs.AI, q-bio.NC
Submitted: 2024-03-17
Updated: 2026-11-01
Code: https://github.com/huggingface/accelerate
Terminology
Sources
- Learning Transferable Visual Models From Natural Language Supervision
- High-Resolution Image Synthesis with Latent Diffusion Models
- Reconstruction of Perceived Images from fMRI Patterns and Semantic Brain Exploration using Instance-Conditioned GANs
- Natural scene reconstruction from fMRI signals using generative latent diffusion
- Decoding natural image stimuli from fMRI data with a surface-based convolutional network
- Second Sight: Using brain-optimized encoding models to align image distributions with human brain activity
- Brain-optimized inference improves reconstructions of fMRI brain activity
- Through their eyes: multi-subject Brain Decoding with simple alignment techniques
- Aligning brain functions boosts the decoding of visual semantics in novel subjects
- Cinematic Mindscapes: High-quality Video Reconstruction from Brain Activity
- Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding
- Contrast, Attend and Diffuse to Decode High-Resolution Images from Brain Activities
- UniBrain: Unify Image Reconstruction and Captioning All in One Diffusion Model from Human Brain Activity
- DREAM: Visual Decoding from Reversing Human Visual System
- From voxels to pixels and back: Self-supervision in natural-image reconstruction from fMRI
- DCNN-GAN: Reconstructing Realistic Image from fMRI
- Hierarchical Text-Conditional Image Generation with CLIP Latents
- LAION-5B: An open large-scale dataset for training next generation image-text models
- ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
- LiT: Zero-Shot Transfer with Locked-image text Tuning
Related papers
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs
- From Formulas to Protocols: A Compositional View of Binary Target Segmentation Metrics