Structured Visual Landscape: Generating Preferred Representations in Biological and Artificial Neural Networks
cs.CV, cs.LG, cs.NE
Submitted: 2026-11-02
Updated: 2026-11-06
License: http://creativecommons.org/licenses/by/4.0/
The gist: Understanding how neurons respond to visual stimulus is an important question in both deep learning and neuroscience.
Terminology
Abstract
Understanding how neurons respond to visual stimulus is an important question in both deep learning and neuroscience. It has significant implications for enhancing the interpretability of black-box artificial neural networks and for understanding the visual representation in the human brain. We propose a structured visual representations landscape and design an activation-based prior that allows us to effectively regularize the landscape with activations from brain regions or neural network units. Our model Vis-Lens integrates a variational autoencoder and a diffusion model into an image generation framework. It enables the generation of natural, realistic preferred images by directly modifying the activation-regularized latent codes without per-image iterative optimization. We demonstrate the effectiveness of our framework on both artificial neural networks and human visual cortex using functional magnetic resonance imaging (fMRI) data. Our framework improves activation modulation and image realism compared with a recent state-of-the-art method for generating preferred visual representations of these neural targets. Code is available at https://github.com/NeuroAIHub/Vis-Lens.
Sources
- Wasserstein GAN
- Crafting Interpretable Embeddings by Asking LLMs Questions
- Seeing Beyond the Brain: Conditional Diffusion Model with Sparse Masked Modeling for Vision Decoding
- The Pictorial Cortex: Zero-Shot Cross-Subject fMRI-to-Image Reconstruction via Compositional Latent Modeling
- Auto-Encoding Variational Bayes
- BrainFLORA: Uncovering Brain Concept Representation via Multimodal Neural Embeddings
- Visual Decoding and Reconstruction via EEG Embeddings with Guided Diffusion
- Microsoft COCO: Common Objects in Context
- BrainCLIP: Bridging Brain and Visual-Linguistic Representation Via CLIP for Generic Natural Visual Stimulus Decoding
- BrainSCUBA: Fine-Grained Natural Language Captions of Visual Cortex Selectivity
- Brain Diffusion for Visual Exploration: Cortical Discovery using Large Scale Generative Models
- SynBrain: Enhancing Visual-to-fMRI Synthesis via Probabilistic Representation Learning
- SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations
- Conditional Generative Adversarial Nets
- Spectral Normalization for Generative Adversarial Networks
- GLIDE: Towards Photorealistic Image Generation and Editing with Text-Guided Diffusion Models
- DINOv2: Learning Robust Visual Features without Supervision
- Natural scene reconstruction from fMRI signals using generative latent diffusion
- MindLLM: A Subject-Agnostic and Versatile Model for fMRI-to-Text Decoding
- Learning Transferable Visual Models From Natural Language Supervision
Related papers
- TEA: Text Encoder Alignment for Robust Concept Erasure in Text-to-Image Models
- Do Egocentric Video-Language Models Capture Both Hand- and Object-Centric Cues?
- Aura: Consistent Multi-Subject Video Generation via VLM-Grounded Semantic Alignment
- PVRF: All-in-one Adverse Weather Removal via Prior-modulated and Velocity-constrained Rectified Flow
- DLWM: Diverse Latent World Models for Efficient Multimodal Reasoning
- Out of Sight, Still in Mind: Token Compression for Omni-LLMs