A probabilistic model of visual segmentation explains early visual cortical dynamics
summary
The gist
A probabilistic model of visual segmentation explains early visual cortical dynamics by coupling feature inference with segment inference to induce neural dynamics The model predicts both gain and…
In short
The model uses a probabilistic framework to link feature recognition with segment identification to explain how visual cortex processes images. It shows that neural responses exhibit gain and variability changes over time, driven by an iterative process where features and segments are inferred together. This suggests cortical dynamics are organized around inferred structure rather than just static features.
Key concepts
- Feature Inference Coupled with Segment Inference
- The model assumes that identifying visual features (like edges or colors) is intrinsically linked to figuring out how those features form meaningful parts or segments of the image. The system constantly cycles between guessing what the basic visual elements are and guessing how they are grouped together, creating dynamic neural activity.
- Iterative Inference Dynamics
- The process involves repeating steps: first, extracting features from deep networks, then using an Expectation-Maximization (EM) algorithm to refine segment labels. This loop induces changes in the neural representation over time as the model refines its understanding of the image structure.
- Quenching of Variability
- The steady decrease in response variability observed during stimulus presentation is explained by a key mechanism: the target probability distribution that neurons are sampling from gradually shifts toward the true, ideal posterior distribution for that specific visual input over time.
Terminology used across episodes
This episode discusses
The paper
A probabilistic model of visual segmentation explains early visual cortical dynamics · Read on arXiv
Tridib K Biswas, Ruben Coen-Cagli
Albert Einstein College of Medicine
Neural dynamics are a central feature of biological neural systems. In the visual cortex, the functional role of neural dynamics is poorly understood, particularly for static stimuli. The statistics of still natural images have long been used by models that successfully predict firing in early visual cortex; those models assume that neurons represent inferences about latent image features, but often ignore the dynamics of said inference. Here, we test if inferential dynamics in images explain visual-cortical dynamics. To do so, we present a biologically plausible model of cortical computation which assumes that the inference of features is coupled with inference about how those features are organized into meaningful parts, a function also known as segmentation. Given an input image, our model iterates between inferring the features and the segments, through recurrence and feedback, thus inducing dynamics in the neural representation of features. For these neural dynamics, we derive theoretical and exact predictions. Our model predicts both gain and variability in evoked responses to natural images, and we show that it captures classical single-neuron experimental findings such as gain dynamics and variability decay during stimulus presentation. The model also predicts that those dynamics are organized around inferred segment structure and more heterogeneous than previously thought, reflecting the complexity of inference for natural scenes. Finally, our analytical formulation provides a clear interpretation of our results. Insights from our normative model could generalize to segmentation problems in other experimental modalities, and help address the cost of inference in artificial networks with iterative computation.
Transcript
Introduction to the show: ident: Genomics Radio. Generated commentary on the latest computational biology and genomics papers.
Ines: I'm Ines.
Marcus: And I'm Marcus, here with Yuki.
Ines: We're looking at the paper: "A probabilistic model of visual segmentation explains early visual cortical dynamics".
Marcus: Here's the gist of the paper. A probabilistic model of visual segmentation explains early visual cortical dynamics by coupling feature inference with segment inference to induce neural dynamics The model predicts both gain and variability in evoked responses to natural images, showing that these dynamics are organized around inferred segment structure and more heterogeneous than previously thought Model Foundation The model is based on a biologically plausible framework assuming that the inference of features is coupled with inference about how those features are organized into meaningful parts, a function also known as segmentation The model iterates between inferring the features and the segments, through recurrence and feedback, thus inducing dynamics in the neural representation of features This approach is inspired by FlexMM, a probabilistic image segmentation algorithm that frames clustering dynamically as variational inference Inference Dynamics The iterative process involves several key steps in the generative model of perceptual segmentation and inference dynamics Feedforward neural inputs from deep nonlinearities, where features are extracted from deep convolutional networks pretrained on the ImageNet-1K database Approximate inference of segments, which involves maximizing the log-likelihood using an Expectation-Maximization (EM) algorithm to update posterior segments The iterative update rules with spatial regularization enforce that π(t)i,k for pixels closer together are more similar through a 2D convolution with a Gaussian kernel Neural Response Statistics The per-segment generative model specifies the relation between segment labels c, observed features x, and per-segment parameters θk The posterior distribution p(g x, c) is derived by assuming that the membrane potential of V1 neurons encodes this distribution via neural sampling Neural response statistics are determined by two factors: first, the parameters θ(t) for a given segment evolve throughout the iterations of EM, and second, the posterior class probabilities also evolve during EM per equation (4) The marginal moments that correspond to our model’s neural response statistics are derived from the mixture of per-segment distributions p(g+ x, c) with weights πk = p(c x) Observed Dynamics The simulation results capture three hallmarks of V1 dynamics, namely onset transients and the subsequent suppression of firing rate as well as the gradual suppression of variability Transient increase and fast adaptation of firing rate is observed, where one segment was typically initialized with a high feature mismatch (τ 2 µ) driving the transient Quenching of variability is demonstrated by a steady decrease in the Fano factor throughout stimulus presentation, reflecting that the target posterior distribution itself changes over time towards the ideal posterior for a given visual input Broader Implications The normative theory explains that the inference of latent features of natural visual stimuli requires iterative computation which, in turn, explains neural dynamics This framework provides a clear interpretation of our results and could generalize to segmentation problems in other experimental modalities and help address the cost of inference in artificial networks with iterative computation The work aims to advance basic understanding of visual cortex function, which is also necessary for understanding dis-function in disease The normative link between cortical dynamics and segmentation has implications for causal inference problems in other areas of the brain Code and Data Availability An implementation of FlexMM inference with multiple deep networks including AlexNet
64: and VGG-19
65: is available at https://github.com/tbiswas97/flexmm The images used in
12: are available through the Berkeley Segmentation Database
75: , and the THINGS image dataset is also publicly available The authors provide some in-development code pertinent to this work at https://github.com/tbiswas97/flexmm The model proceeds with closed form optimization from the written equations, and we do not fit any parameters to neural data Error bars are reported for Figures 4 and 5, as well as for correlation values reported in the main text The model is not memory intensive, being able to run on an M1 Macbook Pro running Numpy in typically less than a minute per image Our work follows the tradition of “probabilistic brain” theories according to which neurons perform inference about latent features The quenching of variability reflects that the target posterior distribution itself changes over time towards the ideal posterior for a given visual input The model does not model spatial correlations explicitly, although neurons within a segment appear correlated The authors have no human participants in this research, and made every effort to follow the NeurIPS Ethics Guidelines The use of previously published models is acknowledged via citation The license for existing assets is explicitly mentioned and properly respected We have no human participants in this research, and made every effort to follow the NeurIPS Ethics Guidelines The core method development in this research does not involve LLMs as any important, original, or non-standard components We do not foresee any negative societal consequences from this work We have no human participants in this research, and made every effort to follow the NeurIPS Ethics Guidelines.
Ines: The paper does not release new assets.
Marcus: The experimental setting should be presented in the core of the paper to a level of detail that is necessary to appreciate the results and make sense of them Errors are reported for Figures 4 and 5, as well as for correlation values reported in the main text.
Ines: The model is not memory intensive, being able to run on an M1 Macbook Pro running Numpy in typically less than a minute per image.
Title and authors: Ines: Let's talk about the title itself, "A probabilistic model of visual segmentation explains early visual cortical dynamics". It sounds very comprehensive, suggesting this isn't just another static feature analysis model.
Marcus: It points toward the idea that we need to look at the dynamics—the changes over time—not just a single snapshot of features in an image. They are testing if inferential dynamics actually explain how neural activity shifts when we see something new or change things in a scene.
Yuki: From a population genetics standpoint, I’m curious if this segmentation process is universal across all visual scenes, or if it's specific to the structure of natural scenery we see every day.
Ines: They are specifically focusing on natural scenes because those are the fundamental challenge that biological visual systems have solved. The paper frames this as a way to capture the rich structure intrinsic to those scenes through recurrent neural dynamics.
Marcus: The mechanism they describe involves feedforward neural inputs from deep networks, like AlexNet or VGG-nineteen which extracts features, and then an approximate inference of segments using something like an Expectation-Maximization algorithm.
Yuki: That sounds computationally heavy for real biological neurons if we take it literally. How does this model actually map onto the physical constraints of a neuron?
Ines: The paper assumes that neural response statistics are determined by two things: how the parameters for each segment evolve during those inference iterations, and also how the class probabilities change during those same steps.
Marcus: So, it’s not just one calculation; it's a feedback loop where inferring segments modulates the inferred features, which then shapes the neural response statistics. It's that coupled inference that matters here.
The paper's summary: Ines: The summary of "A probabilistic model of visual segmentation explains early visual cortical dynamics" boils down to this: they are modeling how the brain uses an iterative process—segmentation and feature inference—to generate the neural data we actually measure in the visual cortex.
Marcus: They show that this iterative process induces gain and variability in evoked responses to natural images, which is a big deal because it shows that these dynamics aren't just random noise; they are organized around inferred segment structures.
Yuki: So, instead of just saying neurons respond to features, they’re suggesting the response pattern itself changes based on the structure the brain infers about those features. That’s a shift in how we think about cortical processing.
Ines: Exactly. The results capture three main hallmarks of visual cortex activity: onset transients and fast adaptation of firing rate, which happens when one segment is initialized with a high feature mismatch, tau squared mu, driving that initial transient response.
Marcus: And they also show the quenching of variability, which is demonstrated by a steady decrease in the Fano factor throughout the entire stimulus presentation. This means the target distribution for what we expect to see changes over time towards the ideal posterior for that input image.
Yuki: The idea that variability quenches isn't just noise reduction; it’s the system settling on a more certain representation of reality as it processes things. That makes sense in terms of how perception works across different life stages or experiences, I suppose.
Ines: It implies that the inference of latent features in natural visual stimuli requires this iterative computation, and this computation directly explains the observed neural dynamics. This framework offers a clear way to interpret those results and could help us understand why certain things go wrong in disease.
The paper's improvements: Marcus: The authors suggest a few improvements to take this model further. They focus on incorporating iterative probabilistic inference directly into artificial neural networks so we can better capture gain dynamics and variability decay during the stimulus presentation, rather than just static feature representation.
Ines: That addresses a limitation of simpler models by making the inference itself dynamic within the network architecture, allowing it to evolve with the input. This is about capturing that time-dependent organization we talked about earlier.
Yuki: I wonder if making this iterative inference explicit helps us get closer to modeling how biological systems handle uncertainty in real-time during perception, which is something we always struggle with in population genetics when looking at dynamic traits.
Marcus: They also propose developing an AI system that predicts firing rate and variability by explicitly modeling that coupling between feature inference and segment inference, which is the core mechanism they found to be important.
Ines: And another point they raise is creating a model capable of predicting context-dependent dynamics where the complexity of the inference itself depends on what visual inputs you give it, and where that segmentation acts as modulatory feedback on those model neuron responses.
Yuki: That sounds like moving beyond a fixed set of rules toward something more adaptive, something that reacts differently depending on the scene’s complexity.
Marcus: So they are pushing for a framework that utilizes hierarchical generative models, similar to FlexMM, to do unsupervised probabilistic segmentation while also characterizing how that segmentation uncertainty modulates the neural response statistics.
Conclusion: Ines: So, wrapping up the "A probabilistic model of visual segmentation explains early visual cortical dynamics," the main message is that inferring features in a natural scene requires iterative computation involving both feature inference and segment inference to explain the observed neural dynamics.
Marcus: They demonstrate that this coupling creates gain and variability in responses, and crucially, they show how variability quenches as the system settles onto a better representation of the input over time.
Yuki: I think this means we should stop thinking of visual processing as a simple mapping from pixels to features, because it seems to be an active process of structuring the scene into meaningful parts.
Ines: It gives us a way to interpret the neural activity—not as just static representation, but as a dynamic inference process that evolves during stimulus presentation. This framework is useful for understanding cortical function and could apply beyond vision to other areas of brain research.
Marcus: It provides a normative link between cortical dynamics and segmentation, which is important because it can help us think about causal inference problems in other complex biological systems.
Yuki: I just want to say that this model shows the power of using probabilistic methods to capture the uncertainty inherent in perception, which is something we see everywhere in how we study complex biological variation across populations.
Ines: It’s a strong step forward for understanding how our brains actually build and maintain representations of the world. We look forward to seeing how these ideas connect with other models next.
More episodes
- 2610.33881-Sensorimotor behavior from an untrained brain-and-nerve-cord connectome
- 2610.40344-Morphological Features of Murine Pulmonary Arterial Networks for Diagnosing Vascular Disease
- Daily Summary for 2026-11-19
- 2609.10193-Are You Learning Biological Signal or Shortcuts? Auditing and Mitigating Bias in Protein-Protein Interaction Datasets
- 2610.38236-Generative discovery of chemical reaction networks
- 2507.11176-An Interpretable AI framework Quantifying Traditional Chinese Medicine Principles Towards Enhancing and Integrating with Modern Biomedicine
- The day in conversation for 2026-11-19
- Daily Summary for 2026-11-19
- 2504.03590-Towards a Unified Framework for Determining Conformational Ensembles of Disordered Proteins
- 2603.15217-A multiscale discrete-to-continuum framework for structured population models