SpikeCLR: Contrastive Self-Supervised Learning for Few-Shot Event-Based Vision using Spiking Neural Networks

arXiv:2603.16338 · cs.CV · Submitted 2026-03-17 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: AI Radio. Generated commentary on the latest Artificial Intelligence papers.

Tom: I'm Tom.

Jane: And I'm Jane, here with Lu, Meng and our in-house large language model, Lalam.

Tom: We're looking at the paper: "SpikeCLR: Contrastive Self-Supervised Learning for Few-Shot Event-Based Vision using Spiking Neural Networks".

Jane: Here's the gist of the paper. Event-based vision sensors provide significant advantages for high-speed perception, including microsecond temporal resolution, high dynamic range, and low power consumption <ref:2603.16338#pg2> In this work, we introduce SpikeCLR, a contrastive self-supervised learning framework that enables SNNs to learn robust visual representations from unlabeled event data <ref:2603.16338#pg3> Our results demonstrate that selfsupervised pretraining with subsequent fine-tuning outperforms supervised learning in low-data regimes, achieving consistent gains in few-shot and semi-supervised settings <ref:2603.16338#pg10> Event cameras depart from frame-based imaging by measuring per-pixel changes in log-intensity asynchronously, producing sparse streams of events rather than full frames at fixed intervals <ref:2603.16338#pg2> The asynchronous, spike-based nature of event data aligns with Spiking Neural Networks (SNNs)

42: , which process information as discrete spike sequences <ref:2603.16338#pg3> Despite these advantages, deploying SNNs for event-based vision faces a critical bottleneck: the scarcity of large-scale labeled datasets <ref:2603.16338#pg3> Self-supervised learning (SSL) has emerged as a powerful paradigm to learn visual representations without labels by exploiting the structure inherent in unlabeled data <ref:2603.16338#pg3> Key challenges include designing meaningful augmentations for sparse, polarity-based event representations and training SNNs with contrastive objectives in the presence of discrete, non-differentiable spike operations <ref:2603.16338#pg3> How the Training Signal is Built SpikeCLR introduces a contrastive self-supervised learning framework for event-based vision with Spiking Neural Networks, which enables effective learning with spike-based representations using surrogate gradient training <ref:2603.16338#pg3> The approach builds on SimCLR

5: , where each event recording is first converted into a grid-based representation (event histogram), on which stochastic augmentations are applied to generate two correlated views <ref:2603.16338#pg5> The contrastive objective is the NT-Xent loss, defined as: 1.

Tom: For an anchor embedding zi, its positive zj corresponds to the other view of the same original sample, while the remaining 2N − 2 embeddings act as negatives <ref:2603.16338#pg7> The final objective averages this loss over all positive pairs in the batch <ref:2603.16338#pg8> Furthermore, temporal variability can be introduced by sampling a random temporal window from the event stream and converting each window to a histogram <ref:2603.16338#pg8> Where the Gains Come From The ablation study reveals that combining spatial and temporal augmentations is critical for learning effective spatio-temporal invariances in event data <ref:2603.16338#pg10> Specifically, temporal augmentations emerge as particularly crucial, achieving the best single-family performance <ref:2603.16338#pg10> The full augmentation pipeline, incorporating all three families, achieves the overall best performance in 9-shot and 45-shot scenarios <ref:2603.16338#pg10> This validates our design choice to leverage the unique characteristics of event data through specialized augmentations across spatial, temporal, and polarity dimensions <ref:2603.16338#pg10> Generalizability and Scalability The transfer experiments show that SpikeCLR representations transfer across datasets When pretraining on N-Caltech101 and adapting to CIFAR10-DVS, we observe consistent gains over training from scratch in the few-shot regime Increasing the amount of unlabeled data during pretraining consistently improves downstream performance across all few-shot regimes This trend confirms the scalability of our self-supervised approach: as more unlabeled data becomes available, the learned representations become increasingly robust and generalizable SpikeCLR improves data efficiency in both few-shot and semi-supervised regimes, while keeping the pipeline compatible with standard SNN training via surrogate gradients <ref:2603.16338#pg3> The current approach inherits a practical constraint of SimCLR-style contrastive learning: strong performance typically relies on large effective batch sizes (or memory banks) to provide enough negatives A possibility to overcome these limitations could be to explore non-contrastive approaches (e.g.

Title and authors: Tom: So the paper summarizes SpikeCLR by showing how it turns an event recording into a grid-based representation, which is called an event histogram, and then applies stochastic augmentations to create two different views of the same original sample. This is where they get their positive pairs.

Jane: That makes sense for someone who doesn't read papers. So they take one stream of events and make two slightly different versions—maybe shifting the time window or changing how intensity is mapped—and they try to pull the representations from those two versions closer together.

Lu: And then, while keeping track of these positive pairs, they use a negative example strategy where all other embeddings in the batch act as negatives to prevent the model from collapsing its features. This is done using the NT-Xent loss, which is a specific contrastive objective.

Meng: I see that part about using negatives—it keeps things from just learning everything as zero or one thing. But what they are doing with the actual event data itself, beyond just making two views? That's where the real novelty should be.

Lalam: They introduce a temporal variability trick too. They sample a random time window from the continuous event stream and turn that window into another histogram, which is then used to create a second correlated view for contrastive learning. This lets them capture some of that temporal aspect directly in the self-supervised setup.

The paper's summary: Tom: The paper highlights a few key improvements they made over the existing methods. First, they show that combining spatial and temporal augmentations is really critical for learning effective spatio-temporal invariances in event data.

Jane: So it’s not just making two different pictures of the same scene; it’s also changing *when* you look at those pictures to make sure the model learns things that aren't dependent on a single moment in time.

Lu: They found that temporal augmentations emerge as particularly crucial, achieving the best single-family performance when looking at just one type of augmentation. But they also show that putting all three families of augmentations—spatial, polarity, and temporal—together gives the overall best performance in scenarios like nine-shot and forty-five-shot settings.

Meng: That’s interesting because it suggests you can’t just focus on one aspect; you need that full pipeline to get good results when you're trying to learn things in a few shots. But how does that translate to deployment?

Lalam: It validates the design choice they made to leverage the unique characteristics of event data by applying these specialized augmentations across spatial, temporal, and polarity dimensions. The point is that this multi-faceted augmentation helps the model generalize better than if you only used one type of change.

The paper's improvements: Tom: So to wrap up, SpikeCLR improves data efficiency in both few-shot and semi-supervised regimes while keeping the pipeline compatible with standard SNN training via surrogate gradients. It’s a solid improvement on how we train these models without needing massive labeled datasets.

Jane: What this means for us is that we can get much better performance on dynamic event data even when we only have very little labeled information, which is a big deal for real-world applications where data collection is hard.

Lu: The paper confirms the scalability of their self-supervised approach; when you pretrain on datasets like N-Caltech101 and then adapt it to things like CIFAR10-DVS, you see consistent gains over training from scratch in those few-shot regimes. As more unlabeled data comes available, the learned representations get more robust.

Meng: From my side at the startup, I’m watching how this helps us because representation quality scales with the amount of unlabeled pretraining data; increasing it from ten thousand to sixty thousand samples on N-MNIST consistently improves downstream performance in those few-shot scenarios. That scaling is what makes it practical for production environments where we don't have endless labeled data.

Lalam: Lalam here, so the overall message of SpikeCLR is that this framework significantly boosts data efficiency in both few-shot and semi-supervised settings, all while staying compatible with the standard SNN training pipeline using surrogate gradients. It shows that learning robust visual representations for event cameras doesn't have to be impossible due to data scarcity.

Conclusion: Tom: So, SpikeCLR is basically a contrastive self supervised learning framework that lets spiking neural networks learn visual representations from unlabeled event data without needing any labels at all.

Jane: That's right, and what it does really well is showing that you can get consistent gains in few-shot and semi supervised settings by using this self supervised pretraining before fine tuning.

Lu: The core idea they introduce is converting an event recording into a grid based representation, called an event histogram, and then applying stochastic augmentations to create two correlated views from that.

Meng: So it’s not just making two pictures of the same thing; they are specifically sampling a random time window from the continuous stream and turning that window into another histogram to create a second view.

Lalam: That temporal variability trick is what makes their contrastive loss, which is called NT-Xent loss, really strong because it captures spatio temporal invariances across all three dimensions—spatial, polarity intensity, and that time aspect.

Tom: And the results show that combining those spatial and temporal augmentations is critical for learning those invariances in event data.

Jane: It’s interesting how they found that the temporal augmentations emerged as particularly crucial when looking at just one type of augmentation alone, but using all three families together gives them the best overall performance in nine shot and forty five shot scenarios.

Lu: Their transfer experiments show that these SpikeCLR representations are transferable across different datasets, like pretraining on N-Caltech101 and then adapting to CIFAR10-DVS, where they see consistent gains over training from scratch.

Meng: But we have to remember the constraint they mention; their SimCLR style contrastive learning typically relies on large effective batch sizes or memory banks to give the model enough negatives.

Tom: Yeah, and that’s a practical thing for deployment because running those massive effective batch sizes might be tricky at scale.

Jane: But they suggest exploring non-contrastive approaches as a way to overcome those limitations, which is where they are headed next in their work.

Lu: From my view, the scalability of their approach is clear; as more unlabeled data becomes available during pretraining, the learned representations become increasingly robust and generalizable.

Lalam: I’m confident that this work on SpikeCLR improves data efficiency in both few shot and semi supervised regimes while keeping the pipeline compatible with standard SNN training via surrogate gradients.

Tom: So, to wrap up, SpikeCLR is a contrastive self supervised learning framework for few shot event based vision using spiking neural networks.

Jane: It shows that we can learn robust visual representations from unlabeled event data by building on the structure inherent in the data itself.

Lu: This moves us closer to deploying SNNs for real world event based vision systems where labeled data is scarce.

Meng: For production, this means a potentially more efficient way to get high quality models without needing huge amounts of expensive labeled event footage upfront.

Lalam: And I'm sure this will open up some really interesting possibilities for how we use these fast event cameras in applications across the board.

CNRS IPAL IRL 2955 Singapore University of Toulouse IRIT France Fédération ENAC ISAE-SUPAERO ONERA Université de Toulouse Institute for Infocomm Research A*STAR Singapore CerCo CNRS UMR 5549 Université de Toulouse

cs.CV

Submitted: 2026-03-17

Updated: 2026-03-17

Comments: 17 pages, 4 figures

DOI: 10.1088/2634-4386/aebe58

Code: https://github.com/maximevaillant/SpikeCLR

License: http://creativecommons.org/licenses/by/4.0/

Importance score: 91/100

The gist: Event-based vision sensors provide significant advantages for high-speed perception, including microsecond temporal resolution, high dynamic range…

Key concepts

Event Cameras
These cameras measure changes in light intensity at each pixel asynchronously, producing sparse streams of events instead of full frames. This allows for microsecond temporal resolution and high dynamic range, which is ideal for processing with Spiking Neural Networks (SNNs).
SpikeCLR
This is a contrastive self-supervised learning framework designed specifically for event-based vision and SNNs. It uses the NT-Xent loss to train the network by making different augmented versions of an event sample similar, forcing the model to learn meaningful visual features without needing human labels.
Contrastive Learning (NT-Xent Loss)
This is a training objective where the model learns by comparing different data samples. For each event, two views are created using augmentations. The network is trained to ensure these two views have similar representations while pushing them away from other random samples in the batch, improving feature discrimination.
Spatiotemporal Augmentations
These are transformations applied to event data that include changes in both space (where events occur) and time (when they occur). The study shows that combining spatial, polarity, and temporal augmentations is crucial for teaching the model how to recognize objects robustly across different viewing conditions.

Terminology

Summary

Event-based vision sensors provide significant advantages for high-speed perception, including microsecond temporal resolution, high dynamic range, and low power consumption <ref:2603. p. 0

Where the paper says it

fr Abstract. Event-based vision sensors provide significant advantages for high-speed perception, including microsecond temporal resolution, high dynamic range, and low power consumption.

16338#pg2> In this work, we introduce SpikeCLR, a contrastive self-supervised learning framework that enables SNNs to learn robust visual representations from unlabeled event data <ref:2603. p. 0
Where the paper says it

However, this potential is severely limited by the scarcity of large-scale labeled datasets required to effectively train such models. In this work, we introduce SpikeCLR, a contrastive self-supervised learning framework that enables SNNs to learn robust visual representations from unlabeled event data.

16338#pg3> Our results demonstrate that selfsupervised pretraining with subsequent fine-tuning outperforms supervised learning in low-data regimes, achieving consistent gains in few-shot and semi-supervised settings <ref:2603. p. 0
Where the paper says it

We adapt prior frame-based methods to the spiking domain using surrogate gradient training and introduce a suite of event-specific augmentations that leverage spatial, temporal, and polarity transformations. Through extensive experiments on CIFAR10-DVS, N-Caltech101, N-MNIST, and DVS-Gesture benchmarks, we demonstrate that selfsupervised pretraining with subsequent fine-tuning outperforms supervised learning in low-data regimes, achieving consistent gains in few-shot and semi-supervised settings.

16338#pg10>

Introduction to the Problem

Event cameras depart from frame-based imaging by measuring per-pixel changes in log-intensity asynchronously, producing sparse streams of events rather than full frames at fixed intervals <ref:2603.16338#pg2> The asynchronous, spike-based nature of event data aligns with Spiking Neural Networks (SNNs) [42], which process information as discrete spike sequences <ref:2603. p. 1

Where the paper says it

This yields microsecond-level temporal resolution, high dynamic range (>120 dB), and low power consumption, making event cameras well-suited for fast motion, challenging lighting, and resource-constrained platforms. The asynchronous, spike-based nature of event data aligns with Spiking Neural Networks (SNNs) [42], which process information as discrete spike sequences.

16338#pg3> Despite these advantages, deploying SNNs for event-based vision faces a critical bottleneck: the scarcity of large-scale labeled datasets <ref:2603. p. 1
Where the paper says it

Because SNNs explicitly model time through spiking dynamics, they are a natural fit for event streams: each neuron integrates incoming currents into its membrane potential and emits a spike when a threshold is exceeded, enabling sparse and energy-efficient processing. Despite these advantages, deploying SNNs for event-based vision faces a critical bottleneck: the scarcity of large-scale labeled datasets.

16338#pg3> Self-supervised learning (SSL) has emerged as a powerful paradigm to learn visual representations without labels by exploiting the structure inherent in unlabeled data <ref:2603. p. 1
Where the paper says it

Models therefore often learn from synthetic motion patterns that only partially reflect deployment conditions, making label-efficient training especially important. Self-supervised learning (SSL) has emerged as a powerful paradigm to learn visual representations without labels by exploiting the structure inherent in unlabeled data.

16338#pg3> Key challenges include designing meaningful augmentations for sparse, polarity-based event representations and training SNNs with contrastive objectives in the presence of discrete, non-differentiable spike operations <ref:2603. p. 1
Where the paper says it

However, their application to event-based data and SNNs remains largely unexplored. Key challenges include: (i) designing meaningful augmentations for sparse, polarity-based event representations, and (ii) training SNNs with contrastive objectives in the presence of discrete, non-differentiable spike operations.

16338#pg3>

How the Training Signal is Built

SpikeCLR introduces a contrastive self-supervised learning framework for event-based vision with Spiking Neural Networks, which enables effective learning with spike-based representations using surrogate gradient training <ref:2603.16338#pg3> The approach builds on SimCLR [5], where each event recording is first converted into a grid-based representation (event histogram), on which stochastic augmentations are applied to generate two correlated views <ref:2603. p. 5

Where the paper says it

4 Contrastive Learning Our approach builds on SimCLR [5], a contrastive self-supervised learning framework that learns visual representations by bringing together different augmented views of the same sample while pushing apart views coming from different samples. In our setting, each event recording is first converted into a grid-based representation (event histogram), on which we apply stochastic augmentations to generate two correlated views.

16338#pg5> The contrastive objective is the NT-Xent loss, defined as:

  1. For an anchor embedding zi, its positive zj corresponds to the other view of the same original sample, while the remaining 2N − 2 embeddings act as negatives <ref:2603. p. 5
    Where the paper says it

    Each sample yields two views, resulting in 2N embeddings. For an anchor embedding zi , its positive zj corresponds to the other view of the same original sample, while the remaining 2N − 2 embeddings act as negatives.

    16338#pg7> The final objective averages this loss over all positive pairs in the batch <ref:2603. p. 5
    Where the paper says it

    The NT-Xent loss is defined as: \label {eq:ntxent} \ell _{i,j}=- \log \frac {\exp \left (\mathrm {sim}(\mathbf {z}_i, \mathbf {z}_j) / \tau \right )} {\sum _{k=1}^{2N}\mathbbm {1}_{[k \neq i]}\exp \left (\mathrm {sim}(\mathbf {z}_i, \mathbf {z}_k) / \tau \right )} , (3) where sim(·, ·) denotes cosine similarity and τ is a temperature hyperparameter. The final objective averages this loss over all positive pairs in the batch.

    16338#pg8> Furthermore, temporal variability can be introduced by sampling a random temporal window from the event stream and converting each window to a histogram <ref:2603. p. 6
    Where the paper says it

    Temporal augmentations. While our main pipeline operates on event histograms, temporal variability can still be introduced before histogram construction by sampling a random temporal window (temporal crop) from the event stream.

    16338#pg8>

Where the Gains Come From

The ablation study reveals that combining spatial and temporal augmentations is critical for learning effective spatio-temporal invariances in event data <ref:2603.16338#pg10> Specifically, temporal augmentations emerge as particularly crucial, achieving the best single-family performance <ref:2603. p. 9

Where the paper says it

This suggests that spatial invariances, while important, are less critical for eventbased representations than their temporal and polarity counterparts. Temporal augmentations emerge as particularly crucial, achieving the best single-family performance and providing gains of 6.

16338#pg10> The full augmentation pipeline, incorporating all three families, achieves the overall best performance in 9-shot and 45-shot scenarios <ref:2603. p. 9
Where the paper says it

Combining augmentation families leads to synergistic improvements. The spatial and temporal combination achieves the best 1-shot performance (20.

16338#pg10> This validates our design choice to leverage the unique characteristics of event data through specialized augmentations across spatial, temporal, and polarity dimensions <ref:2603. p. 0
Where the paper says it

In this work, we introduce SpikeCLR, a contrastive self-supervised learning framework that enables SNNs to learn robust visual representations from unlabeled event data. We adapt prior frame-based methods to the spiking domain using surrogate gradient training and introduce a suite of event-specific augmentations that leverage spatial, temporal, and polarity transformations.

16338#pg10>

Generalizability and Scalability

The transfer experiments show that SpikeCLR representations transfer across datasets When pretraining on N-Caltech101 and adapting to CIFAR10-DVS, we observe consistent gains over training from scratch in the few-shot regime Increasing the amount of unlabeled data during pretraining consistently improves downstream performance across all few-shot regimes This trend confirms the scalability of our self-supervised approach: as more unlabeled data becomes available, the learned representations become increasingly robust and generalizable

Conclusion and Future Work

SpikeCLR improves data efficiency in both few-shot and semi-supervised regimes, while keeping the pipeline compatible with standard SNN training via surrogate gradients <ref:2603.16338#pg3> The current approach inherits a practical constraint of SimCLR-style contrastive learning: strong performance typically relies on large effective batch sizes (or memory banks) to provide enough negatives A possibility to overcome these limitations could be to explore non-contrastive approaches (e. p. 2

Where the paper says it

Contrastive approaches [4,6,21,39] such as SimCLR [5] or MoCo [20] learn representations by maximizing agreement between different augmented views of the same sample, while simultaneously contrasting them against representations of other samples in the batch, which act as negative examples to prevent representational collapse. Non-contrastive methods such as BYOL [17] or SimSiam [7] remove explicit negatives and prevent collapse through architectural asymmetry and optimization constraints.

g.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Improved Data Efficiency in Low-Data Regimes: The system can achieve consistent gains in few-shot and semi-supervised settings by using SpikeCLR for self-supervised pretraining, which outperforms supervised learning when labeled data is scarce.

  2. Enhanced Representation Learning via Augmentation: The model learns robust features by applying a family of event specific augmentations—spatial, polarity/intensity, and temporal transformations—which the ablation study shows are critical for learning effective spatio-temporal invariances in event data.

  3. Superior Temporal Discriminative Signal: By computing the contrastive loss at each timestep independently (Temporal NT-Xent), the system receives a stronger learning signal than optimizing the mean alone, leading to better downstream accuracy on dynamic datasets like DVS-Gesture.

  4. Architecture Agnosticism: The framework demonstrates that the benefits of selfsupervised pretraining generalize across different architectures, meaning the model can be effectively applied to diverse SNN designs, including SEW-ResNet18, Spiking JELLY, and Spiking VGG9.

  5. Cross-Dataset Transfer Capability: The system can adapt representations learned on one event dataset to a different target dataset (e.g., CIFAR10-DVS to DVS-Gesture), confirming that the encoder captures reusable spatio-temporal primitives even when the acquisition protocols differ.

  6. Scalability with Unlabeled Data: Representation quality scales with the amount of unlabeled pretraining data, as increasing it from 10K to 60K samples on N-MNIST consistently improves downstream performance in few-shot scenarios.

Abstract

Event-based vision sensors provide significant advantages for high-speed perception, including microsecond temporal resolution, high dynamic range, and low power consumption. When combined with Spiking Neural Networks (SNNs), they can be deployed on neuromorphic hardware, enabling energy-efficient applications on embedded systems. However, this potential is severely limited by the scarcity of large-scale labeled datasets required to effectively train such models. In this work, we introduce SpikeCLR, a contrastive self-supervised learning framework that enables SNNs to learn robust visual representations from unlabeled event data. We adapt prior frame-based methods to the spiking domain using surrogate gradient training and introduce a suite of event-specific augmentations that leverage spatial, temporal, and polarity transformations. Through extensive experiments on CIFAR10-DVS, N-Caltech101, N-MNIST, and DVS-Gesture benchmarks, we demonstrate that self-supervised pretraining with subsequent fine-tuning outperforms supervised learning in low-data regimes, achieving consistent gains in few-shot and semi-supervised settings. Our ablation studies reveal that combining spatial and temporal augmentations is critical for learning effective spatio-temporal invariances in event data. We further show that learned representations transfer across datasets, contributing to efforts for powerful event-based models in label-scarce settings.

Sources

Related papers