Learning Binary Codes with Spiking Neural Networks for Cross-Modal Hashing Retrieval

arXiv:2606.00740 · cs.IR, cs.MM · Submitted 2026-05-30 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "Learning Binary Codes with Spiking Neural Networks for Cross-Modal Hashing Retrieval".

Title and authors: Tom: So we're looking at this paper, "Learning Binary Codes with Spiking Neural Networks for Cross-Modal Hashing Retrieval," and it's about how they use spiking neural networks to turn things like images and text into compact binary codes for searching in a Hamming space.

Jane: Right, but what they are doing is a big shift—instead of learning continuous features first and then just turning those into a code at the very end, they propose this unified spiking framework that builds the binary code directly from the neural dynamics themselves.

Lu: They call it SpikeHash, and it reformulates hash code generation as an event driven process. This means the binary codes aren't just some post-processed output; they’re formed by integrating spike state evolution, directional cross-modal modulation, and a positive negative spike competition readout.

Meng: That sounds complicated when you think about production deployment. My main concern is that they still need dense artificial neural network computation before the hash generation part happens, which makes it tricky for low power systems.

Tom: Yeah, that’s the engineering hurdle. But what they’re saying is that this unified spiking framework connects the whole process—the semantic interaction, the state evolution, and the final hash generation—all in one place within that spike representation space.

Jane: And they use something called directional spike modulation to make sure the image and text modalities actually influence each other's firing dynamics. It’s not just them evolving separately; they are actively shaping each other’s signals during the process.

Lu: Exactly, and this leads into their main contribution: a competitive reference framework for spiking cross-modal hashing. They’re trying to find that sweet spot between accuracy and efficiency in how you learn the representation and how you form that binary code.

Tom: The numbers they give are interesting here because they show that SpikeHash achieves competitive retrieval performance while using fewer parameters and requiring less computational cost than dense artificial neural network methods like UCMFH.

Jane: And the authors point out that this gain isn't just coming from using pre-trained features or scaling up the number of parameters, but it seems closely related to these specific spiking dynamics they designed.

Lu: That’s because they designed Bit Activation Regularization, or BAR, to reduce a problem where both the positive and negative spike channels go silent at the same time, which encourages those hash bits to actually participate in the readout.

Meng: I’m still watching how stable this spiking mechanism is when we move away from their perfectly controlled training simulations to real traffic where things are way more chaotic.

Tom: That’s a fair concern about robustness, but they did mention that their loss function is optimized end-to-end with a surrogate gradient during training, which helps stabilize the whole process.

Jane: So they’re showing that this approach yields results that are semantically consistent and interpretable, not just good numbers, because they constrain the hash readout and the bit activity itself before it even gets to inference.

Lu: And for those low bit settings, like sixteen bits or fewer, they use a longer time window of eight time steps to give those limited capacity hash codes a richer dynamic representation space.

Meng: If we stick to low bit settings, we need more time steps, which means longer sequences and potentially more energy during that spike generation phase. That adds computational cost back in somewhere.

Tom: It’s a trade-off they have to manage; they’re showing that this extended temporal dimension compensates for the lack of bit capacity, but it doesn't eliminate the underlying computational requirements of the spiking dynamics.

Jane: So, what this means for someone who just listens to the show is that while they are pushing for more sophisticated interaction and better regularity in training, they’re still focused on keeping that whole process as event driven and low power as possible.

Lu: The ultimate goal here is to prove that you can build this unified framework from spike encoding right up to hash generation, which makes it a much more cohesive system overall.

Tom: So the paper "Learning Binary Codes with Spiking Neural Networks for Cross-Modal Hashing Retrieval" shows how you can unify spiking computation with hashing to create a more efficient and inherently discrete representation learning system for cross-modal tasks.

Jane: It’s definitely a step forward in rethinking how we handle discrete code generation in cross-modal hashing, especially when you’re looking at energy-aware retrieval on hardware that supports event driven computation.

Lu: That's the story for today. We've looked at how SpikeHash redefines hash code generation by making it an event driven process that integrates spike state evolution and competition between paired channels.

Meng: So, if I’m running this at scale, I’m still worried about that initial dense artificial neural network computation before the hash generation part happens, which makes it tricky for low power systems.

Tom: That's the path forward then: unifying spiking computation with hashing to create a more efficient and inherently discrete representation learning system for cross-modal tasks.

The paper's summary: Tom: So, to wrap up on this paper, we're talking about how they use spiking neural networks to turn image and text data into efficient binary codes for searching in a Hamming space.

Jane: Right, essentially it shows how you can build a unified spiking framework that directly generates those binary codes using competition between spike channels.

Lu: The big implication here is moving away from learning continuous features first and then post-processing them; they are creating a process where the binary code is formed by integrating spike state evolution and cross-modal modulation.

Meng: So, if I’m running this at scale, I’m still worried about that initial dense artificial neural network computation before the hash generation part happens, which makes it tricky for low power systems.

Tom: It is a hurdle; but the results show they achieve competitive performance with fewer parameters and less computational cost than methods like UCMFH.

Jane: And they emphasize that this gain isn't just from using pre-trained features or scaling up the number of parameters, but it seems closely tied to these specific spiking dynamics.

Lu: They designed Bit Activation Regularization to make sure those hash bits actually participate well by reducing the simultaneous silence of positive and negative spike channels, which is key for reliable code formation.

Meng: I’m still watching how stable this spiking mechanism is when we move away from their perfectly controlled training simulations to real traffic where things are way more chaotic.

Tom: The authors acknowledge that it’s a regularization technique, but they argue it supports the overall goal of getting a more reliable binary code from the spiking process.

Jane: They also addressed the issue of low bit settings by suggesting using a longer time window of eight time steps instead of what might be standard for other systems.

Lu: That’s to compensate for limited capacity when you only have a few hash bits; they are providing a richer dynamic representation space for those codes.

Meng: If we stick to low bit settings, we need more time steps, which means longer sequences and potentially more energy during that spike generation phase. That adds computational cost back in somewhere.

Tom: It’s a trade-off they have to manage; they’re showing that this extended temporal dimension compensates for the lack of bit capacity, but it doesn't eliminate the underlying computational requirements of the spiking dynamics.

Jane: So, what this means for someone who just listens to the show is that while they are pushing for more sophisticated interaction and better regularity in training, they’re still focused on keeping that whole process as event driven and low power as possible.

Lu: The ultimate goal here is to prove that you can build this unified framework from spike encoding right up to hash generation, which makes it a much more cohesive system overall.

Tom: So the paper "Learning Binary Codes with Spiking Neural Networks for Cross-Modal Hashing Retrieval" shows how you can unify spiking computation with hashing to create a more efficient and inherently discrete representation learning system for cross-modal tasks.

Jane: It’s definitely a step forward in rethinking how we handle discrete code generation in cross-modal hashing, especially when you’re looking at energy-aware retrieval on hardware that supports event driven computation.

Tom: That’s the story for today. We've looked at how SpikeHash redefines hash code generation by making it an event driven process that integrates spike state evolution and competition between paired channels.

Jane: It really shows a pathway to making cross-modal retrieval much more practical and efficient in resource constrained environments, provided we can address those scaling concerns and deployment robustness when moving away from the idealized training conditions.

The paper's improvements: Tom: So, we're talking about how they suggest improving SpikeHash and what that means for future work on this kind of system.

Jane: They’re focused on making the framework even more integrated, moving beyond just the initial proposal to refine how it actually learns and operates during training.

Lu: One big improvement they propose is enhancing cross-modal interaction through something called the Cross-Modal Spiking Gated Interaction Module, or CMSGI. This module injects channel conditioned information into the firing process by generating fields like GT to I and GI to T, which helps capture those fine grained complementary relations between modalities.

Meng: That sounds like more complexity that could slow down inference on my end. What’s the actual impact of adding these extra gating modules on the runtime for a production model?

Tom: The authors suggest this enhancement is necessary because it allows the system to model richer semantic interaction, but they’re trying to keep it efficient by designing these interactions specifically within the spiking representation space.

Jane: They also focus on making sure those hash bits actually contribute properly during training with something called Bit Activation Regularization, or BAR. The idea there is to reduce that situation where both the positive and negative spike channels are silent at the same time, which keeps the hash bits from being wasted.

Lu: That regularization aims to ensure effective participation of every hash bit in the final code, which they think is crucial for getting good results from a discrete readout.

Meng: I’m still wary of that because I haven't seen how well that BAR performs when we introduce real-world noise or very different input data compared to their training sets.

Tom: The authors acknowledge that it’s a regularization technique, but they argue it supports the overall goal of getting a more reliable binary code from the spiking process.

Jane: They also address the issue of low bit settings, like sixteen bits or fewer, by suggesting using a longer time window of eight time steps instead of what might be standard for other systems.

Lu: That’s to compensate for the limited capacity when you only have a few hash bits; they’re providing a richer dynamic representation space for those codes.

Meng: So, if we stick to low bit settings, we need more time steps, which means longer sequences and potentially more energy during that spike generation phase. That adds computational cost back in somewhere.

Tom: It’s a trade-off they have to manage; they’re showing that this extended temporal dimension compensates for the lack of bit capacity, but it doesn't eliminate the underlying computational requirements of the spiking dynamics.

Jane: So, what this means for someone who just listens to the show is that while they are pushing for more sophisticated interaction and better regularity in training, they’re still focused on keeping that whole process as event driven and low power as possible.

Lu: The ultimate goal here is to prove that you can build this unified framework from spike encoding right up to hash generation, which makes it a much more cohesive system overall.

Conclusion: Tom: So we're wrapping up on "Learning Binary Codes with Spiking Neural Networks for Cross-Modal Hashing Retrieval," which is about turning image and text data into efficient binary codes using spiking neural networks for retrieval.

Jane: Right, essentially it shows how you can build a unified spiking framework that directly generates those binary codes through neural dynamics using competitive spike readout.

Lu: The big implication here is moving away from learning continuous features first and then post-processing them; they are creating a process where the binary code is formed by integrating spike state evolution and cross-modal modulation.

Meng: So, if I’m running this at scale, I’m still worried about that initial dense artificial neural network computation before the hash generation part happens, which makes it tricky for low power systems.

Tom: It is a hurdle; but the results show they achieve competitive performance with fewer parameters and less computational cost than methods like UCMFH.

Jane: And they emphasize that this gain isn't just from using pre-trained features or scaling up the number of parameters, but it seems closely tied to these specific spiking dynamics.

Lu: They designed Bit Activation Regularization to make sure those hash bits actually participate well by reducing the simultaneous silence of positive and negative spike channels, which is key for reliable code formation.

Meng: I’m still watching how stable this spiking mechanism is when we move away from their perfectly controlled training simulations to real traffic where things are way more chaotic.

Tom: The authors acknowledge that it’s a regularization technique, but they argue it supports the overall goal of getting a more reliable binary code from the spiking process.

Jane: They also addressed the issue of low bit settings by suggesting using a longer time window of eight time steps instead of what might be standard for other systems.

Lu: That’s to compensate for limited capacity when you only have a few hash bits; they are providing a richer dynamic representation space for those codes.

Meng: If we stick to low bit settings, we need more time steps, which means longer sequences and potentially more energy during that spike generation phase. That adds computational cost back in somewhere.

Tom: It’s a trade-off they have to manage; they’re showing that this extended temporal dimension compensates for the lack of bit capacity, but it doesn't eliminate the underlying computational requirements of the spiking dynamics.

Jane: So, what this means for someone who just listens to the show is that while they are pushing for more sophisticated interaction and better regularity in training, they’re still focused on keeping that whole process as event driven and low power as possible.

Lu: The ultimate goal here is to prove that you can build this unified framework from spike encoding right up to hash generation, which makes it a much more cohesive system overall.

Tom: So the paper "Learning Binary Codes with Spiking Neural Networks for Cross-Modal Hashing Retrieval" shows how you can unify spiking computation with hashing to create a more efficient and inherently discrete representation learning system for cross-modal tasks.

Jane: It’s definitely a step forward in rethinking how we handle discrete code generation in cross-modal hashing, especially when you’re looking at energy-aware retrieval on hardware that supports event driven computation.

Tom: That’s the story for today. We've looked at how SpikeHash redefines hash code generation by making it an event driven process that integrates spike state evolution and competition between paired channels.

Jane: It really shows a pathway to making cross-modal retrieval much more practical and efficient in resource constrained environments, provided we can address those scaling concerns and deployment robustness when moving away from the idealized training conditions.

Lu: I think what’s most important is that they built this entire system from spike encoding right up to hash generation, which makes it a much more cohesive system overall.

Meng: From my side, if we can get the computational overhead down to where they show with two point one nine million parameters and over sixty percent fewer operations compared to dense artificial neural network methods, then it becomes a very viable candidate for energy-aware retrieval on edge devices.

Tom: That’s the path forward, so this paper is about unifying spiking computation with hashing to create a more efficient and inherently discrete representation learning system for cross-modal tasks.

Jane: It’s definitely a step forward in rethinking how we handle discrete code generation in cross-modal hashing, especially when you’re looking at energy-aware retrieval on hardware that supports event driven computation.

Yukuan Zhang, Jiarui Zhao, Shangqing Nie, Shengsheng Wang

Jilin University

cs.IR, cs.MM

Submitted: 2026-05-30

Updated: 2026-11-18

Project page: https://shuqiao-111.github.io

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

The gist: Cross-modal hashing retrieval encodes heterogeneous data into compact binary codes for efficient Hammingspace search <ref:2606.00740#pg2>.

Key concepts

Cross-modal Hashing Retrieval
This involves encoding different types of data, like images and text, into compact binary codes. The goal is to allow a system to efficiently search for relevant items across these different data types using the Hamming distance metric.
Spike-state Evolution
Instead of standard neural network layers, SpikeHash treats the interaction between image and text features as the temporal evolution of a shared hash state driven by spike sequences. This models how information is processed over time in a biologically inspired way.
Positive-Negative Spiking Hash Readout
This is the core mechanism where each bit of the binary hash code is determined by the competition between two paired spike channels. One channel fires positively, and the other negatively, and their temporal interaction decides whether that specific hash bit is '1' or '0'.

Terminology

Summary

Cross-modal hashing retrieval encodes heterogeneous data into compact binary codes for efficient Hammingspace search <ref:2606. p. 000740#pg2>. SpikeHash proposes a unified spiking framework that formulates cross-modal hashing as spike-state evolution, directional spike interaction, and competitive spike readout <ref:2606. p. 000740#pg3>.

Pruning redundant reasoning tokens

Existing cross-modal hashing methods are largely built upon continuous representation learning before binary code generation. This separation between continuous semantic learning and discrete hash generation leads to two limitations: (1) Model optimization mainly focuses on continuous representations, whereas retrieval ultimately relies on discrete hash codes, and (2) Although hash codes greatly reduce storage costs and the cost of Hamming distance matching, feature transformation and cross-modal interaction before hash generation still depend on dense artificial neural network (ANN) computation. p. 1

How the training signal is built

SpikeHash converts image and text features into multi-timestep spike sequences. In a shared Hamming space, the two spike sequences jointly drive the temporal evolution of a shared hash state. p. 0 Cross-modal interaction is further performed through directional spike modulation, enabling each modality to influence the firing dynamics of the other. p. 0 Crucially, SpikeHash replaces the conventional continuous hash head with a positive-negative spiking hash readout, where each hash bit is produced by temporal competition between paired spike channels. p. 0

Where the gains come from

The main contributions of this work are summarized as follows: • We propose a competitive reference framework for spiking cross-modal hashing, providing a compact accuracyefficiency trade-off paradigm that couples representation learning with binary code formation. • We propose SpikeHash, which unifies shared spiking hash state evolution and cross-modal spiking gated interaction to jointly model semantic interaction, state evolution, and hash generation in the spike representation space. p. 1 • We design Bit Activation Regularization (BAR) to reduce simultaneous silence of positive and negative spike channels, encouraging effective hash-bit participation. p. 1

The overall training objective is L = Lhral + λsralLsral + λbarLbar. During training, the PLIF firing function is optimized end to end with a surrogate gradient. p. 4 The loss weights are set to λsral = 0. p. 501 and λbar = 0. p. 42.

The experimental results show that SpikeHash achieves competitive retrieval performance with fewer parameters, lower computational cost, and lower energy consumption. The results also indicate that the gain of SpikeHash is closely related to the PLIF-based spiking dynamics, rather than simply coming from CLIP pretrained features or parameter scale. p. 9

The paper concludes that SpikeHash reformulates hash code generation as an event driven process that integrates spiking state evolution, directional cross-modal modulation, and positive negative spike competition readout. This design naturally connects binary hash readout with spiking representation learning, enabling SpikeHash to form a unified framework from spike encoding and cross-modal interaction to hash generation. p. 8

The visualization results further demonstrate that SpikeHash not only achieves competitive quantitative retrieval performance, but also produces semantically consistent and interpretable bidirectional retrieval results. The final objective jointly constrains hash readout, bit activity, and the representation before readout, and these three components together support the training of SpikeHash. p. 7

SpikeHash substantially reduces model size, computation, and energy consumption during hash code generation. It therefore provides a promising direction for energy-aware cross-modal hash retrieval, especially when deployed with hardware that can exploit event-driven computation. p. 8 p. 8 The results show that spiking mechanisms are not only a low power computing strategy, but also an effective way to rethink the mechanism of discrete code generation in cross-modal hashing. p. 3 p. 3

SpikeHash achieves competitive retrieval performance on multiple benchmark datasets and under different code length settings, while showing clear advantages in parameter size, computation, and energy consumption. It therefore provides a promising direction for energy-aware cross-modal hash retrieval, especially when deployed with hardware that can exploit event-driven computation. The results show that spiking mechanisms are not only a low power computing strategy, but also an effective way to rethink the mechanism of discrete code generation in cross-modal hashing.

The paper is supported by the National Natural Science Foundation of China (62376106), The Science and Technology Development Plan of Jilin Province (20250102212JC). This work is 2606. p. 0 p. 000740. The project page is available at https://shuqiao-111. p. 0 p. 0github.io/. The corresponding author is Shengsheng Wang. p. 0 p. 0

The paper is supported by the National Natural Science Foundation of China (62376106), The Science and Technology Development Plan of Jilin Province (20250102212JC). This work is 2606.00741. The project page is available at https://shuqiao-111.github.io/. The corresponding author is Shengsheng Wang.

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Unified Spiking Framework for Hashing: SpikeHash reformulates hash code generation as an event driven process that integrates spiking state evolution, directional cross-modal modulation, and positive negative spike competition readout. This allows binary codes to be directly formed by neural dynamics rather than being a post processed output of continuous representations.

  2. Cross-Modal Interaction Enhancement: The Cross-Modal Spiking Gated Interaction Module (CMSGI) performs directional cross-modal modulation, generating fields like GT->I and GI->T that inject channel-conditioned information from the paired modality into the firing process to capture fine-grained complementary relations between modalities.

  3. Robust Binary Readout: Instead of using a conventional hash head composed of Linear, tanh, and sign operations, SpikeHash uses a positive-negative spiking hash head where each bit is produced by temporal competition between paired spike channels, ensuring the score used for training and the binary code used for inference are from the same process.

  4. Parameter Efficiency: SpikeHash achieves significant reductions in computational overhead, requiring only 2.19 million parameters and reducing operations by over 60% compared to dense ANN methods like UCMFH, making it suitable for low-power and neuromorphic deployment.

  5. Temporal Dimension Compensation: For low bit settings (like 16 bits), the system utilizes a longer time window (NT=8) to provide a richer dynamic representation space for hash codes with limited capacity, showing that the temporal dimension plays a compensatory role in low bit settings.

Sources

Related papers