Programmable Probabilistic Computer with 1,000,000 p-bits
summary
The gist
Programmable probabilistic computers built from p-bits have been proposed as hardware accelerators for sampling and optimizing Ising models…
In short
Researchers overcame hardware limits for probabilistic computers by networking FPGAs into a single machine with one million p-bits. This distributed system performs Gibbs sampling at high speeds while keeping coupling weights local, proving that performance depends on a communication ratio ($\eta$) rather than individual clock speeds.
Key concepts
- p-bits
- These are the fundamental programmable bits used in the proposed computer architecture. They serve as the basic units for performing probabilistic computations, allowing the system to model complex Ising models and optimize them.
- $\eta = fcomm/fp-bit$
- This dimensionless parameter measures how frequently boundary information must be refreshed compared to local updates. A large $\eta$ means frequent refreshing is needed, while a small $\eta$ indicates that boundary states can remain stale between exchanges.
- $C_{max}$
- This metric identifies the worst communication bottleneck in the physical interconnect. It represents the cluster pair that creates the tightest transport bottleneck, helping designers bound local update clocks based on this maximum cost.
Terminology used across episodes
This episode discusses
- Programmable Probabilistic Computer with 1,000,000 p-bits · Paper Radio
- How to Build a Quantum Supercomputer: Scaling from Hundreds to Millions of Qubits · Paper Radio
- The Future of Memory: Limits and Opportunities
- Next-generation Probabilistic Computing Hardware with 3D MOSAICs, Illusion Scale-up, and Co-design
- A Generalized Mean Field Algorithm for Variational Inference in Exponential Families
- KaHIP v3.00 -- Karlsruhe High Quality Partitioning -- User Guide
- Performance report of heuristic algorithm that cracked the largest Gset Ising problems (G81 cut=14060)
- Cosm: Collective Switched Motion for Fast and Accurate Sparse Ising Optimization
The paper
Programmable Probabilistic Computer with 1,000,000 p-bits · Read on arXiv
Navid Anjum Aadit, *Xiuqi Zhang, Shuvro Chowdhury, Kevin Callahan-Coray, Kyle Lee, Saleh Bunaiyan, Sanjay Seshan, Clayton Thomas, Jason Twigg, Andrew Seawright, *Forrest Brewer, Tathagata Srimani and Kerem Y. Camsarı
Department of Electrical & Computer Engineering, University of California, Santa Barbara · Electrical Engineering Department, King Fahd University of Petroleum & Minerals · Department of Electrical and Computer Engineering, Carnegie Mellon University · Siemens Digital Industries Software
Many problems in physics and machine learning require sampling the collective states of large numbers of interacting variables. On modern graphics processors, repeated movement of interaction data between memory and compute units can limit how efficiently these calculations can scale. Here we demonstrate a programmable probabilistic computer with one million interacting probabilistic bits (p-bits) across 18 field-programmable gate arrays, sustaining 10 12 probabilistic updates per second. Coupling weights remain beside the update circuits in on-chip memory, and only single-bit boundary states pass between devices. Experiments on three-dimensional and quasi-two-dimensional spin glasses show that the ratio of boundary-exchange frequency to local update frequency controls the effect of communication delays on optimization. A bound derived from the graph partition and interconnect identifies operating conditions that recover the residual-energy decay of an unpartitioned GPU reference. Infrequent exchange reduces the decay exponent, although faster local updates can shorten the time to reach a target energy. A cluster mean-field model reproduces this dependence, while ferromagnetic sampling experiments reveal a corresponding dependence of magnetization relaxation on the same frequency ratio. These results establish a million-p-bit computing architecture in which processing and local memory scale together, and provide a quantitative design principle for exploiting tolerance to communication delays in larger probabilistic computers.
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "Programmable Probabilistic Computer with 1,000,000 p-bits".
Paper summary: Tom: So we’re looking at this paper today, "Programmable Probabilistic Computer with one million p-bits <ref:2606.25313#pg1,Programmable Probabilistic Computer with 1,000,000 p-bits>." Basically it says we can take these probabilistic computers that are usually stuck on a single chip because of memory limits and make something huge.
Jane: That sounds ambitious. So what’s the main idea here? What is this paper actually claiming about what they built?
Tom: They’re proposing a way to network FPGAs together into one massive Ising machine. They claim this setup realizes a programmable probabilistic computer with one million p-bits, which is far bigger than any single device can handle.
Lu: It’s not just about making it bigger; they are tackling the fundamental question of how to make a partitioned machine behave like one big machine again. They introduce this key dimensionless parameter, eta equals fcomm divided by fp-bit, and how that ratio controls the performance.
Meng: So if I understand right, this whole system is designed for Gibbs sampling at over a trillion flips per second while keeping all those coupling weights in local on-chip memory across the network. What does that mean for deployment?
Lalam: It means they are doing the heavy lifting locally, and only sending small boundary information across devices. The core mechanism involves duplicating cut-edge weights as shadow weights on both sides of each partition so only one bit crosses a device boundary at a time <ref:2606.25313#pg2>.
Tom: That sounds like they’re trying to minimize the communication overhead. But what is the practical catch when you distribute computation like that? How does it actually perform compared to just running it on one big machine?
Jane: The paper explores a tradeoff here, showing that performance depends entirely on this timing ratio, eta equals fcomm divided by fp-bit. If eta is above some threshold, the distributed machine acts exactly like a single monolithic device.
Lu: Below that threshold, the residual energy still decays as a power law but with a reduced exponent. It turns parallelism into something you can measure as throughput versus accuracy <ref:2606.25313#pg3>.
Meng: A tradeoff? That’s what I care about when I’m thinking about running models in production. So, if eta is low, we get less quality for the effort of parallelizing it across FPGAs. How does that translate to actual cost?
Lalam: They define a communication-cost metric called Cmax which identifies the tightest bottleneck created by the physical interconnect when you actually place this design on hardware <ref:2606.25313#pg3>. This bounds the local update clock given by fp-bit less than fcomm squared times Ncolor <ref:2606.25313#pg3>.
Tom: Okay, so we have the ideal theoretical setup, but now we have physical constraints like interconnects and topology. How do they handle that in practice?
Jane: They use a topology-aware Potts partitioning strategy to help manage this cost <ref:2606.25313#pg2>. This function penalizes placing strongly connected partitions on devices that are far apart, which helps concentrate the cut edges at short hop distances.
Paper summary: Lu: That strategy avoids having to search through all possible partition orderings explicitly by aligning the partition with the existing chain topology to minimize worst-path congestion <ref:2606.25313#pg2>. Plus, they confirmed that testing with disconnected links showed that any slope loss comes from those boundary exchanges rather than problems with the local updates themselves.
Meng: So it seems like they solved the partitioning problem by letting the topology guide where we put the partitions, minimizing traffic jams on long wires. But what about the actual results? Did they prove this is good?
Lalam: They validated their findings on three dee Edwards–Anderson spin glasses <ref:2606.25313#pg3>. The optimization quality of a distributed run depends only on that eta ratio, not on how fast you run the clock or how slow you run the local updates individually <ref:2606.25313#pg3>.
Tom: That’s interesting because it means we don't need to tune the clock and the update speed separately as much as we thought. The fitted curves from their hardware implementations and cluster mean-field theory models collapse onto the same functional form after rescaling <ref:2606.25313#pg3>.
Jane: That collapse really confirms what they’re saying, which is that the main thing shaping this distributed system is indeed that delayed boundary information in partitioned stochastic dynamics <ref:2606.25313#pg3>. It shows the system isn't just random noise; it follows a predictable pattern related to how often those boundaries are refreshed relative to the local updates.
Lu: What this implies for distributed sampling is that we have a universal rule for quality control across different graph structures and hardware layouts, as long as you respect that eta ratio <ref:2606.25313#pg3>.
Meng: From a production standpoint, if I were deploying this idea, I’d focus on figuring out the Cmax metric to see where the biggest traffic bottlenecks are going to be when we map our specific graph onto the physical interconnect <ref:2606.25313#pg3>. That seems like a concrete first step for me.
Lalam: It means that for any graph, if you can manage those boundary exchanges efficiently according to the eta ratio, the scaling quality remains predictable regardless of how big the machine gets <ref:2606.25313#pg3>.
Tom: So we’re moving from just hoping it works on a big chip to having a principled way—using eta—to predict if our distributed setup will actually achieve the performance we need <ref:2606.25313#pg1>. That’s a step in the right direction for hardware acceleration.
Jane: It sounds like they took an abstract concept about scaling and turned it into a measurable engineering constraint that any distributed sampler needs to address <ref:2606.25313#pg2>.
Lu: And the paper addresses the practical issue of stale boundary information, which is a huge hurdle in distributed systems where you have to rely on neighboring partitions <ref:2606.25313#pg3>.
Meng: I just hope this doesn't introduce new complexities in the software layer for managing those boundary exchanges <ref:2606.25313#pg1>. Because if the communication management becomes too hard, the whole promise of scaling vanishes <ref:2606.25313#pg4>.
Lalam: Based on what’s written in this paper, they are quite confident that by managing that single timing ratio correctly, you get a solid performance guarantee for any topology you throw at it <ref:2606.25313#pg1>.
Conclusion: Tom: So we’re wrapping up our look at this paper on the programmable probabilistic computer with one million p-bits and what it means for scaling up these types of simulations.
Jane: The authors are showing how you can take these probabilistic computers, which are usually stuck on a single chip because of memory limits, and network them together into something much bigger.
Tom: It’s about using FPGAs to build one massive Ising machine that can do Gibbs sampling at over a trillion flips per second while keeping all the coupling weights local.
Lu: They introduce this idea of how often you have to refresh boundary information when you partition the machine, which they call eta equals fcomm divided by fp-bit.
Meng: That ratio basically tells you if your distributed setup is acting like one big machine or if it’s starting to break down because of communication lag.
Lalam: The key result is that performance only depends on that timing ratio, not how fast you run the clock or how slow your local updates are, as long as you respect eta.
Jane: So what does this mean for someone who just wants to understand the big picture? It suggests we have a universal rule for quality control in these distributed sampling systems based on that one ratio.
Tom: Exactly. They show that whether you’re running on a small setup or this million-bit machine, the scaling quality is determined by how well you manage those boundary exchanges.
Lu: It’s about taking an abstract idea and turning it into a concrete engineering constraint for building any distributed sampler.
Meng: If we can figure out that communication cost metric called Cmax, we can start predicting where the physical bottlenecks will be when we map our specific problem onto actual hardware.
Jane: That seems like a solid next step, moving from theory to figuring out the real-world cost of deployment across different physical layouts.
More episodes
- 2505.16254-Human-AI Writing as Distributed Cognition: Reassessing Collaborative Writing Theories in the Age of LLMs
- 2506.10913-Choreographic Quick Changes: First-Class Location (Set) Polymorphism
- 2507.14812-A Black-Box Approach for Exogenous Replenishment in Online Resource Allocation
- 2507.15448-Frames over finite fields with respect to the Galois inner product and their connections to self-orthogonal and LCD codes
- 2508.21572-NewsReX: An Open-Source Multi-Framework for Neural News Recommendation
- 2509.17028-Impact of Packetization on Network Calculus Analysis
- 2510.17342-AoA Services in 5G Networks: A Framework for Real-World Implementation and Systematic Testing
- 2511.04088-Efficient and rate-optimal list-decoding in the presence of minimal feedback
- 2511.17264-Notes on Stack Machines and Quantum Stack Machines
- 2601.15318-On the closest balanced game