Scaling Up Analog CIM: A Simulation Framework for Topology-Aware Mapping
Listen
Radio episode about this paper
Transcript
Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.
Tom: We're looking at the paper: "Scaling Up Analog CIM".
Jane: Here's the gist of the paper. Analog compute-in-memory (CIM) enables energy-efficient AI inference by performing in-situ matrix multiplication Scaling Up Analog CIM: A Simulation Framework for Topology-Aware Mapping is a simulation platform that quantifies how mapping topology affects system performance across hardware configurations and how mapping optimization translates into latency gains.
Tom: System Modeling and Mapping Space The paper develops Isobar, a simulation platform integrating CIM decompositions, heterogeneous execution, and resource modeling to evaluate topology-aware mapping in analog CIM chiplet systems.
Jane: The mapping space is formulated according to the CIM hardware hierarchy, considering four levels: package, die, macro, and crossbar array.
Tom: A candidate deployment is represented as a triple p = (γ, α, β), where γ partitions the model’s CIM-mapped linear operators into groups Gk.
Paper summary: Tom: So we're looking at this paper called "Scaling Up Analog CIM: A Simulation Framework for Topology-Aware Mapping".
Jane: It looks like they are trying to figure out how mapping the computations on these analog compute-in-memory systems affects how fast they run as you scale up.
Lu: Exactly. The authors are developing a simulation platform called Isobar to evaluate this mapping, looking at it across different hardware setups and seeing how optimizing that mapping actually gives you latency gains > Tom Which is interesting because most simulations just look at local costs or aggregate communication metrics, and this paper argues that ignoring the weight-mapping topology hides the real impact when these systems get bigger.
Meng: From an engineering standpoint, what's important is if you can reliably predict how a change in where you put things on the chip affects performance across different hardware configurations.
Lalam: I think it's about making sure that when we design these massive AI models, we aren't just picking a configuration that looks good locally but one that actually works well when the whole system is running together.
Tom: Right, so they are creating this simulation framework to quantify how mapping topology affects latency across hardware settings. It sounds like they are trying to map out this design space very thoroughly before we commit to physical layouts.
Jane: And they do that by looking at a hierarchy of levels, from the package down to the crossbar array.
Lu: They define the mapping space based on how operators are grouped into different sets, called groups Gk, and then assign those logical die instances to slots in the package and macro instances to physical die slots. That’s a triple p equals gamma alpha beta defining this entire mapping space F > Tom So they are setting up a very formal way to describe all the possible ways you can map things onto the hardware.
Meng: And they quantify execution cost by combining communication, computation, and memory-access costs to see how topology affects latency across different hardware settings. That combination is key because we don't just look at one part of the process in isolation when things are running in real-time.
Lalam: They define a specific inter-die placement comparison cost R s equal to RCIM s plus Rdigital plus RKV plus Rother, where that covers CIM execution and digital computation and KV memory service and other processing.
Jane: That sounds like they’re breaking down the total time into these specific components so we can see exactly what part of the delay is due to where things are placed versus how much it takes to actually do the work.
Paper summary: Tom: And then they estimate communication time H p C based on how many bytes get routed through shared resources under a specific mapping p. That helps us see how traffic flows when we try out different configurations.
Lu: They also introduce an intra-die boundary b beta s, which marks where communication service can be hidden by computation for every co-resident operator at the same time. This boundary is important because it tells you where you might be able to hide communication delays by running computations in parallel with the data movement > Meng So that means if two things are close enough on the same die, they don't have to wait for each other as much.
Tom: They use a phase diagram to quantitatively reveal how mapping topology impacts system performance, which allows them to compare deployments whose differences might be hidden by older metrics. That suggests they found a way to visualize trade-offs that weren't obvious before.
Jane: And they show that the boundary envelope B minus s and B plus s identify where physical assignment becomes critical when communication dominates either intra-die or package level execution.
Lu: They find the ratio between the largest and smallest boundary bandwidths is two point sixty-seven times across seventy intra-die macro assignments, and four point seventeen times across six Gemma seven B inter-die mappings > Tom That factor of four for those inter-die mappings is pretty significant.
Meng: So, what does this mean for me in production? If communication dominates at the package level, that physical assignment becomes very critical because you can't just move things around anymore; you have to be precise about where they sit.
Lalam: They also mention that their optimization uses a hierarchical dynamic programming simulated annealing search over the deployment choices to find these best mappings.
Jane: And the validation part is strong; they validated their predictions against silicon measurements and BookSim simulations, reproducing measured latency increases with chiplet separation with only five point one percent relative error in the latency ratio between most separated and adjacent placements > Tom That level of accuracy, especially when you're dealing with physical hardware, sounds really solid <ref:2611.08399#pg1>.
Lu: The optimization guided by Isobar reduced communication time by twenty-three point three percent and end-to-end latency by eighteen point six percent relative to the strongest baseline for Gemma seven B prefill > Meng That is a big reduction in latency when you are trying to optimize the mapping.
Tom: And they show that analog CIM still has energy efficiency potential at large language model scale, with full request dynamic energy estimates being well below published GPU measurements > Jane So even if we are just looking at these analog systems, they retain a lot of the energy efficiency benefit we expect from them for LLMs.
Paper summary: Lu: They controlled comparisons across capacity, matrix decomposition, and resource configurations to show how those choices shape communication costs and mapping benefits > Meng It really emphasizes that topology-aware mapping isn't just an academic exercise; it’s a key design dimension for efficient execution in these systems.
Jane: So, the main idea of this paper is providing a configurable framework, Isobar, to evaluate and optimize those topology-aware mappings in large scale analog CIM systems > Tom It shows that accounting for shared resource contention actually improves the selection of the best mapping strategy.
Lu: The results establish that topology-aware mapping is an integral part of CIM architecture design when you consider existing hardware capabilities > Meng And they pointed out a limitation, which is that the framework itself still depends on how well you model those initial CIM decompositions and resource costs accurately to get those reliable cost estimates > Jane Which means the accuracy of the final mapping search depends on the quality of your input models.
Tom: That makes sense. It’s not a magic fix; it’s a very thorough way to search that complex design space using simulation before you go into expensive silicon runs > Lu And it shows that even with these complexities, we can get much better performance by being deliberate about the mapping from the start.
Jane: So, looking at this paper, "Scaling Up Analog CIM: A Simulation Framework for Topology-Aware Mapping", it seems like the authors are giving us a very detailed map of how to choose where to put computations when you're scaling up analog compute-in-memory systems > Tom It’s about moving beyond just local costs and seeing how the whole system interacts across hardware layers.
Lu: And the implication is that for future designs, we absolutely need this kind of simulation platform if we want to get the best performance out of these large scale analog CIM chips > Meng Because it clearly shows that topology-aware mapping is a core part of how you build these things efficiently.
Jane: So, to wrap up this discussion on "Scaling Up Analog CIM: A Simulation Framework for Topology-Aware Mapping", the paper gives us a tool to systematically evaluate mapping decisions across different hardware configurations and hardware settings > Tom It’s about making sure that the physical placement of computation is optimized against communication and memory access costs simultaneously.
Lu: And they show that this optimization leads to measurable latency reductions, like eighteen point six percent relative to a strong baseline for Gemma seven B prefill > Meng So, if you're designing these systems today, this framework helps you make smarter decisions about the architecture itself.
Conclusion: Tom: So we're wrapping up this session on "Scaling Up Analog CIM: A Simulation Framework for Topology-Aware Mapping". Jane It boils down to them building a simulation tool called Isobar that lets you test different ways to map computations onto analog compute-in-memory systems at a huge scale.
Lu: Yeah, they are really digging into how the physical layout of those components affects how fast the AI runs when you move up in size.
Meng: I just want to know if this framework can actually predict what breaks when you try to deploy something like this in production.
Lalam: The authors use this simulation to prove that if you map things smartly based on their hardware structure, the latency drops significantly because it handles communication and memory access better.
Jane: So, what's the big picture here? Tom What does this whole study mean for people who are actually building these massive AI chips?
Tom: It means they can finally test these complex physical layouts before you even spend money on actual silicon prototypes.
Lu: They showed that accounting for where things sit—like in the package or the die—is a necessary part of the design process, not just an afterthought.
Meng: So if we want to build something that doesn't bottleneck at scale, we need to use this kind of planning simulation first?
Jane: Exactly. The paper shows that figuring out these topology-aware mappings is a core part of how you design efficient systems for large language models.
Yuxin Ren, Danella Zhao, Huanrui Yang
University of Arizona
cs.AR
Submitted: 2026-11-05
Updated: 2026-11-05
Code: https://github.com/sandialabs/cross-sim
License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/
Importance score: 89/100
The gist: Analog compute-in-memory (CIM) enables energy-efficient AI inference by performing in-situ matrix multiplication Scaling Up Analog CIM…
Key concepts
- Isobar
- A simulation platform that integrates CIM decompositions, heterogeneous execution, and resource modeling. It evaluates how different physical mappings of operations across chiplets impact system performance in analog CIM systems.
- Mapping Space F
- The set of all possible ways to assign logical die instances to package slots and macro instances to physical die slots. This space is structured according to the CIM hardware hierarchy, allowing researchers to systematically test various deployment configurations.
- Execution Cost Quantification
- A method used to measure total latency by combining communication, computation, and memory-access costs. It specifically calculates inter-die placement cost and estimates communication time based on shared resource routing under a given mapping.
- Topology-Aware Mapping
- The process of strategically assigning logical operations to physical hardware locations (like die or package slots) considering the underlying hardware structure. This optimization aims to minimize performance bottlenecks by accounting for how operations interact across different levels of the CIM architecture.
Terminology
Summary
Analog compute-in-memory (CIM) enables energy-efficient AI inference by performing in-situ matrix multiplication Scaling Up Analog CIM: A Simulation Framework for Topology-Aware Mapping is a simulation platform that quantifies how mapping topology affects system performance across hardware configurations and how mapping optimization translates into latency gains. p. 0
System Modeling and Mapping Space
The paper develops Isobar, a simulation platform integrating CIM decompositions, heterogeneous execution, and resource modeling to evaluate topology-aware mapping in analog CIM chiplet systems. The mapping space is formulated according to the CIM hardware hierarchy, considering four levels: package, die, macro, and crossbar array. p. 1 p. 1 A candidate deployment is represented as a triple p = (γ, α, β), where γ partitions the model’s CIM-mapped linear operators into groups Gk. p. 1 p. 1 The assignment of logical die instances to package slots and macro instances to physical die slots defines the mapping space F. p. 1 p. 1
Execution Cost Quantification
The paper combines communication, computation, and memory-access costs to quantify how topology-aware mapping affects latency across hardware settings. Inter-die placement comparison cost is defined by R(s) = RCIM(s) + Rdigital + RKV + Rother. p. 2 p. 2 Communication time estimate H(p; C) is calculated based on the total bytes routed through shared resources under mapping p. p. 2 p. 2 The intra-die boundary b(β; s) marks where communication service can be hidden by computation for every co-resident operator. p. 2 p. 2
Phase Analysis and Design Implications
The paper uses a phase diagram to quantitatively reveal the impact of topology-aware mapping, enabling comparison of deployments whose differences are hidden by previous metrics. The boundary envelope B−(s) and B+(s) identifies where physical assignment becomes critical as communication dominates intra-die or package-level execution. p. 0 p. 0 The ratio between the largest and smallest boundary bandwidths is 2. p. 2 p. 267× across 70 intra-die macro assignments and 4. p. 2 p. 217× across six Gemma-7B inter-die mappings. p. 2 p. 2
Optimization and Validation
Isobar implements a hierarchical dynamic programming–simulated annealing (DP–SA) search over the deployment choices to optimize topology-aware mapping. The framework validates its predictions against silicon measurements and BookSim simulations, reproducing measured latency increases with chiplet separation with only 5. p. 41% relative error in the latency ratio between most separated and adjacent placements. p. 4 Isobar-guided mapping reduces communication time by 23. p. 03% and end-to-end latency by 18. p. 06% relative to the strongest baseline for Gemma-7B prefill. p. 5
Energy Efficiency Potential
The simulation shows that analog CIM retains energy-efficiency potential at LLMs scale, with full-request dynamic-energy estimates well below published GPU measurements. Controlled comparisons across capacity, matrix decomposition, and resource configurations reveal how these choices shape communication costs and mapping benefits. p. 1 p. 1 The results establish topology-aware mapping as a key design dimension for efficient execution. p. 0 p. 0
Conclusion
Isobar provides a configurable framework for evaluating and optimizing topology-aware mappings in largescale analog CIM systems. The phase and configuration studies establish the importance of topology-aware mapping under existing hardware capabilities and identify how capacity, resource organization, and workload govern its system benefits. p. 7 The results demonstrate that accounting for shared-resource contention improves topology-aware mapping selection. p. 5 The paper concludes that topology-aware mapping is an integral part of CIM architecture design. p. 7
Pruning redundant reasoning tokens
The paper develops Isobar, a simulation platform integrating CIM decompositions, heterogeneous execution, and resource modeling to evaluate topology-aware mapping in analog CIM chiplet systems. The mapping space is formulated according to the CIM hardware hierarchy, considering four levels: package, die, macro, and crossbar array. A candidate deployment is represented as a triple p = (γ, α, β), where γ partitions the model’s CIM-mapped linear operators into groups Gk. The assignment of logical die instances to package slots and macro instances to physical die slots defines the mapping space F.
How the training signal is built
The paper combines communication, computation, and memory-access costs to quantify how topology-aware mapping affects latency across hardware settings. Inter-die placement comparison cost is defined by R(s) = RCIM(s) + Rdigital + RKV + Rother. Communication time estimate H(p; C) is calculated based on the total bytes routed through shared resources under mapping p. The intra-die boundary b(β; s) marks where communication service can be hidden by computation for every co-resident operator.
Where the gains come from
The paper uses a phase diagram to quantitatively reveal the impact of topology-aware mapping, enabling comparison of deployments whose differences are hidden by previous metrics. The boundary envelope B−(s) and B+(s) identifies where physical assignment becomes critical as communication dominates intra-die or package-level execution. The ratio between the largest and smallest boundary bandwidths is 2.67× across 70 intra-die macro assignments and 4.17× across six Gemma-7B inter-die mappings.
Energy Comparison with GPUs
The simulation shows that analog CIM retains energy-efficiency potential at LLMs scale, with full-request dynamic-energy estimates well below published GPU measurements. Controlled comparisons across capacity, matrix decomposition, and resource configurations reveal how these choices shape communication costs and mapping benefits. The results establish topology-aware mapping as a key design dimension for efficient execution.
Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.
Improvements for AI systems
-
Topology-aware mapping for analog CIM systems will reduce communication time by 23.3% and end-to-end latency by 18.6% relative to the strongest baseline for Gemma-7B prefill, as shown in Isobar simulations.
-
The system can perform a phase analysis that quantifies
how strongly mapping topology affects system performance across hardware configurations and how mapping optimization translates into latency gains.
-
The AI system will be able to utilize Isobar to guide mapping search through incremental evaluation, enabling millisecond-scale evaluation and second-scale search with thousands of dies.
-
The system can accurately estimate communication costs using the formula H(p; C) = Xj / Pj (p) + max r in Rj Vjr(p) / Cr, which is then used to construct a latency and dynamic-energy model.
-
The system will be able to identify
where physical assignment becomes critical as communication dominates intra-die or package-level execution
by analyzing the boundary envelope B+(s) and B-(s). -
The AI system can select optimal mappings by comparing metrics like Isobar, Word-hops, and Max-hops, which
outperforms Max-hops in all three workloads
based on BookSim communication times. -
The system can optimize resource organization by assigning attention head groups to separate units to
disperse communication while preserving the aggregate throughput.
-
The AI will be able to make capacity choices that account for communication patterns produced by matrix decompositions, recognizing that
Capacity consolidation removes competing handoff flows in OPT-1.3B, whereas wide fan-out handoffs retain concentrated contention in Gemma-7B.
Related papers
- Embedded Arena: Iterative Optimization via Hardware Feedback
- WitCert: Sound Runtime Risk Observability and Gating for KV-Cache Quantization
- SparsePixels: Efficient Convolution for Sparse Data on FPGAs
- Golden Ruler: A Numeric Format Catalog with Bit-Exact Conformance Vectors for FP8, BF16, MXFP4, and Microscaling Formats
- AMMA: A Multi-Chiplet Memory-Centric Architecture for Low-Latency 1M Context Attention Serving
- RFChipAgent: Multi-Agentic AI Flow for Analog/RF Chip Design