SplitCom: Communication-efficient Split Federated Fine-tuning of LLMs via Temporal Compression

arXiv:2602.10564 · cs.NI · Submitted 2026-02-11 · Read on arXiv

Listen

Radio episode about this paper

Transcript

Introduction to the show: ident: Computer Science. The day in computer science: software, systems and theory on arXiv.

Tom: We're looking at the paper: "SplitCom: Communication-efficient Split Federated Fine-tuning of LLMs via Temporal Compression".

Jane: Here's the gist of the paper. Federated fine-tuning of on-device large language models (LLMs) mitigates privacy concerns by preventing raw data sharing However, the intensive computational and memory demands pose significant challenges for resource-constrained edge devices To overcome these limitations, split federated learning (SFL) emerges as a promising solution that partitions the model into lightweight client-side and compute-intensive serverside sub-models, thus offloading the primary training workload to a powerful server.

Tom: Nevertheless, the high-dimension activation exchanges in SFL lead to excessive communication overhead.

Jane: To overcome this, we propose SplitCom, a communication-efficient SFL framework for LLMs that exploits temporal redundancy in activations across consecutive training epochs.

Tom: The core innovation of the framework Inspired by video compression, the core innovation of our framework lies in selective activation uploading only when a noticeable deviation from previous epochs occurs.

Paper summary: Tom: So, wrapping up on SplitCom: Communication-efficient Split Federated Fine-tuning of LLMs via Temporal Compression. The authors showed that by using temporal compression inspired by video techniques, they can significantly reduce the communication cost in split federated learning for large language models.

Jane: They achieved this through a similarity-aware activation reuse mechanism that selectively uploads activations based on whether they have changed much since the last training epoch. They tested this with GPT two models and found substantial reductions in total communication costs, like up to ninety-five point eight percent in the U-shape variant.

Lu: The key insight they highlighted is that parameter-efficient fine-tuning causes only tiny changes to the client side submodels over time, leading to highly correlated activations between adjacent epochs. This correlation is what makes temporal compression work effectively for saving bandwidth.

Meng: For deployment, it means we can potentially use these larger models on edge devices because the training phase doesn't bottleneck on data transfer as much anymore. But they did mention that the method relies on observing this slow evolution, so if the model updates change very rapidly, their compression strategy might not be as effective.

Tom: That’s a fair point about the rate of change. The paper focuses heavily on demonstrating that this framework makes on-device LLM application practical under stringent computational and communication constraints because it tackles those frequent client to server exchanges head-on.

Jane: Ultimately, SplitCom offers a way for privacy-preserving federated fine-tuning to become more accessible by making the resource demands manageable through clever temporal data reuse.

Conclusion: Tom: So, we're wrapping up on SplitCom, which is about making federated fine-tuning of large language models work better on little devices by cutting down on data transfer.

Jane: It sounds like they took this problem where you have huge models and limited memory on edge devices and tried to fix the communication bottleneck that comes with splitting the training between clients and a central server.

Lu: Exactly. The big issue in split federated learning is that all those activations, all those intermediate calculations going back through the layers, just create too much traffic when you send them around.

Meng: From my side, it’s about deployment cost at scale. If every training round requires moving massive activation data back and forth constantly, the whole process grinds to a halt on resource-constrained hardware.

Lalam: I'm looking at the core idea here—they propose using temporal compression, which is basically like video compression where you only send a frame if it’s significantly different from the last one.

Tom: So they're saying that because the model doesn't change drastically between training steps, we can intelligently skip sending most of those activations and just send the important stuff.

Jane: That’s what they call selective activation uploading, and they use a similarity-aware mechanism to decide which activations are worth sending based on how much they deviate from previous epochs.

Lu: They even introduced two ways to control this skipping—one is a simple rule-based thing called bang-bang control, but the other uses deep deterministic policy gradient reinforcement learning to learn the best way to balance speed versus efficiency.

Meng: I’m interested in that learning part. If an AI agent can figure out the optimal threshold dynamically, that means we don't have to tune it manually for every single model.

Tom: The results they show are pretty impressive too—they claim a reduction in uplink communication costs of up to ninety-five point eight percent in their U-shape configuration without hurting the final model performance much.

Jane: It’s a big win because it shows that you can maintain good learning quality while drastically cutting down on the massive bandwidth required for these kinds of AI tasks.

Lu: The way they did this extension to the U-shape architecture is clever because it ensures the central server never even sees the clients' labels, which adds another layer of privacy protection.

Meng: So what does this actually mean for putting these kinds of language models on things like phones or small robots? It sounds like it makes practical deployment much more feasible than before.

Tom: It definitely moves us closer to making large language model applications really accessible outside of super powerful data centers.

Jane: But they also pointed out that the success hinges on the idea that these models evolve slowly during training, which is a real constraint we have to keep in mind for future work.

University of Hong Kong

cs.NI

Submitted: 2026-02-11

Updated: 2026-11-10

License: http://arxiv.org/licenses/nonexclusive-distrib/1.0/

Importance score: 90/100

The gist: Federated fine-tuning of on-device large language models (LLMs) mitigates privacy concerns by preventing raw data sharing However…

Key concepts

Split Federated Learning (SFL)
SFL partitions a large LLM into smaller parts: lightweight client-side models and compute-intensive server-side sub-models. This setup allows for privacy by keeping raw data on devices, but it requires frequent communication between the client and server during training.
Inter-epoch Temporal Compression
This technique mimics video compression by only sending activation data to the central server if there is a noticeable difference compared to previous training epochs. It ensures that redundant information across consecutive steps is reused, drastically lowering the amount of data transmitted over the network.
Similarity-aware Activation Reuse Mechanism
Clients keep a local cache of intermediate activations. A new activation is only sent to the server if its similarity to previously cached activations falls below a certain threshold. This mechanism intelligently decides when to transmit, maximizing communication efficiency by avoiding unnecessary data uploads.

Terminology

Summary

Federated fine-tuning of on-device large language models (LLMs) mitigates privacy concerns by preventing raw data sharing However, the intensive computational and memory demands pose significant challenges for resource-constrained edge devices To overcome these limitations, split federated learning (SFL) emerges as a promising solution that partitions the model into lightweight client-side and compute-intensive serverside sub-models, thus offloading the primary training workload to a powerful server. p. 0 Nevertheless, the high-dimension activation exchanges in SFL lead to excessive communication overhead. p. 0 To overcome this, we propose SplitCom, a communication-efficient SFL framework for LLMs that exploits temporal redundancy in activations across consecutive training epochs. p. 0

The core innovation of the framework

Inspired by video compression, the core innovation of our framework lies in selective activation uploading only when a noticeable deviation from previous epochs occurs. We introduce inter-epoch temporal compression, a module analogous to interframe compression techniques in video compression. p. 1 Concretely, our framework implements a similarity-aware activation reuse mechanism to boost communication efficiency: Each client maintains a local cache of intermediate activations and only transmits a newly computed activation to the central server if notable deviations from previous epochs are detected. p. 1

Adaptive control strategies

To balance communication efficiency and learning performance, we introduce two adaptive threshold control schemes based on 1) bang-bang control or 2) deep deterministic policy gradient (DDPG)-based reinforcement learning. The former is rule-based, which is simple to implement, whereas the latter is learning-based. p. 1 Specifically, the DDPG agent learns a continuous policy to dynamically adjust θ by maximizing a reward function that balances model performance and communication efficiency. p. 4

Dimensionality reduction techniques

We implement dimensionality reduction techniques, such as random projection (RP) [12], to minimize the storage requirements for cached intermediate activations and gradients on clients. We adopt RP for its computational efficiency and suitability for resource-constrained edge devices because it relies solely on matrix multiplication and is mathematically proven to preserve pairwise cosine similarity with high probability. p. 3

U-shape architecture extension

We extend SplitCom to the U-shape architecture, ensuring the server never accesses clients’ labels. This extension enables temporal compression on both activations and gradients, which reduces bidirectional communication costs while ensuring that clients’ labels are never exposed to the server. p. 1 We apply the similarity-aware reuse mechanism symmetrically across all four transmission boundaries in this configuration. p. 5

Experimental validation

Extensive simulations and laboratory experiments demonstrate that SplitCom reduces uplink communication costs by up to 98.6 % in its standard configuration and total communication costs by up to 95. p. 08 % in its U-shape variant without noticeably compromising model performance. p. 0 The results show that the U-shape SplitCom with adaptive threshold control (BBC/DDPG) consistently reduces total communication overhead by 77–80% for GPT-2 XLarge and 84–90% for GPT-2 Small compared to the U-shape SplitLoRA baseline. p. 9 The results support our core insight: LoRA induces minimal updates to clientside sub-models, leading to stable activations across training epochs. p. 9 The results demonstrate that while U-shape SplitCom naturally exhibits asymmetric sparsity (higher gradient sparsity), larger models possess greater robustness to the activation sparsity induced by INT8 quantization compared to their smaller counterparts. p. 6 The paper identifies that existing SFL frameworks suffer from communication bottlenecks due to frequent client–server exchanges. p. 2 The authors conclude that SplitCom, in both its standard and privacy-enhanced U-shape configurations, makes on-device LLM application practical under stringent computational and communication constraints. p. 0

Green — the paper says it, and the page is printed. Red — no passage carries it, so treat that sentence as unverified. The same check runs on your own text: check a summary against its source, $5.

Improvements for AI systems

  1. Temporal Compression for Activations: The system can exploit temporal redundancy in activations across consecutive training epochs by only uploading a newly computed activation to the central server if a noticeable deviation from previous epochs occurs, analogous to video compression.

  2. Adaptive Threshold Control: Implement two adaptive strategies, bang-bang control or deep deterministic policy gradient (DDPG)-based reinforcement learning, to determine the similarity threshold, balancing communication efficiency and model performance by tracking validation perplexity.

  3. Memory Reduction via Dimensionality Reduction: Employ dimensionality reduction techniques like Random Projection (RP) instead of Principal Component Analysis (PCA) to minimize client-side memory requirements for cached intermediate activations and gradients, as RP is mathematically proven to preserve pairwise cosine similarity with high probability.

  4. Privacy-Preserving U-shape Architecture: Extend the framework to a U-shape architecture that relocates loss computation to clients, enabling temporal compression on both activations and gradients, ensuring that clients’ labels are never exposed to the server by establishing four distinct communication interfaces.

  5. Bidirectional Communication Compression: The system can be extended to compress both uplink activations and downlink gradients by maintaining separate cache libraries for each direction and applying independent similarity checks with their own thresholds, allowing the server to reuse cached activation abar jsi for server-side forward and backward passes.

Sources

Related papers