LocalTrain: Integer-Only Local Learning for Energy-Efficient CNN Training on Edge FPGAs

The gist The KV260 prototype demonstrates complete integer local CNN training with shared convolutional and classifier arithmetic, achieving 76.57% accuracy on CIFAR-10 and 91.85% on SVHN at approximately 4.20 W mean module power.

How it works

The system employs a shared integer matrix engine to train both a CNN backbone and its local classifiers using integer-only learning operations. This architecture maps convolutional and local-learning computations to this shared engine, avoiding a separate matrix engine for the local classifiers. The training process is structured such that it completes one layer before advancing, allowing saved input storage to be reused after gradient computation.

The core components of the training involve several key mechanisms:

** Local supervision through a trainable classifier attached to each convolutional layer removes cross-layer error propagation but introduces the computations needed to train each classifier and generate errors for its associated layer. The design reduces data that must be expanded, buffered, or accumulated at once by keeping regional errors compact in DDR and expanding them only for convolutional weight gradients. Tile-wise folding bounds on-chip gradient storage while scales selected from earlier batches allow direct INT32-to-INT8 output conversion after warmup. The local classifier feature errors are computed using preupdate weights, denoted as EF = EVpre. These errors train the associated convolutional layer without propagating through earlier backbone layers. The architecture supports eight 8 × 8 INT8 compute units to handle convolutional and classifier computations, including score generation, errors for the associated convolutional layers, and weight gradients. The shared engine performs 512 MACs per cycle with a theoretical peak of 102.4 GMAC/s at 200 MHz. The hardware organization shows that convolutional and classifier operations reuse the matrix engine. The regional error expansion broadcasts integer codes without performing a division at each spatial position by reducing their shared exponent by log2 A before convolutional weight-gradient computation. This path avoids full spatial error-map writes and reads but still requires compact-error and activation accesses. The architecture folds partial gradients one tile at a time to obtain the complete gradient before GB5 updates. This folding sums corresponding 8 × 8 tiles using a 48-bit accumulator with 384 bytes of logical storage. The final classifier uses flattened features and a batch-shared dropout mask with keep probability 0.5, doubling retained values. The architecture maintains saved inputs and pre-update classifier weights until their final gradient-related reads. This procedure creates three hardware requirements: (1) saved inputs and pre-update classifier weights must remain available until their final gradient-related reads; (2) GB5 updates require the complete batch-gradient range; and (3) PS controller batches, RNG, history DDR: parameters and training tensors,page4>. The output path uses a preselected historical shift to make the conversion scale available without waiting for current-output statistics. After the first eight batches, each completed INT32 convolutional output reaches two units: conversion applies this shift while the histogram observes the raw code before rounding and saturation. The direct conversion avoids 448 MiB of raw-output write/read payload per full batch after warmup. This comparison shows a 50.0% reduction in cycles and 2 MiB less transfer payload for direct conversion versus full-tensor staging and replay. The reserved DDR workspace remains 161.22 MiB, containing features, error buffers, raw outputs, partial gradients, and other state. This logical capacity and payload counts are distinct from peak live storage and measured DDR traffic. The architecture supports the GB5 normalization using a five-bit magnitude scale to produce partial convolutional gradients with matching exponents. These gradients are accumulated before updating weights without aligning the gradient exponent to the fixed weight exponent. The final classifier uses a higher-resolution master state initialized as M = 16W and represents weights at exponent eW − 4. The update calculation involves learning-rate shift λt and d = eG −eW + 4−λt, resulting in ∆ = clip[−2032,2032] RN(g2 d). The final weight update is W+ = sat8(W − g), without aligning the gradient exponent to the fixed weight exponent. The architecture supports scheduled freezing to reduce later training work by skipping selected layers’ local classifiers, gradients, and updates. This procedure creates three hardware requirements: (1) saved inputs and pre-update classifier weights must remain available until their final gradient-related reads; (2) GB5 updates require the complete batch-gradient range; and (3) PS controller batches, RNG, history DDR: parameters and training tensors,page4>. The effective matrix throughput is 31.22 GMAC/s on CIFAR-10 and 31.57 GMAC/s on SVHN. This aggregate effective throughput is 30.49% and 30.83% of the 102.4 GMAC/s peak. The measured training energy is 40.4% and 42.1% lower than the same integer method on Jetson, with runtime being 1.86× and 1.82× longer under the measurement boundaries in Section IV. The final results show that integer local reduces energy relative to FP32 BP by 23.8% on CIFAR-10 and 27.0% on SVHN, with accuracy reductions of 3.30 and 0.53 percentage points. The KV260 prototype demonstrates complete integer local CNN training with shared convolutional and classifier arithmetic. This architecture utilizes compact errors, tile folding, and historical scales to reduce intermediate representation and buffering requirements. This system achieves 31.22–31.57 GMAC/s effective matrix throughput with approximately 4.20 W mean module power. The current evidence covers one FPGA seed, logical storage savings, and isolated conversion tests. This paper concludes by stating that future work will measure board-level mechanism ablations and build-matched memory and result-stall breakdowns. The software comparison shows that integer local reduces energy relative to FP32 BP by 23.8% on CIFAR-10 and 27.0% on SVHN. This method achieves higher accuracy and 25.8% lower peak CUDA allocation compared to Local FP32, but takes 2.10× and 2.25× longer. The integer History policy shows that history reduces peak CUDA allocation from 345.57 to 297.83 MiB. This study demonstrates that freezing affects both workload and learning outcome, with scheduled freezing reducing training time by 28.4% on CIFAR-10 and 32.3% on SVHN. The KV260 prototype demonstrates complete integer local CNN training with shared convolutional and classifier arithmetic. This architecture utilizes compact errors, tile folding, and historical scales to reduce intermediate representation and buffering requirements. This system achieves 31.22–31.57 GMAC/s effective matrix throughput with approximately 4.20 W mean module power. The current evidence covers one FPGA seed, logical storage savings, and isolated conversion tests.

1. No cross-layer error propagation is introduced by mapping computations to a shared integer matrix engine, as stated: This removes cross-layer error propagation but introduces the computations needed to train each classifier and generate errors for its associated layer. This allows for local supervision without waiting for errors from later layers.

2. Regional error storage is significantly reduced compared to traditional methods; For the evaluated network at batch size 256, regional errors require 64–256 KiB per layer, compared with 8–16 MiB for expanded INT8 spatial error maps. This compact error handling minimizes intermediate storage and memory traffic.

3. Historical quantization scales enable direct output conversion without waiting for current statistics: Historical scales permit direct output conversion. This speeds up the training pipeline by avoiding external memory reads and writes after warmup.

4. The architecture supports complete gradient accumulation through tile-wise folding, reducing required storage: Tile accumulation limits on-chip accumulator capacity, is managed by folding partial gradients into a 384-byte accumulator instead of requiring a resident 48-bit accumulator would require 432 KiB of logical storage.

5. The system achieves energy efficiency gains over GPU implementations: Against the same integer method on Jetson Orin Nano, measured training energy is 40.4% and 42.1% lower. This enables on-device training for resource-constrained edge devices with reduced power consumption.

6. The shared engine provides high effective throughput: Effective matrix throughput is 31.22 and 31.57 GMAC/s, which demonstrates a substantial utilization of the hardware resources during training.