Target-Clamped Contrastive Learning in Nonlinear Resistive Networks with Monotone Elements
math.OC
Submitted: 2026-11-18
Updated: 2026-11-18
License: http://creativecommons.org/licenses/by/4.0/
The gist: In the pursuit of energy-efficient hardware for machine learning, nonlinear resistive networks trained via local learning rules are emerging as promising alternatives to digital neural networks.
Terminology
Abstract
In the pursuit of energy-efficient hardware for machine learning, nonlinear resistive networks trained via local learning rules are emerging as promising alternatives to digital neural networks. Learning in these physical systems is often achieved with Equilibrium Propagation (EP), which compares a "free" state to a "nudged" target-driven state to perform gradient descent on an objective defined by an explicit cost function. In contrast, Target-Clamped Contrastive Learning (TCCL) "clamps" output nodes directly to exact target values, performing gradient descent on an implicit contrastive function - an objective whose properties have long remained unknown. Focusing on nonlinear networks composed of resistors with monotonically increasing current-voltage characteristics (such as Ohmic resistors and diodes), our contribution is twofold. First, we establish rigorous convergence guarantees for TCCL. We prove that the contrastive function is convex for nonlinear resistive networks without hidden nodes, and demonstrate that this property does not hold in general for the objective function optimized by EP. For these networks, TCCL converges to a global minimum of the contrastive function under suitable assumptions on the learning rate. In the presence of hidden nodes, the contrastive function is not convex, but we show that TCCL still converges to a stationary point on a subsequence. We also reveal cases where such stationary points are global minimizers of the contrastive function. Second, we characterize the representational capacity of these circuits. We establish a novel function approximation theorem demonstrating that any continuous, monotonically non-decreasing, and shift-equivariant function can be approximated to arbitrary precision, proving that these properties are the sole limitations on the network's computational expressivity.
Related papers
- An Accelerated Proximal Gradient Method for Bures-Wasserstein Space
- FOMTools: Development and AI-Assisted Code Generation for First-Order Methods
- Lions and Muons: Optimization via Stochastic Frank-Wolfe under Heavy-Tailed Noise
- Dynamic Service Recommendation with Congestion-Dependent Joining: Near-Optimal and Constant-Factor Approximation Algorithms
- Integrating Learning and Optimization for Open Transport Capacity Pooling under Disruption
- Zeroth-Order Gradient Estimators under Decision-Dependent Noise