Product Interaction: An Algebraic Formalism for Deep Learning Architectures
Product Interactions propose an algebraic method to unify modern neural network architectures by viewing layers as compositions of multiplication operators. This framework systematically generates networks by increasing the 'interaction order'—the complexity of how data elements multiply together—revealing that CNNs, attention, and Mamba correspond to specific interaction orders.