UNISON: A Unified Framework for Learning from Scored Bags

The gist: UNISON introduces a unified framework that bridges multiple instance learning and recommendation systems by treating both as instances of learning from scored bag collections of (item, score) pairs.

How it works

UNISON decomposes learning into three interconnected stages: (1) learning universal item representations, (2) modeling bag-specific preference patterns through lightweight personalized parameters, and (3) inferring bag-level characteristics from learned preferences This unified framework treats both multiple instance learning and recommendation systems as instances of learning from scored bags of (item, score) pairs. The relation between items and their scores is determined by bag-specific latent variables, which can be used to classify the entire bag.

Key Objectives and Stages

The framework addresses three interconnected objectives: (1) Item Representation, (2) Item Scoring, and (3) Bag Classification.

Objective 1 is to learn an encoding function f: X → R d that maps items to fixed-dimensional embeddings z = f(x).

Objective 2 is to predict the score h: B × X → S, sˆ∗j = h(Bj, x∗), given a scored bag Bj and candidate item x∗.

Objective 3 is to predict a global label yj ∈ Y characterizing the entire bag g: B → Y, yˆj = g(Bj).

The training consists of three stages that map directly to these objectives:

Stage 1 (Item Embedding): Map each item to a latent representation using domain-specific encoders.

Stage 2 (Preference Modeling): Project item embeddings into a lower-dimensional space using a shared MLP ϕ: R d → R d′, then apply bag-specific linear scoring: z˜k = ϕ(zk), sˆk,j = w⊤j z˜k + bj, where (wj, bj) ∈ R d′ × R are personalized parameters for bag j.

Stage 3 (Bag Classification): Use the learned parameters as functional embeddings: yˆj = ψ([wj; bj]), where ψ is an MLP classifier and [·; ·] denotes concatenation.

Theoretical Justification

The architectural choice of linear bag-specific scoring is justified by a first-order analysis, showing that when item embeddings are sufficiently smooth in the score, a linear function approximates the score up to a Taylor remainder controlled by the curvature of the underlying preference function. Theorem 1 establishes this linear sufficiency for preference modeling.

The cold-start fitting problem reduces to ridge-less linear regression with a closed-form solution when the bag is cold, and convergence guarantees are provided for the iterative variant. The optimal choice of learning rate η that minimizes the spectral radius is η∗ = 2σ20 + σ2max.

Empirical Performance and Generalization

UNISON has been evaluated across six diverse datasets spanning entertainment, immunology, and ecology. The framework matches or outperforms state-of-the-art approaches across these domains.

In the controlled synthetic experiment, the bag-specific parameters (wj, bj) recover the latent factors that drive item scores even when the projection ϕ has no access to the original scoring mechanism (AUC > 0.998).

UNISON demonstrates superior computational efficiency and interpretability compared to meta-learning baselines, offering a 100× computational speedup over meta-learning baselines.

Ablation and Trade-offs

The necessity of the MLP projection ϕ is confirmed because removing Stage 2 MLP consistently degrades performance, as shown by the BookCrossing dataset where ρ drops from 0.462 to 0.150. UNISON restricts adaptation to a low-dimensional subspace (wj, bj) ∈ R d′+1, providing implicit regularization that reduces variance compared to gradient-based meta-learners. The framework achieves a trade-off by optimizing for the most challenging double cold-start scenario (CU-CI), reducing the dimension of z˜, and sacrificing some performance in partially warm settings.

Conclusion

UNISON is a unified framework for learning from scored bags that bridges multiple instance learning and recommendation systems. The main contributions are the theoretical justification for linear scoring functions, the introduction of functional embeddings using model parameters as bag representations in MIL, and state-of-the-art empirical performance across diverse domains with 100× computational speedup over meta-learning baselines. The framework achieves a robust generalization that deep meta-learners struggle to match.

1. textbf Weight Adaptation for Cold-Start Scenarios in Recommendation Systems: UNISON can handle cold-start scenarios by adapting only bag-specific linear parameters (wj, bj) during inference time using their support set, contrasting with heavy, gradient-based meta-learning frameworks which are prone to overfitting when the support set is sparse.

2. textbf Unified Framework for Multi-Task Learning: UNISON bridges multiple instance learning and recommendation systems by treating both as instances of learning from scored bag collections of (item, score) pairs, decomposing learning into (1) universal item representations, (2) bag-specific preference patterns through lightweight personalized parameters, and (3) inferring bag-level characteristics.

3. textbf Theoretical Justification for Linear Scoring: The framework justifies the architectural choice of linear bag-specific scoring by showing that when item embeddings are sufficiently smooth in the score, a linear function approximates the score up to a Taylor remainder controlled by the curvature of the underlying preference function (Theorem 1).

4. textbf Robust Cold-Start Performance: For cold-start scenarios, UNISON shows that the cold-bag fitting problem reduces to ridge-less linear regression with a closed-form solution, providing convergence guarantees for the iterative variant and outperforming meta-learning approaches on challenging scenarios like CU-CI.

5. textbf Domain Generality and Transfer Learning: UNISON's architecture is validated across six diverse datasets spanning entertainment, immunology, and ecology, demonstrating that the architecture of Stage II (preference learning) and Stage III (bag classification) remains unchanged across domains; only the initial item encoder (Stage I) is domain-specific.

We introduce UNISON, a unified framework that bridges multiple instance learning and recommendation systems by treating both as instances of learning from scored bag collections of (item, score) pairs. Unlike traditional approaches that address bag-level classification and personalized recommendation as separate problems, UNISON decomposes learning into three interconnected stages: (1) learning universal item representations, (2) modeling bag-specific preference patterns through lightweight personalized parameters, and (3) inferring bag-level characteristics from learned preferences. We justify the architectural choice of linear bag-specific scoring through a first-order analysis: when item embeddings are sufficiently smooth in the score, a linear function approximates the score up to a Taylor remainder controlled by the curvature of the underlying preference function. For cold-start scenarios, we show that the cold-bag fitting problem reduces to ridge-less linear regression with a closed-form solution, and provide convergence guarantees for the iterative variant. Empirically, UNISON matches or outperforms state-of-the-art approaches across diverse domains: entertainment (books, movies), immunology (T cell receptor (TCR) based classification), and microbiome, offering superior computational efficiency and interpretability. A controlled synthetic experiment further confirms that the bag-specific parameters (w j, b j) recover the latent factors that drive item scores even when the projection ϕ has no access to the original scoring mechanism (AUC > 0.998). UNISON is available at https://github.com/louzounlab/UNISON.