Sequential Minimal Optimization Algorithm for One-Class Support Vector Machines With Privileged Information
Andrey Lange, Dmitry Smolyakov, Evgeny Burnaev
cs.LG, cs.AI, stat.ML
Submitted: 2026-06-20
Comments: 19 pages, 4 figures. Author-prepared preprint of the article published in IEEE Access. The published article is licensed under CC BY-NC-ND 4.0
Journal ref: IEEE Access 11 (2023) 128106-128124
DOI: 10.1109/ACCESS.2023.3331685
Code: https://github.com/langeam/Categorical-Hamming-Kernels
License: http://creativecommons.org/licenses/by-nc-nd/4.0/
The gist: One of the powerful techniques in data modeling is accounting for features that are available at the training stage, but are not available when the trained model is used to classify or predict test
Terminology
Abstract
One of the powerful techniques in data modeling is accounting for features that are available at the training stage, but are not available when the trained model is used to classify or predict test data -- the Learning Using Privileged Information paradigm (LUPI). Sequential Minimal Optimization (SMO) methods have been developed for supervised Support Vector Machines (SVM), unsupervised one-class SVM, and SVM with privileged information (SVM+). The missing brick in this research has long been a one-class SVM with privileged information (OC-SVM+). In this paper, we propose an SMO algorithm for OC-SVM+ that significantly outperforms non-sequential algorithms for training the OC-SVM+ model. Its finite-time convergence is established. The experiments show how privileged information affects a descriptive domain in the space of original features. Comparative benchmark tests demonstrate that our algorithm is superior over interior point algorithms.
Sources
- Learning to Transfer Privileged Information
- Unifying distillation and privileged information
- OCKELM+: Kernel Extreme Learning Machine based One-class Classification using Privileged Information (or KOC+: Kernel Ridge Regression or Least Square SVM with zero bias based One-class Classification using Privileged Information)
Related papers
- Polynomial-Augmented Neural Networks (PANNs) with Weak Orthogonality Constraints for Enhanced Function and PDE Approximation
- AIRL-S: Unifying Reinforcement Learning and Search-Based Test-Time Scaling via Adversarial Inverse Reinforcement Learning
- Transformers as Bayesian In-Context Experimenters: Smoothness-Adaptive Efficient ATE Estimation
- Convergence issues in Relational Concept Analysis based on AOC-posets
- Beliefs Beyond Posteriors: The Predictive Cost of ELBO Binding
- Understanding Diffusion Models via Ratio-Based Function Approximation with SignReLU Networks