Many Preferences, Few Policies: Compact Portfolios for Multi-Objective LLM Alignment
This work introduces PALM, an algorithm to select a small portfolio of LLMs that covers diverse user preferences across many traits. It solves the problem of needing a separate LLM for every user by generating a compact set of models that can approximate any desired preference combination, offering theoretical guarantees on size and quality.