11:30 am–12:30 pm
Jones 303 5747 S. Ellis Avenue
Monday, March 24, 2025 at 11:30 AM in Jones 303, 5747 S Ellis Ave.
Victor Veitch, Department of Statistics and the Data Science Institute, University of Chicago
Title: Statistical Views on LLM post training
Abstract: Typically, large language models are refined with a post training procedure aimed at biasing their outputs to have desirable properties---helpfulness, harmlessness, factualness, and so forth. These desiderata are often elicited by pairwise comparisons of LLM responses. This comparative reward signal creates some subtleties in how post training should be conducted. I'll discuss some ways of formalizing the goal of post training and methods for achieving these goals.