BoLT: A Benchmark to Democratize Black-box Optimization Research for Expensive LLM Tasks

In three linesBoLT is an open-source benchmark for black-box optimization applied to LLMs. It covers hyperparameters, data mixtures, and prompts via lightweight surrogate models fitted to thousands of real experiments. Benchmarking Bayesian Optimization and BBO methods reveals gaps in existing approaches.

Summary generated by Claude — human-verified