Symposium 2025 Recordings

From human-in-the-loop to LLM-in-the-loop Bayesian optimisation

Prof. Antonio del Rio Chanona

Abstract

Bayesian optimization has proven effective for optimizing expensive-to-evaluate functions in design of experiments (DoE). However, valuable insights from domain experts are often overlooked. This work introduces a collaborative Bayesian optimization approach that re-integrates human input into the data-driven decision-making process. By combining high-throughput Bayesian optimization with discrete decision theory, experts can influence the selection of experiments via a discrete choice. We propose a multi-objective approach to generate a set of high-utility and distinct experimental designs, from which the expert selects the most promising one for evaluation at each iteration. Our methodology maintains the advantages of Bayesian optimization while incorporating expert knowledge and improving accountability in experimental planning.

Beyond human expertise, we further explore replacing domain experts with multi-agent large language models (LLMs) that simulate expert decision-making. These agents, assess candidate experimental designs, enabling an automated yet interpretable alternative to human-guided DoE. By leveraging a team of specialized LLM agents, we investigate their effectiveness in balancing trade-offs, capturing diverse expert opinions, and accelerating decision-making. Through case studies, including bioprocess optimization and experimental design for new material discovery, we demonstrate that both human and AI-driven expert guidance enhance experimental planning. By continuously incorporating either human or AI-driven expert opinion, the proposed method enables faster convergence and improved accountability for Bayesian optimization in design of experiments.