An AI system to help scientists write expert-level empirical software
Abstract
Abstract The cycle of scientific discovery is frequently bottlenecked by the slow, manual creation of software to support computational experiments 1 . To address this, we present Empirical Research Assistance (ERA), an artificial intelligence (AI) system that creates expert-level scientific software whose goal is to maximize a quality metric. The system uses a large language model (LLM) and tree search 2 to systematically improve the quality metric and intelligently navigate the large space of possible solutions. ERA achieves expert-level results when it explores and integrates complex research ideas from external sources. The effectiveness of tree search is demonstrated across a diverse range of tasks. In bioinformatics, ERA discovered 40 new methods for single-cell data analysis that outperformed the top human-developed methods on a public leaderboard. In epidemiology, ERA generated 14 models that outperformed the Centers for Disease Control and Prevention (CDC) ensemble and all other individual models for forecasting COVID-19 hospitalizations. ERA also produced expert-level software for geospatial analysis, neural activity prediction in zebrafish and numerical solution of integrals, as well as a new rule-based construction for time-series forecasting. By devising and implementing new solutions to diverse tasks, ERA represents a notable step towards accelerating scientific progress.
Article Details
Authors (42)
Eser Aygün
Anastasiya Belyaeva
Gheorghe Comanici
Marc Coram
Hao Cui
Jake Garrison
Renee Johnston
Anton Kast
Cory Y. McLean
Peter Norgaard
Zahra Shamsi
David Smalling
James Thompson
Subhashini Venugopalan
Brian P. Williams
Chujun He
Sarah Martinson
Martyna Plomecka
Lai Wei
Yuchen Zhou
Qian-Ze Zhu
Matthew Abraham
Erica Brand
Anna Bulanova
Jeffrey A. Cardille
Chris Co
Scott Ellsworth
Grace Joseph
Malcolm Kane
Ryan Krueger
Johan Kartiwa
Dan Liebling
Jan-Matthis Lueckmann
Paul Raccuglia
Xuefei Julie Wang
Katherine Chou
James Manyika
Yossi Matias
John C. Platt
Lizzie Dorfman
Shibl Mourad
Michael P. Brenner