Towards end-to-end automation of AI research

A pipeline automates the full scientific research process from idea generation through experiment execution to manuscript writing and peer review. The system named The AI Scientist performs ideation, literature search, code writing, experiment running, result analysis, plotting, paper drafting in LaTeX format and automated peer review using foundation models in an agentic setup. One manuscript generated by the system passed the first round of peer review for a workshop at a top tier machine learning conference. The workshop acceptance rate is 70 percent. The paper exceeded the average human acceptance threshold at the workshop after submission to an International Conference on Learning Representations event with organizer consent. The AI Scientist runs in two modes. The focused mode starts with human provided code templates while the template free mode generates initial code scripts itself and applies tree search with added test time compute for wider exploration. Both modes create diverse ideas that the system tests reports on and evaluates automatically. An automated reviewer component scores papers according to NeurIPS guidelines and predicts conference acceptance decisions at a level comparable to human reviewers. The system performs better with increased compute resources and higher quality base models. It filters ideas against existing literature via the Semantic Scholar API and web access before execution.