The evolving role of programming and LLMs in the development of self-driving laboratories
John R. Kitchin, “The evolving role of programming and LLMs in the development of self-driving laboratories”, APL Machine Learning 3(2), 026111 (2025). https://doi.org/10.1063/5.0266757
This summary was written by a language model from the paper and has not been checked line by line. Rely on the published paper for details.
Note on the source text: the paper is built around executable code examples and their printed outputs. The extracted text available to this wiki contains the prose but not the code blocks or most of their outputs, so this summary describes the examples from the surrounding narrative only.
Question
Self-driving laboratories (SDLs) couple automated experiments to closed-loop machine learning, but they are expensive to build and run and require a mix of domain knowledge, programming and data science that takes a long time to learn. The paper asks how students and researchers can practice automation on a small, cheap, real instrument, and how the programming involved changes as one moves from ordinary scripts to large language models (LLMs) (Kitchin 2025). It is written for scientists and engineers with basic programming skills and is deliberately code-heavy.
Approach
The vehicle is Claude-Light, a remotely accessible instrument derived from the earlier SDL-Light design. A Raspberry Pi runs a Flask web server exposing a REST API; the user sets the R, G and B levels of an LED as floats from 0 to 1, and a photometer returns ten outputs (eight wavelengths, near infrared and a clear channel) as 16-bit intensities. A camera shows the LED color. The photometer is open to room and daylight, so measurements are noisy and some channels saturate at certain times of day; a measurement takes 1 to 2 s. The outputs are close to linear in the inputs, which keeps the modeling easy and puts the emphasis on automation and experimental design (Kitchin 2025).
The paper then walks through a hierarchy of automation. Conventional programming covers raw HTTP access (curl with jq, Python requests), a small Python library with four instrument classes of increasing dimension, a loop with linear regression, a Latin square design of experiments, and a Gaussian process model used to solve an inverse problem. The LLM part, run through the LiteLLM package (results shown use a Claude Sonnet model), covers instrument selection from documentation in a system prompt, structured output extraction with Pydantic, tool (function) calling, code generation, agents, safety, and reproducibility (Kitchin 2025).
Key findings
Claude-Light has supported classes with about 50 simultaneous students with few problems, and since about August 2024 it has been used from more than 1800 unique IP addresses and run more than 73,000 experiments (Kitchin 2025). A nine-run Latin square on the three-input instrument showed the red input dominating the 630 nm output, with smaller effects on the green and blue channels, attributed to overlapping LED emission spectra. A Gaussian process with a linear kernel fit to twenty experiments was inverted to find inputs giving outputs of (10,000, 10,000, 10,000); Nelder-Mead worked better than the default SciPy optimizer, which tended to stall in poor local minima. Scaling the outputs was needed because unscaled data gave ill-conditioned Hessians, and omitting a white-noise kernel gave overconfident uncertainties, a serious issue because uncertainty steers active learning (Kitchin 2025). The author argues that active learning and Bayesian optimization do not change the basic loop-then-analyze structure; they change how the next experiment is chosen and require stopping criteria.
For LLMs, structured extraction removed the need to write pattern-matching code for the many ways a user might specify colors. Tool calling worked, but writing function specifications by hand was verbose, and the class-with-call design of the instrument library needed a registry of instances. Asked to write code for the new library without context, the model invented plausible but nonexistent functions; supplying library usage in the prompt fixed this, and the generated code produced a correct scan of green output against green input after manual inspection (Kitchin 2025). Safety examples showed a user prompt overriding the system prompt and prompt-engineered code writing and executing an extra file. Reproducibility tests showed inconsistent answers to a simple multistep arithmetic task and order-of-magnitude unit errors in a Reynolds number calculation whose expected value is 109,375. Lowering the temperature improves repeatability but can make a model reproducibly wrong (Kitchin 2025).
Significance
The paper frames LLMs as more code, not a replacement for it: the skill shifts from learning syntax to learning how to prompt, validate and constrain models, and programs containing LLMs are more complex than those without. It lays out the practical concerns for SDLs, including prompt injection, sandboxing or human review of generated code, data sharing with cloud vendors, rate limits and backoff, cost guard rails, model retirement that threatens reproducibility, and documenting work in executable documents (Kitchin 2025). Structured outputs and tool calls are recommended as ways to reduce nondeterminism. Claude-Light itself is offered as a low-cost training and prototyping platform, while the author notes that one instrument is far from a lab of interacting instruments.