Large Language Models
Large language models (LLMs) generate text in response to a user prompt, optionally conditioned by a system prompt and limited by a context window measured in tokens. In scientific software they are used in several increasingly powerful modes: answering questions from documentation placed in the prompt, extracting structured data that can be validated against a schema, choosing and parameterizing predefined functions (tool calling), generating executable code, and acting as agents that combine these steps in a program loop with memory and access to external resources. Because outputs are nondeterministic and models know nothing of libraries written after training, using them well requires new skills in prompting, validation and risk management.
LLMs in laboratory automation (2025)
Kitchin used the Claude-Light instrument to show each of these modes in a self-driving laboratory setting, accessing models through LiteLLM (Kitchin 2025). Structured extraction with Pydantic avoided hand-written parsing; tool calling worked but required verbose hand-written function specifications; and code generation hallucinated functions for an unfamiliar library until usage examples were added to the prompt. The paper demonstrates prompt overriding of a system prompt and generated code that writes and runs another file, recommends sandboxing or human inspection of generated code, and discusses vendor data sharing, rate limits, cost, model retirement, and nondeterminism (for example unit errors in a Reynolds number calculation). Low temperature, structured outputs and tool calls are suggested as ways to make results more reproducible, though a model at zero temperature can be consistently wrong. Summary: kitchin2025b.
LLMs as the core of agentic science (2025)
Xin, Kitchin and Kulik describe multimodal LLMs as the core of agentic AI for science, noting that scientific use requires handling symbolic and structured content such as chemical formulas and equations through specialized tokenization, embeddings and external knowledge bases (Xin et al. 2025). They highlight domain models such as LLaMat for materials, warn that hallucination (including invented software packages, the basis of “slopsquatting”) and prompt fragility become dangerous in long multi-step tasks, and recommend grounding outputs in knowledge graphs and reporting the model version and representative prompts so that LLM-assisted work can be reproduced. Summary: xin2025.