Reproducibility
Reproducibility is the ability of others to repeat a computational or experimental study and obtain the same result, and to build on it. For data-driven and AI-based work it depends on sharing far more than a narrative description: trained parameters, hyperparameters, data and preprocessing steps, code, and for language-model systems the model version and prompts. Related discussions appear in Agentic AI and in the summaries xin2025 and kitchin2025b.
Embedding data and code in publications (2015)
Kitchin argues that sharing becomes routine only when data, analysis and code are integrated into the authoring workflow itself (Kitchin 2015). He demonstrates attaching machine-readable data files (JSON, CSV, Excel) to Supporting Information PDFs and writing papers in org-mode so that tables are both human readable and usable by code, and shows how the JSON file from a DFT paper suffices to regenerate VASP inputs for an individual calculation. Where licenses forbid sharing some inputs (VASP pseudopotential files), he notes it is still usually possible to share enough for another expert to reproduce the work. Copying code into LaTeX does not guarantee it is what was run, whereas an executable document keeps code and results synchronized. See Data sharing, Literate programming and the summary kitchin2015.
Reporting machine-learned models (2018)
Kitchin identifies reporting as perhaps the most significant challenge for ML in catalysis (Kitchin 2018). ML models can have hundreds of trained parameters plus many hyperparameters, are expressed as programs rather than equations, are fit to datasets of hundreds to tens of thousands of points, and depend on normalization and scaling pipelines. Narrative Methods sections, figures and current supporting information formats are not effective for conveying all of this, so he argues the community must develop and adopt new ways of sharing data and code. Summary: kitchin2018.
Trust in self-driving laboratory data (2025)
Canty et al. argue that concerns about whether self-driving laboratory data can be trusted must be met by recording not only inputs and outputs but state and environmental metadata (such as temperature and relative humidity), by calibration, benchmarking and control studies built into workflows, and by reporting operational performance, degree of human involvement, resources used, waste and shortcomings (Canty et al. 2025). They note that few SDL technologies currently include repeatability or reproducibility metrics, that no standards or incentives for such studies exist, and recommend releasing code and data at publication and giving SDLs identifiers (analogous to ORCID) for provenance. Summary: canty2025.