Toward Benchmarking in Catalysis Science: Best Practices, Challenges, and Opportunities
Thomas Bligaard, R. Morris Bullock, Charles T. Campbell, Jingguang G. Chen, Bruce C. Gates, Raymond J. Gorte, Christopher W. Jones, William D. Jones, John R. Kitchin and Susannah L. Scott, “Toward Benchmarking in Catalysis Science: Best Practices, Challenges, and Opportunities”, ACS Catalysis 6, 2590-2602 (2016). https://doi.org/10.1021/acscatal.6b00183
This summary was written by a language model from the paper and has not been checked line by line. Rely on the published paper for details.
Question
Catalysis research is now a global enterprise producing very large amounts of data, yet explicit benchmarking (evaluating results by comparison with an accepted standard) appears rarely in the catalysis literature: a Chemical Abstracts search cited by the authors found fewer than 500 mentions among roughly a million catalysis papers over a decade, although the number is rising quickly. This Perspective, written by ten investigators supported by the DOE Basic Energy Sciences Catalysis Science program, asks which practices would enable benchmarking and data sharing within and between catalysis subfields, how benchmarking can accelerate progress, and how the benchmarking tools themselves can be assessed and improved (Bligaard et al. 2016).
Approach
The authors define benchmarking as a community activity with technical parts (agreeing on the central questions) and social parts (agreeing on the rules for answering them), and argue that a good benchmark is accessible, affordable, clear, relevant, solvable, portable, reproducible and scalable. They review historical precedents, then give best practices and open challenges for four subfields: computational catalysis, heterogeneous catalysis, molecular and supported molecular catalysis, and electrocatalysis. Enzyme catalysis and photocatalysis are explicitly excluded. The Supporting Information lists code-sharing, data-sharing and archiving tools.
Key findings
Historical examples include the shared standard catalysts EuroPt-1 (5 wt % Pt/SiO2) and EuroNi-1 (5 wt % Ni/SiO2) from the 1980s, later naphtha reforming and oxide standards, and World Gold Council Au catalysts on TiO2, Fe2O3 and carbon from 2002. In molecular catalysis, zirconocene dichloride with methylaluminoxane serves as a benchmark for single-site ethylene polymerization, and in electrocatalysis Pt on high-surface-area carbon is the reference for hydrogen evolution and oxidation. The authors note that these standard catalysts usually came without prescribed benchmark reactions or test conditions.
For computational catalysis, the paper argues that an adsorbate geometry alone is not enough to reproduce an adsorption energy, and that input and output files, the procedures used to derive energies, frequencies, barriers and entropies, and the kinetic model codes fed by them should be shared, with version control for evolving implementations. Benchmarks against more accurate theory or against single-crystal experiments exist for small adsorbates on late transition metals (including cases with large van der Waals contributions), but not for oxides, carbides and nitrides. Chemical accuracy (1 kcal/mol or better for activation energies) remains out of reach, so theory is better at trends. Linear scaling relations among adsorption and transition-state energies are presented as a set of benchmarks that both explain structure-function relationships and reveal limits on how much a class of materials can be improved. A CH3ReO3 epoxidation example shows how comparing DFT with experiment exposed a solvent-mediated mechanism that resolved rate-constant discrepancies of 7 to 9 orders of magnitude.
For heterogeneous catalysis, the recommendations are standard reactions with accepted area- or site-normalized rates, turnover frequencies measured free of heat and mass transport limits in ideal reactors and checked by varying concentrations, blind round-robin tests across laboratories to set realistic error bars, quantified deactivation, and experimental adsorption and activation energies on well-defined materials to validate theory. For molecular catalysts the authors caution that reproducibility problems are common, and they call for purity data on the material actually used, rate laws rather than single high-conversion points (using “site-time yield” when only such data exist), selectivity reported at stated conversion, TON demonstrably above 1, transparent active-site counting, controls without catalyst, and stability tested by initial rates after recycle or long continuous-flow runs. For electrocatalysis they recommend reporting current versus overpotential normalized to geometric area, electrochemically active surface area (ECSA) and catalyst mass, as in the JCAP database for HER and OER, along with full protocols, composition and morphology data, and transient behavior. Reliable ECSA methods exist for Pt but not for non-Pt materials, which limits benchmarking of those catalysts.
The conclusions note inconsistent terminology across communities (for example, enzymologists use “turnover number” for what others call TOF) and argue that journals, funders and institutions must reward openness and reproducibility, citing National Academies, Nature, Science TOP and funding-agency data policies.
Significance
This multi-author Perspective places reproducible computation and data sharing alongside experimental standards as the foundation of catalysis benchmarking, and it frames shared, curated benchmark databases as a community responsibility.