Neural Network Potentials

A research topic across the group’s papers.

Neural network potentials are machine-learned interatomic potentials fit to a large set of reference electronic structure calculations, usually DFT. In the Behler–Parrinello form, the total energy is a sum of atomic energies, and each atomic energy is predicted by a small feed-forward network from symmetry-function fingerprints of the neighbors within a cutoff radius. This makes the model invariant to translation and rotation and lets the cost grow roughly linearly with system size. Unlike physics-based force fields, the functional form has no built-in physical constraints, so accuracy is limited mainly by how well the training data cover the configurations of interest. Interpolation within the training data can be close to DFT quality, but extrapolation is unreliable. Related learned-representation models are described in Graph neural networks.

A neural network versus ReaxFF for gold (2016)

Boes, Groenenboom, Keith and Kitchin trained a Behler–Parrinello network with AMP (6.5 Å cutoff, four hidden layers of 40 tanh nodes) on 9,734 of 9,972 PBE calculations of Au bulk phases, vacancies, fcc(111) and fcc(100) surfaces and clusters of up to 126 atoms, reaching a training RMSE of 0.017 eV/atom. The network nearly reproduced DFT equations of state (fcc bulk modulus 145 versus 147 GPa) and the bulk vacancy diffusion barrier (0.60 versus 0.56 eV), but overestimated vacancy formation energies by about 0.4 eV and had 0.2 to 0.3 eV errors on an fcc(100) terrace diffusion path that was poorly represented in training. BPNN-driven MD found the untrained planar global minimum of a six-atom cluster with residuals below 0.05 eV/atom. Compared with ReaxFF fit to the same data, the network was more accurate overall but needed far more training data and ran about 15 versus 6,700 MD steps per second in its Python implementation (Boes et al. 2016). Summary: boes2016. See also Reactive force fields.

A Behler–Parrinello potential for bulk zirconia (2018)

Wang, Tharval and Kitchin trained a BPNN with the AMP code on 2178 PBE calculations of ZrO2 polymorphs, vacancy supercells, relaxation steps, short MD snapshots and NEB images. They used a 6.5 Å cutoff and two hidden layers of 13 nodes, and reached a training RMSE of 7.90 meV/atom. The model reproduced lattice parameters, equations of state, vacancy formation energies and migration barriers for the trained polymorphs and ran MD at a fraction of DFT cost. It degraded on the untrained high-pressure Pca21 phase, especially under compression. The authors showed that disagreement between two independently trained networks along a path (up to about 2.29 eV along cubic [110]) flags configurations missing from the training set, and they recommend adding such structures and retraining (Wang et al. 2018). Summary: wang2018.

Oxygen on a dynamic Pd(111) surface (2017)

Boes and Kitchin trained a BPNN with AMP on 11,925 PBE calculations of four-layer Pd(111) slabs with 0 to 1 ML oxygen, using a 6.5 Å cutoff and 20 G2/G4 symmetry functions per element, and reached a validation RMSE of 0.009 eV/atom. The network reproduced bulk Pd properties and the clean-surface energy (0.563 versus 0.561 eV/site), predicted oxygen adsorption energies within about 0.05 to 0.1 eV/O, and was used for NEB diffusion paths, 600 K MD and grand-canonical MC on a 10 × 10 slab, which a fixed-size Cartesian network could not do. It failed where training data were sparse: hcp layer stacking, coverage-induced reconstructions, and subsurface oxygen, which caused MD at 0.33 ML and above to deconstruct the surface. The authors recommend iterative retraining and validation with a second, differently constructed network (Boes 2017). Summary: boes2017a. Comparison with lattice models is discussed in Cluster expansion.

A validated single-element potential for Pd surfaces (2018)

Gao and Kitchin trained a Behler–Parrinello network with AMP on 3035 PBE calculations of Pd bulk phases, defects, (111) and (211) slabs, NEB paths and MD snapshots, using only eight Gaussian symmetry functions, a 6.5 Å cutoff and two hidden layers of 18 nodes (validation error 1.92 meV/atom). The potential reproduced equations of state for five lattices, elastic constants within about 20 GPa, bulk vacancy formation and migration (about 0.676 eV), surface energies and adatom diffusion barriers, but underestimated surface vacancy formation energies by up to 0.22 eV where training data were sparse. Single-point evaluations were about four orders of magnitude faster than DFT. The authors stress that the potential is interpolative and cannot be extended to adsorbates, alloys or oxides without new fingerprints and networks (Gao 2018). Summary: gao2018.

Shared weights across elements: SingleNN (2020)

Liu and Kitchin modified the Behler-Parrinello scheme so that atoms of all elements pass through one network with shared hidden layers and element-specific linear output weights, selected by an element mask, using electronegativity-weighted symmetry functions (10 G2, 4 G4) so the fingerprint length does not grow with the number of elements. On a DFT benchmark of pure Li, Mo, Cu, Ni, Si and Ge (1580 structures), SingleNN with two hidden layers of 20 nodes (846 parameters, versus 4446 for the equivalent six-element BPNN) gave energy MAEs slightly above BPNN (for example 8.25 versus 5.59 meV/atom for Mo) and force MAEs slightly below it (0.071 versus 0.113 eV/Å for Si), showing that one shared nonlinear transformation can serve all six elements (Liu 2020). Summary: liu2020.

SingleNN surrogates for segregation Monte Carlo with adsorbates (2021)

Two SingleNN potentials (two hidden layers of 30 tanh nodes, Behler-Parrinello symmetry functions) were trained on PBE energies of AgPd bulk cells and 3x3x4 AgPd(111) slabs, reaching test errors of 1.6 and 0.5 meV/atom, and drove semi-grand canonical Monte Carlo on 12x12x4 and 30x30x4 slabs (Liu et al. 2021). Variation among five models trained on different data splits was used to estimate the propagated simulation error. Summary: liu2021.

SingleNN as an on-the-fly surrogate for relaxations (2021)

Yang, Jiménez-Negrón and Kitchin used small SingleNN potentials (two hidden layers of 50 neurons, symmetry function fingerprints) not as general-purpose potentials but as local, continually retrained surrogates during DFT relaxations and NEB calculations (Yang et al. 2021). Because the atom-centered model can share information across configurations of different size, one ensemble served many structures at once, and NEB transition states matched the reference within 4 to 6 meV in activation energy. Summary: yang2021.

Uncertainty estimates for a SingleNN AuPd potential (2022)

A [4, 11, 11, 1] SingleNN potential (211 parameters) with weighted G2 symmetry functions, trained on energies and forces of AuPd slabs at a 3.934 Å lattice constant, reached energy RMSEs of about 0.003 eV/atom and was given per-prediction uncertainties with the delta method (Zhan 2021). The uncertainties grew by an order of magnitude on slabs with 4.034 and 4.134 Å lattice constants whose fingerprints lay outside the training range, and computing the Fisher information from energy errors alone was sufficient in practice; force uncertainties can be obtained by differentiating forces with respect to the parameters. Summary: zhan2022.

Benchmarking catalysis potentials on energy differences (2023)

Ock et al. (2023) argued that GNN potentials for catalysis should be judged on energy differences between related systems, not only on individual adsorption energy MAE. On OC20-Dense, newer models reduced the MAE but did not steadily improve error cancellation, and the sign of differences smaller than 0.2 eV was predicted correctly only 54% to 64% of the time overall, rising to about 70% for configurations near the minimum. The authors propose an error cancellation ratio and a marginal correct sign ratio as complementary metrics. Summary: ock2023.

Error plateau of large-scale catalysis potentials (2024)

Abdelmaqsoud and coworkers examined why energy errors of MLPs trained on the Open Catalyst 2020 dataset had leveled off near 0.2 eV, with nonmetals consistently worst. Tightening the DFT cutoff and smearing changed total energies by more than 0.1 eV but adsorption energies by only about 0.05 eV, and did not change model errors, so DFT convergence was ruled out. Adsorbate-induced surface reconstruction, present in about 22% of calculations, made the adsorption energy label ill-posed; removing those systems from validation lowered MAEs of GemNet-OC, eSCN and EquiformerV2 by roughly 35%. Models trained on total energies sidestep the problem and give comparable adsorption energy errors (Abdelmaqsoud et al. 2024). Summary: abdelmaqsoud2024a. See also DFT convergence and Adsorption energy referencing.

Machine-learned potentials as a role for ML in catalysis (2018)

In a Comment on ML in catalysis, Kitchin presents potentials learned from DFT as one of the two main uses of ML in the field, filling the gap between DFT, which is too costly for the many calculations needed in screening, free energy, Monte Carlo or MD work, and classical potentials, which are fast but may lack accuracy and are hard to improve systematically (Kitchin 2018). He names Amp, aenet and Prophet as open-source codes for building them, and highlights published examples: MD of N2 dissociation on Ru(0001) at DFT accuracy including surface phonon coupling, simulation of infrared spectra, and a potential for alloy nanoparticles of 147 to 1,415 atoms in water used to obtain segregation profiles and interfacial water structure. Summary: kitchin2018.

A minimal potential for AuPd(111) segregation (2017)

For AuPd(111) slabs, Boes and Kitchin used AMP to train per-element networks with only two hidden layers of three nodes and eight G2 symmetry functions (6 Å cutoff), 66 weights in total. Candidate structures were all energy-unique configurations of small seven-layer slabs enumerated with EMT at five lattice constants (62,706 structures); DFT was run only where two networks of different architecture disagreed most, and three such iterations produced 3914 training calculations with validation error matching training error. Network-to-network disagreement on about 547,000 configurations of a larger untrained slab was comparable to the DFT residuals, and a much larger 40-input network gave nearly identical segregation profiles. The potential then drove Monte Carlo on 1500-atom slabs whose surface compositions matched LEIS data (Boes 2017). Summary: boes2017; see also Surface segregation.

A ternary potential for Cu-Pd-Au(111) slabs (2022)

Yang and co-workers trained a Behler-Parrinello potential (AMP, one network per element with two hidden layers of ten neurons, G2 fingerprints at three η values and a 6 Å cutoff) on 5278 DFT fcc(111) slabs spanning the Cu-Pd-Au composition triangle and six lattice constants. Training and validation MAEs were about 2 meV/atom, and held-out 12 × 12 × 5 slabs gave a comparable MAE with a small bias. To argue coverage, they showed that at fixed η the fingerprints fall on triangular planes determined by lattice constant and layer, with composition setting the position within each plane, and that the random small-cell training data nearly fill these planes. A second network with the same architecture, trained on 1000 slabs, predicted surface relaxation energies for Monte Carlo on CuPd(110) (Yang et al. 2022). Summary: yang2022; see also Surface segregation.

Direct versus energy-gradient forces in large catalysis potentials (2022)

On OC20, directly predicted forces outperformed forces taken as the negative gradient of the energy: GemNet-dT reached 0.021 eV/Å in-domain force MAE with direct forces against 0.030 eV/Å with gradients, and direct models also had lower energy errors, while gradients cost 2 to 4 times more memory and time and sometimes destabilized training (Kolluru et al. 2022). Direct forces are not energy conserving, so they suit relaxations rather than molecular dynamics, and the reverse trend seen on MD17 remained unexplained. The same work found that adding ab initio MD data helped GemNet-OC forces and relaxations, rattled data mainly helped relaxations, and relaxation-based models trained on a 2M subset stayed competitive with the full 133M set. Summary: kolluru2022.

A graph network potential for ternary segregation Monte Carlo (2024)

Broderick et al. (2024) used a fine-tuned GemNet-OC model in IS2RE mode to predict relaxed slab energies directly from ideal lattice positions inside Monte Carlo on 1500-atom slabs, after training on slabs of at most 35 atoms. Although raw energies were off by more than 1 eV before fine-tuning, swap energy differences were accurate thanks to error cancellation, and Brier scores on Boltzmann transition probabilities (0.05 after fine-tuning) were used as a task-relevant accuracy metric. Summary: broderick2024.

Knowing when to trust relaxed energies from OC20 potentials (2024)

For workflows such as AdsorbML that use GNN potentials to relax adsorbate-slab structures without DFT, Musielewicz and coworkers built an uncertainty benchmark from roughly 25,000 EquiformerV2 relaxations each on the OC20 in-domain and out-of-domain validation sets, with VASP single points as ground truth (Musielewicz et al. 2024). The recalibrated GemNet-OC latent distance gave very large uncertainties for Cu and TiO2 bulk equations of state (outside the training domain), smaller ones for a strained Al slab that grew away from the minimum, and a smooth monotonic rise with increasing O coverage on Pt that tracked the measured errors. Summary: musielewicz2024.

Detecting extrapolation in a fine-tuned SchNet potential (2025)

Vinchurkar and coworkers fine-tuned an OC20-pretrained SchNet on bulk gold frames with a shallow-ensemble output layer and found that, while silver structures were clearly flagged as unfamiliar, amorphous gold clusters received variances as low as bulk gold, and that both variance and error rose for higher-energy frames that are scarce in the data (Vinchurkar et al. 2025). This illustrates that ensemble variance from a potential trained mostly on near-equilibrium data may miss structural extrapolation within one element. Summary: vinchurkar2025.

Pretrained potentials applied zero-shot to transition states (2025)

Wander and coworkers showed that OC20-pretrained graph network potentials, never trained on reaction paths, approximate the potential energy surface well enough to locate transition states. Success on the 932-calculation OC20NEB benchmark rose with each model’s OC20 accuracy: EquiformerV2 (153M and 31M), then GemNet-OC, with PaiNN and DimeNet++ far behind for dissociations and transfers (roughly 12 to 28% success). Accuracy did not differ appreciably between in-domain and out-of-domain materials. A gap remained for spin-polarized systems: since OC20 lacks spin polarization, Co and Ni transition states needed DFT refinement (Wander et al. 2025). Summary: wander2025a.

Hessians from pretrained catalysis potentials (2025)

Wander, Musielewicz, Cheula and Kitchin showed that potentials pretrained only on OC20 energies and forces also give usable second derivatives. Finite-difference Hessians from five models were compared with 4,599 DFT Hessians of relaxed adsorbates and OC20NEB transition states; the 153M-parameter EquiformerV2 reached a frequency MAE of 58 cm^-1 on minima and 88 cm^-1 on transition states, slightly overestimating frequencies, and model ranking tracked OC20 force accuracy. The cheap Hessians also enabled complete potential energy sampling of translational entropy and Hessian-driven transition state refinement with Sella (Wander et al. 2025). Summary: wander2025.