The Kitchin Research Group

Say you have acquired some data, and fitted a nonlinear model to it. The fit looks good, but how good are predictions from the model? In high dimensional space it is tricky to tell if you are extrapolating, what should you do? First, read this paper! We illustrate a simple tool called the delta method that can help you estimate the prediction uncertainty from your model using automatic differentiation to get the required derivatives. We show some examples, and how you can use this method to refine what data you should use in fitting your models. We even show how to handle some tricky cases with models where the Hessian is not invertable! The examples are from molecular simulation, but the approach is general and should work for other models too.

@article{zhan-2021-uncer-quant,
  author =       {Ni Zhan and John R. Kitchin},
  title =        {Uncertainty Quantification in Machine Learning and Nonlinear
                  Least Squares Regression Models},
  journal =      {AIChE Journal},
  volume =       {},
  number =       {},
  pages =        {},
  year =         2021,
  doi =          {10.1002/aic.17516},
  url =          {http://dx.doi.org/10.1002/aic.17516},
  DATE_ADDED =   {Mon Nov 8 08:51:21 2021},
}

Checkout the video brief here:

Copyright (C) 2021 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.5

For fun, I have been live-streaming some of our research talks from the past year. Two of these talks are shown below. This is an experiment of sorts, let me know if you like them!

Copyright (C) 2021 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.4.6

Geometry optimization and transition state searches are two very common tasks in molecular simulation. Typically every one of these is done from scratch, and they can only be made faster by using a better initial guess. Typical optimization algorithms use some variation of gradient descent, and the best ones also use an iterative approach to estimate the Hessian (second derivatives). The problem is we do not know the underlying function that is being optimized, so there is hardly any choice to benefit from the Hessian (which allows bigger, more accurate steps to be taken).

In this paper, we use machine learning to develop a surrogate model that is cheap compared to the DFT calculations, and that has an uncertainty quantification so we can tell when it is accurate. This allows us to take many cheap steps when the surrogate model is accurate, and only do expensive calculations when needed. More importantly though, the surrogate model works across many different geometry optimizations, which allows us to benefit from previous calculations. We show this works on a variety of atomic geometries ranging from metal slabs, slabs with adsorbates, and nanoparticle geometries, as well as with nudged elastic band calculations for transitions state searches.

@article{yang-2021-machin-learn,
  author =       {Yilin Yang and Omar A. Jim{\'e}nez-Negr{\'o}n and John R.
                  Kitchin},
  title =        {Machine-Learning Accelerated Geometry Optimization in
                  Molecular Simulation},
  journal =      {The Journal of Chemical Physics},
  volume =       154,
  number =       23,
  pages =        234704,
  year =         2021,
  doi =          {10.1063/5.0049665},
  url =          {https://doi.org/10.1063/5.0049665},
}

Copyright (C) 2021 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.4.6

Modeling alloys is tricky, and modeling dilute alloys has been an open challenge for a long time. Segregation is one of the biggest challenges; the surface composition is not the same as the bulk, and is influenced by adsorption. Although we know that the composition varies to lower the surface free energy, the configurational degrees of freedom make it difficult to find the lowest energy arrangement of atoms. In the dilute limit, the quantum chemical calculations and Monte Carlo simulates we do become very expensive due to the unit cell size. In this work, we use machine learning to build surrogate models for the configurational energy of an alloy slab in the dilute limit. Then, we use those models in conjunction with a semi-grand canonical Monte Carlo (MC) simulation to solve several of these problems. First, by fixing the alloy chemical potential at the desired dilute limit, we avoid the need for large unit cells. Second, the surrogate models are much more efficient than DFT, so we can use them in the MC simulations to run tens of thousands of simulation steps to get the required samples for reliable statistical averaging. In dilute alloys, the focus is on the unique catalytic properties of single atoms of an active metal like Pd in an inert metal like Ag. In the single atom limit though, there are few active sites. If you increase the bulk concentration, at some point the single atoms begin to aggregate into dimers and trimers, and adsorbates can make the aggregation happen faster. Aggregation is undesirable because it usually leads to lower selectivity, and more bulk like reactivity. An open question has been how do you design alloy catalysts under reaction conditions, given all this complexity. We apply this approach to the adsorption of acrolein on a dilute AgPd alloy, and show how to use the method to identify the bulk concentration where aggregation begins in a reactive environment.

@article{liu-2021-semi-grand,
  author =       {Mingjie Liu and Yilin Yang and John R. Kitchin},
  title =        {Semi-Grand Canonical Monte Carlo Simulation of the Acrolein
                  Induced Surface Segregation and Aggregation of {AgPd} With
                  Machine Learning Surrogate Models},
  journal =      {The Journal of Chemical Physics},
  volume =       154,
  number =       13,
  pages =        134701,
  year =         2021,
  doi =          {10.1063/5.0046440},
  url =          {https://doi.org/10.1063/5.0046440},
}

Copyright (C) 2021 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.4.6

Many machine learned potentials work by creating a numeric fingerprint that represents the local atomic environment around an atom, and then "machine learning" a function that computes the atomic energy for that atom. The total energy of an atomic configuration is then simply the sum of the atomic energies, and the forces are simply the derivative of that energy with respect to the atomic positions. In the Behler-Parrinello formulation, each element gets its own neural network for these calculations. In this work, we show that a single neural network with multiple outputs can be used instead. This means that all the elements share the weights in the neural network, and the atomic energy of each element is linearly proportional to the output of the last hidden layer. This has some benefits for transferability and suggests that there is a common nonlinear dimensional transform of the numeric fingerprints for the elements in this study.

@article{liu-2020-singl,
  author =       {Mingjie Liu and John R. Kitchin},
  title =        {Singlenn: Modified Behler-Parrinello Neural Network With
                  Shared Weights for Atomistic Simulations With Transferability},
  journal =      {The Journal of Physical Chemistry C},
  volume =       124,
  number =       32,
  pages =        {17811-17818},
  year =         2020,
  doi =          {10.1021/acs.jpcc.0c04225},
  url =          {https://doi.org/10.1021/acs.jpcc.0c04225},
}

Discuss on Twitter

Copyright (C) 2021 by John Kitchin. See the License for information about copying.

org-mode source

Org-mode version = 9.4