Beyond the fourth paradigm of modeling in chemical engineering
John R. Kitchin, Victor Alves and Carl D. Laird, “Beyond the fourth paradigm of modeling in chemical engineering”, Nature Chemical Engineering 2(1), 11–13 (2025). https://doi.org/10.1038/s44286-024-00170-x
This summary was written by a language model from the paper and has not been checked line by line. Rely on the published paper for details.
This is a short Comment (perspective) piece rather than a research article, so it reports no new data; the summary below describes its arguments and the examples it points to.
Question
The Comment asks what comes after machine learning as a way of building models in chemical engineering, and what the field must do to take advantage of it (Kitchin et al. 2025). The authors frame the history of modeling as four paradigms: correlations used as tables or charts (steam tables, for example), which need no understanding of the underlying physics; manual fitting of mathematical models with tools such as log-log paper; computer-enabled multivariate and nonlinear regression and large first-principles models; and data-driven machine learning (ML), which, like the first paradigm, can work without physical understanding. They propose that the next step merges domain knowledge with data-driven models, enabled by differentiable programming.
Approach
The piece is an argument built from examples, many from the authors’ own groups. It defines differentiable programming as software built so that derivatives are available natively at any point of a program’s execution, made possible by automatic differentiation (AD), which applies the chain rule to give derivatives equivalent to analytical ones without hand derivation, finite differences or symbolic algebra. It then discusses opportunities for AD beyond ML training, what can be gained by looking inside ML models rather than treating them as black boxes, the data and team requirements for leveraging these tools, and the implications for education and workforce training.
Key findings
The authors argue that AD, not any particular model architecture, is the key innovation behind the success of ML: AD is decades old, but its commoditization in Python and Julia and its integration into ML frameworks is recent, and it is what allowed rapid prototyping of new architectures from multilayer perceptrons to transformers. Because derivatives appear throughout science and engineering (thermodynamics, driving forces, transport, sensitivity and uncertainty analysis, control and robotics), AD should also accelerate computational analysis beyond simulation. Examples cited include computing the degree of rate control in a complex reaction network, which was not practical with finite differences; an AD-based inverse problem method that maps two state spaces and was more efficient than nonlinear programming approaches; forces in machine-learned interatomic potentials obtained as derivatives of the energy; hybrid models that combine neural networks with mechanistic dynamics; and process control applications that use derivative information. AD also provides Jacobians, Hessians and their vector products, which opens new routes to optimization and regularization. A practical barrier is that large legacy codes in languages without AD support would need to be rewritten, a cost that must be weighed against long-term gains; the JAX ecosystem (differentiable NumPy and parts of SciPy) and Julia scientific ML are given as examples of this transition.
On opening the black box, the authors note that a regression neural network can be viewed as a nonlinear transform into a latent space in which the output is linear, equivalent for a single layer to an expansion in a basis of activation functions. The activation function therefore sets the mathematical properties of the trained model: hyperbolic tangent gives a smooth but non-convex model that is hard to optimize globally, while ReLU gives a piecewise linear model whose global optimum can be found provably with tools such as OMLT, with similar progress for linear decision trees. Uncertainty quantification remains unresolved: Bayesian and Gaussian process methods are skill-intensive and costly, Hessian-based methods fail for large networks with roughly 106 to 107 parameters where the Hessian cannot even be computed, and Hessians of smaller models are often singular.
The authors also stress that ML needs the right data, not merely large amounts of it, that experiment and model design are coupled, that many experimental domains cannot reach the data scales of text or molecular simulation (so methods for smaller datasets such as transfer, multimodal or foundation-model learning are needed), and that teams are required because one person rarely holds all the necessary skills.
Significance
The Comment’s central claim is that productive use of these tools requires an intersection of domain knowledge, ML and data science, and that most of the current academic and industrial workforce learned none of the latter two. The authors call on chemical engineering to innovate in education so that curricula fit both four-year degrees and lifelong upskilling, while acknowledging that any curriculum change carries opportunity costs. The piece serves as a programmatic statement for the group’s work on AD-enabled modeling, inverse problems and uncertainty quantification.