Differentiable Programming
Differentiable programming means writing software (a function, a package or a whole language) so that derivatives of its outputs with respect to its inputs are available natively wherever the code runs. It rests on automatic differentiation (AD), which applies the chain rule through the operations a program actually executes. The result is numerically equivalent to an analytical derivative, but nobody has to derive or code that derivative by hand, and it avoids the truncation error and cost of finite differences and the expression swell of symbolic differentiation. AD can deliver gradients, Jacobians, Hessians and their vector products. Frameworks such as JAX (a differentiable NumPy with parts of SciPy), PyTorch, TensorFlow and Julia’s scientific ML ecosystem have made it widely available.
Automatic differentiation for regression uncertainty (2022)
Zhan and Kitchin used automatic differentiation (autograd in a short example, PyTorch for a neural network potential) to obtain, in one line each, the loss Hessian and prediction gradients required by the delta method, turning a classical nonlinear regression error estimate into a tool usable on machine learning models (Zhan 2021). For an 861-parameter model with 1900 training points the inverse Fisher information took about 5 minutes on a laptop CPU, with cost scaling quadratically in parameters. Summary: zhan2022.
Differentiable process models for inverse mapping (2023)
Alves, Kitchin and Lima show a concrete use of differentiable programming in process systems engineering (Alves et al. 2023). They rewrote a CSTR model and an eight-ODE membrane reactor model in JAX (using its functional array updates and its differentiable Dormand-Prince ODE solver) so that Jacobians could be obtained by AD, then used those Jacobians with the implicit function theorem to integrate from output space back to input space. Exact AD derivatives also gave the sensitivities of each input to each output along the whole path and a phase portrait of the inverse map, revealing stiffness (derivatives differing by orders of magnitude) in both models. They propose JAX-compatible surrogates, such as Gaussian processes, for models trapped in closed-source simulators where AD cannot be applied directly. Summary: alves2023.
Differentiable programming as the next modeling paradigm (2025)
Kitchin, Alves and Laird argue that AD is the core technology behind modern machine learning, and that its most important future impact in chemical engineering lies beyond training black-box models (Kitchin et al. 2025). Derivatives appear throughout the discipline (thermodynamics, driving forces, transport, sensitivity and uncertainty analysis, process control), so AD-enabled codes can speed up analyses such as degree of rate control in complex reaction networks, inverse problems that map one state space onto another, forces for machine-learned potentials, and hybrid models that embed neural networks in mechanistic dynamics. They identify the cost of rewriting large legacy codes in languages without AD as the main practical barrier, and argue that using these tools well requires combining domain knowledge, ML and data science, which in turn requires changes to chemical engineering education and workforce training. Summary: kitchin2025.
Differentiating through NLP solutions to train neural DAEs (2026)
Lueg, Alves, Schicksnus, Kitchin, Laird and Biegler treat the solution of a constrained NLP as a differentiable layer (Lueg et al. 2026). In their bi-level training scheme each per-trajectory collocation NLP is solved with the network weights fixed, and the gradient of the loss with respect to the weights follows from the implicit function theorem applied to the KKT conditions; the required terms reduce to vector-Jacobian products of the network, evaluated in parallel with JAX. JAX also supplies the values, Jacobians and Hessians of the network constraints in the fully simultaneous NLP through Pyomo gray-box models. Summary: lueg2026.
See also Inverse problems, Interpretable machine learning and Neural network potentials.