Solving an inverse problem with generative models
John R. Kitchin, “Solving an inverse problem with generative models”, Digital Discovery 4(7), 1856–1869 (2025). https://doi.org/10.1039/d5dd00137d
This summary was written by a language model from the paper and has not been checked line by line. Rely on the published paper for details.
Note on the source text: the extracted text is the accepted manuscript version. It is complete and readable, including the code listings and their printed outputs, although the two-column layout interleaves code with prose in places.
Question
An inverse problem asks which inputs to a system produce a desired set of outputs. The usual routes are to fit a forward model (inputs to outputs) and invert it with an iterative optimizer, or to fit a backward model (outputs to inputs) and simply evaluate it. This paper asks whether generative machine learning models, which learn to produce samples of input and output values together, can solve the same problem by conditional generation: fix the desired outputs and let the model generate the inputs (Kitchin 2025). The paper deliberately restricts itself to a well-behaved case with a unique, one-to-one solution, and frames the work as an introduction and motivation for the idea rather than a treatment of nonlinear or many-to-one problems.
Approach
The data came from Claude-Light, a remotely accessible instrument in which an RGB LED with three inputs (red, green and blue intensities between 0 and 1) illuminates a light sensor reporting 16-bit counts at eight visible wavelengths plus near-IR and clear channels. One hundred input settings were chosen by Latin hypercube sampling and measured through the instrument’s REST API. Exploratory analysis with a pair plot showed that the red input correlates with the 590 and 630 nm outputs, while green and blue each correlate with one channel (515 and 445 nm). The analysis then kept three inputs and four outputs, with an 80/20 train/test split.
The inverse problem was to find RGB settings giving 214 (16384) counts in the 445, 515 and 630 nm channels, leaving 590 nm unconstrained. Four methods were compared with short Python listings: a least-squares linear forward model inverted with scipy’s minimizer; a three-component partial least squares (PLS) backward model from scikit-learn mapping the three target outputs to the inputs; a Gaussian mixture model (GMM, via the gmr package, with a single component) of the joint distribution of inputs and outputs, conditioned analytically on the target outputs; and a ForestDiffusion model, which uses gradient-boosted trees with flow matching to learn a transformation from a reference distribution to the data distribution, trained to generate inputs conditioned on the three outputs. Appendices illustrate the underlying ideas with a conditioned two-dimensional Gaussian, an inverse-transform mapping of a uniform to a Gaussian distribution, and the interpolation and extrapolation behavior of a GMM and a ForestDiffusion model fit to a parabola with a gap in the data.
Key findings
All four methods gave essentially the same answer: about (0.49, 0.24, 0.52) for the forward model with optimization, and (0.47, 0.23, 0.52) for PLS and the GMM, with ForestDiffusion giving (0.47, 0.23, 0.51). The linear forward model and the PLS backward model both had test-set R2 near 1, and the conditionally generated outputs (GMM) and inputs (ForestDiffusion) also showed good parity with measured values.
The generative approaches added features the conventional ones lacked in this implementation. Drawing 100 conditional samples from the GMM gave a mean of (0.4686, 0.2342, 0.5158) with standard deviations near 0.002, so the solution is known to about three decimal places; repeated ForestDiffusion generation gave a similarly narrow spread. The GMM also lets one condition on any subset of variables, so the same model serves for forward prediction (condition on inputs) or inverse solution (condition on outputs), whereas the PLS model had to be constructed in advance around the specific outputs to be specified.
All data-driven methods misbehaved outside the training domain. PLS returned negative inputs for negative target counts and inputs above 4 for counts beyond the sensor’s saturation limit; the GMM similarly returned impossible inputs while reporting small uncertainties, extrapolating linearly from the nearest Gaussian. ForestDiffusion returned values inside the 0 to 1 range for an unattainable target, but these were wrong and still reported with high confidence, because a tree model outputs its outermost leaf values away from the data. The parabola example showed that neither generative model interpolates across the gap reliably either.
Significance
The paper argues that generative models are neither forward nor backward models but models of the joint distribution (or of a transformation between distributions), and that the choice of conditioning variables at generation time decides whether a sample is a forward prediction or an inverse solution. Each method needed about the same number of user code lines, thanks to library abstractions. Kitchin suggests that generative approaches could shift effort from choosing a model form to generating representative data and training and assessing the generative model, with possible uses in parameter estimation, optimization and uncertainty propagation. Open challenges named include how to sample efficiently for generative models, how to detect extrapolation, how to estimate uncertainty reliably, and how to handle nonlinear and many-to-one mappings. The manuscript itself was written as a literate org-mode document with code and data shared on Figshare.