Surrogate-Based Optimization
Surrogate-based optimization replaces expensive, nonlinear, implicit or simulation-based relations in an optimization problem with a cheaper data-driven model fitted to samples of that relation. When the surrogate is piecewise linear (for example a ReLU neural network or a linear model decision tree), it can be written as mixed-integer linear constraints, so the problem can be solved to global optimality with mixed-integer solvers, with the number of binary variables and the surrogate accuracy both governing the cost and quality of the solution.
Hyperplane decision trees for process design (2025)
Sunshine, Colombo Tedesco, Akhade, McNenly, Kitchin and Laird introduced hyperplane trees, linear model decision trees whose splits act on an enumerated set of linear combinations of the inputs, and embedded them through OMLT and Pyomo as disjunctive, then mixed-integer, constraints (Sunshine et al. 2025). In an IDAES heat exchanger cost minimization where the water pressure-enthalpy-temperature relation was replaced by a surrogate, LMDT, hyperplane tree and ReLU network surrogates all reached optima within 1% of the original NLP (about 4.76 versus 4.78 million dollars). The hyperplane trees needed about half as many leaves as the LMDT at similar error, yet were not faster to solve because of the added transformed variables (BARON times of 1.4 to 6.4 s versus 1.3 s), while the ReLU network took about 5098 s. The work shows that surrogate size, number of auxiliary variables and solver time must be weighed together. Summary: sunshine2025.
Multiscale co-optimization with linked LMDT surrogates (2025)
Sunshine and coworkers trained linear model decision trees on three black-box models of a formic acid dehydrogenation process (a DFT and microkinetic catalyst model, a CSTR flowsheet model and a techno-economic cost model), embedded them through OMLT in a single Pyomo generalized disjunctive program with a discrete choice of Pd, Cu or Pt, and solved the resulting MIP with GLPK in seconds (Sunshine et al. 2025). Optimizing all three surrogates together gave 14.01 $/kg H2 with a cheap Cu catalyst, about 40% below the 23.32 $/kg H2 of a sequential design that first maximized catalyst activity (Pt) and then yield. The paper also shows practical details of linking surrogates, such as using an inequality to cap a surrogate output (hydrogen yield) that unbounded leaf regressions can push above 100%, and notes that optima of discontinuous surrogates tend to lie on leaf boundaries where errors are largest. Summary: sunshine2025a.
Neural networks as constraints in a nonlinear program (2026)
In training neural DAEs, Lueg and coworkers embed a smooth feed-forward network as equality constraints in a large NLP solved by an interior point method, with the weights themselves as decision variables (Lueg et al. 2026). They identify the practical obstacles of this embedding (dense Jacobian and Hessian blocks in the weights, costly and frequently regularized factorizations) and show that external gray-box evaluation of the network plus an L-BFGS Hessian approximation make it tractable, echoing results by others for embedding trained networks in optimization problems. Summary: lueg2026.