Surrogate-Based Optimization
Surrogate-based optimization replaces expensive, nonlinear, implicit or simulation-based relations in an optimization problem with a cheaper data-driven model fitted to samples of that relation. When the surrogate is piecewise linear (for example a ReLU neural network or a linear model decision tree), it can be written as mixed-integer linear constraints, so the problem can be solved to global optimality with mixed-integer solvers, with the number of binary variables and the surrogate accuracy both governing the cost and quality of the solution.
Hyperplane decision trees for process design (2025)
Sunshine, Colombo Tedesco, Akhade, McNenly, Kitchin and Laird introduced hyperplane trees, linear model decision trees whose splits act on an enumerated set of linear combinations of the inputs, and embedded them through OMLT and Pyomo as disjunctive, then mixed-integer, constraints (Sunshine et al. 2025). In an IDAES heat exchanger cost minimization where the water pressure-enthalpy-temperature relation was replaced by a surrogate, LMDT, hyperplane tree and ReLU network surrogates all reached optima within 1% of the original NLP (about 4.76 versus 4.78 million dollars). The hyperplane trees needed about half as many leaves as the LMDT at similar error, yet were not faster to solve because of the added transformed variables (BARON times of 1.4 to 6.4 s versus 1.3 s), while the ReLU network took about 5098 s. The work shows that surrogate size, number of auxiliary variables and solver time must be weighed together. Summary: sunshine2025.
Neural networks as constraints in a nonlinear program (2026)
In training neural DAEs, Lueg and coworkers embed a smooth feed-forward network as equality constraints in a large NLP solved by an interior point method, with the weights themselves as decision variables (Lueg et al. 2026). They identify the practical obstacles of this embedding (dense Jacobian and Hessian blocks in the weights, costly and frequently regularized factorizations) and show that external gray-box evaluation of the network plus an L-BFGS Hessian approximation make it tractable, echoing results by others for embedding trained networks in optimization problems. Summary: lueg2026.