Hyperplane decision trees as piecewise linear surrogate models for chemical process design
Ethan M. Sunshine, Carolina Colombo Tedesco, Sneha A. Akhade, Matthew J. McNenly, John R. Kitchin, Carl D. Laird, “Hyperplane decision trees as piecewise linear surrogate models for chemical process design”, Computers & Chemical Engineering 202, 109204 (2025). https://doi.org/10.1016/j.compchemeng.2025.109204
This summary was written by a language model from the paper and has not been checked line by line. Rely on the published paper for details.
Note on the source text: the extracted text is the journal pre-proof (accepted manuscript), so page layout and minor wording may differ from the final version of record. The prose and tables are readable; figures are not available in the text.
Question
Piecewise linear approximations let a nonlinear or simulation-based constraint be written as a mixed-integer linear one, which global solvers can handle, but fitting a piecewise linear model to data is hard. Globally optimal mixed-integer formulations of piecewise regression scale poorly with the number of data points, and linear model decision trees (LMDTs), which split the domain greedily and fit a linear regression in each leaf, only produce axis-aligned rectangular subdomains. Since each leaf becomes a binary variable in the optimization problem, a model that fits data accurately with fewer leaves is valuable. The paper asks whether allowing oblique (hyperplane) splits can give smaller, more accurate piecewise linear surrogates that can still be embedded in optimization problems (Sunshine et al. 2025).
Approach
The hyperplane tree (HT) keeps the standard greedy LMDT algorithm but first lifts the inputs with a fixed linear transformation, so that univariate splits on the new features are oblique splits in the original space. The transformation is enumerated exhaustively, in a scheme the authors liken to Miller indices: two hyperparameters set the maximum absolute integer weight W and the maximum number of nonzero terms N in each hyperplane, all valid integer weight vectors are generated and de-duplicated, and each column is scaled by the range of that input feature so the hyperplanes cover a spread of angles regardless of units. For three inputs with W = 2 and N = 2 this gives 21 candidate directions. The tree is then grown recursively: for each subdomain, B quantile thresholds are considered on each of the K transformed features, ridge-regularized linear regressions on the original inputs are fit on both sides of every candidate split, and the split with lowest loss (mean absolute error by default) is kept, subject to a minimum leaf size S and an optional maximum depth. The candidate regressions are vectorized into batched least-squares calls in PyTorch. The resulting leaves are convex polytopes. Trained trees are embedded in Pyomo models through OMLT as generalized disjunctive programs and reformulated (big-M or convex hull) into mixed-integer constraints. The code is released as the HyperplaneTree package in the LLNL systems2atoms repository.
Key findings
On a two-variable analytic test function with coupled nonlinear terms and several minima, trees of about 150 leaves gave mean absolute errors of 1.180 for a constant-leaf decision tree, 0.147 for an LMDT and 0.100 for an HT, with training times of 0.02, 0.74 and 2.30 s. HTs were the most accurate at a given leaf count up to roughly 2000 leaves, beyond which they overfit more readily than LMDTs. Optimizing over the HT surrogate as a MILP gave a minimum closer to the true one (found with IPOPT) than the LMDT surrogate. The authors note that functions whose nonlinearity comes from variable interactions benefit most, and that the discontinuous surrogates can change sharply near leaf boundaries.
The case study replaces the pressure-enthalpy-temperature relation of water (10,000 IAPWS-97 points on a 100 by 100 grid) in an IDAES heat exchanger cost minimization taken from earlier LMDT work. The HT reached an MAE of 0.453 K with 119 leaves, against 0.521 K with 181 leaves for an LMDT and 1.961 K with 469 leaves for a plain decision tree. Its oblique cuts follow the edge of the vapor-liquid envelope, where an LMDT needs many small rectangles. Embedded in the design problem (now a MINLP solved with BARON), all surrogates gave a total cost of about 4.76 million dollars, within 1% of the original NLP (4.78). HTs used 168 to 174 leaves versus 337 for the LMDT at similar MAE (about 0.3 K), but were not cheaper to solve: with W = 3 the 14 extra transformed variables raised BARON time to 6.4 s versus 1.3 s for the LMDT, while W = 2 (six extra variables) took 1.4 s. A ReLU neural network with four hidden layers (375 nonlinear units) gave a slightly lower cost but MAE 1.92 K and a BARON time of about 5098 s.
Significance
Hyperplane trees offer a cheap, greedy route to compact piecewise linear surrogates for global process optimization, and the comparison shows that tree surrogates can be far easier to optimize over than ReLU networks of similar accuracy. The authors stress that the gains over LMDTs are case dependent and that fewer leaves do not automatically mean faster optimization, so better reformulations of the hyperplane constraints are left as future work (Sunshine et al. 2025).