Decision Trees
Decision trees partition the input space by a sequence of binary splits and assign a simple model, often a constant or a linear regression, to each resulting region (leaf). They are fast to train greedily and to evaluate, and their structure is easy to inspect; splits are usually axis-aligned, though oblique trees allow splits on linear combinations of features.
Hyperplane trees with linear leaves (2025)
Sunshine and coworkers made oblique splits available to the standard greedy linear model decision tree algorithm by first lifting the data with an exhaustively enumerated, range-normalized set of integer-weighted linear combinations of the inputs (controlled by a maximum weight W and maximum number of terms N), and vectorized the candidate-split regressions in PyTorch (Sunshine et al. 2025). Leaves become convex polytopes rather than rectangles. On a two-variable test function, about 150-leaf trees gave mean absolute errors of 1.180 (constant leaves), 0.147 (linear leaves) and 0.100 (hyperplane tree), at the cost of roughly three times the training time of a linear tree; for water P-H-T data the hyperplane tree reached 0.453 K with 119 leaves against 0.521 K with 181 leaves for a linear tree. Hyperplane trees overfit sooner than linear trees beyond about 2000 leaves in the test case. Summary: sunshine2025.
Linear model decision trees as optimization surrogates (2025)
In a multiscale formic acid dehydrogenation study, LMDTs (regression trees with a linear fit in each leaf, grown greedily by testing candidate splits and keeping the one with the lowest regression error) were trained with the linear-tree package on 3600 to 100,000 samples per model, giving trees of 78 to 439 leaves and depths of 11 to 13 (Sunshine et al. 2025). The authors chose LMDTs over ReLU networks because they were easier and more reliable to train on these small datasets, and they flag two weaknesses: leaf regressions are unbounded, so predictions can leave the physical range, and extrapolation beyond the training domain becomes linear and can be unphysical (for example negative costs at large plant sizes). Summary: sunshine2025a.