Decision Trees
Decision trees partition the input space by a sequence of binary splits and assign a simple model, often a constant or a linear regression, to each resulting region (leaf). They are fast to train greedily and to evaluate, and their structure is easy to inspect; splits are usually axis-aligned, though oblique trees allow splits on linear combinations of features.
Hyperplane trees with linear leaves (2025)
Sunshine and coworkers made oblique splits available to the standard greedy linear model decision tree algorithm by first lifting the data with an exhaustively enumerated, range-normalized set of integer-weighted linear combinations of the inputs (controlled by a maximum weight W and maximum number of terms N), and vectorized the candidate-split regressions in PyTorch (Sunshine et al. 2025). Leaves become convex polytopes rather than rectangles. On a two-variable test function, about 150-leaf trees gave mean absolute errors of 1.180 (constant leaves), 0.147 (linear leaves) and 0.100 (hyperplane tree), at the cost of roughly three times the training time of a linear tree; for water P-H-T data the hyperplane tree reached 0.453 K with 119 leaves against 0.521 K with 181 leaves for a linear tree. Hyperplane trees overfit sooner than linear trees beyond about 2000 leaves in the test case. Summary: sunshine2025.