Sequential Sampling Methods for Finding Classification Boundaries in Engineering Applications
Maya Bhat and John R. Kitchin, “Sequential Sampling Methods for Finding Classification Boundaries in Engineering Applications”, Industrial & Engineering Chemistry Research 62(37), 15326-15339 (2023). https://doi.org/10.1021/acs.iecr.3c02362
This summary was written by a language model from the paper and has not been checked line by line. Rely on the published paper for details.
Question
Many engineering problems ask where in a parameter space the outcome switches from acceptable to unacceptable: safe versus unsafe operation, a target product specification met or not, an alloy protected from oxidation or not. These are binary classification problems, and the usual way to map them is a full factorial grid of experiments followed by analysis. Active learning is normally applied to regression and optimization with continuous outputs, and Bayesian optimization is aimed at finding a best point rather than a boundary. The paper asks whether sequential batch sampling, steered by an interpretable classifier and its uncertainty, can locate a classification boundary more accurately and with fewer experiments than grid sampling (Bhat 2023).
Approach
The authors introduce three sequential sampling strategies, implemented in their Python package boundarysearch. All use a scikit-learn support vector classifier with a radial basis function kernel, chosen for interpretability and built-in class probabilities; the boundary is extracted as the 0.5 probability contour with matplotlib’s marching squares routine. The methods assume known domain limits and a boundary that forms a closed polygon. Broad initial sampling (BIS) starts from a sparse Latin hypercube over the whole domain, fits a boundary, then samples randomly among boundary points that lack a minimum number of nearby samples, repeating until converged. Low point density (LPD) starts from a tight cluster around one point near the boundary, grows a bounding box around the sampled region by a small increment each iteration, and chooses the next center on the predicted boundary where sample density is low and nearby classes are imbalanced, preferring points farthest from existing samples. Path tracing (PT) also needs a starting point near the boundary; it steps along the local tangent of the predicted boundary, tests two points perpendicular to it, and adapts the step length depending on whether the two points fall in the same or different classes, until the path closes.
Four metrics judge each run: the number of experiments, the area of the predicted region overlapping the true region, the mean distance of samples from the boundary, and the area of the 95% confidence band (from class probabilities) scaled by boundary length. These are combined in a weighted score with weights 0.34, 0.34, 0.16 and 0.16, lower being better. Artificial noise is added through a sigmoid in the distance to the true boundary, so points near the boundary are more often misclassified. Each method is run 10 times and medians are reported against a full factorial control. The three two-dimensional test cases are an analytic shower mixing problem (hot water at 115 °F and cold at 65 °F, target 98 to 105 °F and 10 to 30 flow units/min), the bulk oxidation protection boundary in ternary Ni-Fe-Al alloys digitized from Payne et al., and the liquid-gas coexistence region of the reduced van der Waals equation of state found by the Maxwell construction.
Key findings
For the shower problem the control used 100 grid points and recovered an overlap area of 1.29 against a true area of 1.48. PT needed a median of 70 experiments and BIS 73, while LPD needed 116; all three recovered overlap areas of 1.40 to 1.41, and LPD gave the smallest mean distance to the boundary (0.07). For the oxidation boundary the control was the 169-point uniform ternary grid used experimentally, with an overlap of 0.08 against a true 0.12. BIS reached the boundary with a median of 49 experiments, more than a threefold reduction, and PT with 72, whereas LPD used 186; the sampling methods recovered 0.108 to 0.112. For the van der Waals case PT needed 80 experiments versus 100 for the control and 146 for LPD, and all methods recovered 0.409 to 0.412 against a true 0.43, compared with 0.343 for the grid. LPD’s extra cost came from extrapolating the boundary too far and then correcting it. On the combined score PT was best in all three applications (0.53, 0.33 and 0.28), with BIS a close second, and the grid control was worst for the shower and equation of state cases.
Significance
The work reframes boundary mapping as an active learning task with acquisition rules designed for boundaries rather than optima, and gives practical guidance: PT when high confidence along a single boundary matters, BIS when experiments are scarce, little is known in advance, or several boundary regions may exist, and LPD when balanced exploration is wanted and experiments are cheap. Limitations noted by the authors are the closed polygon assumption, the need for a starting point near the boundary for LPD and PT, PT’s blindness to other boundaries, and only two-dimensional demonstrations. They suggest combining BIS for discovery with PT or LPD for refinement and extending to higher dimensions.